GitHub/school-of-sre
1
0
Fork 0
mirror of https://github.com/linkedin/school-of-sre synced 2026-01-17 05:58:02 +00:00
Code Issues Packages Projects Releases Wiki Activity

Deployed 52e7ed5 with MkDocs version: 1.1.2

Browse Source
This commit is contained in:
github-actions
2021-02-24 16:02:49 +00:00
parent 65fe7bf20b
commit bc0f89d4c8
22 changed files with 629 additions and 625 deletions
Download Patch File Download Diff File Expand all files Collapse all files

99
linux_networking/dns/index.html
View File

@@ -1269,63 +1269,64 @@
<h1 id="dns">DNS</h1>
<p>Domain Names are the simple human-readable names for websites. The Internet understands only IP addresses, but since memorizing incoherent numbers is not practical, domain names are used instead. These domain names are translated into IP addresses by the DNS infrastructure. When somebody tries to open <a href="https://www.linkedin.com">www.linkedin.com</a> in the browser, the browser tries to convert <a href="https://www.linkedin.com">www.linkedin.com</a> to an IP Address. This process is called DNS resolution. A simple pseudocode depicting this process looks this</p>
<div class="highlight"><pre><span></span><code><span class="n">ip</span><span class="p">,</span> <span class="n">err</span> <span class="o">=</span> <span class="n">getIPAddress</span><span class="p">(</span><span class="n">domainName</span><span class="p">)</span>
<span class="k">if</span> <span class="n">err</span><span class="p">:</span>
<span class="nb">print</span><span class="p">(</span><span class="err">“</span><span class="n">unknown</span> <span class="n">Host</span> <span class="ne">Exception</span> <span class="k">while</span> <span class="n">trying</span> <span class="n">to</span> <span class="n">resolve</span><span class="p">:</span><span class="o">%</span><span class="n">s</span><span class="err">”</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">domainName</span><span class="p">))</span>
</code></pre></div>
<pre><code class="language-python">ip, err = getIPAddress(domainName)
if err:
print(“unknown Host Exception while trying to resolve:%s”.format(domainName))
</code></pre>
<p>Now let’s try to understand what happens inside the getIPAddress function. The browser would have a DNS cache of its own where it checks if there is a mapping for the domainName to an IP Address already available, in which case the browser uses that IP address. If no such mapping exists, the browser calls gethostbyname syscall to ask the operating system to find the IP address for the given domainName</p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">getIPAddress</span><span class="p">(</span><span class="n">domainName</span><span class="p">):</span>
<span class="n">resp</span><span class="p">,</span> <span class="n">fail</span> <span class="o">=</span> <span class="n">lookupCache</span><span class="p">(</span><span class="n">domainName</span><span class="p">)</span>
<span class="n">If</span> <span class="ow">not</span> <span class="n">fail</span><span class="p">:</span>
<span class="k">return</span> <span class="n">resp</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">resp</span><span class="p">,</span> <span class="n">err</span> <span class="o">=</span> <span class="n">gethostbyname</span><span class="p">(</span><span class="n">domainName</span><span class="p">)</span>
<span class="k">if</span> <span class="n">err</span><span class="p">:</span>
<span class="k">return</span> <span class="n">null</span><span class="p">,</span> <span class="n">err</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">return</span> <span class="n">resp</span>
</code></pre></div>
<pre><code class="language-python">def getIPAddress(domainName):
resp, fail = lookupCache(domainName)
If not fail:
return resp
else:
resp, err = gethostbyname(domainName)
if err:
return null, err
else:
return resp
</code></pre>
<p>Now lets understand what operating system kernel does when the <a href="https://man7.org/linux/man-pages/man3/gethostbyname.3.html">gethostbyname</a> function is called. The Linux operating system looks at the file <a href="https://man7.org/linux/man-pages/man5/nsswitch.conf.5.html">/etc/nsswitch.conf</a> file which usually has a line</p>
<div class="highlight"><pre><span></span><code>hosts: files dns
</code></pre></div>
<pre><code class="language-bash">hosts: files dns
</code></pre>
<p>This line means the OS has to look up first in file (/etc/hosts) and then use DNS protocol to do the resolution if there is no match in /etc/hosts. </p>
<p>The file /etc/hosts is of format</p>
<p>IPAddress FQDN [FQDN].*</p>
<div class="highlight"><pre><span></span><code><span class="m">127</span>.0.0.1 localhost.localdomain localhost
<pre><code class="language-bash">127.0.0.1 localhost.localdomain localhost
::1 localhost.localdomain localhost
</code></pre></div>
</code></pre>
<p>If a match exists for a domain in this file then that IP address is returned by the OS. Lets add a line to this file</p>
<div class="highlight"><pre><span></span><code><span class="m">127</span>.0.0.1 test.linkedin.com
</code></pre></div>
<pre><code class="language-bash">127.0.0.1 test.linkedin.com
</code></pre>
<p>And then do ping test.linkedin.com</p>
<div class="highlight"><pre><span></span><code>ping test.linkedin.com -n
</code></pre></div>
<div class="highlight"><pre><span></span><code>PING test.linkedin.com <span class="o">(</span><span class="m">127</span>.0.0.1<span class="o">)</span> <span class="m">56</span><span class="o">(</span><span class="m">84</span><span class="o">)</span> bytes of data.
<span class="m">64</span> bytes from <span class="m">127</span>.0.0.1: <span class="nv">icmp_seq</span><span class="o">=</span><span class="m">1</span> <span class="nv">ttl</span><span class="o">=</span><span class="m">64</span> <span class="nv">time</span><span class="o">=</span><span class="m">0</span>.047 ms
<span class="m">64</span> bytes from <span class="m">127</span>.0.0.1: <span class="nv">icmp_seq</span><span class="o">=</span><span class="m">2</span> <span class="nv">ttl</span><span class="o">=</span><span class="m">64</span> <span class="nv">time</span><span class="o">=</span><span class="m">0</span>.036 ms
<span class="m">64</span> bytes from <span class="m">127</span>.0.0.1: <span class="nv">icmp_seq</span><span class="o">=</span><span class="m">3</span> <span class="nv">ttl</span><span class="o">=</span><span class="m">64</span> <span class="nv">time</span><span class="o">=</span><span class="m">0</span>.037 ms
</code></pre></div>
<pre><code class="language-bash">ping test.linkedin.com -n
</code></pre>
<pre><code class="language-bash">PING test.linkedin.com (127.0.0.1) 56(84) bytes of data.
64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.047 ms
64 bytes from 127.0.0.1: icmp_seq=2 ttl=64 time=0.036 ms
64 bytes from 127.0.0.1: icmp_seq=3 ttl=64 time=0.037 ms
</code></pre>
<p>As mentioned earlier, if no match exists in /etc/hosts, the OS tries to do a DNS resolution using the DNS protocol. The linux system makes a DNS request to the first IP in /etc/resolv.conf. If there is no response, requests are sent to subsequent servers in resolv.conf. These servers in resolv.conf are called DNS resolvers. The DNS resolvers are populated by <a href="https://en.wikipedia.org/wiki/Dynamic_Host_Configuration_Protocol">DHCP</a> or statically configured by an administrator.
<a href="https://linux.die.net/man/1/dig">Dig</a> is a userspace DNS system which creates and sends request to DNS resolvers and prints the response it receives to the console.</p>
<div class="highlight"><pre><span></span><code><span class="c1">#run this command in one shell to capture all DNS requests</span>
sudo tcpdump -s <span class="m">0</span> -A -i any port <span class="m">53</span>
<span class="c1">#make a dig request from another shell</span>
<pre><code class="language-bash">#run this command in one shell to capture all DNS requests
sudo tcpdump -s 0 -A -i any port 53
#make a dig request from another shell
dig linkedin.com
</code></pre></div>
<div class="highlight"><pre><span></span><code><span class="m">13</span>:19:54.432507 IP <span class="m">172</span>.19.209.122.56497 &gt; <span class="m">172</span>.23.195.101.53: <span class="m">527</span>+ <span class="o">[</span>1au<span class="o">]</span> A? linkedin.com. <span class="o">(</span><span class="m">41</span><span class="o">)</span>
....E..E....@.n....z...e...5.1.:... .........linkedin.com.......<span class="o">)</span>........
<span class="m">13</span>:19:54.485131 IP <span class="m">172</span>.23.195.101.53 &gt; <span class="m">172</span>.19.209.122.56497: <span class="m">527</span> <span class="m">1</span>/0/1 A <span class="m">108</span>.174.10.10 <span class="o">(</span><span class="m">57</span><span class="o">)</span>
....E..U..@.<span class="p">|</span>. ....e...z.5...A...............linkedin.com..............3..l.
</code></pre>
<pre><code class="language-bash">13:19:54.432507 IP 172.19.209.122.56497 &gt; 172.23.195.101.53: 527+ [1au] A? linkedin.com. (41)
....E..E....@.n....z...e...5.1.:... .........linkedin.com.......)........
13:19:54.485131 IP 172.23.195.101.53 &gt; 172.19.209.122.56497: 527 1/0/1 A 108.174.10.10 (57)
....E..U..@.|. ....e...z.5...A...............linkedin.com..............3..l.
..<span class="o">)</span>........
</code></pre></div>
..)........
</code></pre>
<p>The packet capture shows a request is made to 172.23.195.101:53 (this is the resolver in /etc/resolv.conf) for linkedin.com and a response is received from 172.23.195.101 with the IP address of linkedin.com 108.174.10.10</p>
<p>Now let's try to understand how DNS resolver tries to find the IP address of linkedin.com. DNS resolver first looks at its cache. Since many devices in the network can query for the domain name linkedin.com, the name resolution result may already exist in the cache. If there is a cache miss, it starts the DNS resolution process. The DNS server breaks “linkedin.com” to “.”, “com.” and “linkedin.com.” and starts DNS resolution from “.”. The “.” is called root domain and those IPs are known to the DNS resolver software. DNS resolver queries the root domain Nameservers to find the right nameservers which could respond regarding details for "com.". The address of the authoritative nameserver of “com.” is returned. Now the DNS resolution service contacts the authoritative nameserver for “com.” to fetch the authoritative nameserver for “linkedin.com”. Once an authoritative nameserver of “linkedin.com” is known, the resolver contacts Linkedin’s nameserver to provide the IP address of “linkedin.com”. This whole process can be visualized by running </p>
<div class="highlight"><pre><span></span><code>dig +trace linkedin.com
</code></pre></div>
<p><div class="highlight"><pre><span></span><code>linkedin.com. <span class="m">3600</span> IN A <span class="m">108</span>.174.10.10
</code></pre></div>
This DNS response has 5 fields where the first field is the request and the last field is the response. The second field is the Time to Live which says how long the DNS response is valid in seconds. In this case this mapping of linkedin.com is valid for 1 hour. This is how the resolvers and application(browser) maintain their cache. Any request for linkedin.com beyond 1 hour will be treated as a cache miss as the mapping has expired its TTL and the whole process has to be redone.
<pre><code class="language-bash">dig +trace linkedin.com
</code></pre>
<pre><code class="language-bash">linkedin.com. 3600 IN A 108.174.10.10
</code></pre>
<p>This DNS response has 5 fields where the first field is the request and the last field is the response. The second field is the Time to Live which says how long the DNS response is valid in seconds. In this case this mapping of linkedin.com is valid for 1 hour. This is how the resolvers and application(browser) maintain their cache. Any request for linkedin.com beyond 1 hour will be treated as a cache miss as the mapping has expired its TTL and the whole process has to be redone.
The 4th field says the type of DNS response/request. Some of the various DNS query types are
A, AAAA, NS, TXT, PTR, MX and CNAME.
- A record returns IPV4 address of the domain name
@@ -1333,12 +1334,12 @@ A, AAAA, NS, TXT, PTR, MX and CNAME.
- NS record returns the authoritative nameserver for the domain name
- CNAME records are aliases to the domain names. Some domains point to other domain names and resolving the latter domain name gives an IP which is used as an IP for the former domain name as well. Example www.linkedin.com’s IP address is the same as 2-01-2c3e-005a.cdx.cedexis.net.
- For the brevity we are not discussing other DNS record types, the RFC of each of these records are available <a href="https://en.wikipedia.org/wiki/List_of_DNS_record_types">here</a>.</p>
<p><div class="highlight"><pre><span></span><code>dig A linkedin.com +short
<span class="m">108</span>.174.10.10
<pre><code class="language-bash">dig A linkedin.com +short
108.174.10.10
dig AAAA linkedin.com +short
<span class="m">2620</span>:109:c002::6cae:a0a
2620:109:c002::6cae:a0a
dig NS linkedin.com +short
@@ -1352,9 +1353,9 @@ ns3.p43.dynect.net.
dns1.p09.nsone.net.
dig www.linkedin.com CNAME +short
<span class="m">2</span>-01-2c3e-005a.cdx.cedexis.net.
</code></pre></div>
Armed with these fundamentals of DNS lets see usecases where DNS is used by SREs.</p>
2-01-2c3e-005a.cdx.cedexis.net.
</code></pre>
<p>Armed with these fundamentals of DNS lets see usecases where DNS is used by SREs.</p>
<h2 id="applications-in-sre-role">Applications in SRE role</h2>
<p>This section covers some of the common solutions SRE can derive from DNS</p>
<ol>

149
linux_networking/http/index.html
View File

@@ -1226,112 +1226,113 @@
<h1 id="http">HTTP</h1>
<p>Till this point we have only got the IP address of linkedin.com. The HTML page of linkedin.com is served by HTTP protocol which the browser renders. Browser sends a HTTP request to the IP of the server determined above.
Request has a verb GET, PUT, POST followed by a path and query parameters and lines of key value pair which gives information about the client and capabilities of the client like contents it can accept and a body (usually in POST or PUT)</p>
<p><div class="highlight"><pre><span></span><code><span class="c1"># Eg run the following in your container and have a look at the headers </span>
<pre><code class="language-bash"># Eg run the following in your container and have a look at the headers
curl linkedin.com -v
</code></pre></div>
<div class="highlight"><pre><span></span><code>* Connected to linkedin.com <span class="o">(</span><span class="m">108</span>.174.10.10<span class="o">)</span> port <span class="m">80</span> <span class="o">(</span><span class="c1">#0)</span>
</code></pre>
<pre><code class="language-bash">* Connected to linkedin.com (108.174.10.10) port 80 (#0)
&gt; GET / HTTP/1.1
&gt; Host: linkedin.com
&gt; User-Agent: curl/7.64.1
&gt; Accept: */*
&gt;
&lt; HTTP/1.1 <span class="m">301</span> Moved Permanently
&lt; Date: Mon, <span class="m">09</span> Nov <span class="m">2020</span> <span class="m">10</span>:39:43 GMT
&lt; HTTP/1.1 301 Moved Permanently
&lt; Date: Mon, 09 Nov 2020 10:39:43 GMT
&lt; X-Li-Pop: prod-esv5
&lt; X-LI-Proto: http/1.1
&lt; Location: https://www.linkedin.com/
&lt; Content-Length: <span class="m">0</span>
&lt; Content-Length: 0
&lt;
* Connection <span class="c1">#0 to host linkedin.com left intact</span>
* Closing connection <span class="m">0</span>
</code></pre></div></p>
* Connection #0 to host linkedin.com left intact
* Closing connection 0
</code></pre>
<p>Here, in the first line GET is the verb, / is the path and 1.1 is the HTTP protocol version. Then there are key value pairs which give client capabilities and some details to the server. The server responds back with HTTP version, <a href="https://en.wikipedia.org/wiki/List_of_HTTP_status_codes">Status Code and Status message</a>. Status codes 2xx means success, 3xx denotes redirection, 4xx denotes client side errors and 5xx server side errors.</p>
<p>We will now jump in to see the difference between HTTP/1.0 and HTTP/1.1. </p>
<div class="highlight"><pre><span></span><code><span class="c1">#On the terminal type</span>
telnet www.linkedin.com <span class="m">80</span>
<span class="c1">#Copy and paste the following with an empty new line at last in the telnet STDIN</span>
<pre><code class="language-bash">#On the terminal type
telnet www.linkedin.com 80
#Copy and paste the following with an empty new line at last in the telnet STDIN
GET / HTTP/1.1
HOST:linkedin.com
USER-AGENT: curl
</code></pre></div>
</code></pre>
<p>This would get server response and waits for next input as the underlying connection to www.linkedin.com can be reused for further queries. While going through TCP, we can understand the benefits of this. But in HTTP/1.0 this connection will be immediately closed after the response meaning new connection has to be opened for each query. HTTP/1.1 can have only one inflight request in an open connection but connection can be reused for multiple requests one after another. One of the benefits of HTTP/2.0 over HTTP/1.1 is we can have multiple inflight requests on the same connection. We are restricting our scope to generic HTTP and not jumping to the intricacies of each protocol version but they should be straight forward to understand post the course.</p>
<p>HTTP is called <strong>stateless protocol</strong>. This section we will try to understand what stateless means. Say we logged in to linkedin.com, each request to linkedin.com from the client will have no context of the user and it makes no sense to prompt user to login for each page/resource. This problem of HTTP is solved by <em>COOKIE</em>. A user is created a session when a user logs in. This session identifier is sent to the browser via <em>SET-COOKIE</em> header. The browser stores the COOKIE till the expiry set by the server and sends the cookie for each request from hereon for linkedin.com. More details on cookies are available <a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Cookies">here</a>. Cookies are a critical piece of information like password and since HTTP is a plain text protocol, any man in the middle can capture either password or cookies and can breach the privacy of the user. Similarly as discussed during DNS a spoofed IP of linkedin.com can cause a phishing attack on users where an user can give linkedin’s password to login on the malicious site. To solve both problems HTTPs came in place and HTTPs has to be mandated.</p>
<p>HTTPS has to provide server identification and encryption of data between client and server. The server administrator has to generate a private public key pair and certificate request. This certificate request has to be signed by a certificate authority which converts the certificate request to a certificate. The server administrator has to update the certificate and private key to the webserver. The certificate has details about the server (like domain name for which it serves, expiry date), public key of the server. The private key is a secret to the server and losing the private key loses the trust the server provides. When clients connect, the client sends a HELLO. The server sends its certificate to the client. The client checks the validity of the cert by seeing if it is within its expiry time, if it is signed by a trusted authority and the hostname in the cert is the same as the server. This validation makes sure the server is the right server and there is no phishing. Once that is validated, the client negotiates a symmetrical key and cipher with the server by encrypting the negotiation with the public key of the server. Nobody else other than the server who has the private key can understand this data. Once negotiation is complete, that symmetric key and algorithm is used for further encryption which can be decrypted only by client and server from thereon as they only know the symmetric key and algorithm. The switch to symmetric algorithm from asymmetric encryption algorithm is to not strain the resources of client devices as symmetric encryption is generally less resource intensive than asymmetric. </p>
<p><div class="highlight"><pre><span></span><code><span class="c1">#Try the following on your terminal to see the cert details like Subject Name(domain name), Issuer details, Expiry date</span>
<pre><code class="language-bash">#Try the following on your terminal to see the cert details like Subject Name(domain name), Issuer details, Expiry date
curl https://www.linkedin.com -v
</code></pre></div>
<div class="highlight"><pre><span></span><code>* Connected to www.linkedin.com <span class="o">(</span><span class="m">13</span>.107.42.14<span class="o">)</span> port <span class="m">443</span> <span class="o">(</span><span class="c1">#0)</span>
</code></pre>
<pre><code class="language-bash">* Connected to www.linkedin.com (13.107.42.14) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully <span class="nb">set</span> certificate verify locations:
* successfully set certificate verify locations:
* CAfile: /etc/ssl/cert.pem
CApath: none
* TLSv1.2 <span class="o">(</span>OUT<span class="o">)</span>, TLS handshake, Client hello <span class="o">(</span><span class="m">1</span><span class="o">)</span>:
<span class="o">}</span> <span class="o">[</span><span class="m">230</span> bytes data<span class="o">]</span>
* TLSv1.2 <span class="o">(</span>IN<span class="o">)</span>, TLS handshake, Server hello <span class="o">(</span><span class="m">2</span><span class="o">)</span>:
<span class="o">{</span> <span class="o">[</span><span class="m">90</span> bytes data<span class="o">]</span>
* TLSv1.2 <span class="o">(</span>IN<span class="o">)</span>, TLS handshake, Certificate <span class="o">(</span><span class="m">11</span><span class="o">)</span>:
<span class="o">{</span> <span class="o">[</span><span class="m">3171</span> bytes data<span class="o">]</span>
* TLSv1.2 <span class="o">(</span>IN<span class="o">)</span>, TLS handshake, Server key exchange <span class="o">(</span><span class="m">12</span><span class="o">)</span>:
<span class="o">{</span> <span class="o">[</span><span class="m">365</span> bytes data<span class="o">]</span>
* TLSv1.2 <span class="o">(</span>IN<span class="o">)</span>, TLS handshake, Server finished <span class="o">(</span><span class="m">14</span><span class="o">)</span>:
<span class="o">{</span> <span class="o">[</span><span class="m">4</span> bytes data<span class="o">]</span>
* TLSv1.2 <span class="o">(</span>OUT<span class="o">)</span>, TLS handshake, Client key exchange <span class="o">(</span><span class="m">16</span><span class="o">)</span>:
<span class="o">}</span> <span class="o">[</span><span class="m">102</span> bytes data<span class="o">]</span>
* TLSv1.2 <span class="o">(</span>OUT<span class="o">)</span>, TLS change cipher, Change cipher spec <span class="o">(</span><span class="m">1</span><span class="o">)</span>:
<span class="o">}</span> <span class="o">[</span><span class="m">1</span> bytes data<span class="o">]</span>
* TLSv1.2 <span class="o">(</span>OUT<span class="o">)</span>, TLS handshake, Finished <span class="o">(</span><span class="m">20</span><span class="o">)</span>:
<span class="o">}</span> <span class="o">[</span><span class="m">16</span> bytes data<span class="o">]</span>
* TLSv1.2 <span class="o">(</span>IN<span class="o">)</span>, TLS change cipher, Change cipher spec <span class="o">(</span><span class="m">1</span><span class="o">)</span>:
<span class="o">{</span> <span class="o">[</span><span class="m">1</span> bytes data<span class="o">]</span>
* TLSv1.2 <span class="o">(</span>IN<span class="o">)</span>, TLS handshake, Finished <span class="o">(</span><span class="m">20</span><span class="o">)</span>:
<span class="o">{</span> <span class="o">[</span><span class="m">16</span> bytes data<span class="o">]</span>
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
} [230 bytes data]
* TLSv1.2 (IN), TLS handshake, Server hello (2):
{ [90 bytes data]
* TLSv1.2 (IN), TLS handshake, Certificate (11):
{ [3171 bytes data]
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
{ [365 bytes data]
* TLSv1.2 (IN), TLS handshake, Server finished (14):
{ [4 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
} [102 bytes data]
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.2 (OUT), TLS handshake, Finished (20):
} [16 bytes data]
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
{ [1 bytes data]
* TLSv1.2 (IN), TLS handshake, Finished (20):
{ [16 bytes data]
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server accepted to use h2
* Server certificate:
* subject: <span class="nv">C</span><span class="o">=</span>US<span class="p">;</span> <span class="nv">ST</span><span class="o">=</span>California<span class="p">;</span> <span class="nv">L</span><span class="o">=</span>Sunnyvale<span class="p">;</span> <span class="nv">O</span><span class="o">=</span>LinkedIn Corporation<span class="p">;</span> <span class="nv">CN</span><span class="o">=</span>www.linkedin.com
* start date: Oct <span class="m">2</span> <span class="m">00</span>:00:00 <span class="m">2020</span> GMT
* expire date: Apr <span class="m">2</span> <span class="m">12</span>:00:00 <span class="m">2021</span> GMT
* subjectAltName: host <span class="s2">&quot;www.linkedin.com&quot;</span> matched cert<span class="s1">&#39;s &quot;www.linkedin.com&quot;</span>
<span class="s1">* issuer: C=US; O=DigiCert Inc; CN=DigiCert SHA2 Secure Server CA</span>
<span class="s1">* SSL certificate verify ok.</span>
<span class="s1">* Using HTTP2, server supports multi-use</span>
<span class="s1">* Connection state changed (HTTP/2 confirmed)</span>
<span class="s1">* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0</span>
<span class="s1">* Using Stream ID: 1 (easy handle 0x7fb055808200)</span>
<span class="s1">* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!</span>
<span class="s1"> 0 82117 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0</span>
<span class="s1">* Connection #0 to host www.linkedin.com left intact</span>
<span class="s1">HTTP/2 200 </span>
<span class="s1">cache-control: no-cache, no-store</span>
<span class="s1">pragma: no-cache</span>
<span class="s1">content-length: 82117</span>
<span class="s1">content-type: text/html; charset=utf-8</span>
<span class="s1">expires: Thu, 01 Jan 1970 00:00:00 GMT</span>
<span class="s1">set-cookie: JSESSIONID=ajax:2747059799136291014; SameSite=None; Path=/; Domain=.www.linkedin.com; Secure</span>
<span class="s1">set-cookie: lang=v=2&amp;lang=en-us; SameSite=None; Path=/; Domain=linkedin.com; Secure</span>
<span class="s1">set-cookie: bcookie=&quot;v=2&amp;70bd59e3-5a51-406c-8e0d-dd70befa8890&quot;; domain=.linkedin.com; Path=/; Secure; Expires=Wed, 09-Nov-2022 22:27:42 GMT; SameSite=None</span>
<span class="s1">set-cookie: bscookie=&quot;v=1&amp;202011091050107ae9b7ac-fe97-40fc-830d-d7a9ccf80659AQGib5iXwarbY8CCBP94Q39THkgUlx6J&quot;; domain=.www.linkedin.com; Path=/; Secure; Expires=Wed, 09-Nov-2022 22:27:42 GMT; HttpOnly; SameSite=None</span>
<span class="s1">set-cookie: lissc=1; domain=.linkedin.com; Path=/; Secure; Expires=Tue, 09-Nov-2021 10:50:10 GMT; SameSite=None</span>
<span class="s1">set-cookie: lidc=&quot;b=VGST04:s=V:r=V:g=2201:u=1:i=1604919010:t=1605005410:v=1:sig=AQHe-KzU8i_5Iy6MwnFEsgRct3c9Lh5R&quot;; Expires=Tue, 10 Nov 2020 10:50:10 GMT; domain=.linkedin.com; Path=/; SameSite=None; Secure</span>
<span class="s1">x-fs-txn-id: 2b8d5409ba70</span>
<span class="s1">x-fs-uuid: 61bbf94956d14516302567fc882b0000</span>
<span class="s1">expect-ct: max-age=86400, report-uri=&quot;https://www.linkedin.com/platform-telemetry/ct&quot;</span>
<span class="s1">x-xss-protection: 1; mode=block</span>
<span class="s1">content-security-policy-report-only: default-src &#39;</span>none<span class="s1">&#39;; connect-src &#39;</span>self<span class="s1">&#39; www.linkedin.com www.google-analytics.com https://dpm.demdex.net/id lnkd.demdex.net blob: https://linkedin.sc.omtrdc.net/b/ss/ static.licdn.com static-exp1.licdn.com static-exp2.licdn.com static-exp3.licdn.com; script-src &#39;</span>sha256-THuVhwbXPeTR0HszASqMOnIyxqEgvGyBwSPBKBF/iMc<span class="o">=</span><span class="s1">&#39; &#39;</span>sha256-PyCXNcEkzRWqbiNr087fizmiBBrq9O6GGD8eV3P09Ik<span class="o">=</span><span class="s1">&#39; &#39;</span>sha256-2SQ55Erm3CPCb+k03EpNxU9bdV3XL9TnVTriDs7INZ4<span class="o">=</span><span class="s1">&#39; &#39;</span>sha256-S/KSPe186K/1B0JEjbIXcCdpB97krdzX05S+dHnQjUs<span class="o">=</span><span class="s1">&#39; platform.linkedin.com platform-akam.linkedin.com platform-ecst.linkedin.com platform-azur.linkedin.com static.licdn.com static-exp1.licdn.com static-exp2.licdn.com static-exp3.licdn.com; img-src data: blob: *; font-src data: *; style-src &#39;</span>self<span class="s1">&#39; &#39;</span>unsafe-inline<span class="s1">&#39; static.licdn.com static-exp1.licdn.com static-exp2.licdn.com static-exp3.licdn.com; media-src dms.licdn.com; child-src blob: *; frame-src &#39;</span>self<span class="s1">&#39; lnkd.demdex.net linkedin.cdn.qualaroo.com; manifest-src &#39;</span>self<span class="s1">&#39;; report-uri https://www.linkedin.com/platform-telemetry/csp?f=g</span>
<span class="s1">content-security-policy: default-src *; connect-src &#39;</span>self<span class="s1">&#39; https://media-src.linkedin.com/media/ www.linkedin.com s.c.lnkd.licdn.com m.c.lnkd.licdn.com s.c.exp1.licdn.com s.c.exp2.licdn.com m.c.exp1.licdn.com m.c.exp2.licdn.com wss://*.linkedin.com dms.licdn.com https://dpm.demdex.net/id lnkd.demdex.net blob: https://accounts.google.com/gsi/status https://linkedin.sc.omtrdc.net/b/ss/ www.google-analytics.com static.licdn.com static-exp1.licdn.com static-exp2.licdn.com static-exp3.licdn.com media.licdn.com media-exp1.licdn.com media-exp2.licdn.com media-exp3.licdn.com; img-src data: blob: *; font-src data: *; style-src &#39;</span>unsafe-inline<span class="s1">&#39; &#39;</span>self<span class="s1">&#39; static-src.linkedin.com *.licdn.com; script-src &#39;</span>report-sample<span class="s1">&#39; &#39;</span>unsafe-inline<span class="s1">&#39; &#39;</span>unsafe-eval<span class="s1">&#39; &#39;</span>self<span class="s1">&#39; spdy.linkedin.com static-src.linkedin.com *.ads.linkedin.com *.licdn.com static.chartbeat.com www.google-analytics.com ssl.google-analytics.com bcvipva02.rightnowtech.com www.bizographics.com sjs.bizographics.com js.bizographics.com d.la4-c1-was.salesforceliveagent.com slideshare.www.linkedin.com https://snap.licdn.com/li.lms-analytics/ platform.linkedin.com platform-akam.linkedin.com platform-ecst.linkedin.com platform-azur.linkedin.com; object-src &#39;</span>none<span class="s1">&#39;; media-src blob: *; child-src blob: lnkd-communities: voyager: *; frame-ancestors &#39;</span>self<span class="err">&#39;</span><span class="p">;</span> report-uri https://www.linkedin.com/platform-telemetry/csp?f<span class="o">=</span>l
* subject: C=US; ST=California; L=Sunnyvale; O=LinkedIn Corporation; CN=www.linkedin.com
* start date: Oct 2 00:00:00 2020 GMT
* expire date: Apr 2 12:00:00 2021 GMT
* subjectAltName: host &quot;www.linkedin.com&quot; matched cert's &quot;www.linkedin.com&quot;
* issuer: C=US; O=DigiCert Inc; CN=DigiCert SHA2 Secure Server CA
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7fb055808200)
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
0 82117 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
* Connection #0 to host www.linkedin.com left intact
HTTP/2 200
cache-control: no-cache, no-store
pragma: no-cache
content-length: 82117
content-type: text/html; charset=utf-8
expires: Thu, 01 Jan 1970 00:00:00 GMT
set-cookie: JSESSIONID=ajax:2747059799136291014; SameSite=None; Path=/; Domain=.www.linkedin.com; Secure
set-cookie: lang=v=2&amp;lang=en-us; SameSite=None; Path=/; Domain=linkedin.com; Secure
set-cookie: bcookie=&quot;v=2&amp;70bd59e3-5a51-406c-8e0d-dd70befa8890&quot;; domain=.linkedin.com; Path=/; Secure; Expires=Wed, 09-Nov-2022 22:27:42 GMT; SameSite=None
set-cookie: bscookie=&quot;v=1&amp;202011091050107ae9b7ac-fe97-40fc-830d-d7a9ccf80659AQGib5iXwarbY8CCBP94Q39THkgUlx6J&quot;; domain=.www.linkedin.com; Path=/; Secure; Expires=Wed, 09-Nov-2022 22:27:42 GMT; HttpOnly; SameSite=None
set-cookie: lissc=1; domain=.linkedin.com; Path=/; Secure; Expires=Tue, 09-Nov-2021 10:50:10 GMT; SameSite=None
set-cookie: lidc=&quot;b=VGST04:s=V:r=V:g=2201:u=1:i=1604919010:t=1605005410:v=1:sig=AQHe-KzU8i_5Iy6MwnFEsgRct3c9Lh5R&quot;; Expires=Tue, 10 Nov 2020 10:50:10 GMT; domain=.linkedin.com; Path=/; SameSite=None; Secure
x-fs-txn-id: 2b8d5409ba70
x-fs-uuid: 61bbf94956d14516302567fc882b0000
expect-ct: max-age=86400, report-uri=&quot;https://www.linkedin.com/platform-telemetry/ct&quot;
x-xss-protection: 1; mode=block
content-security-policy-report-only: default-src 'none'; connect-src 'self' www.linkedin.com www.google-analytics.com https://dpm.demdex.net/id lnkd.demdex.net blob: https://linkedin.sc.omtrdc.net/b/ss/ static.licdn.com static-exp1.licdn.com static-exp2.licdn.com static-exp3.licdn.com; script-src 'sha256-THuVhwbXPeTR0HszASqMOnIyxqEgvGyBwSPBKBF/iMc=' 'sha256-PyCXNcEkzRWqbiNr087fizmiBBrq9O6GGD8eV3P09Ik=' 'sha256-2SQ55Erm3CPCb+k03EpNxU9bdV3XL9TnVTriDs7INZ4=' 'sha256-S/KSPe186K/1B0JEjbIXcCdpB97krdzX05S+dHnQjUs=' platform.linkedin.com platform-akam.linkedin.com platform-ecst.linkedin.com platform-azur.linkedin.com static.licdn.com static-exp1.licdn.com static-exp2.licdn.com static-exp3.licdn.com; img-src data: blob: *; font-src data: *; style-src 'self' 'unsafe-inline' static.licdn.com static-exp1.licdn.com static-exp2.licdn.com static-exp3.licdn.com; media-src dms.licdn.com; child-src blob: *; frame-src 'self' lnkd.demdex.net linkedin.cdn.qualaroo.com; manifest-src 'self'; report-uri https://www.linkedin.com/platform-telemetry/csp?f=g
content-security-policy: default-src *; connect-src 'self' https://media-src.linkedin.com/media/ www.linkedin.com s.c.lnkd.licdn.com m.c.lnkd.licdn.com s.c.exp1.licdn.com s.c.exp2.licdn.com m.c.exp1.licdn.com m.c.exp2.licdn.com wss://*.linkedin.com dms.licdn.com https://dpm.demdex.net/id lnkd.demdex.net blob: https://accounts.google.com/gsi/status https://linkedin.sc.omtrdc.net/b/ss/ www.google-analytics.com static.licdn.com static-exp1.licdn.com static-exp2.licdn.com static-exp3.licdn.com media.licdn.com media-exp1.licdn.com media-exp2.licdn.com media-exp3.licdn.com; img-src data: blob: *; font-src data: *; style-src 'unsafe-inline' 'self' static-src.linkedin.com *.licdn.com; script-src 'report-sample' 'unsafe-inline' 'unsafe-eval' 'self' spdy.linkedin.com static-src.linkedin.com *.ads.linkedin.com *.licdn.com static.chartbeat.com www.google-analytics.com ssl.google-analytics.com bcvipva02.rightnowtech.com www.bizographics.com sjs.bizographics.com js.bizographics.com d.la4-c1-was.salesforceliveagent.com slideshare.www.linkedin.com https://snap.licdn.com/li.lms-analytics/ platform.linkedin.com platform-akam.linkedin.com platform-ecst.linkedin.com platform-azur.linkedin.com; object-src 'none'; media-src blob: *; child-src blob: lnkd-communities: voyager: *; frame-ancestors 'self'; report-uri https://www.linkedin.com/platform-telemetry/csp?f=l
x-frame-options: sameorigin
x-content-type-options: nosniff
strict-transport-security: max-age<span class="o">=</span><span class="m">2592000</span>
strict-transport-security: max-age=2592000
x-li-fabric: prod-lva1
x-li-pop: afd-prod-lva1
x-li-proto: http/2
x-li-uuid: <span class="nv">Ybv5SVbRRRYwJWf8iCsAAA</span><span class="o">==</span>
x-msedge-ref: Ref A: CFB9AC1D2B0645DDB161CEE4A4909AEF Ref B: BOM02EDGE0712 Ref C: <span class="m">2020</span>-11-09T10:50:10Z
date: Mon, <span class="m">09</span> Nov <span class="m">2020</span> <span class="m">10</span>:50:10 GMT
x-li-uuid: Ybv5SVbRRRYwJWf8iCsAAA==
x-msedge-ref: Ref A: CFB9AC1D2B0645DDB161CEE4A4909AEF Ref B: BOM02EDGE0712 Ref C: 2020-11-09T10:50:10Z
date: Mon, 09 Nov 2020 10:50:10 GMT
* Closing connection <span class="m">0</span>
</code></pre></div></p>
* Closing connection 0
</code></pre>
<p>Here my system has a list of certificate authorities it trusts in this file /etc/ssl/cert.pem. Curl validates the certificate is for www.linkedin.com by seeing the CN section of the subject part of the certificate. It also makes sure the certificate is not expired by seeing the expire date. It also validates the signature on the certificate by using the public key of issuer Digicert in /etc/ssl/cert.pem. Once this is done, using the public key of www.linkedin.com it negotiates cipher TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 with a symmetric key. Subsequent data transfer including first HTTP request uses the same cipher and symmetric key.</p>

12
linux_networking/ipr/index.html
View File

@@ -1269,14 +1269,14 @@
<h1 id="ip-routing-and-data-link-layer">IP Routing and Data Link Layer</h1>
<p>We will dig how packets that leave the client reach the server and vice versa. When the packet reaches the IP layer, the transport layer populates source port, destination port. IP/Network layer populates destination IP(discovered from DNS) and then looks up the route to the destination IP on the routing table. </p>
<div class="highlight"><pre><span></span><code><span class="c1">#Linux route -n command gives the default routing table</span>
<pre><code class="language-bash">#Linux route -n command gives the default routing table
route -n
</code></pre></div>
<div class="highlight"><pre><span></span><code>Kernel IP routing table
</code></pre>
<pre><code class="language-bash">Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
<span class="m">0</span>.0.0.0 <span class="m">172</span>.17.0.1 <span class="m">0</span>.0.0.0 UG <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> eth0
<span class="m">172</span>.17.0.0 <span class="m">0</span>.0.0.0 <span class="m">255</span>.255.0.0 U <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> eth0
</code></pre></div>
0.0.0.0 172.17.0.1 0.0.0.0 UG 0 0 0 eth0
172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
</code></pre>
<p>Here the destination IP is bitwise AND’d with the Genmask and if the answer is the destination part of the table then that gateway and interface is picked for routing. Here linkedin.com’s IP 108.174.10.10 is AND’d with 255.255.255.0 and the answer we get is 108.174.10.0 which doesn’t match with any destination in the routing table. Then Linux does an AND of destination IP with 0.0.0.0 and we get 0.0.0.0. This answer matches the default row</p>
<p>Routing table is processed in the order of more octets of 1 set in genmask and genmask 0.0.0.0 is the default route if nothing matches.
At the end of this operation Linux figured out that the packet has to be sent to next hop 172.17.0.1 via eth0. The source IP of the packet will be set as the IP of interface eth0.

8
linux_networking/tcp/index.html
View File

@@ -1271,11 +1271,11 @@
<p>TCP is a transport layer protocol like UDP but it guarantees reliability, flow control and congestion control.
TCP guarantees reliable delivery by using sequence numbers. A TCP connection is established by a three way handshake. In our case, the client sends a SYN packet along with the starting sequence number it plans to use, the server acknowledges the SYN packet and sends a SYN with its sequence number. Once the client acknowledges the syn packet, the connection is established. Each data transferred from here on is considered delivered reliably once acknowledgement for that sequence is received by the concerned party</p>
<p><img alt="3-way handshake" src="../images/established.png" /></p>
<div class="highlight"><pre><span></span><code><span class="c1">#To understand handshake run packet capture on one bash session</span>
tcpdump -S -i any port <span class="m">80</span>
<span class="c1">#Run curl on one bash session</span>
<pre><code class="language-bash">#To understand handshake run packet capture on one bash session
tcpdump -S -i any port 80
#Run curl on one bash session
curl www.linkedin.com
</code></pre></div>
</code></pre>
<p><img alt="tcpdump-3way" src="../images/pcap.png" /></p>
<p>Here client sends a syn flag shown by [S] flag with a sequence number 1522264672. The server acknowledges receipt of SYN with an ack [.] flag and a Syn flag for its sequence number[S]. The server uses the sequence number 1063230400 and acknowledges the client it’s expecting sequence number 1522264673 (client sequence+1). Client sends a zero length acknowledgement packet to the server(server sequence+1) and connection stands established. This is called three way handshake. The client sends a 76 bytes length packet after this and increments its sequence number by 76. Server sends a 170 byte response and closes the connection. This was the difference we were talking about between HTTP/1.1 and HTTP/1.0. In HTTP/1.1 this same connection can be reused which reduces overhead of 3 way handshake for each HTTP request. If a packet is missed between client and server, server won’t send an ack to the client and client would retry sending the packet till the ACK is received. This guarantees reliability.
The flow control is established by the win size field in each segment. The win size says available TCP buffer length in the kernel which can be used to buffer received segments. A size 0 means the receiver has a lot of lag to catch from its socket buffer and the sender has to pause sending packets so that receiver can cope up. This flow control protects from slow receiver and fast sender problem</p>