mirror of
https://github.com/linkedin/school-of-sre
synced 2026-01-12 19:48:02 +00:00
Deployed 52e7ed5 with MkDocs version: 1.1.2
This commit is contained in:
github-actions
@@ -1414,28 +1414,28 @@
|
||||
<p>This might sound a little weird to you: python, in a way is a compiled language! Python has a compiler built-in! It is obvious in the case of java since we compile it using a separate command ie: <code>javac helloWorld.java</code> and it will produce a <code>.class</code> file which we know as a <em>bytecode</em>. Well, python is very similar to that. One difference here is that there is no separate compile command/binary needed to run a python program.</p>
|
||||
<p><strong>What is the difference then, between java and python?</strong>
|
||||
Well, Java's compiler is more strict and sophisticated. As you might know Java is a statically typed language. So the compiler is written in a way that it can verify types related errors during compile time. While python being a <em>dynamic</em> language, types are not known until a program is run. So in a way, python compiler is dumb (or, less strict). But there indeed is a compile step involved when a python program is run. You might have seen python bytecode files with <code>.pyc</code> extension. Here is how you can see bytecode for a given python program.</p>
|
||||
<div class="highlight"><pre><span></span><code><span class="c1"># Create a Hello World</span>
|
||||
$ <span class="nb">echo</span> <span class="s2">"print('hello world')"</span> > hello_world.py
|
||||
<pre><code class="language-bash"># Create a Hello World
|
||||
$ echo "print('hello world')" > hello_world.py
|
||||
|
||||
<span class="c1"># Making sure it runs</span>
|
||||
# Making sure it runs
|
||||
$ python3 hello_world.py
|
||||
hello world
|
||||
|
||||
<span class="c1"># The bytecode of the given program</span>
|
||||
# The bytecode of the given program
|
||||
$ python -m dis hello_world.py
|
||||
<span class="m">1</span> <span class="m">0</span> LOAD_NAME <span class="m">0</span> <span class="o">(</span>print<span class="o">)</span>
|
||||
<span class="m">2</span> LOAD_CONST <span class="m">0</span> <span class="o">(</span><span class="s1">'hello world'</span><span class="o">)</span>
|
||||
<span class="m">4</span> CALL_FUNCTION <span class="m">1</span>
|
||||
<span class="m">6</span> POP_TOP
|
||||
<span class="m">8</span> LOAD_CONST <span class="m">1</span> <span class="o">(</span>None<span class="o">)</span>
|
||||
<span class="m">10</span> RETURN_VALUE
|
||||
</code></pre></div>
|
||||
1 0 LOAD_NAME 0 (print)
|
||||
2 LOAD_CONST 0 ('hello world')
|
||||
4 CALL_FUNCTION 1
|
||||
6 POP_TOP
|
||||
8 LOAD_CONST 1 (None)
|
||||
10 RETURN_VALUE
|
||||
</code></pre>
|
||||
<p>Read more about dis module <a href="https://docs.python.org/3/library/dis.html">here</a></p>
|
||||
<p>Now coming to C/C++, there of course is a compiler. But the output is different than what java/python compiler would produce. Compiling a C program would produce what we also know as <em>machine code</em>. As opposed to bytecode.</p>
|
||||
<h3 id="running-the-programs">Running The Programs</h3>
|
||||
<p>We know compilation is involved in all 3 languages we are discussing. Just that the compilers are different in nature and they output different types of content. In case of C/C++, the output is machine code which can be directly read by your operating system. When you execute that program, your OS will know how exactly to run it. <strong>But this is not the case with bytecode.</strong></p>
|
||||
<p>Those bytecodes are language specific. Python has its own set of bytecode defined (more in <code>dis</code> module) and so does java. So naturally, your operating system will not know how to run it. To run this bytecode, we have something called Virtual Machines. Ie: The JVM or the Python VM (CPython, Jython). These so called Virtual Machines are the programs which can read the bytecode and run it on a given operating system. Python has multiple VMs available. Cpython is a python VM implemented in C language, similarly Jython is a Java implementation of python VM. <strong>At the end of the day, what they should be capable of is to understand python language syntax, be able to compile it to bytecode and be able to run that bytecode.</strong> You can implement a python VM in any language! (And people do so, just because it can be done)</p>
|
||||
<div class="highlight"><pre><span></span><code> The Operating System
|
||||
<pre><code> The Operating System
|
||||
|
||||
+------------------------------------+
|
||||
| |
|
||||
@@ -1465,7 +1465,7 @@ hello_world.c OS Specific machinecode | A New Pr
|
||||
| |
|
||||
| |
|
||||
+------------------------------------+
|
||||
</code></pre></div>
|
||||
</code></pre>
|
||||
<p>Two things to note for above diagram:</p>
|
||||
<ol>
|
||||
<li>Generally, when we run a python program, a python VM process is started which reads the python source code, compiles it to byte code and run it in a single step. Compiling is not a separate step. Shown only for illustration purpose.</li>
|
||||
|
||||
@@ -1338,84 +1338,84 @@
|
||||
<p><strong>Everything in Python is an object.</strong></p>
|
||||
<p>That includes the functions, lists, dicts, classes, modules, a running function (instance of function definition), everything. In the CPython, it would mean there is an underlying struct variable for each object.</p>
|
||||
<p>In python's current execution context, all the variables are stored in a dict. It'd be a string to object mapping. If you have a function and a float variable defined in the current context, here is how it is handled internally.</p>
|
||||
<div class="highlight"><pre><span></span><code><span class="o">>>></span> <span class="n">float_number</span><span class="o">=</span><span class="mf">42.0</span>
|
||||
<span class="o">>>></span> <span class="k">def</span> <span class="nf">foo_func</span><span class="p">():</span>
|
||||
<span class="o">...</span> <span class="k">pass</span>
|
||||
<span class="o">...</span>
|
||||
<pre><code class="language-python">>>> float_number=42.0
|
||||
>>> def foo_func():
|
||||
... pass
|
||||
...
|
||||
|
||||
<span class="c1"># NOTICE HOW VARIABLE NAMES ARE STRINGS, stored in a dict</span>
|
||||
<span class="o">>>></span> <span class="nb">locals</span><span class="p">()</span>
|
||||
<span class="p">{</span><span class="s1">'__name__'</span><span class="p">:</span> <span class="s1">'__main__'</span><span class="p">,</span> <span class="s1">'__doc__'</span><span class="p">:</span> <span class="kc">None</span><span class="p">,</span> <span class="s1">'__package__'</span><span class="p">:</span> <span class="kc">None</span><span class="p">,</span> <span class="s1">'__loader__'</span><span class="p">:</span> <span class="o"><</span><span class="k">class</span> <span class="err">'</span><span class="nc">_frozen_importlib</span><span class="o">.</span><span class="n">BuiltinImporter</span><span class="s1">'>, '</span><span class="n">__spec__</span><span class="s1">': None, '</span><span class="vm">__annotations__</span><span class="s1">': </span><span class="si">{}</span><span class="s1">, '</span><span class="n">__builtins__</span><span class="s1">': <module '</span><span class="n">builtins</span><span class="s1">' (built-in)>, '</span><span class="n">float_number</span><span class="s1">': 42.0, '</span><span class="n">foo_func</span><span class="s1">': <function foo_func at 0x1055847a0>}</span>
|
||||
</code></pre></div>
|
||||
# NOTICE HOW VARIABLE NAMES ARE STRINGS, stored in a dict
|
||||
>>> locals()
|
||||
{'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <class '_frozen_importlib.BuiltinImporter'>, '__spec__': None, '__annotations__': {}, '__builtins__': <module 'builtins' (built-in)>, 'float_number': 42.0, 'foo_func': <function foo_func at 0x1055847a0>}
|
||||
</code></pre>
|
||||
<h2 id="python-functions">Python Functions</h2>
|
||||
<p>Since functions too are objects, we can see what all attributes a function contains as following</p>
|
||||
<div class="highlight"><pre><span></span><code><span class="o">>>></span> <span class="k">def</span> <span class="nf">hello</span><span class="p">(</span><span class="n">name</span><span class="p">):</span>
|
||||
<span class="o">...</span> <span class="nb">print</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Hello, </span><span class="si">{</span><span class="n">name</span><span class="si">}</span><span class="s2">!"</span><span class="p">)</span>
|
||||
<span class="o">...</span>
|
||||
<span class="o">>>></span> <span class="nb">dir</span><span class="p">(</span><span class="n">hello</span><span class="p">)</span>
|
||||
<span class="p">[</span><span class="s1">'__annotations__'</span><span class="p">,</span> <span class="s1">'__call__'</span><span class="p">,</span> <span class="s1">'__class__'</span><span class="p">,</span> <span class="s1">'__closure__'</span><span class="p">,</span> <span class="s1">'__code__'</span><span class="p">,</span> <span class="s1">'__defaults__'</span><span class="p">,</span> <span class="s1">'__delattr__'</span><span class="p">,</span> <span class="s1">'__dict__'</span><span class="p">,</span>
|
||||
<span class="s1">'__dir__'</span><span class="p">,</span> <span class="s1">'__doc__'</span><span class="p">,</span> <span class="s1">'__eq__'</span><span class="p">,</span> <span class="s1">'__format__'</span><span class="p">,</span> <span class="s1">'__ge__'</span><span class="p">,</span> <span class="s1">'__get__'</span><span class="p">,</span> <span class="s1">'__getattribute__'</span><span class="p">,</span> <span class="s1">'__globals__'</span><span class="p">,</span> <span class="s1">'__gt__'</span><span class="p">,</span>
|
||||
<span class="s1">'__hash__'</span><span class="p">,</span> <span class="s1">'__init__'</span><span class="p">,</span> <span class="s1">'__init_subclass__'</span><span class="p">,</span> <span class="s1">'__kwdefaults__'</span><span class="p">,</span> <span class="s1">'__le__'</span><span class="p">,</span> <span class="s1">'__lt__'</span><span class="p">,</span> <span class="s1">'__module__'</span><span class="p">,</span> <span class="s1">'__name__'</span><span class="p">,</span>
|
||||
<span class="s1">'__ne__'</span><span class="p">,</span> <span class="s1">'__new__'</span><span class="p">,</span> <span class="s1">'__qualname__'</span><span class="p">,</span> <span class="s1">'__reduce__'</span><span class="p">,</span> <span class="s1">'__reduce_ex__'</span><span class="p">,</span> <span class="s1">'__repr__'</span><span class="p">,</span> <span class="s1">'__setattr__'</span><span class="p">,</span> <span class="s1">'__sizeof__'</span><span class="p">,</span> <span class="s1">'__str__'</span><span class="p">,</span>
|
||||
<span class="s1">'__subclasshook__'</span><span class="p">]</span>
|
||||
</code></pre></div>
|
||||
<pre><code class="language-python">>>> def hello(name):
|
||||
... print(f"Hello, {name}!")
|
||||
...
|
||||
>>> dir(hello)
|
||||
['__annotations__', '__call__', '__class__', '__closure__', '__code__', '__defaults__', '__delattr__', '__dict__',
|
||||
'__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__get__', '__getattribute__', '__globals__', '__gt__',
|
||||
'__hash__', '__init__', '__init_subclass__', '__kwdefaults__', '__le__', '__lt__', '__module__', '__name__',
|
||||
'__ne__', '__new__', '__qualname__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__',
|
||||
'__subclasshook__']
|
||||
</code></pre>
|
||||
<p>While there are a lot of them, let's look at some interesting ones</p>
|
||||
<h4 id="globals"><strong>globals</strong></h4>
|
||||
<p>This attribute, as the name suggests, has references of global variables. If you ever need to know what all global variables are in the scope of this function, this will tell you. See how the function start seeing the new variable in globals</p>
|
||||
<div class="highlight"><pre><span></span><code><span class="o">>>></span> <span class="n">hello</span><span class="o">.</span><span class="vm">__globals__</span>
|
||||
<span class="p">{</span><span class="s1">'__name__'</span><span class="p">:</span> <span class="s1">'__main__'</span><span class="p">,</span> <span class="s1">'__doc__'</span><span class="p">:</span> <span class="kc">None</span><span class="p">,</span> <span class="s1">'__package__'</span><span class="p">:</span> <span class="kc">None</span><span class="p">,</span> <span class="s1">'__loader__'</span><span class="p">:</span> <span class="o"><</span><span class="k">class</span> <span class="err">'</span><span class="nc">_frozen_importlib</span><span class="o">.</span><span class="n">BuiltinImporter</span><span class="s1">'>, '</span><span class="n">__spec__</span><span class="s1">': None, '</span><span class="vm">__annotations__</span><span class="s1">': </span><span class="si">{}</span><span class="s1">, '</span><span class="n">__builtins__</span><span class="s1">': <module '</span><span class="n">builtins</span><span class="s1">' (built-in)>, '</span><span class="n">hello</span><span class="s1">': <function hello at 0x7fe4e82554c0>}</span>
|
||||
<pre><code class="language-python">>>> hello.__globals__
|
||||
{'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <class '_frozen_importlib.BuiltinImporter'>, '__spec__': None, '__annotations__': {}, '__builtins__': <module 'builtins' (built-in)>, 'hello': <function hello at 0x7fe4e82554c0>}
|
||||
|
||||
<span class="c1"># adding new global variable</span>
|
||||
<span class="o">>>></span> <span class="n">GLOBAL</span><span class="o">=</span><span class="s2">"g_val"</span>
|
||||
<span class="o">>>></span> <span class="n">hello</span><span class="o">.</span><span class="vm">__globals__</span>
|
||||
<span class="p">{</span><span class="s1">'__name__'</span><span class="p">:</span> <span class="s1">'__main__'</span><span class="p">,</span> <span class="s1">'__doc__'</span><span class="p">:</span> <span class="kc">None</span><span class="p">,</span> <span class="s1">'__package__'</span><span class="p">:</span> <span class="kc">None</span><span class="p">,</span> <span class="s1">'__loader__'</span><span class="p">:</span> <span class="o"><</span><span class="k">class</span> <span class="err">'</span><span class="nc">_frozen_importlib</span><span class="o">.</span><span class="n">BuiltinImporter</span><span class="s1">'>, '</span><span class="n">__spec__</span><span class="s1">': None, '</span><span class="vm">__annotations__</span><span class="s1">': </span><span class="si">{}</span><span class="s1">, '</span><span class="n">__builtins__</span><span class="s1">': <module '</span><span class="n">builtins</span><span class="s1">' (built-in)>, '</span><span class="n">hello</span><span class="s1">': <function hello at 0x7fe4e82554c0>, '</span><span class="n">GLOBAL</span><span class="s1">': '</span><span class="n">g_val</span><span class="s1">'}</span>
|
||||
</code></pre></div>
|
||||
# adding new global variable
|
||||
>>> GLOBAL="g_val"
|
||||
>>> hello.__globals__
|
||||
{'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <class '_frozen_importlib.BuiltinImporter'>, '__spec__': None, '__annotations__': {}, '__builtins__': <module 'builtins' (built-in)>, 'hello': <function hello at 0x7fe4e82554c0>, 'GLOBAL': 'g_val'}
|
||||
</code></pre>
|
||||
<h3 id="code"><strong>code</strong></h3>
|
||||
<p>This is an interesting one! As everything in python is an object, this includes the bytecode too. The compiled python bytecode is a python code object. Which is accessible via <code>__code__</code> attribute here. A function has an associated code object which carries some interesting information.</p>
|
||||
<div class="highlight"><pre><span></span><code><span class="c1"># the file in which function is defined</span>
|
||||
<span class="c1"># stdin here since this is run in an interpreter</span>
|
||||
<span class="o">>>></span> <span class="n">hello</span><span class="o">.</span><span class="vm">__code__</span><span class="o">.</span><span class="n">co_filename</span>
|
||||
<span class="s1">'<stdin>'</span>
|
||||
<pre><code class="language-python"># the file in which function is defined
|
||||
# stdin here since this is run in an interpreter
|
||||
>>> hello.__code__.co_filename
|
||||
'<stdin>'
|
||||
|
||||
<span class="c1"># number of arguments the function takes</span>
|
||||
<span class="o">>>></span> <span class="n">hello</span><span class="o">.</span><span class="vm">__code__</span><span class="o">.</span><span class="n">co_argcount</span>
|
||||
<span class="mi">1</span>
|
||||
# number of arguments the function takes
|
||||
>>> hello.__code__.co_argcount
|
||||
1
|
||||
|
||||
<span class="c1"># local variable names</span>
|
||||
<span class="o">>>></span> <span class="n">hello</span><span class="o">.</span><span class="vm">__code__</span><span class="o">.</span><span class="n">co_varnames</span>
|
||||
<span class="p">(</span><span class="s1">'name'</span><span class="p">,)</span>
|
||||
# local variable names
|
||||
>>> hello.__code__.co_varnames
|
||||
('name',)
|
||||
|
||||
<span class="c1"># the function code's compiled bytecode</span>
|
||||
<span class="o">>>></span> <span class="n">hello</span><span class="o">.</span><span class="vm">__code__</span><span class="o">.</span><span class="n">co_code</span>
|
||||
<span class="sa">b</span><span class="s1">'t</span><span class="se">\x00</span><span class="s1">d</span><span class="se">\x01</span><span class="s1">|</span><span class="se">\x00\x9b\x00</span><span class="s1">d</span><span class="se">\x02\x9d\x03\x83\x01\x01\x00</span><span class="s1">d</span><span class="se">\x00</span><span class="s1">S</span><span class="se">\x00</span><span class="s1">'</span>
|
||||
</code></pre></div>
|
||||
# the function code's compiled bytecode
|
||||
>>> hello.__code__.co_code
|
||||
b't\x00d\x01|\x00\x9b\x00d\x02\x9d\x03\x83\x01\x01\x00d\x00S\x00'
|
||||
</code></pre>
|
||||
<p>There are more code attributes which you can enlist by <code>>>> dir(hello.__code__)</code></p>
|
||||
<h2 id="decorators">Decorators</h2>
|
||||
<p>Related to functions, python has another feature called decorators. Let's see how that works, keeping <code>everything is an object</code> in mind.</p>
|
||||
<p>Here is a sample decorator:</p>
|
||||
<div class="highlight"><pre><span></span><code><span class="o">>>></span> <span class="k">def</span> <span class="nf">deco</span><span class="p">(</span><span class="n">func</span><span class="p">):</span>
|
||||
<span class="o">...</span> <span class="k">def</span> <span class="nf">inner</span><span class="p">():</span>
|
||||
<span class="o">...</span> <span class="nb">print</span><span class="p">(</span><span class="s2">"before"</span><span class="p">)</span>
|
||||
<span class="o">...</span> <span class="n">func</span><span class="p">()</span>
|
||||
<span class="o">...</span> <span class="nb">print</span><span class="p">(</span><span class="s2">"after"</span><span class="p">)</span>
|
||||
<span class="o">...</span> <span class="k">return</span> <span class="n">inner</span>
|
||||
<span class="o">...</span>
|
||||
<span class="o">>>></span> <span class="nd">@deco</span>
|
||||
<span class="o">...</span> <span class="k">def</span> <span class="nf">hello_world</span><span class="p">():</span>
|
||||
<span class="o">...</span> <span class="nb">print</span><span class="p">(</span><span class="s2">"hello world"</span><span class="p">)</span>
|
||||
<span class="o">...</span>
|
||||
<span class="o">>>></span>
|
||||
<span class="o">>>></span> <span class="n">hello_world</span><span class="p">()</span>
|
||||
<span class="n">before</span>
|
||||
<span class="n">hello</span> <span class="n">world</span>
|
||||
<span class="n">after</span>
|
||||
</code></pre></div>
|
||||
<pre><code class="language-python">>>> def deco(func):
|
||||
... def inner():
|
||||
... print("before")
|
||||
... func()
|
||||
... print("after")
|
||||
... return inner
|
||||
...
|
||||
>>> @deco
|
||||
... def hello_world():
|
||||
... print("hello world")
|
||||
...
|
||||
>>>
|
||||
>>> hello_world()
|
||||
before
|
||||
hello world
|
||||
after
|
||||
</code></pre>
|
||||
<p>Here <code>@deco</code> syntax is used to decorate the <code>hello_world</code> function. It is essentially same as doing</p>
|
||||
<div class="highlight"><pre><span></span><code><span class="o">>>></span> <span class="k">def</span> <span class="nf">hello_world</span><span class="p">():</span>
|
||||
<span class="o">...</span> <span class="nb">print</span><span class="p">(</span><span class="s2">"hello world"</span><span class="p">)</span>
|
||||
<span class="o">...</span>
|
||||
<span class="o">>>></span> <span class="n">hello_world</span> <span class="o">=</span> <span class="n">deco</span><span class="p">(</span><span class="n">hello_world</span><span class="p">)</span>
|
||||
</code></pre></div>
|
||||
<pre><code class="language-python">>>> def hello_world():
|
||||
... print("hello world")
|
||||
...
|
||||
>>> hello_world = deco(hello_world)
|
||||
</code></pre>
|
||||
<p>What goes inside the <code>deco</code> function might seem complex. Let's try to uncover it.</p>
|
||||
<ol>
|
||||
<li>Function <code>hello_world</code> is created</li>
|
||||
@@ -1429,7 +1429,7 @@
|
||||
<li><code>hello_world</code> is replaced with above function</li>
|
||||
</ol>
|
||||
<p>Let's visualize it for better understanding</p>
|
||||
<div class="highlight"><pre><span></span><code> BEFORE function_object (ID: 100)
|
||||
<pre><code> BEFORE function_object (ID: 100)
|
||||
|
||||
"hello_world" +--------------------+
|
||||
+ |print("hello_world")|
|
||||
@@ -1456,7 +1456,7 @@
|
||||
|
|
||||
|
|
||||
"hello_world" +-------------+
|
||||
</code></pre></div>
|
||||
</code></pre>
|
||||
<p>Note how the <code>hello_world</code> name points to a new function object but that new function object knows the reference (ID) of the original function.</p>
|
||||
<h2 id="some-gotchas">Some Gotchas</h2>
|
||||
<ul>
|
||||
|
||||
@@ -1284,31 +1284,31 @@
|
||||
<p>Since serving web requests is no longer a simple task like reading files from disk and return contents, we need to process each http request, perform some operations programmatically and construct a response.</p>
|
||||
<h2 id="sockets">Sockets</h2>
|
||||
<p>Though we have frameworks like flask, HTTP is still a protocol that works over TCP protocol. So let us setup a TCP server and send an HTTP request and inspect the request's payload. Note that this is not a tutorial on socket programming but what we are doing here is inspecting HTTP protocol at its ground level and look at what its contents look like. (Ref: <a href="https://realpython.com/python-sockets/">Socket Programming in Python (Guide) on RealPython</a>)</p>
|
||||
<div class="highlight"><pre><span></span><code><span class="kn">import</span> <span class="nn">socket</span>
|
||||
<pre><code class="language-python">import socket
|
||||
|
||||
<span class="n">HOST</span> <span class="o">=</span> <span class="s1">'127.0.0.1'</span> <span class="c1"># Standard loopback interface address (localhost)</span>
|
||||
<span class="n">PORT</span> <span class="o">=</span> <span class="mi">65432</span> <span class="c1"># Port to listen on (non-privileged ports are > 1023)</span>
|
||||
HOST = '127.0.0.1' # Standard loopback interface address (localhost)
|
||||
PORT = 65432 # Port to listen on (non-privileged ports are > 1023)
|
||||
|
||||
<span class="k">with</span> <span class="n">socket</span><span class="o">.</span><span class="n">socket</span><span class="p">(</span><span class="n">socket</span><span class="o">.</span><span class="n">AF_INET</span><span class="p">,</span> <span class="n">socket</span><span class="o">.</span><span class="n">SOCK_STREAM</span><span class="p">)</span> <span class="k">as</span> <span class="n">s</span><span class="p">:</span>
|
||||
<span class="n">s</span><span class="o">.</span><span class="n">bind</span><span class="p">((</span><span class="n">HOST</span><span class="p">,</span> <span class="n">PORT</span><span class="p">))</span>
|
||||
<span class="n">s</span><span class="o">.</span><span class="n">listen</span><span class="p">()</span>
|
||||
<span class="n">conn</span><span class="p">,</span> <span class="n">addr</span> <span class="o">=</span> <span class="n">s</span><span class="o">.</span><span class="n">accept</span><span class="p">()</span>
|
||||
<span class="k">with</span> <span class="n">conn</span><span class="p">:</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s1">'Connected by'</span><span class="p">,</span> <span class="n">addr</span><span class="p">)</span>
|
||||
<span class="k">while</span> <span class="kc">True</span><span class="p">:</span>
|
||||
<span class="n">data</span> <span class="o">=</span> <span class="n">conn</span><span class="o">.</span><span class="n">recv</span><span class="p">(</span><span class="mi">1024</span><span class="p">)</span>
|
||||
<span class="k">if</span> <span class="ow">not</span> <span class="n">data</span><span class="p">:</span>
|
||||
<span class="k">break</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">data</span><span class="p">)</span>
|
||||
</code></pre></div>
|
||||
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
|
||||
s.bind((HOST, PORT))
|
||||
s.listen()
|
||||
conn, addr = s.accept()
|
||||
with conn:
|
||||
print('Connected by', addr)
|
||||
while True:
|
||||
data = conn.recv(1024)
|
||||
if not data:
|
||||
break
|
||||
print(data)
|
||||
</code></pre>
|
||||
<p>Then we open <code>localhost:65432</code> in our web browser and following would be the output:</p>
|
||||
<div class="highlight"><pre><span></span><code>Connected by <span class="o">(</span><span class="s1">'127.0.0.1'</span>, <span class="m">54719</span><span class="o">)</span>
|
||||
b<span class="s1">'GET / HTTP/1.1\r\nHost: localhost:65432\r\nConnection: keep-alive\r\nDNT: 1\r\nUpgrade-Insecure-Requests: 1\r\nUser-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.83 Safari/537.36 Edg/85.0.564.44\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9\r\nSec-Fetch-Site: none\r\nSec-Fetch-Mode: navigate\r\nSec-Fetch-User: ?1\r\nSec-Fetch-Dest: document\r\nAccept-Encoding: gzip, deflate, br\r\nAccept-Language: en-US,en;q=0.9\r\n\r\n'</span>
|
||||
</code></pre></div>
|
||||
<pre><code class="language-bash">Connected by ('127.0.0.1', 54719)
|
||||
b'GET / HTTP/1.1\r\nHost: localhost:65432\r\nConnection: keep-alive\r\nDNT: 1\r\nUpgrade-Insecure-Requests: 1\r\nUser-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.83 Safari/537.36 Edg/85.0.564.44\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9\r\nSec-Fetch-Site: none\r\nSec-Fetch-Mode: navigate\r\nSec-Fetch-User: ?1\r\nSec-Fetch-Dest: document\r\nAccept-Encoding: gzip, deflate, br\r\nAccept-Language: en-US,en;q=0.9\r\n\r\n'
|
||||
</code></pre>
|
||||
<p>Examine closely and the content will look like the HTTP protocol's format. ie:</p>
|
||||
<div class="highlight"><pre><span></span><code>HTTP_METHOD URI_PATH HTTP_VERSION
|
||||
<pre><code>HTTP_METHOD URI_PATH HTTP_VERSION
|
||||
HEADERS_SEPARATED_BY_SEPARATOR
|
||||
</code></pre></div>
|
||||
</code></pre>
|
||||
<p>So though it's a blob of bytes, knowing <a href="https://tools.ietf.org/html/rfc2616">http protocol specification</a>, you can parse that string (ie: split by <code>\r\n</code>) and get meaningful information out of it.</p>
|
||||
<h2 id="flask">Flask</h2>
|
||||
<p>Flask, and other such frameworks does pretty much what we just discussed in the last section (with added more sophistication). They listen on a port on a TCP socket, receive an HTTP request, parse the data according to protocol format and make it available to you in a convenient manner.</p>
|
||||
|
||||
@@ -1369,87 +1369,87 @@
|
||||
<h3 id="5-other">5. Other</h3>
|
||||
<p>We are not accounting for users into our app and other possible features like rate limiting, customized links etc but it will eventually come up with time. Depending on the requirements, they too might need to get incorporated.</p>
|
||||
<p>The minimal working code is given below for reference but I'd encourage you to come up with your own.</p>
|
||||
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">flask</span> <span class="kn">import</span> <span class="n">Flask</span><span class="p">,</span> <span class="n">redirect</span><span class="p">,</span> <span class="n">request</span>
|
||||
<pre><code class="language-python">from flask import Flask, redirect, request
|
||||
|
||||
<span class="kn">from</span> <span class="nn">hashlib</span> <span class="kn">import</span> <span class="n">md5</span>
|
||||
from hashlib import md5
|
||||
|
||||
<span class="n">app</span> <span class="o">=</span> <span class="n">Flask</span><span class="p">(</span><span class="s2">"url_shortener"</span><span class="p">)</span>
|
||||
app = Flask("url_shortener")
|
||||
|
||||
<span class="n">mapping</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
mapping = {}
|
||||
|
||||
<span class="nd">@app</span><span class="o">.</span><span class="n">route</span><span class="p">(</span><span class="s2">"/shorten"</span><span class="p">,</span> <span class="n">methods</span><span class="o">=</span><span class="p">[</span><span class="s2">"POST"</span><span class="p">])</span>
|
||||
<span class="k">def</span> <span class="nf">shorten</span><span class="p">():</span>
|
||||
<span class="k">global</span> <span class="n">mapping</span>
|
||||
<span class="n">payload</span> <span class="o">=</span> <span class="n">request</span><span class="o">.</span><span class="n">json</span>
|
||||
@app.route("/shorten", methods=["POST"])
|
||||
def shorten():
|
||||
global mapping
|
||||
payload = request.json
|
||||
|
||||
<span class="k">if</span> <span class="s2">"url"</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">payload</span><span class="p">:</span>
|
||||
<span class="k">return</span> <span class="s2">"Missing URL Parameter"</span><span class="p">,</span> <span class="mi">400</span>
|
||||
if "url" not in payload:
|
||||
return "Missing URL Parameter", 400
|
||||
|
||||
<span class="c1"># TODO: check if URL is valid</span>
|
||||
# TODO: check if URL is valid
|
||||
|
||||
<span class="n">hash_</span> <span class="o">=</span> <span class="n">md5</span><span class="p">()</span>
|
||||
<span class="n">hash_</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">payload</span><span class="p">[</span><span class="s2">"url"</span><span class="p">]</span><span class="o">.</span><span class="n">encode</span><span class="p">())</span>
|
||||
<span class="n">digest</span> <span class="o">=</span> <span class="n">hash_</span><span class="o">.</span><span class="n">hexdigest</span><span class="p">()[:</span><span class="mi">5</span><span class="p">]</span> <span class="c1"># limiting to 5 chars. Less the limit more the chances of collission</span>
|
||||
hash_ = md5()
|
||||
hash_.update(payload["url"].encode())
|
||||
digest = hash_.hexdigest()[:5] # limiting to 5 chars. Less the limit more the chances of collission
|
||||
|
||||
<span class="k">if</span> <span class="n">digest</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">mapping</span><span class="p">:</span>
|
||||
<span class="n">mapping</span><span class="p">[</span><span class="n">digest</span><span class="p">]</span> <span class="o">=</span> <span class="n">payload</span><span class="p">[</span><span class="s2">"url"</span><span class="p">]</span>
|
||||
<span class="k">return</span> <span class="sa">f</span><span class="s2">"Shortened: r/</span><span class="si">{</span><span class="n">digest</span><span class="si">}</span><span class="se">\n</span><span class="s2">"</span>
|
||||
<span class="k">else</span><span class="p">:</span>
|
||||
<span class="c1"># TODO: check for hash collission</span>
|
||||
<span class="k">return</span> <span class="sa">f</span><span class="s2">"Already exists: r/</span><span class="si">{</span><span class="n">digest</span><span class="si">}</span><span class="se">\n</span><span class="s2">"</span>
|
||||
if digest not in mapping:
|
||||
mapping[digest] = payload["url"]
|
||||
return f"Shortened: r/{digest}\n"
|
||||
else:
|
||||
# TODO: check for hash collission
|
||||
return f"Already exists: r/{digest}\n"
|
||||
|
||||
|
||||
<span class="nd">@app</span><span class="o">.</span><span class="n">route</span><span class="p">(</span><span class="s2">"/r/<hash_>"</span><span class="p">)</span>
|
||||
<span class="k">def</span> <span class="nf">redirect_</span><span class="p">(</span><span class="n">hash_</span><span class="p">):</span>
|
||||
<span class="k">if</span> <span class="n">hash_</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">mapping</span><span class="p">:</span>
|
||||
<span class="k">return</span> <span class="s2">"URL Not Found"</span><span class="p">,</span> <span class="mi">404</span>
|
||||
<span class="k">return</span> <span class="n">redirect</span><span class="p">(</span><span class="n">mapping</span><span class="p">[</span><span class="n">hash_</span><span class="p">])</span>
|
||||
@app.route("/r/<hash_>")
|
||||
def redirect_(hash_):
|
||||
if hash_ not in mapping:
|
||||
return "URL Not Found", 404
|
||||
return redirect(mapping[hash_])
|
||||
|
||||
|
||||
<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s2">"__main__"</span><span class="p">:</span>
|
||||
<span class="n">app</span><span class="o">.</span><span class="n">run</span><span class="p">(</span><span class="n">debug</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
|
||||
if __name__ == "__main__":
|
||||
app.run(debug=True)
|
||||
|
||||
<span class="sd">"""</span>
|
||||
<span class="sd">OUTPUT:</span>
|
||||
"""
|
||||
OUTPUT:
|
||||
|
||||
|
||||
<span class="sd">===> SHORTENING</span>
|
||||
===> SHORTENING
|
||||
|
||||
<span class="sd">$ curl localhost:5000/shorten -H "content-type: application/json" --data '{"url":"https://linkedin.com"}'</span>
|
||||
<span class="sd">Shortened: r/a62a4</span>
|
||||
$ curl localhost:5000/shorten -H "content-type: application/json" --data '{"url":"https://linkedin.com"}'
|
||||
Shortened: r/a62a4
|
||||
|
||||
|
||||
<span class="sd">===> REDIRECTING, notice the response code 302 and the location header</span>
|
||||
===> REDIRECTING, notice the response code 302 and the location header
|
||||
|
||||
<span class="sd">$ curl localhost:5000/r/a62a4 -v</span>
|
||||
<span class="sd">* Uses proxy env variable NO_PROXY == '127.0.0.1'</span>
|
||||
<span class="sd">* Trying ::1...</span>
|
||||
<span class="sd">* TCP_NODELAY set</span>
|
||||
<span class="sd">* Connection failed</span>
|
||||
<span class="sd">* connect to ::1 port 5000 failed: Connection refused</span>
|
||||
<span class="sd">* Trying 127.0.0.1...</span>
|
||||
<span class="sd">* TCP_NODELAY set</span>
|
||||
<span class="sd">* Connected to localhost (127.0.0.1) port 5000 (#0)</span>
|
||||
<span class="sd">> GET /r/a62a4 HTTP/1.1</span>
|
||||
<span class="sd">> Host: localhost:5000</span>
|
||||
<span class="sd">> User-Agent: curl/7.64.1</span>
|
||||
<span class="sd">> Accept: */*</span>
|
||||
<span class="sd">></span>
|
||||
<span class="sd">* HTTP 1.0, assume close after body</span>
|
||||
<span class="sd">< HTTP/1.0 302 FOUND</span>
|
||||
<span class="sd">< Content-Type: text/html; charset=utf-8</span>
|
||||
<span class="sd">< Content-Length: 247</span>
|
||||
<span class="sd">< Location: https://linkedin.com</span>
|
||||
<span class="sd">< Server: Werkzeug/0.15.4 Python/3.7.7</span>
|
||||
<span class="sd">< Date: Tue, 27 Oct 2020 09:37:12 GMT</span>
|
||||
<span class="sd"><</span>
|
||||
<span class="sd"><!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN"></span>
|
||||
<span class="sd"><title>Redirecting...</title></span>
|
||||
<span class="sd"><h1>Redirecting...</h1></span>
|
||||
<span class="sd">* Closing connection 0</span>
|
||||
<span class="sd"><p>You should be redirected automatically to target URL: <a href="https://linkedin.com">https://linkedin.com</a>. If not click the link.</span>
|
||||
<span class="sd">"""</span>
|
||||
</code></pre></div>
|
||||
$ curl localhost:5000/r/a62a4 -v
|
||||
* Uses proxy env variable NO_PROXY == '127.0.0.1'
|
||||
* Trying ::1...
|
||||
* TCP_NODELAY set
|
||||
* Connection failed
|
||||
* connect to ::1 port 5000 failed: Connection refused
|
||||
* Trying 127.0.0.1...
|
||||
* TCP_NODELAY set
|
||||
* Connected to localhost (127.0.0.1) port 5000 (#0)
|
||||
> GET /r/a62a4 HTTP/1.1
|
||||
> Host: localhost:5000
|
||||
> User-Agent: curl/7.64.1
|
||||
> Accept: */*
|
||||
>
|
||||
* HTTP 1.0, assume close after body
|
||||
< HTTP/1.0 302 FOUND
|
||||
< Content-Type: text/html; charset=utf-8
|
||||
< Content-Length: 247
|
||||
< Location: https://linkedin.com
|
||||
< Server: Werkzeug/0.15.4 Python/3.7.7
|
||||
< Date: Tue, 27 Oct 2020 09:37:12 GMT
|
||||
<
|
||||
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
|
||||
<title>Redirecting...</title>
|
||||
<h1>Redirecting...</h1>
|
||||
* Closing connection 0
|
||||
<p>You should be redirected automatically to target URL: <a href="https://linkedin.com">https://linkedin.com</a>. If not click the link.
|
||||
"""
|
||||
</code></pre>
|
||||
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user