mirror of
https://github.com/linkedin/school-of-sre
synced 2026-01-09 01:58:03 +00:00
docs (level 101): fix typos, punctuation, formatting (#160)
* docs: formatted for readability * docs: rephrased and added punctuation * docs: fix typos, punctuation, formatting * docs: fix typo and format * docs: fix caps and formatting * docs: fix punctuation and formatting * docs: capitalized SQL commands, fixed puntuation, formatting * docs: fix punctuation * docs: fix punctuation and formatting * docs: fix caps,punctuation and formatting * docs: fix links, punctuation, formatting * docs: fix code block formatting * docs: fix punctuation, indentation and formatting
This commit is contained in:
Jana R
GitHub
@@ -2,24 +2,24 @@
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Basic understanding of python language.
|
||||
- Basic familiarity with flask framework.
|
||||
- Basic understanding of Python language.
|
||||
- Basic familiarity with Flask framework.
|
||||
|
||||
## What to expect from this course
|
||||
|
||||
This course is divided into two high level parts. In the first part, assuming familiarity with python language’s basic operations and syntax usage, we will dive a little deeper into understanding python as a language. We will compare python with other programming languages that you might already know like Java and C. We will also explore concepts of Python objects and with help of that, explore python features like decorators.
|
||||
This course is divided into two high-level parts. In the first part, assuming familiarity with Python language’s basic operations and syntax usage, we will dive a little deeper into understanding Python as a language. We will compare Python with other programming languages that you might already know like Java and C. We will also explore concepts of Python objects and with help of that, explore Python features like decorators.
|
||||
|
||||
In the second part which will revolve around the web, and also assume familiarity with the Flask framework, we will start from the socket module and work with HTTP requests. This will demystify how frameworks like flask work internally.
|
||||
In the second part which will revolve around the web, and also assume familiarity with the Flask framework, we will start from the `socket` module and work with HTTP requests. This will demystify how frameworks like Flask work internally.
|
||||
|
||||
And to introduce SRE flavour to the course, we will design, develop and deploy (in theory) a URL shortening application. We will emphasize parts of the whole process that are more important as an SRE of the said app/service.
|
||||
And to introduce SRE flavour to the course, we will design, develop and deploy (in theory) a URL-shortening application. We will emphasize parts of the whole process that are more important as an SRE of the said app/service.
|
||||
|
||||
## What is not covered under this course
|
||||
|
||||
Extensive knowledge of python internals and advanced python.
|
||||
Extensive knowledge of Python internals and advanced Python.
|
||||
|
||||
## Lab Environment Setup
|
||||
|
||||
Have latest version of python installed
|
||||
Have latest version of Python installed
|
||||
|
||||
## Course Contents
|
||||
|
||||
@@ -29,21 +29,21 @@ Have latest version of python installed
|
||||
2. [Python and Web](https://linkedin.github.io/school-of-sre/level101/python_web/python-web-flask/)
|
||||
1. [Sockets](https://linkedin.github.io/school-of-sre/level101/python_web/python-web-flask/#sockets)
|
||||
2. [Flask](https://linkedin.github.io/school-of-sre/level101/python_web/python-web-flask/#flask)
|
||||
3. [The URL Shortening App](https://linkedin.github.io/school-of-sre/level101/python_web/url-shorten-app/)
|
||||
3. [The URL-Shortening App](https://linkedin.github.io/school-of-sre/level101/python_web/url-shorten-app/)
|
||||
1. [Design](https://linkedin.github.io/school-of-sre/level101/python_web/url-shorten-app/#design)
|
||||
2. [Scaling The App](https://linkedin.github.io/school-of-sre/level101/python_web/sre-conclusion/#scaling-the-app)
|
||||
3. [Monitoring The App](https://linkedin.github.io/school-of-sre/level101/python_web/sre-conclusion/#monitoring-strategy)
|
||||
|
||||
## The Python Language
|
||||
|
||||
Assuming you know a little bit of C/C++ and Java, let's try to discuss the following questions in context of those two languages and python. You might have heard that C/C++ is a compiled language while python is an interpreted language. Generally, with compiled language we first compile the program and then run the executable while in case of python we run the source code directly like `python hello_world.py`. While Java, being an interpreted language, still has a separate compilation step and then its run. So what's really the difference?
|
||||
Assuming you know a little bit of C/C++ and Java, let's try to discuss the following questions in context of those two languages and Python. You might have heard that C/C++ is a compiled language while Python is an interpreted language. Generally, with compiled language we first compile the program and then run the executable while in case of Python we run the source code directly like `python hello_world.py`. While Java, being an interpreted language, still has a separate compilation step and then it's run. So, what's really the difference?
|
||||
|
||||
### Compiled vs. Interpreted
|
||||
|
||||
This might sound a little weird to you: python, in a way is a compiled language! Python has a compiler built-in! It is obvious in the case of java since we compile it using a separate command ie: `javac helloWorld.java` and it will produce a `.class` file which we know as a _bytecode_. Well, python is very similar to that. One difference here is that there is no separate compile command/binary needed to run a python program.
|
||||
This might sound a little weird to you: Python, in a way is a compiled language! Python has a compiler built-in! It is obvious in the case of Java since we compile it using a separate command, ie: `javac helloWorld.java` and it will produce a `.class` file which we know as a _bytecode_. Well, Python is very similar to that. One difference here is that there is no separate compile command/binary needed to run a Python program.
|
||||
|
||||
**What is the difference then, between java and python?**
|
||||
Well, Java's compiler is more strict and sophisticated. As you might know Java is a statically typed language. So the compiler is written in a way that it can verify types related errors during compile time. While python being a _dynamic_ language, types are not known until a program is run. So in a way, python compiler is dumb (or, less strict). But there indeed is a compile step involved when a python program is run. You might have seen python bytecode files with `.pyc` extension. Here is how you can see bytecode for a given python program.
|
||||
**What is the difference then, between Java and Python?**
|
||||
Well, Java's compiler is more strict and sophisticated. As you might know Java is a statically typed language. So the compiler is written in a way that it can verify types-related errors during compile time. While Python being a _dynamic_ language, types are not known until a program is run. So in a way, Python compiler is dumb (or, less strict). But there indeed is a compile step involved when a Python program is run. You might have seen Python bytecode files with `.pyc` extension. Here is how you can see bytecode for a given Python program.
|
||||
|
||||
```bash
|
||||
# Create a Hello World
|
||||
@@ -63,15 +63,15 @@ $ python -m dis hello_world.py
|
||||
10 RETURN_VALUE
|
||||
```
|
||||
|
||||
Read more about dis module [here](https://docs.python.org/3/library/dis.html)
|
||||
Read more about `dis` module [here](https://docs.python.org/3/library/dis.html).
|
||||
|
||||
Now coming to C/C++, there of course is a compiler. But the output is different than what java/python compiler would produce. Compiling a C program would produce what we also know as _machine code_. As opposed to bytecode.
|
||||
Now coming to C/C++, there of course is a compiler. But the output is different than what Java/Python compiler would produce. Compiling a C program would produce what we also know as _machine code_, as opposed to _bytecode_.
|
||||
|
||||
### Running The Programs
|
||||
|
||||
We know compilation is involved in all 3 languages we are discussing. Just that the compilers are different in nature and they output different types of content. In case of C/C++, the output is machine code which can be directly read by your operating system. When you execute that program, your OS will know how exactly to run it. **But this is not the case with bytecode.**
|
||||
|
||||
Those bytecodes are language specific. Python has its own set of bytecode defined (more in `dis` module) and so does java. So naturally, your operating system will not know how to run it. To run this bytecode, we have something called Virtual Machines. Ie: The JVM or the Python VM (CPython, Jython). These so called Virtual Machines are the programs which can read the bytecode and run it on a given operating system. Python has multiple VMs available. Cpython is a python VM implemented in C language, similarly Jython is a Java implementation of python VM. **At the end of the day, what they should be capable of is to understand python language syntax, be able to compile it to bytecode and be able to run that bytecode.** You can implement a python VM in any language! (And people do so, just because it can be done)
|
||||
Those bytecodes are language specific. Python has its own set of bytecode defined (more in `dis` module) and so does Java. So naturally, your operating system will not know how to run it. To run this bytecode, we have something called Virtual Machines. Ie: The JVM or the Python VM (CPython, Jython). These so-called Virtual Machines are the programs which can read the bytecode and run it on a given operating system. Python has multiple VMs available. CPython is a Python VM implemented in C language, similarly Jython is a Java implementation of Python VM. **At the end of the day, what they should be capable of is to understand Python language syntax, be able to compile it to bytecode and be able to run that bytecode.** You can implement a Python VM in any language! (And people do so, just because it can be done)
|
||||
|
||||
```
|
||||
The Operating System
|
||||
@@ -108,5 +108,5 @@ hello_world.c OS Specific machinecode | A New Pr
|
||||
|
||||
Two things to note for above diagram:
|
||||
|
||||
1. Generally, when we run a python program, a python VM process is started which reads the python source code, compiles it to byte code and run it in a single step. Compiling is not a separate step. Shown only for illustration purpose.
|
||||
1. Generally, when we run a Python program, a Python VM process is started which reads the Python source code, compiles it to bytecode and run it in a single step. Compiling is not a separate step. Shown only for illustration purpose.
|
||||
2. Binaries generated for C like languages are not _exactly_ run as is. Since there are multiple types of binaries (eg: ELF), there are more complicated steps involved in order to run a binary but we will not go into that since all that is done at OS level.
|
||||
|
||||
@@ -1,12 +1,12 @@
|
||||
# Some Python Concepts
|
||||
|
||||
Though you are expected to know python and its syntax at basic level, let us discuss some fundamental concepts that will help you understand the python language better.
|
||||
Though you are expected to know python and its syntax at basic level, let us discuss some fundamental concepts that will help you understand the Python language better.
|
||||
|
||||
**Everything in Python is an object.**
|
||||
|
||||
That includes the functions, lists, dicts, classes, modules, a running function (instance of function definition), everything. In the CPython, it would mean there is an underlying struct variable for each object.
|
||||
That includes the functions, lists, dicts, classes, modules, a running function (instance of function definition), everything. In the CPython, it would mean there is an underlying `struct` variable for each object.
|
||||
|
||||
In python's current execution context, all the variables are stored in a dict. It'd be a string to object mapping. If you have a function and a float variable defined in the current context, here is how it is handled internally.
|
||||
In Python's current execution context, all the variables are stored in a dict. It'd be a string to object mapping. If you have a function and a float variable defined in the current context, here is how it is handled internally.
|
||||
|
||||
```python
|
||||
>>> float_number=42.0
|
||||
@@ -35,7 +35,7 @@ Since functions too are objects, we can see what all attributes a function conta
|
||||
'__subclasshook__']
|
||||
```
|
||||
|
||||
While there are a lot of them, let's look at some interesting ones
|
||||
While there are a lot of them, let's look at some interesting ones.
|
||||
|
||||
#### __globals__
|
||||
|
||||
@@ -53,7 +53,7 @@ This attribute, as the name suggests, has references of global variables. If you
|
||||
|
||||
### __code__
|
||||
|
||||
This is an interesting one! As everything in python is an object, this includes the bytecode too. The compiled python bytecode is a python code object. Which is accessible via `__code__` attribute here. A function has an associated code object which carries some interesting information.
|
||||
This is an interesting one! As everything in Python is an object, this includes the bytecode too. The compiled Python bytecode is a Python code object. Which is accessible via `__code__` attribute here. A function has an associated code object which carries some interesting information.
|
||||
|
||||
```python
|
||||
# the file in which function is defined
|
||||
@@ -74,11 +74,11 @@ This is an interesting one! As everything in python is an object, this includes
|
||||
b't\x00d\x01|\x00\x9b\x00d\x02\x9d\x03\x83\x01\x01\x00d\x00S\x00'
|
||||
```
|
||||
|
||||
There are more code attributes which you can enlist by `>>> dir(hello.__code__)`
|
||||
There are more code attributes which you can enlist by `>>> dir(hello.__code__)`.
|
||||
|
||||
## Decorators
|
||||
|
||||
Related to functions, python has another feature called decorators. Let's see how that works, keeping `everything is an object` in mind.
|
||||
Related to functions, Python has another feature called decorators. Let's see how that works, keeping `everything is an object` in mind.
|
||||
|
||||
Here is a sample decorator:
|
||||
|
||||
@@ -115,7 +115,7 @@ What goes inside the `deco` function might seem complex. Let's try to uncover it
|
||||
1. Function `hello_world` is created
|
||||
2. It is passed to `deco` function
|
||||
3. `deco` create a new function
|
||||
1. This new function is calls `hello_world` function
|
||||
1. This new function calls `hello_world` function
|
||||
2. And does a couple other things
|
||||
4. `deco` returns the newly created function
|
||||
5. `hello_world` is replaced with above function
|
||||
@@ -156,7 +156,7 @@ Note how the `hello_world` name points to a new function object but that new fun
|
||||
|
||||
## Some Gotchas
|
||||
|
||||
- While it is very quick to build prototypes in python and there are tons of libraries available, as the codebase complexity increases, type errors become more common and will get hard to deal with. (There are solutions to that problem like type annotations in python. Checkout [mypy](http://mypy-lang.org/).)
|
||||
- Because python is dynamically typed language, that means all types are determined at runtime. And that makes python run very slow compared to other statically typed languages.
|
||||
- While it is very quick to build prototypes in Python and there are tons of libraries available, as the codebase complexity increases, type errors become more common and will get hard to deal with. (There are solutions to that problem like type annotations in Python. Checkout [mypy](http://mypy-lang.org/).)
|
||||
- Because Python is dynamically typed language, that means all types are determined at runtime. And that makes Python run very slow compared to other statically typed languages.
|
||||
- Python has something called [GIL](https://www.dabeaz.com/python/UnderstandingGIL.pdf) (global interpreter lock) which is a limiting factor for utilizing multiple CPU cores for parallel computation.
|
||||
- Some weird things that python does: https://github.com/satwikkansal/wtfpython
|
||||
- Some weird things that Python does: [https://github.com/satwikkansal/wtfpython](https://github.com/satwikkansal/wtfpython).
|
||||
|
||||
@@ -1,12 +1,12 @@
|
||||
# Python, Web and Flask
|
||||
|
||||
Back in the old days, websites were simple. They were simple static html contents. A webserver would be listening on a defined port and according to the HTTP request received, it would read files from disk and return them in response. But since then, complexity has evolved and websites are now dynamic. Depending on the request, multiple operations need to be performed like reading from database or calling other API and finally returning some response (HTML data, JSON content etc.)
|
||||
Back in the old days, websites were simple. They were simple static html contents. A webserver would be listening on a defined port and according to the HTTP request received, it would read files from disk and return them in response. But since then, complexity has evolved and websites are now dynamic. Depending on the request, multiple operations need to be performed like reading from database or calling other API and finally returning some response (HTML data, JSON content, etc.)
|
||||
|
||||
Since serving web requests is no longer a simple task like reading files from disk and return contents, we need to process each http request, perform some operations programmatically and construct a response.
|
||||
Since serving web requests is no longer a simple task like reading files from disk and return contents, we need to process each HTTP request, perform some operations programmatically and construct a response.
|
||||
|
||||
## Sockets
|
||||
|
||||
Though we have frameworks like flask, HTTP is still a protocol that works over TCP protocol. So let us setup a TCP server and send an HTTP request and inspect the request's payload. Note that this is not a tutorial on socket programming but what we are doing here is inspecting HTTP protocol at its ground level and look at what its contents look like. (Ref: [Socket Programming in Python (Guide) on RealPython](https://realpython.com/python-sockets/))
|
||||
Though we have frameworks like Flask, HTTP is still a protocol that works over TCP protocol. So, let us setup a TCP server and send an HTTP request and inspect the request's payload. Note that this is not a tutorial on socket programming but what we are doing here is inspecting HTTP protocol at its ground level and look at what its contents look like. (Ref: [Socket Programming in Python (Guide) on RealPython](https://realpython.com/python-sockets/))
|
||||
|
||||
```python
|
||||
import socket
|
||||
@@ -27,7 +27,7 @@ with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
|
||||
print(data)
|
||||
```
|
||||
|
||||
Then we open `localhost:65432` in our web browser and following would be the output:
|
||||
Then, we open `localhost:65432` in our web browser and following would be the output:
|
||||
|
||||
```bash
|
||||
Connected by ('127.0.0.1', 54719)
|
||||
@@ -47,10 +47,10 @@ So though it's a blob of bytes, knowing [http protocol specification](https://to
|
||||
|
||||
Flask, and other such frameworks does pretty much what we just discussed in the last section (with added more sophistication). They listen on a port on a TCP socket, receive an HTTP request, parse the data according to protocol format and make it available to you in a convenient manner.
|
||||
|
||||
ie: you can access headers in flask by `request.headers` which is made available to you by splitting above payload by `/r/n`, as defined in http protocol.
|
||||
That is you can access headers in Flask by `request.headers` which is made available to you by splitting above payload by `/r/n`, as defined in HTTP protocol.
|
||||
|
||||
Another example: we register routes in flask by `@app.route("/hello")`. What flask will do is maintain a registry internally which will map `/hello` with the function you decorated with. Now whenever a request comes with the `/hello` route (second component in the first line, split by space), flask calls the registered function and returns whatever the function returned.
|
||||
Another example: we register routes in Flask by `@app.route("/hello")`. What Flask will do is maintain a registry internally which will map `/hello` with the function you decorated with. Now, whenever a request comes with the `/hello` route (second component in the first line, split by space), Flask calls the registered function and returns whatever the function returned.
|
||||
|
||||
Same with all other web frameworks in other languages too. They all work on similar principles. What they basically do is understand the HTTP protocol, parses the HTTP request data and gives us programmers a nice interface to work with HTTP requests.
|
||||
|
||||
Not so much of magic, innit?
|
||||
Not so much of magic in it?
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
The design and development is just a part of the journey. We will need to setup continuous integration and continuous delivery pipelines sooner or later. And we have to deploy this app somewhere.
|
||||
|
||||
Initially we can start with deploying this app on one virtual machine on any cloud provider. But this is a `Single point of failure` which is something we never allow as an SRE (or even as an engineer). So an improvement here can be having multiple instances of applications deployed behind a load balancer. This certainly prevents problems of one machine going down.
|
||||
Initially, we can start with deploying this app on one virtual machine on any cloud provider. But this is a `Single point of failure` which is something we never allow as an SRE (or even as an engineer). So an improvement here can be having multiple instances of applications deployed behind a load balancer. This certainly prevents problems of one machine going down.
|
||||
|
||||
Scaling here would mean adding more instances behind the load balancer. But this is scalable upto only a certain point. After that, other bottlenecks in the system will start appearing. ie: DB will become the bottleneck, or perhaps the load balancer itself. How do you know what is the bottleneck? You need to have observability into each aspects of the application architecture.
|
||||
|
||||
@@ -14,32 +14,32 @@ Get deeper insights into scaling from School Of SRE's [Scalability module](../sy
|
||||
|
||||
## Monitoring Strategy
|
||||
|
||||
Once we have our application deployed. It will be working ok. But not forever. Reliability is in the title of our job and we make systems reliable by making the design in a certain way. But things still will go down. Machines will fail. Disks will behave weirdly. Buggy code will get pushed to production. And all these possible scenarios will make the system less reliable. So what do we do? **We monitor!**
|
||||
Once we have our application deployed. It will be working okay. But not forever. Reliability is in the title of our job and we make systems reliable by making the design in a certain way. But things still will go down. Machines will fail. Disks will behave weirdly. Buggy code will get pushed to production. And all these possible scenarios will make the system less reliable. So what do we do? **We monitor!**
|
||||
|
||||
We keep an eye on the system's health and if anything is not going as expected, we want ourselves to get alerted.
|
||||
|
||||
Now let's think in terms of the given url shortening app. We need to monitor it. And we would want to get notified in case something goes wrong. But we first need to decide what is that _something_ that we want to keep an eye on.
|
||||
Now let's think in terms of the given URL-shortening app. We need to monitor it. And we would want to get notified in case something goes wrong. But we first need to decide what is that _something_ that we want to keep an eye on.
|
||||
|
||||
1. Since it's a web app serving HTTP requests, we want to keep an eye on HTTP Status codes and latencies
|
||||
2. Request volume again is a good candidate, if the app is receiving an unusual amount of traffic, something might be off.
|
||||
3. We also want to keep an eye on the database so depending on the database solution chosen. Query times, volumes, disk usage etc.
|
||||
3. We also want to keep an eye on the database so depending on the database solution chosen. Query times, volumes, disk usage, etc.
|
||||
4. Finally, there also needs to be some external monitoring which runs periodic tests from devices outside of your data centers. This emulates customers and ensures that from customer point of view, the system is working as expected.
|
||||
|
||||
## Applications in SRE role
|
||||
|
||||
In the world of SRE, python is a widely used language. For small scripts and tooling developed for various purposes. Since tooling developed by SRE works with critical pieces of infrastructure and has great power (to bring things down), it is important to know what you are doing while using a programming language and its features. Also it is equally important to know the language and its characteristics while debugging the issues. As an SRE having a deeper understanding of python language, it has helped me a lot to debug very sneaky bugs and be generally more aware and informed while making certain design decisions.
|
||||
In the world of SRE, Python is a widely used language for small scripts and tooling developed for various purposes. Since tooling developed by SRE works with critical pieces of infrastructure and has great power (to bring things down), it is important to know what you are doing while using a programming language and its features. Also it is equally important to know the language and its characteristics while debugging the issues. As an SRE having a deeper understanding of Python language, it has helped me a lot to debug very sneaky bugs and be generally more aware and informed while making certain design decisions.
|
||||
|
||||
While developing tools may or may not be part of SRE job, supporting tools or services is more likely to be a daily duty. Building an application or tool is just a small part of productionization. While there is certainly that goes in the design of the application itself to make it more robust, as an SRE you are responsible for its reliability and stability once it is deployed and running. And to ensure that, you’d need to understand the application first and then come up with a strategy to monitor it properly and be prepared for various failure scenarios.
|
||||
|
||||
## Optional Exercises
|
||||
|
||||
1. Make a decorator that will cache function return values depending on input parameters.
|
||||
2. Host the URL shortening app on any cloud provider.
|
||||
3. Setup monitoring using many of the tools available like catchpoint, datadog etc.
|
||||
4. Create a minimal flask-like framework on top of TCP sockets.
|
||||
2. Host the URL-shortening app on any cloud provider.
|
||||
3. Setup monitoring using many of the tools available like Catchpoint, Datadog, etc.
|
||||
4. Create a minimal Flask-like framework on top of TCP sockets.
|
||||
|
||||
## Conclusion
|
||||
|
||||
This module, in the first part, aims to make you more aware of the things that will happen when you choose python as your programming language and what happens when you run a python program. With the knowledge of how python handles things internally as objects, lot of seemingly magic things in python will start to make more sense.
|
||||
This module, in the first part, aims to make you more aware of the things that will happen when you choose Python as your programming language and what happens when you run a Python program. With the knowledge of how Python handles things internally as objects, lot of seemingly magic things in Python will start to make more sense.
|
||||
|
||||
The second part will first explain how a framework like flask works using the existing knowledge of protocols like TCP and HTTP. It then touches the whole lifecycle of an application development lifecycle including the SRE parts of it. While the design and areas in architecture considered will not be exhaustive, it will give a good overview of things that are also important being an SRE and why they are important.
|
||||
The second part will first explain how a framework like Flask works using the existing knowledge of protocols like TCP and HTTP. It then touches the whole lifecycle of an application development lifecycle including the SRE parts of it. While the design and areas in architecture considered will not be exhaustive, it will give a good overview of things that are also important being an SRE and why they are important.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# The URL Shortening App
|
||||
|
||||
Let's build a very simple URL shortening app using flask and try to incorporate all aspects of the development process including the reliability aspects. We will not be building the UI and we will come up with a minimal set of API that will be enough for the app to function well.
|
||||
Let's build a very simple URL-shortening app using Flask and try to incorporate all aspects of the development process including the reliability aspects. We will not be building the UI and we will come up with a minimal set of API that will be enough for the app to function well.
|
||||
|
||||
## Design
|
||||
|
||||
@@ -8,19 +8,19 @@ We don't jump directly to coding. First thing we do is gather requirements. Come
|
||||
|
||||
### 1. High Level Operations and API Endpoints
|
||||
|
||||
Since it's a URL shortening app, we will need an API for generating the shorten link given an original link. And an API/Endpoint which will accept the shorten link and redirect to original URL. We are not including the user aspect of the app to keep things minimal. These two API should make app functional and usable by anyone.
|
||||
Since it's a URL-shortening app, we will need an API for generating the shorten link given an original link. And an API/Endpoint which will accept the shorten link and redirect to original URL. We are not including the user aspect of the app to keep things minimal. These two API should make app functional and usable by anyone.
|
||||
|
||||
### 2. How to shorten?
|
||||
|
||||
Given a url, we will need to generate a shortened version of it. One approach could be using random characters for each link. Another thing that can be done is to use some sort of hashing algorithm. The benefit here is we will reuse the same hash for the same link. ie: if lot of people are shortening `https://www.linkedin.com` they all will have the same value, compared to multiple entries in DB if chosen random characters.
|
||||
Given a URL, we will need to generate a shortened version of it. One approach could be using random characters for each link. Another thing that can be done is to use some sort of hashing algorithm. The benefit here is we will reuse the same hash for the same link. Ie: if lot of people are shortening `https://www.linkedin.com`, they all will have the same value, compared to multiple entries in DB if chosen random characters.
|
||||
|
||||
What about hash collisions? Even in random characters approach, though there is a less probability, hash collisions can happen. And we need to be mindful of them. In that case we might want to prepend/append the string with some random value to avoid conflict.
|
||||
What about hash collisions? Even in random characters approach, though there is a less probability, hash collisions can happen. And we need to be mindful of them. In that case, we might want to prepend/append the string with some random value to avoid conflict.
|
||||
|
||||
Also, choice of hash algorithm matters. We will need to analyze algorithms. Their CPU requirements and their characteristics. Choose one that suits the most.
|
||||
|
||||
### 3. Is URL Valid?
|
||||
|
||||
Given a URL to shorten, how do we verify if the URL is valid? Do we even verify or validate? One basic check that can be done is see if the URL matches a regex of a URL. To go even further we can try opening/visiting the URL. But there are certain gotchas here.
|
||||
Given a URL to shorten, how do we verify if the URL is valid? Do we even verify or validate? One basic check that can be done is see if the URL matches a regex of a URL. To go even further, we can try opening/visiting the URL. But there are certain gotchas here.
|
||||
|
||||
1. We need to define success criteria. ie: HTTP 200 means it is valid.
|
||||
2. What if the URL is in private network?
|
||||
@@ -32,13 +32,12 @@ Finally, storage. Where will we store the data that we will generate over time?
|
||||
|
||||
### 5. Other
|
||||
|
||||
We are not accounting for users into our app and other possible features like rate limiting, customized links etc but it will eventually come up with time. Depending on the requirements, they too might need to get incorporated.
|
||||
We are not accounting for users into our app and other possible features like rate limiting, customized links, etc. but it will eventually come up with time. Depending on the requirements, they too might need to get incorporated.
|
||||
|
||||
The minimal working code is given below for reference but I'd encourage you to come up with your own.
|
||||
The minimal working code is given below for reference, but I'd encourage you to come up with your own.
|
||||
|
||||
```python
|
||||
from flask import Flask, redirect, request
|
||||
|
||||
from hashlib import md5
|
||||
|
||||
app = Flask("url_shortener")
|
||||
|
||||
Reference in New Issue
Block a user