Learn to build a Python web framework from scratch

How to write a Python web framework. Part I.
"Don't reinvent the wheel" is one of the most frequent mantras we hear every day. But
what if I want to learn more about the wheel? What if I want to learn how to make this
damn wheel? I think it is a great idea to reinvent it for the purpose of learning. Thus, in
this series, we will write our own Python web framework to see how all that magic is done
in Flask, Django and other frameworks.
In this first part of the series, we will build the most important parts of the framework. At
the end of it, we will have request handlers (think Django views) and routing: both simple
(like /books/) and parameterized (like /greet/{name}). If you like it after reading, please let
me know in the comments what other features we should implement next.
Before I start doing something new, I like to think about the end result. In this case, at
the end of the day, we want to be able to use this framework in production and thus we
want our framework to be served by a fast, lightweight, production-level application
server. I have been using gunicorn in all of my projects in the last few years and I am
very satisfied with the results. So, let's go with gunicorn.
Gunicornis a WSGI HTTP Server, so it expects a specific entrypoint to our application. If

you don't know what WSGI is go find out, I will wait. Otherwise, you will not understand a
huge chunk of this blog post.
Have you learnt what WSGI is? Good. Let's continue.
To be WSGI-compatible, we need a callable object (a function or a class) that expects

two parameters (environ and start_response) and returns a WSGI-compatible response.
Don't worry if it doesn't make sense yet. Hopefully it will "click" for you while writing the
actual code. So, let's get started with the code.
Think of a name for your framework and create a folder with that name. I named it bumbo:
mkdir bumbo
Go into this folder, create a virtual env and activate it:

cd bumbo
python3.6 -m venv venv
source venv/bin/activate
Now, create the file named app.py where we will store our entrypoint for gunicorn:
touch app.py
Inside this app.py, let's write a simple function to see if it works with gunicorn:
# app.py
def app(environ, start_response):

response_body = b"Hello, World!"
status = "200 OK"
start_response(status, headers=[])
return iter([response_body])
As mentioned above, this entrypoint callable receives two params. One of them
is environ where all kinds of info about request is stored such as a request method, url,
query params and the like. The second is start_response which starts the response as
the name suggests. Now, let's try to run this code with gunicorn. For that
install gunicorn and run it like so:
pip install gunicorn
gunicorn app:app
The first app is the file which we created and the second app is the name of the function
we just wrote. If all is good, you will see something like the following in the output:
[2019-02-09 17:58:56 +0500] [30962] [INFO] Starting gunicorn 19.9.0
[2019-02-09 17:58:56 +0500] [30962] [INFO] Listening at: http://127.0.0.1:8000 (30962)
[2019-02-09 17:58:56 +0500] [30962] [INFO] Using worker: sync
[2019-02-09 17:58:56 +0500] [30966] [INFO] Booting worker with pid: 30966
If you see this, open your browser and go to http://localhost:8000. You should see our
good old friend: the Hello, World!message. Awesome! We will build off of this.
Now, let's turn this function into a class because we will need quite a few helper methods
and they are much easier to write inside a class. Create an api.py file:
touch api.py
Inside this file, create the following API class. I will explain what it does in a bit:
# api.py
class API:
def __call__(self, environ, start_response):
response_body = b"Hello, World!"
status = "200 OK"
start_response(status, headers=[])
return iter([response_body])
Now, delete everything inside app.py and write the following:

# app.py
from api import API
app = API()
Restart your gunicorn and check the result in the browser. It should be the same as before
because we simply converted our function named app to a class called API and overrode
its __call__ method which is called when you call the instances of this class:
app = API()
app() # this is where __call__ is called
Now that we created our class, I want to make the code more elegant because all those
bytes (b"Hello World") and start_response seem confusing to me. Thankfully, there is a
cool package called WebOb that provides objects for HTTP requests and responses by
wrapping the WSGI request environment and response status, headers and body. By using
this package, we can pass the environ and start_response to the classes provided by this
package and not have to deal with them ourselves. Before we continue, I suggest you
take a look at the documentation of WebOb to understand what I am talking about and
the API of WebOb more.
Here is how we will go about refactoring this code. First, install WebOb:
pip install webob
Import the Request and Response classes at the beginning of the api.py file:
# api.py
from webob import Request, Response
...
and now we can use them inside the __call__ method:

# api.py
class API:
request = Request(environ)
response = Response()
response.text = "Hello, World!"
return response(environ, start_response)
Looks much better! Restart the gunicorn and you should see the same result as before.
And the best part is I don't have to explain what is being done here. It is all self-
explanatory. We are creating a request, a response and then returning that response.
Awesome! I do have to note that request is not being used here yet because we are not
doing anything with it. So, let's use this chance and use the request object as well. Also,
let's refactor the response creation into its own method. We will see why it is better later:
# api.py
class API:
response = self.handle_request(request)
def handle_request(self, request):

user_agent = request.environ.get("HTTP_USER_AGENT", "No User Agent Found")
response.text = f"Hello, my friend with this user agent: {user_agent}"
return response
Restart your gunicorn and you should see this new message in the browser. Did you see
it? Cool. Let's go on.
At this point, we handle all the requests in the same way. Whatever request we receive,
we simply return the same response which is created in the handle_request method.
Ultimately, we want it to be dynamic. That is, we want to serve the request coming
from /home/ differently than the one coming from /about/.
To that end, inside app.py, let's create two methods that will handle those two requests:
# app.py
from api.py import API
app = API()
def home(request, response):

response.text = "Hello from the HOME page"
def about(request, response):

response.text = "Hello from the ABOUT page"
Now, we need to somehow associate these two methods with the above mentioned
paths: /home/ and /about/. I like the Flask way of doing it that would look like this:
# app.py
from api.py import API
app = API()
@app.route("/home")
@app.route("/about")
def about(request, response):
response.text = "Hello from the ABOUT page"
What do you think? Looks good? Then let's implement this bad boy!
As you can see, the route method is a decorator, accepts a path and wraps the methods.
It shouldn't be too difficult to implement:
# api.py
class API:
def __init__(self):
self.routes = {}
def route(self, path):

def wrapper(handler):
self.routes[path] = handler
return handler
return wrapper
...
Here is what we did here. In the __init__ method, we simply defined

a dict called self.routes where we will be storing paths as keys and handlers as values.
It can look like this:
print(self.routes)
{
"/home": <function home at 0x1100a70c8>,
"/about": <function about at 0x1101a80c3>
}
In the route method, we took path as an argument and in the wrapper method simply put
this path in the self.routesdictionary as a key and the handler as a value.
At this point, we have all the pieces of the puzzle. We have the handlers and the paths
associated with them. Now, when a request comes in, we need to check its path, find an
appropriate handler, call that handler and return an appropriate response. Let's do that:
# api.py
class API:
...

for path, handler in self.routes.items():

if path == request.path:
handler(request, response)
return response
...
Wasn't too difficult, was it? We simply iterated over self.routes, compared paths with the
path of the request, if there is a match, called the handler associated with that path.
Restart the gunicorn and try those paths in the browser. First, go
to http://localhost:8000/home/ and then go to http://localhost:8000/about/. You should
see the corresponding messages. Pretty cool, right?
As the next step, we can answer the question of "What happens if the path is not found?".
Let's create a method that returns a simple HTTP response of "Not found." with the status
code of 404:
# api.py
class API:
...
def default_response(self, response):

response.status_code = 404
response.text = "Not found."
...
Now, let's use it in our handle_request method:

# api.py
class API:
...

if path == request.path:
return response
self.default_response(response)
return response
...
Restart the gunicorn and try some nonexistent routes. You should see this lovely "Not
found." page. Now, let's refactor out finding a handler to its own method for the sake of
readability:
# api.py
class API:
...
def find_handler(self, request_path):

if path == request_path:
return handler
...
Just like before, it is simply iterating over self.route, comparing paths with the request
path and returning the handler if paths are the same. It returns None if no handler was
found. Now, we can use it in our handle_request method:
# api.py
class API:
...

handler = self.find_handler(request_path=request.path)
if handler is not None:

else:
return response
...
I think it looks much better and is pretty self explanatory. Restart your gunicorn to see that
everything is working just like before.
At this point, we have routes and handlers. It is pretty awesome but our routes are simple.
They don't support keyword parameters in the url path. What if we want to have this route
of @app.route("/hello/{person_name}") and be able to use this person_name inside our
handlers like this:
def say_hello(request, response, person_name):
resp.text = f"Hello, {person_name}"
For that, if someone goes to the /hello/Matthew/, we need to be able to match this path
with the registered /hello/{person_name}/ and find the appropriate handler. Thankfully,
there is already a package called parse that does exactly that for us. Let's go ahead and
install it:
pip install parse
Let's test it out:

>>> from parse import parse
>>> result = parse("Hello, {name}", "Hello, Matthew")
>>> print(result.named)
{'name': 'Matthew'}
As you can see, it parsed the string Hello, Matthew and was able to identify
that Matthew corresponds to the {name} that we provided.
Let's use it in our find_handler method to find not only the method that corresponds to the
path but also the keyword params that were provided:
# api.py
from parse import parse
class API:
...

parse_result = parse(path, request_path)
if parse_result is not None:
return handler, parse_result.named
return None, None
...
We are still iterating over self.routes and now instead of comparing the path to the
request path, we are trying to parse it and if there is a result, we are returning both the
handler and keyword params as a dictionary. Now, we can use this
inside handle_request to send those params to the handlers like this:
# api.py
class API:
...

handler, kwargs = self.find_handler(request_path=request.path)

handler(request, response, **kwargs)
else:
return response
...
The only changes are, we are getting both handler and kwargs from self.find_handler, and
passing that kwargs to the handler like this **kwargs.
Let's write a handler with this type of route and try it out:
# app.py
...
@app.route("/hello/{name}")
def greeting(request, response, name):
response.text = f"Hello, {name}"
...
Restart your gunicorn and go to http://localhost:8000/hello/Matthew/. You should the

wonderful message of Hello, Matthew. Awesome, right? Add a couple more such handlers
of yours. You can also indicate the type of the given params. For example you can
do @app.route("/tell/{age:d}") so that you have the param age inside the handler as a
digit.
Conclusion
This was a long ride but I think it was great. I personally learned a lot while writing this. If
you liked this blog post, please let me know in the comments what other features we
should implement in our framework. I am thinking of class based handlers, support for
templates and static files.
Fight on!
How to write a Python web framework. Part II.
In the first part, we started writing our own Python framework and implemented the
following features:
 WSGI compatibility
 Request Handlers
 Routing: simple and parameterized
Make sure to read Part I of these series before this one.
This part will be no less exciting and we will add the following features in it:
 Check for duplicate routes

 Class Based Handlers
 Unit tests
Ready? Let's get started.
Duplicate routes
Right now, our framework allows to add the same route any number of times. So, the
following will work:
@app.route("/home")
@app.route("/home")
def home2(request, response):
response.text = "Hello from the SECOND HOME page"
The framework will not complain and because we use a Python dictionary to store routes,
only the last one will work if you go to http://localhost:8000/home/. Obviously, this is not
good. We want to make sure that the framework complains if the user tries to add an
existing route. As you can imagine, it is not very difficult to implement. Because we are
using a Python dict to store routes, we can simply check if the given path already exists
in the dictionary. If it does, we throw an exception, if it does not we let it add a route.
Before we write any code, let's remember our main API class:
# api.py
class API:
def __init__(self):
self.routes = {}

return handler
return wrapper


parse_result = parse(path, request_path)
if parse_result is not None:
return handler, parse_result.named
return None, None


else:
return response
def default_response(self, response):

response.status_code = 404
response.text = "Not found."
We need to change the route function so that it throws an exception if an existing route
is being added again:
# api.py
class API:
...

if path in self.routes:
raise AssertionError("Such route already exists.")
return handler
return wrapper
...
Now, try adding the same route twice and restart your gunicorn. You should see the
following exception thrown:
Traceback (most recent call last):
...
AssertionError: Such route already exists.
We can refactor it to decrease it to one line:

# api.py
class API:
...

assert path not in self.routes, "Such route already exists."
...
Voilà! Onto the next feature.
Class Based Handlers
If you know Django, you know that it supports both function based and class based views
(our handlers). We already have function based handlers. Now we will add class based
ones which are more suitable if the handler is more complicated and bigger. Our class
based handlers will look like this:
# app.py
...
@app.route("/book")
class BooksHandler:
def get(self, req, resp):
resp.text = "Books Page"
def post(self, req, resp):

resp.text = "Endpoint to create a book"
...
It means that our dict where we store routes self.routes can contain both classes and
functions as values. Thus, when we find a handler in the handle_request() method, we
need to check if the handler is a function or if it is a class. If it is a function, it should work
just like now. If it is a class, depending on the request method, we should call the
appropriate method of the class. That is, if the request method is GET, we should call
the get() method of the class, if it is POST we should call the postmethod and etc. Here is
how the handle_request() method looks like now:
# api.py
class API:
...


else:
return response
...
The first thing we will do is check if the found handler is a class. For that, we use
the inspect module like this:
# api.py
...
import inspect
class API:
...


if inspect.isclass(handler):
pass # class based handler is being used
else:
else:
return response
...
Now, if a class based handler is being used, we need to find the appropriate method of
the class depending on the request method. For that we can use the built-
in getattr function:
# api.py
...
class API:
...


handler_function = getattr(handler(), request.method.lower(), None)
pass
else:
else:
return response
...
getattr accepts an object instance as the first param and the attribute name to get as the
second. The third argument is the value to return if nothing is found. So, GET will
return get, POST will return post and some_other_attribute will return None. If
the handler_function is None, it means that such function was not implemented in the class
and that this request method is not allowed:
if handler_function is None:
raise AttributeError("Method now allowed", request.method)
If the handler_function was actually found, then we simply call it:

handler_function(request, response, **kwargs)
Now the whole method looks like this:

...
class API:
...


handler_function(request, response, **kwargs)
else:
else:
...
I don't like that we have both handler_function and handler. We can refactor them to make
it more elegant:
# api.py
...
class API:
...


handler = getattr(handler(), request.method.lower(), None)
if handler is None:

else:
return response
...
And that's it. We can now test the support for class based handlers. First, if you haven't
already, add this handler to app.py:
# app.py
...
@app.route("/book")
class BooksResource:
def get(self, req, resp):
resp.text = "Books Page"
Now, restart your gunicorn and go to the page http://localhost:8000/book and you should
see the message Books Page. And there you go. We have added support for class based
handlers. Play with them a little bit by implementing other methods such
as post and delete as well.
Onto the next feature!
Unit Tests
What project is reliable if it has no unit tests, right? So let's add a couple. I like
using pytest, so let's install it:
pip install pytest
and create a file where we will write our tests:

touch test_bumbo.py
Just to remind you, bumbo is the name of the framework. You may have named it
differently. Also, if you don't know what pytest is, I strongly recommend you look at it to
understand how unit tests are written below.
First of all, let's create a fixture for our API class that we can use in every test:
# test_bumbo.py
import pytest
from api import API
@pytest.fixture
def api():
return API()
Now, for our first unit test, let's start with something simple. Let's test if we can add a
route. If it doesn't throw an exception, it means that the test passes successfully:
...
def test_basic_route(api):
@api.route("/home")
def home(req, resp):
resp.text = "YOLO"
Run the test like this: pytest test_bumbo.py and you should see something like the
following:
collected 1 item
test_bumbo.py .
[100%]
====== 1 passed in 0.09 seconds ======
Now, let's test that it throws an exception if we try to add an existing route:
# test_bumbo.py
...
def test_route_overlap_throws_exception(api):
@api.route("/home")
resp.text = "YOLO"
with pytest.raises(AssertionError):
@api.route("/home")
def home2(req, resp):
resp.text = "YOLO"
Run the tests again and you will see that both of them pass.
We can add a lot more tests such as the default response, parameterized routing, status
codes and etc. However, all of them require that we send an HTTP request to our
handlers. For that we need to have a test client. But I think this post will become too big
if we do it here. We will do it in the next post in this series. We will also add support for
templates and a couple of other interesting stuff. So, stay tuned.
As usual, if you want to see some feature implemented please let me know in the
comments section.
A little reminder that this series is based on the Alcazar framework that I am writing for
learning purposes. If you liked this series, show some love by starring the repo.
Fight on!
How to write a Python web framework. Part III.
In the previous blog posts in the series, we started writing our own Python framework
and implemented the following features:
 Unit tests
In this part, we will add a few awesome features to the list:
 Test Client
 Alternative way to add routes (like Django)
 Support for templates
Test Client
In the part 2, we wrote a couple of unit tests. However, we stopped when we needed to
send HTTP requests to our handlers because we didn't have a test client that could do
that. Let's add one then.
By far the most popular way of sending HTTP requests in Python is the Requests library
by Kenneth Reitz. However, for us to be able to use it in the unit tests, we should always
have our app up and running (i.e. start gunicorn before running tests). The reason is
that Requests only ships with a single Transport Adapter, the HTTPAdapter. That defeats
the purpose of unit tests. Unit tests should be self sustained. Fortunately for us, Sean
Brant wrote a WSGI Transport Adapter for Requests that we can use to create a test
client. Go ahead and install both of these wonderful libraries:
pip install requests requests-wsgi-adapter
Now, let's write the code first and then discuss.
Add the following method to the main API class in api.py:

# api.py
...
from requests import Session as RequestsSession
from wsgiadapter import WSGIAdapter as RequestsWSGIAdapter
class API:
...
def test_session(self, base_url="http://testserver"):

session = RequestsSession()
session.mount(prefix=base_url, adapter=RequestsWSGIAdapter(self))
return session
...
As written here, to use the Requests WSGI Adapter, we need to mount it to a Session
object. This way, any request made using this test_session whose URL starts with the
given prefix will use the given RequestsWSGIAdapter. Great, now we can use
this test_session to create a test client. Create a conftest.py file and move the api fixture
to this file so that it looks like this:
# conftest.py
import pytest
from api import API
@pytest.fixture
def api():
return API()
In case you didn't know, this file is where pytest looks for fixtures by default. Remember
to delete this api fixture from test_bumbo.py. Now, let's create the test client fixture:
# conftest.py
...
@pytest.fixture
def client(api):
return api.test_session()
Our client needs the api fixture and returns the test_session that we wrote earlier. Now
we can use this client fixture in our unit tests. Let's go right ahead to the test_bumbo.py file
and write a unit test that tests if the client can send a request:
# test_bumbo.py
...
def test_bumbo_test_client_can_send_requests(api, client):

RESPONSE_TEXT = "THIS IS COOL"
@api.route("/hey")
def cool(req, resp):
resp.text = RESPONSE_TEXT
assert client.get("http://testserver/hey").text == RESPONSE_TEXT
Run the unit tests by pytest test_bumbo.py and voila. We see that all the tests pass. Let's
add a couple more unit tests for the most important parts:
# test_bumbo.py
...
def test_parameterized_route(api, client):

@api.route("/{name}")
def hello(req, resp, name):
resp.text = f"hey {name}"
assert client.get("http://testserver/matthew").text == "hey matthew"

assert client.get("http://testserver/ashley").text == "hey ashley"
This tests that the parameters that we send in the url are working.
# test_bumbo.py
...
def test_default_404_response(client):
response = client.get("http://testserver/doesnotexist")
assert response.status_code == 404

assert response.text == "Not found."
This one tests that if a request is sent to a non existent route, 404(Not Found) response
is returned.
The rest I will leave to you. Try to write a couple more tests and let me know in the
comments if you need any help. Here are some ideas for unit tests:
 test that class based handlers are working with a GET request
 test that class based handlers are working with a POST request
 test that class based handlers are returning Method Not Allowed. response if an invalid
request method is used
 test that status code is being returned properly
Alternative way to add routes
Right now, here is how routes are added:

@api.route("/home")
def handler(req, resp):
resp.text = "YOLO"
That is, routes are added as decorators, like in Flask. Some people may like the Django
way of registering urls. So, let's give them a choice to add routes like this:
resp.text = "YOLO"
def handler2(req, resp):

resp.text = "YOLO2"
api.add_route("/home", handler)
api.add_route("/about", handler2)
This add_route method should do two things. Check if the route is already registered or
not and if not, register it:
# api.py
class API:
...
def add_route(self, path, handler):

...
Pretty simple. Does this code look familiar to you? It is because we already wrote such
code in the route decorator. We can now follow the DRY principle and use
this add_route method inside the route decorator:
# api.py
class API:
...
def add_route(self, path, handler):


self.add_route(path, handler)
return handler
return wrapper
...
And let's add a unit test to check if it is working:

# test_bumbo.py
...
def test_alternative_route(api, client):

response_text = "Alternative way to add a route"

resp.text = response_text
api.add_route("/alternative", home)
assert client.get("http://testserver/alternative").text == response_text
...
Run your tests and you will see that all of them pass.
Templates support
When I am implementing something new, I like to do something called README driven

development. It is a technique where you write down how you want your API to look like
before implementing. Let's do just that for this feature. Say we have this template that we
want to use in our handler:
<html>
<header>
<title>{{ title }}</title>
</header>
<body>
The name of the framework is {{ name }}
</body>
</html>
{{ title }} and {{ name }} are variables that are sent from a handler and here is how a
handler looks like:
api = API(templates_dir="templates")
@api.route("/home")
resp.body = api.template("home.html", context={"title": "Awesome Framework", "name":
"Alcazar"})
I want it to be as simple as possible so I just need one method that takes template name
and context as params and renders that template with the given params. Also, we want
templates directory to be configurable just like above.
With the API designed, we can now implement it.
For templates support, I think that Jinja2 is the best choice. It is a modern and designer-
friendly templating language for Python, modelled after Django’s templates. So, if you
know Django it should feel right at home.
Jinja2uses a central object called the template Environment. We will configure this
environment upon application initialization and load templates with the help of this
environment. Here is how to create and configure one:
import os
from jinja2 import Environment, FileSystemLoader
templates_env = Environment(loader=FileSystemLoader(os.path.abspath("templates")))
FileSystemLoader loads templates from the file system. This loader can find templates in
folders on the file system and is the preferred way to load them. It takes the path to the
templates directory as a parameter. Now we can use this templates_envlike so:
templates_env.get_template("index.html").render({"title": "Awesome Framework", "name":
"Alcazar"})
Now that we understand how everything works in Jinja2, let's add it to our own
framework. First, let's install Jinja2:
pip install Jinja2
Then, create the Environment object in the __init__ method of our API class:
# api.py
...
import os
from jinja2 import Environment, FileSystemLoader
class API:
def __init__(self, templates_dir="templates"):
self.routes = {}
self.templates_env =
Environment(loader=FileSystemLoader(os.path.abspath(templates_dir)))
...
We did almost the same thing as above except that we gave templates_dir a default value
of templates so that users don't have to write it if they don't want to. Now we have
everything to implement the template method we designed earlier:
# api.py
...
class API:
...
def template(self, template_name, context=None):

if context is None:
context = {}
return self.templates_env.get_template(template_name).render(**context)
...
I don't think there is a need to explain anything here. The only thing you may wonder
about is why I gave context a default value of None, checked if it is None and then set the
value to an empty dictionary {}. You may say I could have given it the default value
of {} in the declaration. But dict is a mutable object and it is a bad practice to set a
mutable object as a default value in Python. Read more about this here.
With everything ready, we can create templates and handlers. First, create
the templates folder:
mkdir templates
Create the index.html file by doing touch templates/index.html and put the following
inside:
<html>
<header>
</header>
<body>
<h1>The name of the framework is {{ name }}</h1>
</body>
</html>
Now we can create a handler in our app.py:

# app.py
...
@app.route("/template")
def template_handler(req, resp):
resp.body = app.template("index.html", context={"name": "Alcazar", "title": "Best
Framework"})
...
That's it (well, almost). Start gunicorn and go to http://localhost:8000/template. You will

see a big bold Internal Server Error. That's because resp.body expects bytes and
our template method returns a unicode string. Thus, we will need to encode it:
# app.py
@app.route("/template")
def template_handler(req, resp):
resp.body = app.template("index.html", context={"name": "Alcazar", "title": "Best
Framework"}).encode()
Restart gunicorn and you will see our template in all its glory. In the future posts, we will
remove the need to encode and make our API prettier.
Conclusion
We have implemented three new features in this post:
 Test Client
Make sure to let me know in the comments what other features we should implement in
this series. For the next part, we will definitely add support for static files but I am not
sure what other features we should add.
How to write a Python web framework. Part IV.
In the previous blog posts in the series, we started writing our own Python framework
and implemented the following features:
 Unit tests
 Test Client
In this part, we will add a few more awesome features to the list:
 Custom exception handler

 Support for static files
 Middleware
Custom exception handler
Exceptions inevitably happen. Users may do something that we didn't expect. We may
write some code that doesn't work on some occasions. Users may go to a non existent
page. With what we have right now, if some exception happens, we show a big
ugly Internal Server Error message. Instead, we could show some nice one. Something
along the lines of Oops! Something went wrong. or Please, contact our customer support. For
that, we need to be able to catch those exceptions and handle them however we want.
It will look like this:

# app.py
from api import API
app = API()
def custom_exception_handler(request, response, exception_cls):

response.text = "Oops! Something went wrong. Please, contact our customer support at
+1-202-555-0127."
app.add_exception_handler(custom_exception_handler)
Here we create a custom exception handler. It looks almost like our simple request
handlers, except that it has exception_clsas its third argument. Now, if we have a request
handler that throws an exception, this above-mentioned custom exception handler
should be called.
# app.py
@app.route("/home")
def exception_throwing_handler(request, response):
raise AssertionError("This handler should not be user")
If we go to http://localhost:8000/home, instead of our previous big ugly Internal Server

Error, we should be able to see our custom message of Oops! Something went wrong.
Please, contact our customer support at +1-202-555-0127. . Does it look good enough?
Let's go ahead and implement it.
The first thing we need is a variable inside our main API class where we will store our
exception handler:
# api.py
class API:
def __init__(self, templates_dir="templates"):
...
self.exception_handler = None
Now we need to add the add_exception_handler method:

# api.py
class API:
...
def add_exception_handler(self, exception_handler):

self.exception_handler = exception_handler
Having registered our custom exception handler, we need to call when an exception
happens. Where do exceptions happen? That's right: when handlers are called. We call
the handlers inside our handle_request method. So, we need to wrap it with a try/except
clause and call our custom exception handler in the except part:
# api.py
class API:
...

try:
handler = getattr(handler(), request.method.lower(), None)
if handler is None:

else:
except Exception as e:
if self.exception_handler is None:
raise e
else:
self.exception_handler(request, response, e)
return response
We also need to make sure that if no exception handler has been registered, the
exception is propagated.
We have everything in place. Go ahead and restart your gunicorn and go

to http://localhost:8000/home. You should see our little cute message instead of the big
ugly default one. Of course, make sure that you have the above mentioned exception
handler and the errorful request handler in the app.py.
If you want to go one step further, create a nice template and use
our api.template() method inside the exception handler. However, our framework doesn't
support static files and thus you will have hard time designing your template with CSS
and JavaScript. Don't get sad because this is exactly what we are doing next.
Support for static files
Templates are not truly templates without good CSS and JavaScript, are they? Shall we
add a support for such files then?
Just like we used Jinja2 for template support, we will use WhiteNoise for static file
serving. Install it:
pip install whitenoise
WhiteNoise is pretty simple. The only thing that we need to do is wrap our WSGI app and
give it the static folder path as a parameter. Before we do that, let's remember how
our __call__ method looks like:
# api.py
class API:
...

...
This is basically an entrypoint to our WSGI app and this is exactly what we need to wrap
with WhiteNoise. Thus, let's refactor its content to a separate method so that it will be
easier to wrap it with WhiteNoise:
# api.py
class API:
...
def wsgi_app(self, environ, start_response):


return self.wsgi_app(environ, start_response)
Now, in our constructor, we can initialize a WhiteNoise instance:

# api.py
...
from whitenoise import WhiteNoise
class API:
...
def __init__(self, templates_dir="templates", static_dir="static"):
self.routes = {}
self.templates_env = Environment(loader=FileSystemLoader(templates_dir))
self.exception_handler = None
self.whitenoise = WhiteNoise(self.wsgi_app, root=static_dir)
As you can see, we wrapped our wsgi_app with WhiteNoise and gave it a path to the static
folder as the second param. The only thing left to do is make this self.whitenoise an
entrypoint to our framework:
# api.py
class API:
...
return self.whitenoise(environ, start_response)
With everything in place, create static folder in the project root, create the main.css file
inside and put the following into it:
body {
background-color: chocolate;
}
In the third blog post, we created the templates/index.html. Now we can put our newly
created css file inside this template:
<html>
<header>
<link href="/main.css" type="text/css" rel="stylesheet">

</header>
<body>
<h1>The name of the framework is {{ name }}</h1>
</body>
</html>
Restart your gunicorn and go to http://localhost/template. You should see that the color
of the whole background is chocolate, not white, meaning that our static file is being
served. Awesome!
Middleware
If you need a little recap of what middlewares are and how they work, go read this
post first. Otherwise, this part may seem a little confusing. I will wait. Back? Great. Let's
go.
You know what they are and how they work but you may be wondering what they are
used for. Basically, middleware is a component that can modify an HTTP request and/or
response and is designed to be chained together to form a pipeline of behavioral changes
during request processing. Examples of middleware tasks can be request logging and
HTTP authentication. The main point is that none of these is fully responsible for
responding to a client. Instead, each middleware changes the behavior in some way as
part of the pipeline, leaving the actual response to come from something later in the
pipeline. In our case, that something that actually responds to a client is our request
handlers. Middlewares are wrappers around our WSGI app that have the ability to modify
requests and responses.
From the bird's eye view, the code will look like this:
FirstMiddleware(SecondMiddleware(our_wsgi_app))
So, when a request comes in, it first goes to FirstMiddleware. It modifies the request and
sends it over to SecondMiddleware. Now, SecondMiddleware modifies the request and sends
it over to our_wsgi_app. Our app handles the request, prepares the response and sends it
back to SecondMiddleware. It can modify the response if it wants and send it back
to FirstMiddleware. It modifies the response and sends it back to the web server (e.g.
gunicorn).
Let's go ahead and create a Middleware class that other middlewares will inherit from and
that wraps our wsgi app.
Create a middleware.py file first:

touch middleware.py
Now, we can begin our Middleware class:
# middleware.py
class Middleware:
def __init__(self, app):
self.app = app
As we mentioned above, it should wrap a wsgi app and in case of multiple middlewares
that app can also be another middleware.
As a base middleware class, it should also have the ability to add another middleware to
the stack:
# middleware.py
class Middleware:
...
def add(self, middleware_cls):
self.app = middleware_cls(self.app)
It is simply wrapping the given middleware class around our current app.
It should also have its main methods which are request processing and response
processing. For now, they will do nothing. The child classes that will inherit from this class
will implement these methods:
# middleware.py
class Middleware:
...
def process_request(self, req):

pass
def process_response(self, req, resp):

pass
Now, the most important part, the method that handles incoming requests:
# middleware.py
class Middleware:
...

self.process_request(request)
response = self.app.handle_request(request)
self.process_response(request, response)
return response
It first calls the self.process_request to do something with the request. Then delegates
the response creation to the app that it is wrapping. Finally, it calls the process_response to
do something with the response object. Then simply returns the response upward.
As middlewares are the first entrypoint to our app now, they are the ones called by our
web server (e.g. gunicorn). Thus, middlewares should implement the WSGI entrypoint
interface:
# middleware.py
from webob import Request
class Middleware:

response = self.app.handle_request(request)
It is just a copy of the wsgi_app function we created above.
With our Middleware class implemented, let's add it to our main API class:
# api.py
...
from middleware import Middleware
class API:
def __init__(self, templates_dir="templates", static_dir="static"):
...
self.middleware = Middleware(self)
It wraps around self which is our wsgi app. Now, let's give it the ability to add
middlewares:
# api.py
class API:
...
def add_middleware(self, middleware_cls):

self.middleware.add(middleware_cls)
The only thing left to do is call this middleware in the entrypoint instead of our own wsgi
app:
# api.py
class API:
...

return self.middleware(environ, start_response)
Why do you ask? Because we are delegating the job of being an entrypoint to the
middlewares now. Remember that we implemented WSGI entrypoint interface inside
our Middleware class. Let's go ahead now and create a simple middleware that simply
prints to the console:
# app.py
from api import API
from middleware import Middleware
app = API()
...
class SimpleCustomMiddleware(Middleware):
def process_request(self, req):
print("Processing request", req.url)
def process_response(self, req, res):

print("Processing response", req.url)
app.add_middleware(SimpleCustomMiddleware)
...
Restart your gunicorn and go to any url (e.g. http://localhost:8000/home). Everything

should work just like before. The only exception is that those texts should appear in the
console. Open your console and you should see the following:
Processing request http://localhost:8000/home
Processing response http://localhost:8000/home
There is a catch. Have you found it? Static files don't work now. The reason is that we
stopped using WhiteNoise. We removed it. Instead of calling WhiteNoise, we are calling the
middleware. Here is what we should do. We need to distinguish between requests for
static files and the others. When a request is coming in for a static file, we should
call WhiteNoise. For others, we should call the middleware. The question is how do we
distinguish between them. Right now, requests for static files look like
this: http://localhost:8000/main.css. Other requests look like
this http://localhost:8000/home. They look the same for our APIclass. Thus we will add a
root to the URLs of static files so that they look like
this http://localhost:8000/static/main.css. We will check if the request path starts
with /static. If so, we will call WhiteNoise, otherwise we will call the middleware. We
should also make sure to cut the /static part. Otherwise WhiteNoise won't find the files:
# api.py
class API:
...

path_info = environ["PATH_INFO"]
if path_info.startswith("/static"):
environ["PATH_INFO"] = path_info[len("/static"):]
return self.whitenoise(environ, start_response)
return self.middleware(environ, start_response)
Now, in the templates, we should call static files like so:

<link href="/static/main.css" type="text/css" rel="stylesheet">
Go ahead and change your index.html.
Restart your gunicorn and check that everything is working properly.
We will use this middleware feature in the future posts to add authentication to our apps.
I think that this middleware part is more difficult to understand compared to others. I also
think that I didn't do a great job explaining it. Thus, please write the code, let it sink in
and ask me questions in the comments if something is not clear.
That's it from me today.
Fight on!

Learn to build a Python web framework from scratch

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Learn to build a Python web framework from scratch

Uploaded by

Copyright:

Available Formats

How to write a Python web framework. Part I.

Gunicornis a WSGI HTTP Server, so it expects a specific entrypoint to our application. If

Have you learnt what WSGI is? Good. Let's continue.

To be WSGI-compatible, we need a callable object (a function or a class) that expects

Go into this folder, create a virtual env and activate it:

def app(environ, start_response):

Now, delete everything inside app.py and write the following:

and now we can use them inside the __call__ method:

return response(environ, start_response)

return response(environ, start_response)

def handle_request(self, request):

def home(request, response):

def about(request, response):

def route(self, path):

Here is what we did here. In the __init__ method, we simply defined

def handle_request(self, request):

for path, handler in self.routes.items():

def default_response(self, response):

Now, let's use it in our handle_request method:

for path, handler in self.routes.items():

def find_handler(self, request_path):

def handle_request(self, request):

if handler is not None:

Let's test it out:

def find_handler(self, request_path):

return None, None

def handle_request(self, request):

handler, kwargs = self.find_handler(request_path=request.path)

if handler is not None:

Restart your gunicorn and go to http://localhost:8000/hello/Matthew/. You should the

Make sure to read Part I of these series before this one.

 Check for duplicate routes

Ready? Let's get started.

def route(self, path):

def __call__(self, environ, start_response):

return response(environ, start_response)

def find_handler(self, request_path):

return None, None

def handle_request(self, request):

handler, kwargs = self.find_handler(request_path=request.path)

if handler is not None:

def default_response(self, response):

def route(self, path):

We can refactor it to decrease it to one line:

def route(self, path):

Voilà! Onto the next feature.

Class Based Handlers

def post(self, req, resp):

def handle_request(self, request):

handler, kwargs = self.find_handler(request_path=request.path)

if handler is not None:

def handle_request(self, request):

handler, kwargs = self.find_handler(request_path=request.path)

if handler is not None:

def handle_request(self, request):

handler, kwargs = self.find_handler(request_path=request.path)

if handler is not None:

If the handler_function was actually found, then we simply call it:

Now the whole method looks like this:

def handle_request(self, request):

handler, kwargs = self.find_handler(request_path=request.path)

if handler is not None:

def handle_request(self, request):

and now we can use them inside the call method:

Here is what we did here. In the init method, we simply defined

def call(self, environ, start_response):

def call(self, environ, start_response):

def call(self, environ, start_response):

def call(self, environ, start_response):

def call(self, environ, start_response):