Learning Django: the hard way (1)

Learning Django: the hard way (1)

What does "runserver" do?

Django provides a light-weight web server for development and test use. You can just start it using the command something like this-

python manage.py runserver 0.0.0.0:8000

manage.py just calls -

django.core.management.execute_from_command_line(sys.argv)

OK, so we need to dig into it a little to bit to see where the runserver comes into play. Finally, it turns out the parameter "runserver" would create a new instance of Command from the module django.core.management.commands.runserver.py.

There is one method to dynamically load the command class based on the command parameter, like runserver. The code is like this -

def load_command_class(app_name, name):
	"""
	Given a command name and an application name, returns the Command
	class instance. All errors raised by the import process
	(ImportError, AttributeError) are allowed to propagate.
	"""
 	module = import_module('%s.management.commands.%s' % (app_name, name))
	return module.Command()

OK, so far the corresponding command object of runserver has been created. Next, the method handle of the command object is called which in turns would call the method inner_run -

def inner_run(self, *args, **options):
    from django.conf import settings
    from django.utils import translation
    threading = options.get('use_threading')
    shutdown_message = options.get('shutdown_message', '')
    quit_command = 'CTRL-BREAK' if sys.platform == 'win32' else 'CONTROL-C'
    self.stdout.write("Performing system checks...\n\n")
    self.validate(display_num_errors=True)
    try:
        self.check_migrations()
    except ImproperlyConfigured:
        pass
    now = datetime.now().strftime('%B %d, %Y - %X')
    if six.PY2:
        now = now.decode(get_system_encoding())
    self.stdout.write((
        "%(started_at)s\n"
        "Django version %(version)s, using settings %(settings)r\n"
        "Starting development server at http://%(addr)s:%(port)s/\n"
        "Quit the server with %(quit_command)s.\n"
    ) % {
        "started_at": now,
        "version": self.get_version(),
        "settings": settings.SETTINGS_MODULE,
        "addr": '[%s]' % self.addr if self._raw_ipv6 else self.addr,
        "port": self.port,
        "quit_command": quit_command,
    })
    # django.core.management.base forces the locale to en-us. We should
    # set it up correctly for the first request (particularly important
    # in the "--noreload" case).
    translation.activate(settings.LANGUAGE_CODE)
    try:
        handler = self.get_handler(*args, **options)
        run(self.addr, int(self.port), handler,
            ipv6=self.use_ipv6, threading=threading)
    except socket.error as e:
        # Use helpful error messages instead of ugly tracebacks.
        ERRORS = {
            errno.EACCES: "You don't have permission to access that port.",
            errno.EADDRINUSE: "That port is already in use.",
            errno.EADDRNOTAVAIL: "That IP address can't be assigned-to.",
        }
        try:
            error_text = ERRORS[e.errno]
        except KeyError:
            error_text = str(e)
        self.stderr.write("Error: %s" % error_text)
        # Need to use an OS exit because sys.exit doesn't work in a thread
        os._exit(1)
    except KeyboardInterrupt:
        if shutdown_message:
            self.stdout.write(shutdown_message)
        sys.exit(0)

The meat and potatoes of this code is the following two lines -

handler = self.get_handler(*args, **options)
run(self.addr, int(self.port), handler, ipv6=self.use_ipv6, threading=threading)

Before looking into the code, it's better to have a review about PEP333 (WSGI), or just take a look at here

What are the two roles played by Django?

Django, itself, is a web framwork which complies with WSGI. From the framework side, it should provides one application handler(callable) wich takes in two parameters: environ and start_response. At the same time, as Django also provides one web server, it needs to provide a mechnisam to invoke the application handler(callable) for each request it receives from an Http client.

OK, after understanding these two roles can we move on to look at the code mentioned above.

How the application and web server interact?

First, let's focus on the application handler.

handler = self.get_handler(*args, **options)

The handler is the one either we configured in Django settings.py or by calling "get_wsgi_application()", seen from the method get_internal_wsgi_application() which is called by get_handler()

Note Django project will create an application by default, and it's also created by calling

"get_wsgi_application()"

In "settings.py" -

WSGI_APPLICATION = '.wsgi.application'

In "wsgi.py":

from django.core.wsgi import get_wsgi_application

application = get_wsgi_application()

def get_internal_wsgi_application():
    """
    Loads and returns the WSGI application as configured by the user in
    ``settings.WSGI_APPLICATION``. With the default ``startproject`` layout,
    this will be the ``application`` object in ``projectname/wsgi.py``.
    This function, and the ``WSGI_APPLICATION`` setting itself, are only useful
    for Django's internal servers (runserver, runfcgi); external WSGI servers
    should just be configured to point to the correct application object
    directly.
    If settings.WSGI_APPLICATION is not set (is ``None``), we just return
    whatever ``django.core.wsgi.get_wsgi_application`` returns.
    """
    from django.conf import settings
    app_path = getattr(settings, 'WSGI_APPLICATION')
    if app_path is None:
        return get_wsgi_application()
    try:
        return import_string(app_path)
    except ImportError as e:
        msg = (
            "WSGI application '%(app_path)s' could not be loaded; "
            "Error importing module: '%(exception)s'" % ({
                'app_path': app_path,
                'exception': e,
            })
        )
        six.reraise(ImproperlyConfigured, ImproperlyConfigured(msg),
                    sys.exc_info()[2])

Let's continue looking at function get_wsgi_application() -

def get_wsgi_application():
    """
    The public interface to Django's WSGI support. Should return a WSGI
    callable.
    Allows us to avoid making django.core.handlers.WSGIHandler public API, in
    case the internal WSGI implementation changes or moves in the future.
    """
    django.setup()
    return WSGIHandler()

Please note the specification of WSGIHandler does comply with PEP333: take in "environ" and "start_response".

class WSGIHandler(base.BaseHandler):
    initLock = Lock()
    request_class = WSGIRequest
    def __call__(self, environ, start_response):
        # Set up middleware if needed. We couldn't do this earlier, because
        # settings weren't available.
        if self._request_middleware is None:
            with self.initLock:
                try:
                    # Check that middleware is still uninitialized.
                    if self._request_middleware is None:
                        self.load_middleware()
                except:
                    # Unload whatever middleware we got
                    self._request_middleware = None
                    raise
        set_script_prefix(get_script_name(environ))
        signals.request_started.send(sender=self.__class__)
        try:
            request = self.request_class(environ)
        except UnicodeDecodeError:
            logger.warning('Bad Request (UnicodeDecodeError)',
                exc_info=sys.exc_info(),
                extra={
                    'status_code': 400,
                }
            )
            response = http.HttpResponseBadRequest()
        else:
            response = self.get_response(request)
        response._handler_class = self.__class__
        status = '%s %s' % (response.status_code, response.reason_phrase)
        response_headers = [(str(k), str(v)) for k, v in response.items()]
        for c in response.cookies.values():
            response_headers.append((str('Set-Cookie'), str(c.output(header=''))))
        start_response(force_str(status), response_headers)
        return response

So far, we have figure out the application handler. Now let's turn to web server and see the function "run" in the module "django.core.servers.basehttp.py".

def run(addr, port, wsgi_handler, ipv6=False, threading=False):
    server_address = (addr, port)
    if threading:
        httpd_cls = type(str('WSGIServer'), (socketserver.ThreadingMixIn, WSGIServer), {})
    else:
        httpd_cls = WSGIServer
    httpd = httpd_cls(server_address, WSGIRequestHandler, ipv6=ipv6)
    httpd.set_app(wsgi_handler)
    httpd.serve_forever()

Here it will create a WSGIServer object, please note that the server will have a "WSGIRequestHandler" and also take into account wsgi appliation (wsgi_handler) just created.

Note: Django WSGIServer inherited from python provided "WSGIServer" - WSGIServer from the module

"simple_server.py" which can be found in the folder "/Lib/wsgiref/"

"WSGIRequestHandler" also inherited from python provided "WSGIRequestHandler", just like "WSGIServer"

WSGIRequestHandler has a method "run", which will do the real stuff. Please note this function specification also complies with PEP333, and it will call the WSGI application handler passing in the environ and start_response function.

def run(self, application):
    """Invoke the application"""
    # Note to self: don't move the close()!  Asynchronous servers shouldn't
    # call close() from finish_response(), so if you close() anywhere but
    # the double-error branch here, you'll break asynchronous servers by
    # prematurely closing.  Async servers must return from 'run()' without
    # closing if there might still be output to iterate over.
    try:
        self.setup_environ()
        self.result = application(self.environ, self.start_response)
        self.finish_response()
    except:
        try:
            self.handle_error()
        except:
            # If we get an error handling an error, just give up already!
            self.close()
            raise   # ...and let the actual server figure it out.