django源码分析——静态文件staticfiles中间件

本文环境python3.5.2，django1.10.x系列

1.在上一篇文章中已经分析过handler的处理过程，其中load_middleware就是将配置的中间件进行初始化，然后调用相应的设置方法。
django框架提供的认证，回话保持，静态文件调试处理等都是通过以中间件的形式来处理。
2.本节就分析一下django框架提供的staticfiles中间件，该中间件分别实现了三个框架的命令，分别为collectstatic,findstatic,runserver。
其中，runserver方法是使用框架的开发者在本地调试使用的方法，使用该方式替换django.core中的runserver是为了使开发时，Django框架能够在本地调试时处理静态文件，这样更有利于提升本地开发的效率。
3.下面就一起分析一下该runserver的执行过程。

分析

1.该代码位于django/contrib/staticfiles/目录下，首先来看management/commands/runserver.py

class Command(RunserverCommand):                                                                # 继承自核心包的runserver
    help = "Starts a lightweight Web server for development and also serves static files."

    def add_arguments(self, parser):
        super(Command, self).add_arguments(parser)
        parser.add_argument(
            '--nostatic', action="store_false", dest='use_static_handler', default=True,
            help='Tells Django to NOT automatically serve static files at STATIC_URL.',
        )                                                                                       # 新增是否使用默认静态文件处理handler
        parser.add_argument(
            '--insecure', action="store_true", dest='insecure_serving', default=False,
            help='Allows serving static files even if DEBUG is False.',
        )                                                                                       # 是否使用server处理静态文件，这样调试的环境可以访问静态文件

    def get_handler(self, *args, **options):
        """
        Returns the static files serving handler wrapping the default handler,
        if static files should be served. Otherwise just returns the default
        handler.
        """
        handler = super(Command, self).get_handler(*args, **options)                            # 获取core中的runserver处理对象
        use_static_handler = options['use_static_handler']                                      # 是否使用静态handler处理，默认使用
        insecure_serving = options['insecure_serving']                                          # 是否使用静态handler处理静态文件，默认不使用
        if use_static_handler and (settings.DEBUG or insecure_serving):                         # 如果使用静态handler，并且在调试或者设置使用静态handler处理则使用静态handler
            return StaticFilesHandler(handler)
        return handler

首先该runserver是为了实现在处理接口的同时，处理静态文件，所以Command继承了core核心中的RunserverCommand类，这样只需要在已有的基础上，改写是该类处理静态文件即可。 
该类又增加了两个参数，nostatic表示不自动处理静态文件，insecure表示就算不是调试模式Django框架也要处理静态文件。 
当调用get_handler的时候，先判断是否配置了自动处理静态文件，或者是否开启了insecure模式，如果自动处理静态文件，并且调试为true或者开启了自动处理静态文件，就StaticFilesHandler(handler)处理返回。 
我们分析一下StaticFilesHandler(handler) 
该类位于staticfiles/handlers.py中

from django.conf import settings
from django.contrib.staticfiles import utils
from django.contrib.staticfiles.views import serve
from django.core.handlers.wsgi import WSGIHandler, get_path_info
from django.utils.six.moves.urllib.parse import urlparse
from django.utils.six.moves.urllib.request import url2pathname

class StaticFilesHandler(WSGIHandler):                                          # 继承自wsgi
    """
    WSGI middleware that intercepts calls to the static files directory, as
    defined by the STATIC_URL setting, and serves those files.
    """
    # May be used to differentiate between handler types (e.g. in a
    # request_finished signal)
    handles_files = True

    def __init__(self, application):
        self.application = application                                          # 传入处理handler
        self.base_url = urlparse(self.get_base_url())                           # 解析配置的静态文件路径
        super(StaticFilesHandler, self).__init__()

    def get_base_url(self):
        utils.check_settings()                                                  # 检查静态文件相关配置是否正确
        return settings.STATIC_URL                                              # 返回配置中的静态文件

    def _should_handle(self, path):
        """
        Checks if the path should be handled. Ignores the path if:

        * the host is provided as part of the base_url
        * the request's path isn't under the media path (or equal)
        """
        return path.startswith(self.base_url[2]) and not self.base_url[1]       # 路径是否以静态路径开头，并且配置文件没有给出静态文件的Host

    def file_path(self, url):
        """
        Returns the relative path to the media file on disk for the given URL.
        """
        relative_url = url[len(self.base_url[2]):]                              # 获取文件的相对路径
        return url2pathname(relative_url)

    def serve(self, request):
        """
        Actually serves the request path.
        """
        return serve(request, self.file_path(request.path), insecure=True)      # 启动server处理静态文件

    def get_response(self, request):
        from django.http import Http404

        if self._should_handle(request.path):                                       # 如果是静态文件路径则使用server处理
            try:
                return self.serve(request)
            except Http404 as e:
                if settings.DEBUG:
                    from django.views import debug
                    return debug.technical_404_response(request, e)
        return super(StaticFilesHandler, self).get_response(request)

    def __call__(self, environ, start_response):
        if not self._should_handle(get_path_info(environ)):                         # 先判断请求url是否是静态文件路径
            return self.application(environ, start_response)                        # 如果不是静态文件路径，则正常处理
        return super(StaticFilesHandler, self).__call__(environ, start_response)    # 如果是静态文件路径则调用父方法处理

该类继承自WSGIHandler,此时当调用该handler的call方法时，会调用该类的__call__，会先获取environ中的请求路径，判断该url是否是配置文件中静态文件路径开头。
如果是静态文件路径开头则使用传入的handler直接处理不执行一下步骤，如果是静态文件路径则调用该了的父类的处理方法。
只不过处理过程调用该类的get_response方法，该方法的主要作用是：

    def get_response(self, request):
        from django.http import Http404

        if self._should_handle(request.path):                                       # 如果是静态文件路径则使用server处理
            try:
                return self.serve(request)
            except Http404 as e:
                if settings.DEBUG:
                    from django.views import debug
                    return debug.technical_404_response(request, e)
        return super(StaticFilesHandler, self).get_response(request)

如果给url是静态文件路径则调用self.server方法处理，否则调用父类正常的get_response方法。 
当调用self.server方法时，就使用了导入的django.contrib.staticfiles.views中的server方法处理。 
分析该server的内容如下：

def serve(request, path, insecure=False, **kwargs):
    """
    Serve static files below a given point in the directory structure or
    from locations inferred from the staticfiles finders.

    To use, put a URL pattern such as::

        from django.contrib.staticfiles import views

        url(r'^(?P<path>.*)$', views.serve)

    in your URLconf.

    It uses the django.views.static.serve() view to serve the found files.
    """
    if not settings.DEBUG and not insecure:                                     # 再次检查配置是否为调试模式，是否设置框架处理静态文件
        raise Http404
    normalized_path = posixpath.normpath(unquote(path)).lstrip('/')             # 解析url并分解出路径,并去除最左边的/
    absolute_path = finders.find(normalized_path)                               # 查找静态文件，如果查找到文件就返回文件的绝对地址
    if not absolute_path:
        if path.endswith('/') or path == '':
            raise Http404("Directory indexes are not allowed here.")
        raise Http404("'%s' could not be found" % path)
    document_root, path = os.path.split(absolute_path)                          # 返回匹配上的文件夹，与文件
    return static.serve(request, path, document_root=document_root, **kwargs)   # 处理该静态文件的response

再次检查是否是处理静态文件，如果不是则直接报错404，否则调用finders去查找该静态文件，我们继续查看finders.find方法。
当找到该静态文件时候，调用static.server处理该静态文件。

def find(path, all=False):
    """
    Find a static file with the given path using all enabled finders.

    If ``all`` is ``False`` (default), return the first matching
    absolute path (or ``None`` if no match). Otherwise return a list.
    """
    searched_locations[:] = []
    matches = []
    for finder in get_finders():                                        # 获取配置的finder类
        result = finder.find(path, all=all)                             # 通过finder来查找静态文件
        if not all and result:                                          # 如果不是全部查找，找到对应文件就返回数据
            return result
        if not isinstance(result, (list, tuple)):                       # 如果是全部查找，而result不是列表或元组，则手动转换
            result = [result]
        matches.extend(result)                                          # 将查找到的结果，加入到列表中
    if matches:                                                         # 如果有结果就返回
        return matches
    # No match.
    return [] if all else None                                          # 如果全局查找就返回空列表，否则返回None

get_finders()该函数返回finder对象，我们查看该方法

def get_finders():
    for finder_path in settings.STATICFILES_FINDERS:                    # 获取配置文件中的查找文件对象，默认配置在conf/global_settings.py中
        yield get_finder(finder_path)                                   # 获取finder对象   django.contrib.staticfiles.finders.FileSystemFinder，django.contrib.staticfiles.finders.AppDirectoriesFinder

@lru_cache.lru_cache(maxsize=None)
def get_finder(import_path):
    """
    Imports the staticfiles finder class described by import_path, where
    import_path is the full Python path to the class.
    """
    Finder = import_string(import_path)                                         # 通过配置的路径，导入finder
    if not issubclass(Finder, BaseFinder):                                      # 检查导入Finder是否是BaseFinder子类
        raise ImproperlyConfigured('Finder "%s" is not a subclass of "%s"' %
                                   (Finder, BaseFinder))
    return Finder()                                                             # 返回finder实例

如果配置文件中没有配置，则使用conf/global_settings.py中的配置文件，配置的两个类就位于该FileSystemFinder和AppDirectoriesFinder两个类。
FileSystemFinder主要是查找文件系统的静态文件，AppDirectoriesFinder主要是查找位于app应用中的静态文件。
其中FileSystemFinder分析如下：

class FileSystemFinder(BaseFinder):
    """
    A static files finder that uses the ``STATICFILES_DIRS`` setting
    to locate files.
    """
    def __init__(self, app_names=None, *args, **kwargs):
        # List of locations with static files
        self.locations = []
        # Maps dir paths to an appropriate storage instance
        self.storages = OrderedDict()
        if not isinstance(settings.STATICFILES_DIRS, (list, tuple)):            # 检查配置文件中的静态文件处理是否是列表或者元组
            raise ImproperlyConfigured(
                "Your STATICFILES_DIRS setting is not a tuple or list; "
                "perhaps you forgot a trailing comma?")
        for root in settings.STATICFILES_DIRS:                                  # 获取配置的文件路径
            if isinstance(root, (list, tuple)):                                 # 如果配置的静态路径是列表或者元组
                prefix, root = root                                             # 获取前缀与路径
            else:
                prefix = ''                                                     # 如果不是列表或元组则为''
            if settings.STATIC_ROOT and os.path.abspath(settings.STATIC_ROOT) == os.path.abspath(root):   # 判断静态文件static是否与media重合
                raise ImproperlyConfigured(
                    "The STATICFILES_DIRS setting should "
                    "not contain the STATIC_ROOT setting")
            if (prefix, root) not in self.locations:                            # 如果解析出来的前缀与路径不再loactions中则添加进去
                self.locations.append((prefix, root))
        for prefix, root in self.locations:                                     # 遍历locations
            filesystem_storage = FileSystemStorage(location=root)               # 给每个值生成一个FileSystemStorage实例
            filesystem_storage.prefix = prefix
            self.storages[root] = filesystem_storage                            # 将生成实例保存进字典中
        super(FileSystemFinder, self).__init__(*args, **kwargs)

    def find(self, path, all=False):
        """
        Looks for files in the extra locations
        as defined in ``STATICFILES_DIRS``.
        """
        matches = []
        for prefix, root in self.locations:                                     # 根据locations的值匹配
            if root not in searched_locations:                                  # 如果root不再全局搜索路径中，则添加到搜索路径中
                searched_locations.append(root)
            matched_path = self.find_location(root, path, prefix)               # 查找文件
            if matched_path:                                                    # 如果找到
                if not all:                                                     # 如果不是查找全部则找到第一个就返回
                    return matched_path
                matches.append(matched_path)                                    # 如果查找全部则添加到返回数组中
        return matches

    def find_location(self, root, path, prefix=None):
        """
        Finds a requested static file in a location, returning the found
        absolute path (or ``None`` if no match).
        """
        if prefix:                                                              # 是否有前缀
            prefix = '%s%s' % (prefix, os.sep)                                  # 添加前缀加系统的分隔符， '/'
            if not path.startswith(prefix):                                     # 如果路径不是前缀开头则直接返回
                return None
            path = path[len(prefix):]                                           # 获取除去前缀的路径
        path = safe_join(root, path)                                            # 获取最终的文件路径
        if os.path.exists(path):
            return path

    def list(self, ignore_patterns):
        """
        List all files in all locations.
        """
        for prefix, root in self.locations:                                     # 获取所有文件的设置的静态文件处理
            storage = self.storages[root]
            for path in utils.get_files(storage, ignore_patterns):
                yield path, storage

主要是查找配置的静态文件查找目录，匹配当前是否找到文件。
AppDirectoriesFinder主要是查找配置在app中的静态文件。

class AppDirectoriesFinder(BaseFinder):
    """
    A static files finder that looks in the directory of each app as
    specified in the source_dir attribute.
    """
    storage_class = FileSystemStorage
    source_dir = 'static'                                                       # 源文件夹

    def __init__(self, app_names=None, *args, **kwargs):
        # The list of apps that are handled
        self.apps = []                                                          # 需要查找文件的应用
        # Mapping of app names to storage instances
        self.storages = OrderedDict()                                           # 存储需要查找的应用
        app_configs = apps.get_app_configs()                                    # 获取所有app的配置
        if app_names:                                                           # 如果有传入值，
            app_names = set(app_names)
            app_configs = [ac for ac in app_configs if ac.name in app_names]    # 将app_configs设置为在默认配置中的项目
        for app_config in app_configs:                                          # 遍历筛选出来的应用配置
            app_storage = self.storage_class(
                os.path.join(app_config.path, self.source_dir))                 # 将应用下面的static目录初始化一个app_storage对象
            if os.path.isdir(app_storage.location):                             # 检查生成的静态文件夹是否存在
                self.storages[app_config.name] = app_storage                    # 根据配置应用的名称对应，app_storage对象
                if app_config.name not in self.apps:                            # 如果app没在app列表中，则将该应用的名称添加到列表
                    self.apps.append(app_config.name)
        super(AppDirectoriesFinder, self).__init__(*args, **kwargs)

    def list(self, ignore_patterns):
        """
        List all files in all app storages.
        """
        for storage in six.itervalues(self.storages):                           # 迭代列表中的应用下的app_storage实例
            if storage.exists(''):  # check if storage location exists          # 检查app_storage实例是否存在当前目录
                for path in utils.get_files(storage, ignore_patterns):          # 获取返回的路径
                    yield path, storage                                         # 返回当前路径，与app_storage实例

    def find(self, path, all=False):
        """
        Looks for files in the app directories.
        """
        matches = []
        for app in self.apps:                                                   # 查找app中的文件
            app_location = self.storages[app].location                          # 获取app的绝对路径
            if app_location not in searched_locations:                          # 如果当前路径不在搜索路径中则添加到全局搜索列表中
                searched_locations.append(app_location)
            match = self.find_in_app(app, path)                                 # 在app中的路径中查找
            if match:                                                           # 如果匹配
                if not all:                                                     # 如果不是全局搜索，则立马返回第一个匹配的
                    return match
                matches.append(match)                                           # 如果是全局搜索则添加到返回列表中
        return matches                                                          # 返回所有匹配的数据

    def find_in_app(self, app, path):
        """
        Find a requested static file in an app's static locations.
        """
        storage = self.storages.get(app)                                        # 获取app_storage实例
        if storage:
            # only try to find a file if the source dir actually exists
            if storage.exists(path):                                            # 检查当前文件是否存在
                matched_path = storage.path(path)                               # 返回匹配后的文件路径
                if matched_path:
                    return matched_path

当调用find时会调用find_in_app方法，该方法中的每个实例都是FileSystemStorage，storage.path(path)调用该方法

    def path(self, name):
        return safe_join(self.location, name)

查找当前文件夹路径，当找到时就返回。
当通过这两种方式找到文件时，返回文件时，

    document_root, path = os.path.split(absolute_path)                          # 返回匹配上的文件夹，与文件
    return static.serve(request, path, document_root=document_root, **kwargs)   # 处理该静态文件的response

执行到该方法，django.views.static.server的代码为

def serve(request, path, document_root=None, show_indexes=False):
    """
    Serve static files below a given point in the directory structure.

    To use, put a URL pattern such as::

        from django.views.static import serve

        url(r'^(?P<path>.*)$', serve, {'document_root': '/path/to/my/files/'})

    in your URLconf. You must provide the ``document_root`` param. You may
    also set ``show_indexes`` to ``True`` if you'd like to serve a basic index
    of the directory.  This index view will use the template hardcoded below,
    but if you'd like to override it, you can create a template called
    ``static/directory_index.html``.
    """
    path = posixpath.normpath(unquote(path)).lstrip('/')
    fullpath = safe_join(document_root, path)                                           # 获取文件的全路径
    if os.path.isdir(fullpath):                                                         # 判断是否是文件夹
        if show_indexes:                                                                # 如果显示文件的树结构则显示
            return directory_index(path, fullpath)
        raise Http404(_("Directory indexes are not allowed here."))
    if not os.path.exists(fullpath):                                                    # 如果不存在则报错
        raise Http404(_('"%(path)s" does not exist') % {'path': fullpath})
    # Respect the If-Modified-Since header.
    statobj = os.stat(fullpath)                                                         # 获取文件的状态
    if not was_modified_since(request.META.get('HTTP_IF_MODIFIED_SINCE'),               # 判断该文件是否已经客户端缓存过期，如果还在缓存期就直接返回
                              statobj.st_mtime, statobj.st_size):
        return HttpResponseNotModified()
    content_type, encoding = mimetypes.guess_type(fullpath)                             # 获取文件的文件类型，获取文件的编码格式
    content_type = content_type or 'application/octet-stream'                           # 设置返回文件的文件类型
    response = FileResponse(open(fullpath, 'rb'), content_type=content_type)            # 将文件读入到缓存流中，并返回response
    response["Last-Modified"] = http_date(statobj.st_mtime)                             # 添加最后的文件modified的时间
    if stat.S_ISREG(statobj.st_mode):                                                   # 是否是一般文件
        response["Content-Length"] = statobj.st_size                                    # 设置返回文件的长度
    if encoding:
        response["Content-Encoding"] = encoding                                         # 如果返回有文件的编码格式就设置文件的编码格式
    return response  

当找到文件后，获取到文件路径后，FileResponse来进行文件返回
FileResponse的代码为

class FileResponse(StreamingHttpResponse):
    """
    A streaming HTTP response class optimized for files.
    """
    block_size = 4096

    def _set_streaming_content(self, value):                                        # 重写父类的设置stream方法
        if hasattr(value, 'read'):                                                  # 如果有read方法
            self.file_to_stream = value                                             # 将file_to_stream设值
            filelike = value                                                        #
            if hasattr(filelike, 'close'):                                          # 如果有close方法，添加到完成时关闭
                self._closable_objects.append(filelike)
            value = iter(lambda: filelike.read(self.block_size), b'')               # 迭代读文件的block_size大小的文件，直到读为空为止
        else:
            self.file_to_stream = None
        super(FileResponse, self)._set_streaming_content(value)                     # 调用父类方法处理value

 我们查看StreamingHttpResponse

class StreamingHttpResponse(HttpResponseBase):
    """
    A streaming HTTP response class with an iterator as content.

    This should only be iterated once, when the response is streamed to the
    client. However, it can be appended to or replaced with a new iterator
    that wraps the original content (or yields entirely new content).
    """

    streaming = True

    def __init__(self, streaming_content=(), *args, **kwargs):
        super(StreamingHttpResponse, self).__init__(*args, **kwargs)
        # `streaming_content` should be an iterable of bytestrings.
        # See the `streaming_content` property methods.
        self.streaming_content = streaming_content                                  # 设置stream调用streaming_content.setter方法

    @property
    def content(self):
        raise AttributeError(
            "This %s instance has no `content` attribute. Use "
            "`streaming_content` instead." % self.__class__.__name__
        )

    @property
    def streaming_content(self):
        return map(self.make_bytes, self._iterator)

    @streaming_content.setter
    def streaming_content(self, value):
        self._set_streaming_content(value)                                          # 调用_set_streaming_content

    def _set_streaming_content(self, value):
        # Ensure we can never iterate on "value" more than once.
        self._iterator = iter(value)                                                # 设置可迭代对象
        if hasattr(value, 'close'):                                                 # 如果对象有close方法则在迭代结束后关闭
            self._closable_objects.append(value)

    def __iter__(self):
        return self.streaming_content                                               # 迭代streaming_content

    def getvalue(self):
        return b''.join(self.streaming_content)

通过将response生成一个可迭代对象，将返回的数据进行分块发送，文件块大小为4096，此时就将文件内容分块发送出去，此时一个静态文件的响应过程完成。