Django and webpack now work together seamlessly

Note: this post has all the details you might want or need. For a quick start, check out this post here.

If you’re a full stack developer who works with Django, then it’s likely you’ve encountered the problem of combining it with a webpack managed frontend. In my recent quest to make the two play nice together I encountered this blog post by developer Valentino Gagliardi. Valentino argues that there are no good way to manage assets produced by webpack with Django, and goes on to explain that while django-webpack-loader used to be a good option, it’s sadly now outdated and in need of replacement.

One of the primary problems Valentino lays out is how as soon as you start getting into split chunks with webpack, all good options fail. So, in honor of Hacktoberfest this year I decided to build and release a solution to these problems.

Please note that this solution is specifically for having Django serve your front end assets. If you want your front end to be on a separate server this is not for you.

Introducing Django Manifest Loader

Webpack powered cache busting and splitChunks are now ready to work with Django.

There are two ways to use Django Manifest Loader:

• As a replacement for Django’s {% static %} template tag, to generate the url for an asset with a hashed name. E.g. {% manifest 'main.js' %} turns into /static/main.26f45137e39ed4cb3a9f.js
• Or as a tool to output the tags and urls needed to integrate with webpack’s split chunks feature.

E.g. {% manifest_match '*.js' '<script src="{match}"/>' %} turns into:

<script src="/static/vendors~main.91de137e39ed4cb3b2a.js"/>
<script src="/static/main.26f45137e39ed4cb3a9f.js"/>

I’m going to talk about these features in two parts. For while they are powered by roughly the same code, the way they are used, and who will be using them may be quite different. But before we get into that let’s talk installation and setup (it’s pretty straight forward, I promise).

Installation and Setup

First install the package, ideally into your virtual environment.

pip install django-manifest-loader

Now we need to make some small modifications to your Django settings.py file.

# settings.py
INSTALLED_APPS = [
    ...
    'manifest_loader',  # add the app
    ...
]

Now add the STATICFILES_DIRS var if you don’t have it already. In it you need to point to the folder webpack is outputting to. The below example is for if output is going to a directory dist/ that is directly inside your BASE_DIR

# settings.py
STATICFILES_DIRS = [BASE_DIR / 'dist']

👏 That’s 👏 it! 👏 (for the python config)

Webpack Setup

If you don’t have webpack installed yet, do so with

npm i --save-dev webpack webpack-cli

Django Manifest Loader requires the WebpackManifestPlugin for webpack, but the CleanWebpackPlugin is also highly recommended. Install those with:

npm i --save-dev webpack-manifest-plugin clean-webpack-plugin

Now let’s take a look at the webpack configuration itself. There are two essential parts here. The first is that we are using the WebpackManifestPlugin, and the second is that we’re outputting the files into a directory that Django is looking for with the STATICFILES_DIRS setting we setup earlier. As long as these two things are in place, the manifest loader will work.

// webpack.config.js example
const path = require('path');
const {CleanWebpackPlugin} = require('clean-webpack-plugin');
const {WebpackManifestPlugin} = require('webpack-manifest-plugin');

module.exports = {
  entry: './src/index.js',
  plugins: [
      new CleanWebpackPlugin(),  
      new WebpackManifestPlugin(), 
  ],
  output: {
    publicPath: '',
    filename: '[name].[contenthash].js', 
    path: path.resolve(__dirname, 'dist'),
  },
};

Please note that as of webpack-manifest-plugin v3.0.0, it is no longer a default export. As of webpack v5 you need to have output.publicPath set to an empty string.

If you feel like you want some details on what the above is doing, read this next section. Otherwise, skip ahead to the usage sections.

About the webpack Configuration

• Imports: path is a package that comes with node that we will use to determine the correct location to output the processed files. CleanWebpackPlugin and WebpackManifestPlugin are what we installed in a previous step.
• module.exports is exporting the configuration object so webpack can consume it.
• entry: './src/index.js', is telling webpack to find a file called index.js that is in a directory src. It will expect to find src on the same level as the webpack.config.js file. index.js is the entry point for the javascript. Webpack will process this file along with any files used by index.js
• plugins: [...] is instantiating the plugins we installed and imported. CleanWebpackPlugin deletes all webpack output that is no longer needed every time you run webpack. This prevents unused and unneeded files from collecting. WebpackManifestPlugin creates a manifest.json file in the output directory. django-manifest-loader needs this to know what files it’s working with, so don’t leave this out!
• output: {} everything in here is telling webpack how to output the files
• output.filename: '[name].[contenthash].js', is telling webpack to rename all output files based on the name of the file AND the content hash of the file. The content hash is super important here, it’s what allows cache busting and is what makes django-manifest-loader worth using. If you’re not cache busting and you’re not using split chunks, you probably don’t need django-manifest-loader.
• output.path is telling webpack to output to the directory dist/ that is on the same level as the webpack.config.js file.

If you have any other questions about this setup, please feel free to leave a comment and I’ll do my best to help you out.

Basic Single File Usage

Django Manifest Loader is all about template tags. So I made a template tag for your templates. Check it out:

<!-- index.html -->
{% load manifest %}
<script src="{% manifest 'index.js' %}"/>

That’s it. The above code will now render as:

<!-- index.html -->
<script src="/static/index.26f45136e49ed5cb5a9f.js"/>

And every time you make an update to your index.js file, the content hash will change here as well. This is called cache busting and ensures that the browser always has the latest version of this file.

How this works

Django Manifest Loader finds the manifest.json file that is generated every time you compile assets. It then takes the input, index.js in this example, and looks it up against the manifest. The manifest will point to the file’s new name, such as index.26f45136e49ed5cb5a9f.js, and then the manifest loader swaps out one string for the other. It then passes along this string with the hashed version of the file name directly to Django’s built-in {% static %} template tag.

I think Django’s existing static template tag works great, and that there’s no way I can build something better. So the best option here seems to be to leverage that.

So it might be even more accurate to say that this:

{% manifest 'index.js' %}
Turns into this:
{% static 'index.26f45136e49ed5cb5a9f.js' %}
Which finally turns into
/static/index.26f45136e49ed5cb5a9f.js

Using for Split Chunks and Pattern Matching

Since you’re reading about split chunks I’m going to assume you know what they are, why you want them, and how to use them. I found this article about them very good if you want to learn or need a refresher.

So you took the step. You split your chunks and your front end app is in tiny pieces with all their tiny hashed names. Good on you. And now, good thing there’s a way to get those chunks dynamically into your templates ✨.

The {% manifest_match %} tag leverages basic pattern matching, courtesy of the fnmatch Python package to let you pull in multiple files listed in your manifest in a single line. The tag takes two required arguments, the first is the pattern you want to match against, such as '*.js' to get all JavaScript files, and the second argument is what you want to inject the url it generates for that file into, such as '<script src="{match}"/>'. Note that Django Manifest Loader is looking for the exact string {match} in order to know where to inject the url. If {match} isn’t present it will throw an exception.

So this:

<!-- index.html -->
{% load manifest %}
{% manifest_match '*.js' '<script src="{match}"/>' %}

will load up the manifest, find all the files in it that match *.js according to the rules of fnmatch, get the urls for those files, and output them embedded in the string of your choice. The above example will then render as

<!-- index.html -->
<script src="/static/index.f3bd219.js"/>
<script src="/static/foo.2d0af93.js"/>
<script src="/static/bar.21fab11.js"/>
<!-- etc, etc -->

How this works

Just like the {% manifest %} tag, the {% manifest_match %} tag is leveraging Django’s built-in {% static %} tag. The static tag has no issue resolving urls, and so instead of rolling a custom solution, Easy-Django-webpack just leverages that capability.

A visualization might help. It turns this:

{% manifest_match '*.js' '<script src="{match}"/>' %} 
into this
<script src="{% manifest 'index.js' %}"/>
<script src="{% manifest 'foo.js' %}"/>
<script src="{% manifest 'bar.js' %}"/>
which turns into this
<script src="{% static 'index.f3bd219.js' %}"/>
<script src="{% static 'foo.2d0af93.js' %}"/>
<script src="{% static 'bar.21fab11.js' %}"/>
which finally turns into 
<script src="/static/index.f3bd219.js"/>
<script src="/static/foo.2d0af93.js"/>
<script src="/static/bar.21fab11.js"/>

And just like that, with a single line in your template, you have imported all your split chunks into your html.

More Info

Resources

• Django Manifest Loader on Github
• The 100% correct way to split your chunks with webpack (maybe)
• What is cache busting?

Optional Settings

The following options are available to you to customize the behavior of Django Manifest Loader. Below are the default values if you don’t override them.

# settings.py
MANIFEST_LOADER = {
    'output_dir': None,
    'manifest_file': 'manifest.json',
    'cache': False
}

output_dir: if set, it will not comb the directories defined in STATICFILES_DIRS for your manifest.json file, and will just look in the location you defined. Not super necessary unless you have a lot of directories to look in.

manifest_file: if you choose to configure your manifest file to have a name other than the default (manifest.json)as defined by WebpackManifestPlugin then you can set that here.

cache: if set to True it will use the default cache defined in your Django settings to store a copy of your manifest file. This will prevent it from both looking for the manifest file and opening it from the filesystem. Not recommended for development, as it will prevent the latest assets from loading until cleared.

How to run the tests locally and check code coverage

It’s never a bad idea to download a project you’re considering using and running their tests yourself. You can also choose to run a test coverage report against this project to see its 100% test coverage.

git clone https://github.com/shonin/django-manifest-loader.git
cd django-manifest-loader
pip install -e .

# run tests
python runtests.py

# check code coverage
pip install coverage
coverage run --source=manifest_loader/ runtests.py
coverage report