Mastering Django: Structure

Django’s Structure—A Heretic’s Eye View

The most common complaints from those new to Django is “it’s too hard to understand”, or “it’s too complex”. You may even be thinking this yourself right now.

At the fundamental level, Django isn’t complicated. Yes, it has its quirks and, yes, big Django projects can be very complex beasts, but bottom line: Django is a logically structured framework built on the easiest to learn programming language available (Python).

As an educator, I have spent countless hours trying to work out why people find Django complex and hard to learn. Thinking on this has led me to commit heresy #1: it’s not your fault, we’ve just been teaching it wrong.

Remember all the books and tutorials that start with “Django is a Model-View-Controller (MVC) framework…”? (This is cut and paste from one of my books, so I am as guilty as anyone in this).

Stating up front that Django is an MVC framework gets one of two responses:

1. Beginners say “What the heck is MVC? *groan*. Guess that’s one more fricking thing I have to learn!”
2. More experienced programmers say “Aha! It’s just like Framework X.”

In both cases, they’re almost entirely wrong.

Django is a Loosely Coupled Framework

If you can indulge me a minute, and purge your brain of your favorite Three Letter Acronyms (TLAs), there’s an easier way to understand this.

The first step is to understand that Django is not the result of an academic exercise, nor is it some guru’s idea of cool—Django’s creators designed Django to solve a particular set of problems in a busy and complex news organization. At the center of this set of problems were three very different needs:

1. The data guys (and gals) needed a common interface to work with disparate data sources, formats and database software.
2. The design teams needed to manage the user experience with the tools they already had (HTML, CSS, JavaScript etc.).
3. The hard-core coders required a framework that allowed them to deploy system changes rapidly and keep everyone happy.

Crucial to making this all work was ensuring each of these core components—data, design and business logic—could be managed independently, or to use the correct computer parlance—the framework had to employ loose coupling.

Now, it’s important to understand that I’m not trying to say Django is doing anything magic or new here, nor were the problems Django’s creators faced unique. The creators of Django are brilliant guys, and they certainly knew MVC was a well-established design pattern that would help solve their problems.

My point is it’s highly unlikely any of them ever said, “Hang on boys, we need to change this code because Wikipedia says a controller should …”.

You need not get hung up on semantics—you can safely forget about the confusing TLAs and whether Django is like Framework X and concentrate on what Django is.

Django’s architecture comprises three major parts:

• Part 1 is a set of tools that make working with data and databases much easier.
• Part 2 is a plain-text template system suitable for non-programmers; and
• Part 3 is a framework that handles communication between the user and the database, and automates many of the painful parts of managing a complex website.

Parts 1 and 2 are instantly relatable (see Figure 3.1):

• Django Models are the tools we use to work with data and databases; and
• Django Templates provide a designer-friendly plain-text template system.

But what about Part 3? I hear you ask, isn’t that the controller, or a Django view?

Well, no. Which leads me to heresy #2:

A Django View is Not a Controller

Check out Figure 3.1, does it look familiar?

Figure 3.1: The somewhat misleading Django MTV diagram.

This is one of my diagrams, but there are plenty of similar versions out there. A common way of explaining Django’s architecture in terms of MVC is to describe it as a Model-Template-View (MTV) or Model-View-Template (MVT). There’s no difference between MTV and MVT—they’re two different ways to describe the same thing, which adds to the confusion.

The misleading part of this diagram is the view. The view in Django is most often described as being equivalent to the controller in MVC, but it’s not—it’s still the view.

Figure 3.2 is a variation on Figure 3.1 to illustrate my point.

Figure 3.2: A slightly different view of Django’s MTV “stack”.

Note how I have drawn a line between the client- and server-side. Like all client/server architectures, Django uses request and response objects to communicate between the client and the server. As Django is a web framework, we’re talking about HTTP request and response objects.

So, in this simplified process, the view retrieves data from the database via the model, formats it, bundles it up in an HTTP response object and sends it to the client (browser).

In other words, the view presents the model to the client as an HTTP response. This is also the exact definition of the view in MVC, or to quote Wikipedia (not the most definitive source, I know, but close enough):

“The view means presentation of the model in a particular format”

Trying to bend the definition of a Django view to fit a particular viewpoint inevitably leads to one of two things:

1. Confused programmer puts everything in views module; or
2. Confused programmer says “Django is too hard!”, and watches TV instead

So getting away from our M’s and T’s and V’s and C’s, Figure 3.3 presents a more holistic view of Django’s architecture.

Figure 3.3: A more holistic view of Django’s architecture.

The first point of confusion we can clear up is where to put a particular function or class:

Does the function/class return a response?

• YES—it’s a view. Put it in the views module (views.py).
• NO—it’s not a view, it’s app logic. Put it somewhere else (somewhere_else.py).

We’ll discuss the somewhere else part in the next section of this chapter.

The next point to note is that the Django framework encapsulates the model, view logic and business logic. In some tutorials, it’s said that the Django framework is the controller, but that isn’t true either—the Django framework can do much more than respond to user input and interact with data.

A perfect example of this extra power is Django’s middleware, which sits between the view and the client-side. Django’s middleware performs critical security and authentication checks before sending the response to the browser.

So, returning to the two confused responses from the beginning of the chapter:

1. Beginners—no, you don’t have to learn about MVC because it’s more than likely going to confuse you and lead to more questions than answers
2. Programmers—no, Django is not like Framework X, and trying to think it is, is likely to confuse you and lead to more questions than answers

Now we’ve got that out of the way, let’s have a look at the structure of a Django project.

Django Project Structure

Django doesn’t require you to build web applications in any particular way. In fact, billions of electrons have been sacrificed discussing the One Best Way to structure a Django project. We’re all pragmatic programmers here, so we won’t play that game.

Django does, however, have a default way of doing things, and there is a definite underlying logic to it you need to understand to become a professional Django programmer.

The fundamental unit of a Django web application is a Django project. A Django project comprises one or more Django apps (Figure 3.4)

Figure 3.4: Django’s project structure.

A Django app is a self-contained package that should only do one thing. For example, a blog, a membership app or an event calendar. Notice at the bottom of Figure 3.4, there’s an extra package called Django Apps.

This is another case where Django’s logic carries right through the framework—Django itself is a collection of apps, each designed to do one thing. With Django’s built-in apps, they’re all designed to make your life easier, which is a Good Thing.

While the built-in apps are invisible in your project tree, you can see them in your settings.py file:

# ...\myclub_project\myclub_site\myclub_site\settings.py

# partial listing

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
]

You can see that Django has added several apps to your project automatically. There are also many other built-in Django apps you can add to the INSTALLED_APPS list. When you add your apps to a Django project, you also add a link to the app configuration class to this list.

You can see this logic in what Django has created for you so far. Open up your \myclub_project folder. The folder structure should look something like this:

# ...\my_clubproject

\env_myclub
\myclub_site        <= This is your Django project
    \myclub_site    <= This is a Django app
    db.sqlite3      <= Your project database
    manage.py       <= Django project management utility

Let’s examine these files and folders in more detail:

• The env_myclub folder is where Django stores your virtual environment files. Generally, you should leave everything inside this folder alone.
• The outer myclub_site folder is your Django project. Django created this folder and its contents when you ran the startproject command in the last chapter. Django doesn’t care about the folder name, so you can rename it to something meaningful to you.
• Inside the outer myclub_site folder are two files:
  • db.sqlite3. The database created when you ran the migrate command; and
  • manage.py. A command-line utility for executing Django commands from within your project.
• The inner myclub_site folder is your Django website application. This is the one application that Django creates automatically for you. Because Django is a web framework, it assumes you want a website app.

This should make more sense by now, but I bet there is one thing that’s still a bit confusing—the two myclub_site folders.

A very common complaint from programmers new to Django is how confusing it is to know which folder they should work in when there are two folders named the same. Django is not alone with this convention—Integrated Development Environments (IDEs) like Visual Studio create a project folder and application folder with the same name. But just because it’s common, that doesn’t mean it isn’t confusing.

As I said a moment ago, Django doesn’t care what you name this folder—so let’s commit heresy #3, breaking thirteen years of Django tutorial convention while we are at it, and rename the folder!

Here, we’re renaming it to “myclub_root”.

Once you have made the change, your folder structure should go from this:

\myclub_project
    \myclub_site
        \myclub_site

To this:

\myclub_project
    \myclub_root
        \myclub_site

Now we’ve taken care of that source of confusion, let’s have a look inside the myclub_site website app Django created for us:

# \myclub_project\myclub_root\

\myclub_site
    __init__.py
    asgi.py # Django 3 only
    settings.py
    urls.py
    wsgi.py

Looking closer at these files:

• The __init__.py file tells Python that this folder (your Django app) is a Python package.
• asgi.py enables ASGI compatible web servers to serve your project.
• settings.py contains the settings for your Django project. Every Django project must have a settings file. By convention, Django puts it in your website app, but it doesn’t have to live there. There are proponents for other structures as I mentioned earlier, but here we’re using the default.
• urls.py contains project-level URL configurations. By default, this contains a single URL pattern for the admin. We will cover more on URLs later in the chapter, and in great detail in Chapter 5.
• wsgi.py enables WSGI compatible web servers to serve your project.

Now that we’ve had a good look at the basic structure of a Django project, it’s time to take the next step and add our own Django app.

Creating Your Own Django Apps

You might have noticed that there is no real program code in your project so far. There is a settings file with configuration information, an almost empty URLs file, and a command-line utility that launches a website that doesn’t do much.

This is because to create a functioning Django website, you need to create Django applications. A Django application (or app for short) is where the work gets done. Apps are one of Django’s killer features. Not only do they allow you to add functionality to a Django project without interfering with other parts of the project, but apps are designed to be portable so you can use one app in multiple projects.

So, let’s create our first custom Django app. Our social club website needs an events calendar to show upcoming events for the club, so we’re creating a Django app called events.

Fire up your Python virtual environment, switch into the \myclub_root folder and run the command:

python manage.py startapp events

This is what your command shell output should look like:

(env_myclub) ...> cd myclub_root
(env_myclub) ...\myclub_root> python manage.py startapp events
(env_myclub) ...\myclub_root>

Once you have created your app, you must tell Django to install it into your project. This is easy to do—inside your settings.py file is a list named INSTALLED_APPS. This list contains all the apps installed in your Django project. Django comes with a few apps pre-installed, we just have to add your new events app to the list (change in bold):

1 INSTALLED_APPS = [
2     'events.apps.EventsConfig',
3     'django.contrib.admin',
4     # more apps
5 ]

Inside every app, Django creates a file, apps.py, containing a configuration class named after your app. Here, the class is named EventsConfig. To register our app with Django, we need to point to the EventsConfig class—which is what we are doing in line 2 of our modified INSTALLED_APPS list.

If you were wondering, EventsConfig contains two configuration options by default—the name of the app (“events”), and the default autofield data type.

Now let’s look inside the \myclub_root folder to see what Django has created for us:

\events
    \migrations
    __init__.py
    admin.py
    apps.py
    models.py
    tests.py
    views.py

• The migrations folder is where Django stores migrations, or changes to your database. There’s nothing in here you need to worry about right now.
• __init__.py tells Python that your events app is a package.
• admin.py is where you register your app’s models with the Django admin application.
• apps.py is a configuration file common to all Django apps.
• models.py is the module containing the models for your app.
• tests.py contains test procedures that run when testing your app.
• views.py is the module containing the views for your app.

Now we have a complete picture of a Django project, we can also answer the question from earlier in the chapter: “well, if it’s not a view, where does it go?”

When you have code that isn’t a view, you create a new Python module (.py file) inside your app and put related functions and classes inside the file. Note the emphasis on related. If you have a bunch of functions that provide database management utilities, for example, put them all in one file. Functions and classes not related to database management should go in another file. You should also try to be descriptive in naming modules—after all, it’s more sensible to put your database functions in a file called db_utils.py than a file called monkeys.py…

When creating new modules for your Django project, you should also consider scope. While adding custom modules to apps is far more common (and more portable), you can have project-level modules (e.g., Django’s manage.py) and site-level modules. In the latter case, your custom modules should go in the same folder as your settings.py file.

The last couple of points might seem blindingly obvious, but it’s important to understand that, while Django has a default logic to its structure, nothing is cast in stone. Django is flexible and allows you to expand and change your project structure to suit the logic of your web application.

Now we have a thorough understanding of the structure of Django’s projects and apps, the next obvious question, given we are building web applications is “how do we navigate a Django project?”

To answer this question, we need to check out the final piece of the Django big picture puzzle—URL configurations.

URLconfs—Django’s Navigator

There’s one last piece to the Django framework puzzle—the critical communication pathway that matches a request on the client-side with a project resource (the arrows between the view and the template in Figure 3.3). Like all web applications, Django uses Uniform Resource Locators (URLs) to match content with a request.

Django’s urls package provides dozens of functions and classes for working with different URL formats, name resolution, exception handling and other navigational utilities. However, at its most basic, it allows you to map a URL to a function or class within your Django project.

A Django URL configuration (or URLconf for short) matches a unique URL with a project resource. You can think of it being like matching a person’s name with their address. Except in Django, we’re not matching a street address—we’re matching a Python path using Python’s dot notation.

Assuming you’re not familiar with dot notation, it’s a common idiom in object-oriented programming. I like to think of the dot like a point because the dot points to something. With Python, the dot operator points to the next object in the object chain.

In Django classes, the object chain is like this:

package.module.class.method

Or with functions:

package.module.function.attribute

Some real-life examples:

• forms.Form points to the Form class in the forms package.
• events.apps.EventsConfig points to the EventsConfig class in the apps sub-package of the events package (i.e., the apps.py file in your events app).
• django.conf.urls points to the urls package inside the conf package inside Django, which is also a Python package!

This can sometimes get a bit confusing, but if you remember to join the dots (sorry, a bad pun there), you can usually find out what the dot operator is referring to.

With a URLconf, the path points to a function or class inside a module (.py file). Let’s look at our Django project diagram again (Figure 3.5).

Figure 3.5: Finding functions and classes with Django’s URLconfs.

To create a URLconf, we use the path() function. The first part of the function is the URL, so in Figure 3.5 the URL is app1/. The path() function then maps this URL to app1.views.some_view().

Assuming your site address is http://www.mycoolsite.com, in plain English we’re saying:

“When someone navigates to http://www.mycoolsite.com/app1/, run the some_view() function inside app1’s views.py file”.

Note a URL doesn’t have to map to a view—it can map to any module in your Django app. For example, you may have a set of wireless environmental sensors that post data back to the server. You could have a custom module called sensors.py that has a function or class to record the sensor data to your database, all without ever touching a view.

And that’s all there is to it. Of course, URLconfs can do a lot more than map a static URL to a function or class, but if you can understand the basics—that Django’s incredibly fast and powerful navigation system is based on the simple concept of matching a URL with a resource—then you have all you need to tie all your Django apps together into a navigable web project.

A Final Note on Writing Django Apps

A common and inevitable question arises once you get your head around Django’s basic structure:

“Where do I start? Should I start with writing my models, the URL configurations, my views? Or what?”

Well, here’s your final heresy for the chapter: it doesn’t matter.

Some people like to start by building all the models so they can see how the data structure looks; others prefer to build the visual layout first, so they start with templates. Others might like to get the basic communication framework in place, so they start with views and URLconfs. Others will start at whatever point seems logical for the project.

Being pragmatic to the bone, I am usually in the last group. I try not to get fixated on what someone else thinks is the right or the wrong way to do things and try to find the simplest and quickest way to achieve the result I want. I also like to work incrementally starting small getting the flow right and building on it to create the complete application. This approach means I inevitably end up jumping from one element to another as the application grows.

Your brain is wired differently to mine and every other programmer. This is a Good Thing. Just remember, an imperfect start to a project is way better than not starting at all. Do what works for you.

Chapter Summary

In this chapter, I gave you a high-level overview of how Django projects are structured, how each component works with other parts of Django to create a web application, and how to create a Django app.

In the next chapter, we will start diving into the inner working of Django’s core modules by exploring the fundamentals of Django’s models.

Up Next…