Running tasks concurrently in Django asynchronous views

Async support has really been improving and expanding in Django! Since Django 3.0 with the addition of ASGI support (Asynchronous Server Gateway Interface) there has been a steady march of improvements that bring Django closer to having a full asynchronous request-response cycle.

Now we’re to the point where there’s enough foundational support that interesting things are possible for our “normal web apps.”. This is where it gets really exciting for people! Here we’ll take a look at how we can start using async views with Django.

Reviewing async code can be challenging. So, together, we’ll walk through creating our first asynchronous view in Django.

Buckle-up for our async-journey together!

Brief history of async support

The brief timeline for adding async support to Django shows it’s been careful, steady, and intentional.

• Django 3.0 → ASGI support
• Django 3.1 → Asynchronous views and middlewares
• Django 4.0 → Asynchronous API for cache backends
• Django 4.1 → Asynchronous ORM interface
• Django 4.2 →Asynchronous streaming responses, asynchronous Model and related manager interfaces, and psycopg version 3 support which provides asynchronous connections and cursors
• Django 5.0 (currently under development 🏗️) → Asynchronous signal dispatch, coroutine request.auser(), and more (?)

Seeing the trend and what’s under development is exciting!

Problem

With all the excitement around asynchronous web development, you’d like to get some of those benefits in your own Django app.

These are the questions we’re setting out to answer:

• How do we write an asynchronous view in Django?
• What can be done asynchronously in Django?
• How to use Django with ASGI (Asynchronous Server Gateway Interface) servers?

Solution

We need an existing or new Django project. Here are some great resources for getting started with Django or deploying your Django app to Fly.io.

With a project ready, let’s get started!

How to write an asynchronous view?

First, an asynchronous view is a coroutine function defined with the async def syntax that accepts a request and returns a response. We can use it directly in a URL configuration as any other standard view.

This is what it looks like:

# urls.py
from django.http import HttpResponse
from django.urls import path


async def my_view(request):
    ...
    return HttpResponse(...)


urlpatterns = [
    ...
    path('my_view/', my_view),
]

Django automatically detects async views and runs them in an async context, so we don’t have to do anything else to make them work! These are also supported under ASGI and WSGI mode. However, Django emulates ASGI style when running async views under WSGI, and this kind of context-switching causes a performance penalty. That’s why it’s more efficient to run async views under ASGI.

We have many functionalities that can be asynchronous or at least async-compatible, and as such can be used in async views. If we want to use part of our code or part of Django that is still synchronous, we can wrap it with asgiref.sync.sync_to_async() from asgiref package (asgiref is mandatory for any Django installation):

# views.py
from asgiref.sync import sync_to_async


@sync_to_async
def my_sync_function():
    ...


async def my_view(request):
    ...
    result = await my_sync_function()
    ...
    return HttpResponse(...)

Notice how our synchronous function my_sync_function() is decorated with @sync_to_async. This creates a bridge between the sync and async contexts, allowing us to use synchronous functions in async views.

Example

Let’s write an asynchronous view together with some of available async options. For this example, we have a Django project for gathering information about open source contributors. The first step for a new user is providing an email address. Our view should do a few things:

• check if an email is not already registered,
• download an avatar,
• return a web response.

There’s no reason we cannot perform the first two tasks separately. They don’t rely on each other. We can’t finish our render until both are done, so this is a great opportunity to let those tasks run concurrently and speed up our overall response!

Our view will cover asynchronous examples for:

• using the ORM, and
• calling an external API.

Both are I/O bound tasks and the real work is being done outside, so they are great tasks for being asynchronous.

Using the ORM

First, we define a coroutine to check if an email is not already registered. QuerySet has an asynchronous interface for all data access operations. Methods are named as synchronous operations but with an a prefix (this is a general rule for creating async variants in Django). We’ll use QuerySet.aexists(), that is an async version of .exists(), and filter against an email address:

# helpers.py
from django.contrib.auth import get_user_model


async def is_email_registered(email):
    return await get_user_model().objects.filter(email=email).aexists()

Unfortunately, the underlying database operation is synchronous because it uses the sync_to_async() wrapper and a synchronous connection (as asynchronous database drivers are not yet integrated, or even exist for most databases).

For Django 4.2+, when using newly introduced psycopg version 3 support and a PostgreSQL database you can make it fully asynchronous! It takes some effort, as you have to initialize a new connection and perform a raw SQL statement, but it’s possible to do this fully asynchronously. This is how we have to do it until async connections are not supported in the Django ORM:

# helpers.py
import psycopg

from django.contrib.auth import get_user_model
from django.db import connection

async def is_email_registered(email):
    # Find and quote a database table name for a Model with users.
    user_db_table = connection.ops.quote_name(get_user_model()._meta.db_table)
    # Create a new async connection.
    aconnection = await psycopg.AsyncConnection.connect(
        **{
            **connection.get_connection_params(),
            "cursor_factory": psycopg.AsyncCursor,
        },
    )
    async with aconnection:
        # Create a new async cursor and execute a query.
        async with aconnection.cursor() as cursor:
            await cursor.execute(
                f'SELECT TRUE FROM {user_db_table} WHERE "email" = %s',
                [email],
            )
            row = await cursor.fetchone()
            return row[0] if row else False

We have a database operation that can happen asynchronously! Now, let’s turn our focus to the next asynchronous task we want perform, which is the API call.

Calling external API

Let’s start by defining an async function to fetch an avatar from an external URL. The httpx package is a great solution for this as it defines AsyncClient that provides async methods for all types of requests:

# helpers.py
import base64
import hashlib

import httpx

async def get_gravatar(email):
    # URL with the avatar for the given email address
    # see https://gravatar.com/site/implement/images/.
    gravatar_hash = hashlib.md5(
        email.lower().strip().encode(),
        usedforsecurity=False,
    ).hexdigest()
    gravatar_url = f"https://www.gravatar.com/avatar/{gravatar_hash}.png"

    # Make HTTP GET request asynchronously.
    async with httpx.AsyncClient() as client:
        response = await client.get(gravatar_url)

    if response.status_code == httpx.codes.OK:
        # Return an avatar encoded with base64.
        return base64.b64encode(response.content).decode()
    return None

Awesome! With our external API task written, we’re ready to try running both of them concurrently.

Running coroutines concurrently

Now, we have all steps covered by coroutine functions and we can gather them together in an asynchronous view new_contributor():

# forms.py
from django import forms

class NewContributorForm(forms.Form):
    email = forms.EmailField(required=True, label="Email address:")

# views.py
from django.shortcuts import render

from .forms import NewContributorForm
from .helpers import get_gravatar, is_email_registered


async def new_contributor(request):
    if request.method == "POST":
        form = NewContributorForm(request.POST)
        if form.is_valid():
            email = form.cleaned_data["email"]
            # Check if the given email address is already registered.
            is_registered = await is_email_registered(email)
            # Fetch an avatar.
            avatar = await get_gravatar(email)
            # Render the second registration step.
            return render(
                request,
                "registration/new_contributor_step_2.html",
                {
                    "is_registered": is_registered,
                    "email": email,
                    "avatar": avatar,
                },
            )
    else:
        # Return an empty form.
        form = NewContributorForm()
    return render(request, "registration/new_contributor.html", {"form": form})

The above view has an important disadvantage, it does not run coroutines concurrently. To do this we can use:

• asyncio.gather() which runs awaitables concurrently and returns an aggregated list of values when all have succeeded:

is_registered, avatar = await asyncio.gather(
    is_email_registered(email),
    get_gravatar(email),
)

• or we can do this in a more modern way with asyncio.TaskGroup() (available in Python 3.11+) where all tasks are awaited when the context manager exits:

async with asyncio.TaskGroup() as task_group:
    task1 = task_group.create_task(is_email_registered(email))
    task2 = task_group.create_task(get_gravatar(email))
is_registered = task1.result()
avatar = task2.result()

Our final asynchronous view 🎉:

# views.py
import asyncio

from django.shortcuts import render

from .forms import NewContributorForm
from .helpers import get_gravatar, is_email_registered

async def new_contributor(request):
    if request.method == "POST":
        form = NewContributorForm(request.POST)
        if form.is_valid():
            email = form.cleaned_data["email"]
            async with asyncio.TaskGroup() as task_group:
                task1 = task_group.create_task(is_email_registered(email))
                task2 = task_group.create_task(get_gravatar(email))
            return render(
                request,
                "registration/new_contributor_step_2.html",
                {
                    "is_registered": task1.result(),
                    "email": email,
                    "avatar": task2.result(),
                },
            )
    else:
        form = NewContributorForm()
    return render(request, "registration/new_contributor.html", {"form": form})

Let’s take a look how it works:

We did it! When our users enter their email address, we perform two separate and concurrent tasks. One task uses the email to fetch an avatar from an external service through an API call. The other task does a database search.

When both tasks are done, we finish rendering the page.

Let’s check the options for deploying our project with ASGI.

ASGI servers options

Django supports deploying with ASGI by creating an entry-point (<your_project>/asgi.py file) with an application callable for ASGI web servers. The official Django documentation contains details how to deploy with the following servers:

• Daphne
• Hypercorn
• Uvicorn

Personally, I prefer daphne as it provides the built-in runserver command that allows to run your project under ASGI during development.

Closing thoughts

There are of course many other use cases for asynchronous stuff in Django 🔥, here we touched only the tip of the iceberg! Performing standalone I/O bound tasks are great for being asynchronous, we can highlight here:

• using a database,
• making an API call,
• reading a file (see aiofiles), or
• executing external shell commands, etc.

The first two of which are discussed in this article.

Async support is getting broader with each new version of Django, but it’s pretty mature already 🧑. You no longer need to leave your favorite framework if you have (or simply want) to use asynchronous code.

Async-world is dynamically developed both in Python and Django, so it’s time give it a try.