Tag: Django

This week, I’m back at my tremendously irregular Django tips series, where I share small pieces of code and approaches to common themes that developers face when working on their web applications.

The topic of today’s post is how to implement a counter that isn’t vulnerable to race conditions. Counting is everywhere, when handling money, when controlling access limits, when scoring, etc.

One common rookie mistake is to do it like this:

model = MyModel.objects.get(id=id)
model.count += 1
model.save(update_fields=["count"])

An approach that is subject to race conditions, as described below:

• Process 1 gets count value (let’s say it is currently 5)
• Process 2 gets counts value
• Process 1 increments and saves
• Process 2 increments and saves

Instead of 7, you end up with 6 in your records.

On a low stakes project or in a situation where precision is not that important, this might do the trick and not become a problem. However, if you need accuracy, you will need to do it differently.

Approach 1

with transaction.atomic():
    model = (
        MyModel.objects.select_for_update()
        .get(id=id)
    )
    model.count += 1
    model.save(update_fields=["count"])

In this approach, when you first fetch the record, you ask for the database to lock it. While you are handling it, no one else can access it.

Since it locks the records, it can create a bottleneck. You will have to evaluate if fits your application’s access patterns. As a rule of thumb, it should be used when you require access to the final value.

Approach 2

from django.db.models import F

model = MyModel.objects.filter(id=id).update(
    count=F("count") + 1
)

In this approach, you don’t lock any values or need to explicitly work inside a transaction. Here, you just tell the database, that it should add 1 to the value that is currently there. The database will take care of atomically incrementing the value.

It should be faster, since multiple processes can access and modify the record “at the same time”. Ideally, you would use it when you don’t need to access the final value.

Approach 3

from django.core.cache import cache

cache.incr(f"mymodel_{id}_count", 1)

If your counter has a limited life-time, and you would rather not pay the cost of a database insertion, using your cache backend could provide you with an even faster method.

The downside, is the level of persistence and the fact that your cache backend needs to support atomic increments. As far as I can tell, you are well served with Redis and Memcached.

For today, this is it. Please let me know if I forgot anything.

Django management commands can be very helpful while developing your application or website, we are very used to runserver, makemigrations, migrate, shell and others. Third party packages often provide extra commands and you can easily add new commands to your own apps.

Today lets take a look at some less known and yet very useful commands that Django provides out of the box.

diffsettings

$ python manage.py diffsettings --default path.to.module --output unified

Dealing with multiple environments and debugging their differences is not as rare as we would like. In that particular scenario diffsettings can become quite handy.

Basically, it displays the differences between the current configuration and another settings file. The default settings are used if a module is not provided.

- DEBUG = False
+ DEBUG = True
- EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
+ EMAIL_BACKEND = 'django.core.mail.backends.console.EmailBackend'
- TIME_ZONE = 'America/Chicago'
+ TIME_ZONE = 'UTC'
+ USE_SRI = True
- USE_TZ = False
+ USE_TZ = True
...

sendtestemail

$ python manage.py sendtestemail my@address.com

This one does not require an extensive explanation. It lets you test and debug your email configuration by using it to send the following message:

Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Subject: Test email from host on 2022-04-28 19:08:56.968492+00:00
From: webmaster@localhost
To: my@address.com
Date: Thu, 28 Apr 2022 19:08:56 -0000
Message-ID: <165117293696.405310.3477251481753991809@host>

If you're reading this, it was successful.
-----------------------------------------------------------

inspectdb

$ python manage.py inspectdb

If you are building your project on top of an existing database (managed by other system), inspectdb can look into the schema and generate the respective models for Django’s ORM, making it very easy to start using the data right away. Here’s an example:

# This is an auto-generated Django model module.
# You'll have to do the following manually to clean this up:
#   * Rearrange models' order
#   * Make sure each model has one field with primary_key=True
#   * Make sure each ForeignKey and OneToOneField has `on_delete` set to the desired behavior
#   * Remove `managed = False` lines if you wish to allow Django to create, modify, and delete the table
# Feel free to rename the models, but don't rename db_table values or field names.

...

class AuthPermission(models.Model):
    content_type = models.ForeignKey('DjangoContentType', models.DO_NOTHING)
    codename = models.CharField(max_length=100)
    name = models.CharField(max_length=255)

    class Meta:
        managed = False
        db_table = 'auth_permission'
        unique_together = (('content_type', 'codename'),)
...

showmigrations

$ python manage.py showmigrations --verbosity 2

When you need to inspect the current state of the project’s migrations in a given environment the above command is the easiest way to get that information. It will tell you what migrations exist, which ones were applied and when.

admin
 [X] 0001_initial (applied at 2021-01-13 19:49:24)
 [X] 0002_logentry_remove_auto_add (applied at 2021-01-13 19:49:24)
 [X] 0003_logentry_add_action_flag_choices (applied at 2021-01-13 19:49:24)
auth
 [X] 0001_initial (applied at 2021-01-13 19:49:24)
 [X] 0002_alter_permission_name_max_length (applied at 2021-01-13 19:49:24)
 [X] 0003_alter_user_email_max_length (applied at 2021-01-13 19:49:24)
...

There are many other useful management commands that are missing in the base Django package, to fill that gap there are some external packages available such as django-extensions. But I will leave those to a future post.

While the admin is a well known and very useful app for your projects, Django also includes another admin package that isn’t as popular (at least I never seen it being heavily used) but that can also be quite handy.

I’m talking about the admindocs app. What it does is to provide documentation for the main components of your project in the Django administration itself.

It takes the existing documentation provided in the code to developers and exposes it to users that have the is_staff flag enabled.

This is what they see:

I can see this being very helpful for small websites that are operated by teams of “non-developers” or even for people providing support to customers. At least when a dedicated and more mature solution for documentation is not available.

To install you just have to:

• Install the docutils package.
• Add django.contrib.admindocs to the installed apps.
• Add path('admin/doc/', include('django.contrib.admindocs.urls')) to the URLs.
• Document your code with “docstrings” and “help_text” attributes.

More details documentation can be found here. And for today, this is it.