To power effective content moderation and data annotation programs, our users often need a large amount of contextual data. More data enables higher accuracy and label fidelity that can be used to confidently train AI models and take real-time enforcement. The challenge is that all that contextual data often requires heavy queries that can bog down the system and lead to slow user experiences.

In the spirit of open source and sharing best practices with the industry, we’re excited to publicly release django-zeal, a Python package used extensively at Cinder to supercharge database queries. django-zeal, inspired by the popular nplusone Python package, quickly locates and alerts our team to costly N+1 queries that could slow our product and our users down.

What are N+1 queries?

N+1 queries are a common pitfall of Django. They occur when the app makes one query to fetch items (the "1" query) and then makes additional queries (the "n" queries) to fetch related data for each item. Often, this happens in the context of querying related tables.

Here’s a concrete potential example: Cinder has a module that shows dozens or even hundreds of pieces of potentially-violating content for reviewers to quickly scan and, where relevant, apply appropriate policy labels. We don’t only show the content, though: we also show related information, such as information about the content author and even things like comments on the content. These pieces of information - author and comments - are in related models. If our app queries for the content and then makes additional queries for each content’s author and comments, we have an N+1: many queries that could have been grouped into a single database call. By fetching all relevant data in one query, we ensure Cinder continues running fast and users are able to do their work with incredible speed and efficiency.

How django-zeal helps

django-zeal detects a wide range of N+1 pitfalls and alerts developers with useful error messages. It goes further than other, similar Python packages by detecting even more types of N+1 queries, with more actionable error messages and advanced configurability. It has extensive test coverage, no dependencies, and has been put to the test on Cinder’s real codebase.

If your app runs on Django, we highly recommend giving django-zeal a try. And be sure to let us know what you think.

Are you an engineer interested in building software use by the most cutting edge trust and safety teams? Apply to our open roles here: https://jobs.ashbyhq.com/cinder

‍