The topic guide on Django’s database-abstraction API\ndescribed the way that you can use Django queries that create,\nretrieve, update and delete individual objects. However, sometimes you will\nneed to retrieve values that are derived by summarizing or aggregating a\ncollection of objects. This topic guide describes the ways that aggregate values\ncan be generated and returned using Django queries.
\nThroughout this guide, we’ll refer to the following models. These models are\nused to track the inventory for a series of online bookstores:
\nfrom django.db import models\n\nclass Author(models.Model):\n name = models.CharField(max_length=100)\n age = models.IntegerField()\n\nclass Publisher(models.Model):\n name = models.CharField(max_length=300)\n\nclass Book(models.Model):\n name = models.CharField(max_length=300)\n pages = models.IntegerField()\n price = models.DecimalField(max_digits=10, decimal_places=2)\n rating = models.FloatField()\n authors = models.ManyToManyField(Author)\n publisher = models.ForeignKey(Publisher, on_delete=models.CASCADE)\n pubdate = models.DateField()\n\nclass Store(models.Model):\n name = models.CharField(max_length=300)\n books = models.ManyToManyField(Book)\nIn a hurry? Here’s how to do common aggregate queries, assuming the models\nabove:
\n# Total number of books.\n>>> Book.objects.count()\n2452\n\n# Total number of books with publisher=BaloneyPress\n>>> Book.objects.filter(publisher__name='BaloneyPress').count()\n73\n\n# Average price across all books.\n>>> from django.db.models import Avg\n>>> Book.objects.all().aggregate(Avg('price'))\n{'price__avg': 34.35}\n\n# Max price across all books.\n>>> from django.db.models import Max\n>>> Book.objects.all().aggregate(Max('price'))\n{'price__max': Decimal('81.20')}\n\n# Difference between the highest priced book and the average price of all books.\n>>> from django.db.models import FloatField\n>>> Book.objects.aggregate(\n... price_diff=Max('price', output_field=FloatField()) - Avg('price'))\n{'price_diff': 46.85}\n\n# All the following queries involve traversing the Book<->Publisher\n# foreign key relationship backwards.\n\n# Each publisher, each with a count of books as a "num_books" attribute.\n>>> from django.db.models import Count\n>>> pubs = Publisher.objects.annotate(num_books=Count('book'))\n>>> pubs\n<QuerySet [<Publisher: BaloneyPress>, <Publisher: SalamiPress>, ...]>\n>>> pubs[0].num_books\n73\n\n# Each publisher, with a separate count of books with a rating above and below 5\n>>> from django.db.models import Q\n>>> above_5 = Count('book', filter=Q(book__rating__gt=5))\n>>> below_5 = Count('book', filter=Q(book__rating__lte=5))\n>>> pubs = Publisher.objects.annotate(below_5=below_5).annotate(above_5=above_5)\n>>> pubs[0].above_5\n23\n>>> pubs[0].below_5\n12\n\n# The top 5 publishers, in order by number of books.\n>>> pubs = Publisher.objects.annotate(num_books=Count('book')).order_by('-num_books')[:5]\n>>> pubs[0].num_books\n1323\nQuerySetLink to this heading#Django provides two ways to generate aggregates. The first way is to generate\nsummary values over an entire QuerySet. For example, say you wanted to\ncalculate the average price of all books available for sale. Django’s query\nsyntax provides a means for describing the set of all books:
>>> Book.objects.all()\nWhat we need is a way to calculate summary values over the objects that\nbelong to this QuerySet. This is done by appending an aggregate()\nclause onto the QuerySet:
>>> from django.db.models import Avg\n>>> Book.objects.all().aggregate(Avg('price'))\n{'price__avg': 34.35}\nThe all() is redundant in this example, so this could be simplified to:
>>> Book.objects.aggregate(Avg('price'))\n{'price__avg': 34.35}\nThe argument to the aggregate() clause describes the aggregate value that\nwe want to compute - in this case, the average of the price field on the\nBook model. A list of the aggregate functions that are available can be\nfound in the QuerySet reference.
aggregate() is a terminal clause for a QuerySet that, when invoked,\nreturns a dictionary of name-value pairs. The name is an identifier for the\naggregate value; the value is the computed aggregate. The name is\nautomatically generated from the name of the field and the aggregate function.\nIf you want to manually specify a name for the aggregate value, you can do so\nby providing that name when you specify the aggregate clause:
>>> Book.objects.aggregate(average_price=Avg('price'))\n{'average_price': 34.35}\nIf you want to generate more than one aggregate, you add another argument to\nthe aggregate() clause. So, if we also wanted to know the maximum and\nminimum price of all books, we would issue the query:
>>> from django.db.models import Avg, Max, Min\n>>> Book.objects.aggregate(Avg('price'), Max('price'), Min('price'))\n{'price__avg': 34.35, 'price__max': Decimal('81.20'), 'price__min': Decimal('12.99')}\nQuerySetLink to this heading#The second way to generate summary values is to generate an independent\nsummary for each object in a QuerySet. For example, if you are\nretrieving a list of books, you may want to know how many authors contributed\nto each book. Each Book has a many-to-many relationship with the Author; we\nwant to summarize this relationship for each book in the QuerySet.
Per-object summaries can be generated using the\nannotate() clause. When an annotate() clause is\nspecified, each object in the QuerySet will be annotated with the\nspecified values.
The syntax for these annotations is identical to that used for the\naggregate() clause. Each argument to annotate() describes\nan aggregate that is to be calculated. For example, to annotate books with the\nnumber of authors:
# Build an annotated queryset\n>>> from django.db.models import Count\n>>> q = Book.objects.annotate(Count('authors'))\n# Interrogate the first object in the queryset\n>>> q[0]\n<Book: The Definitive Guide to Django>\n>>> q[0].authors__count\n2\n# Interrogate the second object in the queryset\n>>> q[1]\n<Book: Practical Django Projects>\n>>> q[1].authors__count\n1\nAs with aggregate(), the name for the annotation is automatically derived\nfrom the name of the aggregate function and the name of the field being\naggregated. You can override this default name by providing an alias when you\nspecify the annotation:
>>> q = Book.objects.annotate(num_authors=Count('authors'))\n>>> q[0].num_authors\n2\n>>> q[1].num_authors\n1\nUnlike aggregate(), annotate() is not a terminal clause. The output\nof the annotate() clause is a QuerySet; this QuerySet can be\nmodified using any other QuerySet operation, including filter(),\norder_by(), or even additional calls to annotate().
Combining multiple aggregations with annotate() will yield the\nwrong results because joins are used instead of subqueries:
>>> book = Book.objects.first()\n>>> book.authors.count()\n2\n>>> book.store_set.count()\n3\n>>> q = Book.objects.annotate(Count('authors'), Count('store'))\n>>> q[0].authors__count\n6\n>>> q[0].store__count\n6\nFor most aggregates, there is no way to avoid this problem, however, the\nCount aggregate has a distinct parameter that\nmay help:
>>> q = Book.objects.annotate(Count('authors', distinct=True), Count('store', distinct=True))\n>>> q[0].authors__count\n2\n>>> q[0].store__count\n3\nSo far, we have dealt with aggregates over fields that belong to the\nmodel being queried. However, sometimes the value you want to aggregate\nwill belong to a model that is related to the model you are querying.
\nWhen specifying the field to be aggregated in an aggregate function, Django\nwill allow you to use the same double underscore notation that is used when referring to related fields in\nfilters. Django will then handle any table joins that are required to retrieve\nand aggregate the related value.
\nFor example, to find the price range of books offered in each store,\nyou could use the annotation:
\n>>> from django.db.models import Max, Min\n>>> Store.objects.annotate(min_price=Min('books__price'), max_price=Max('books__price'))\nThis tells Django to retrieve the Store model, join (through the\nmany-to-many relationship) with the Book model, and aggregate on the\nprice field of the book model to produce a minimum and maximum value.
The same rules apply to the aggregate() clause. If you wanted to\nknow the lowest and highest price of any book that is available for sale\nin any of the stores, you could use the aggregate:
>>> Store.objects.aggregate(min_price=Min('books__price'), max_price=Max('books__price'))\nJoin chains can be as deep as you require. For example, to extract the\nage of the youngest author of any book available for sale, you could\nissue the query:
\n>>> Store.objects.aggregate(youngest_age=Min('books__authors__age'))\nIn a way similar to Lookups that span relationships, aggregations and\nannotations on fields of models or models that are related to the one you are\nquerying can include traversing “reverse” relationships. The lowercase name\nof related models and double-underscores are used here too.
\nFor example, we can ask for all publishers, annotated with their respective\ntotal book stock counters (note how we use 'book' to specify the\nPublisher -> Book reverse foreign key hop):
>>> from django.db.models import Avg, Count, Min, Sum\n>>> Publisher.objects.annotate(Count('book'))\n(Every Publisher in the resulting QuerySet will have an extra attribute\ncalled book__count.)
We can also ask for the oldest book of any of those managed by every publisher:
\n>>> Publisher.objects.aggregate(oldest_pubdate=Min('book__pubdate'))\n(The resulting dictionary will have a key called 'oldest_pubdate'. If no\nsuch alias were specified, it would be the rather long 'book__pubdate__min'.)
This doesn’t apply just to foreign keys. It also works with many-to-many\nrelations. For example, we can ask for every author, annotated with the total\nnumber of pages considering all the books the author has (co-)authored (note how we\nuse 'book' to specify the Author -> Book reverse many-to-many hop):
>>> Author.objects.annotate(total_pages=Sum('book__pages'))\n(Every Author in the resulting QuerySet will have an extra attribute\ncalled total_pages. If no such alias were specified, it would be the rather\nlong book__pages__sum.)
Or ask for the average rating of all the books written by author(s) we have on\nfile:
\n>>> Author.objects.aggregate(average_rating=Avg('book__rating'))\n(The resulting dictionary will have a key called 'average_rating'. If no\nsuch alias were specified, it would be the rather long 'book__rating__avg'.)
QuerySet clausesLink to this heading#filter() and exclude()Link to this heading#Aggregates can also participate in filters. Any filter() (or\nexclude()) applied to normal model fields will have the effect of\nconstraining the objects that are considered for aggregation.
When used with an annotate() clause, a filter has the effect of\nconstraining the objects for which an annotation is calculated. For example,\nyou can generate an annotated list of all books that have a title starting\nwith “Django” using the query:
>>> from django.db.models import Avg, Count\n>>> Book.objects.filter(name__startswith="Django").annotate(num_authors=Count('authors'))\nWhen used with an aggregate() clause, a filter has the effect of\nconstraining the objects over which the aggregate is calculated.\nFor example, you can generate the average price of all books with a\ntitle that starts with “Django” using the query:
>>> Book.objects.filter(name__startswith="Django").aggregate(Avg('price'))\nAnnotated values can also be filtered. The alias for the annotation can be\nused in filter() and exclude() clauses in the same way as any other\nmodel field.
For example, to generate a list of books that have more than one author,\nyou can issue the query:
\n>>> Book.objects.annotate(num_authors=Count('authors')).filter(num_authors__gt=1)\nThis query generates an annotated result set, and then generates a filter\nbased upon that annotation.
\nIf you need two annotations with two separate filters you can use the\nfilter argument with any aggregate. For example, to generate a list of\nauthors with a count of highly rated books:
>>> highly_rated = Count('book', filter=Q(book__rating__gte=7))\n>>> Author.objects.annotate(num_books=Count('book'), highly_rated_books=highly_rated)\nEach Author in the result set will have the num_books and\nhighly_rated_books attributes. See also Conditional aggregation.
annotate() and filter() clausesLink to this heading#When developing a complex query that involves both annotate() and\nfilter() clauses, pay particular attention to the order in which the\nclauses are applied to the QuerySet.
When an annotate() clause is applied to a query, the annotation is computed\nover the state of the query up to the point where the annotation is requested.\nThe practical implication of this is that filter() and annotate() are\nnot commutative operations.
Given:
\nPublisher A has two books with ratings 4 and 5.
Publisher B has two books with ratings 1 and 4.
Publisher C has one book with rating 1.
Here’s an example with the Count aggregate:
>>> a, b = Publisher.objects.annotate(num_books=Count('book', distinct=True)).filter(book__rating__gt=3.0)\n>>> a, a.num_books\n(<Publisher: A>, 2)\n>>> b, b.num_books\n(<Publisher: B>, 2)\n\n>>> a, b = Publisher.objects.filter(book__rating__gt=3.0).annotate(num_books=Count('book'))\n>>> a, a.num_books\n(<Publisher: A>, 2)\n>>> b, b.num_books\n(<Publisher: B>, 1)\nBoth queries return a list of publishers that have at least one book with a\nrating exceeding 3.0, hence publisher C is excluded.
\nIn the first query, the annotation precedes the filter, so the filter has no\neffect on the annotation. distinct=True is required to avoid a query\nbug.
The second query counts the number of books that have a rating exceeding 3.0\nfor each publisher. The filter precedes the annotation, so the filter\nconstrains the objects considered when calculating the annotation.
\nHere’s another example with the Avg aggregate:
>>> a, b = Publisher.objects.annotate(avg_rating=Avg('book__rating')).filter(book__rating__gt=3.0)\n>>> a, a.avg_rating\n(<Publisher: A>, 4.5) # (5+4)/2\n>>> b, b.avg_rating\n(<Publisher: B>, 2.5) # (1+4)/2\n\n>>> a, b = Publisher.objects.filter(book__rating__gt=3.0).annotate(avg_rating=Avg('book__rating'))\n>>> a, a.avg_rating\n(<Publisher: A>, 4.5) # (5+4)/2\n>>> b, b.avg_rating\n(<Publisher: B>, 4.0) # 4/1 (book with rating 1 excluded)\nThe first query asks for the average rating of all a publisher’s books for\npublisher’s that have at least one book with a rating exceeding 3.0. The second\nquery asks for the average of a publisher’s book’s ratings for only those\nratings exceeding 3.0.
\nIt’s difficult to intuit how the ORM will translate complex querysets into SQL\nqueries so when in doubt, inspect the SQL with str(queryset.query) and\nwrite plenty of tests.
order_by()Link to this heading#Annotations can be used as a basis for ordering. When you\ndefine an order_by() clause, the aggregates you provide can reference\nany alias defined as part of an annotate() clause in the query.
For example, to order a QuerySet of books by the number of authors\nthat have contributed to the book, you could use the following query:
>>> Book.objects.annotate(num_authors=Count('authors')).order_by('num_authors')\nvalues()Link to this heading#Ordinarily, annotations are generated on a per-object basis - an annotated\nQuerySet will return one result for each object in the original\nQuerySet. However, when a values() clause is used to constrain the\ncolumns that are returned in the result set, the method for evaluating\nannotations is slightly different. Instead of returning an annotated result\nfor each result in the original QuerySet, the original results are\ngrouped according to the unique combinations of the fields specified in the\nvalues() clause. An annotation is then provided for each unique group;\nthe annotation is computed over all members of the group.
For example, consider an author query that attempts to find out the average\nrating of books written by each author:
\n>>> Author.objects.annotate(average_rating=Avg('book__rating'))\nThis will return one result for each author in the database, annotated with\ntheir average book rating.
\nHowever, the result will be slightly different if you use a values() clause:
>>> Author.objects.values('name').annotate(average_rating=Avg('book__rating'))\nIn this example, the authors will be grouped by name, so you will only get\nan annotated result for each unique author name. This means if you have\ntwo authors with the same name, their results will be merged into a single\nresult in the output of the query; the average will be computed as the\naverage over the books written by both authors.
\nannotate() and values() clausesLink to this heading#As with the filter() clause, the order in which annotate() and\nvalues() clauses are applied to a query is significant. If the\nvalues() clause precedes the annotate(), the annotation will be\ncomputed using the grouping described by the values() clause.
However, if the annotate() clause precedes the values() clause,\nthe annotations will be generated over the entire query set. In this case,\nthe values() clause only constrains the fields that are generated on\noutput.
For example, if we reverse the order of the values() and annotate()\nclause from our previous example:
>>> Author.objects.annotate(average_rating=Avg('book__rating')).values('name', 'average_rating')\nThis will now yield one unique result for each author; however, only\nthe author’s name and the average_rating annotation will be returned\nin the output data.
You should also note that average_rating has been explicitly included\nin the list of values to be returned. This is required because of the\nordering of the values() and annotate() clause.
If the values() clause precedes the annotate() clause, any annotations\nwill be automatically added to the result set. However, if the values()\nclause is applied after the annotate() clause, you need to explicitly\ninclude the aggregate column.
order_by()Link to this heading#Fields that are mentioned in the order_by() part of a queryset are used\nwhen selecting the output data, even if they are not otherwise specified in the\nvalues() call. These extra fields are used to group “like” results together\nand they can make otherwise identical result rows appear to be separate. This\nshows up, particularly, when counting things.
By way of example, suppose you have a model like this:
\nfrom django.db import models\n\nclass Item(models.Model):\n name = models.CharField(max_length=10)\n data = models.IntegerField()\nIf you want to count how many times each distinct data value appears in an\nordered queryset, you might try this:
items = Item.objects.order_by('name')\n# Warning: not quite correct!\nitems.values('data').annotate(Count('id'))\n…which will group the Item objects by their common data values and\nthen count the number of id values in each group. Except that it won’t\nquite work. The ordering by name will also play a part in the grouping, so\nthis query will group by distinct (data, name) pairs, which isn’t what you\nwant. Instead, you should construct this queryset:
items.values('data').annotate(Count('id')).order_by()\n…clearing any ordering in the query. You could also order by, say, data\nwithout any harmful effects, since that is already playing a role in the\nquery.
This behavior is the same as that noted in the queryset documentation for\ndistinct() and the general rule is the\nsame: normally you won’t want extra columns playing a part in the result, so\nclear out the ordering, or at least make sure it’s restricted only to those\nfields you also select in a values() call.
You can also generate an aggregate on the result of an annotation. When you\ndefine an aggregate() clause, the aggregates you provide can reference\nany alias defined as part of an annotate() clause in the query.
For example, if you wanted to calculate the average number of authors per\nbook you first annotate the set of books with the author count, then\naggregate that author count, referencing the annotation field:
\n>>> from django.db.models import Avg, Count\n>>> Book.objects.annotate(num_authors=Count('authors')).aggregate(Avg('num_authors'))\n{'num_authors__avg': 1.66}\n