Django attempts to support as many features as possible on all database\nbackends. However, not all database backends are alike, and we've had to make\ndesign decisions on which features to support and which assumptions we can make\nsafely.
\nThis file describes some of the features that might be relevant to Django\nusage. Of course, it is not intended as a replacement for server-specific\ndocumentation or reference manuals.
\nPersistent connections avoid the overhead of re-establishing a connection to\nthe database in each request. They're controlled by the\nCONN_MAX_AGE parameter which defines the maximum lifetime of a\nconnection. It can be set independently for each database.
The default value is 0, preserving the historical behavior of closing the\ndatabase connection at the end of each request. To enable persistent\nconnections, set CONN_MAX_AGE to a positive number of seconds. For\nunlimited persistent connections, set it to None.
Django opens a connection to the database when it first makes a database\nquery. It keeps this connection open and reuses it in subsequent requests.\nDjango closes the connection once it exceeds the maximum age defined by\nCONN_MAX_AGE or when it isn't usable any longer.
In detail, Django automatically opens a connection to the database whenever it\nneeds one and doesn't have one already — either because this is the first\nconnection, or because the previous connection was closed.
\nAt the beginning of each request, Django closes the connection if it has\nreached its maximum age. If your database terminates idle connections after\nsome time, you should set CONN_MAX_AGE to a lower value, so that\nDjango doesn't attempt to use a connection that has been terminated by the\ndatabase server. (This problem may only affect very low traffic sites.)
At the end of each request, Django closes the connection if it has reached its\nmaximum age or if it is in an unrecoverable error state. If any database\nerrors have occurred while processing the requests, Django checks whether the\nconnection still works, and closes it if it doesn't. Thus, database errors\naffect at most one request; if the connection becomes unusable, the next\nrequest gets a fresh connection.
\nSince each thread maintains its own connection, your database must support at\nleast as many simultaneous connections as you have worker threads.
\nSometimes a database won't be accessed by the majority of your views, for\nexample because it's the database of an external system, or thanks to caching.\nIn such cases, you should set CONN_MAX_AGE to a low value or even\n0, because it doesn't make sense to maintain a connection that's unlikely\nto be reused. This will help keep the number of simultaneous connections to\nthis database small.
The development server creates a new thread for each request it handles,\nnegating the effect of persistent connections. Don't enable them during\ndevelopment.
\nWhen Django establishes a connection to the database, it sets up appropriate\nparameters, depending on the backend being used. If you enable persistent\nconnections, this setup is no longer repeated every request. If you modify\nparameters such as the connection's isolation level or time zone, you should\neither restore Django's defaults at the end of each request, force an\nappropriate value at the beginning of each request, or disable persistent\nconnections.
\nDjango assumes that all databases use UTF-8 encoding. Using other encodings may\nresult in unexpected behavior such as "value too long" errors from your\ndatabase for data that is valid in Django. See the database specific notes\nbelow for information on how to set up your database correctly.
\nDjango supports PostgreSQL 9.4 and higher. psycopg2 2.5.4 through 2.7.7 is\nrequired, though the 2.7.7 is recommended.
\nSee HOST for details.
Django needs the following parameters for its database connections:
\nclient_encoding: 'UTF8',
default_transaction_isolation: 'read committed' by default,\nor the value set in the connection options (see below),
timezone: 'UTC' when USE_TZ is True, value of\nTIME_ZONE otherwise.
If these parameters already have the correct values, Django won't set them for\nevery new connection, which improves performance slightly. You can configure\nthem directly in postgresql.conf or more conveniently per database\nuser with ALTER ROLE.
Django will work just fine without this optimization, but each new connection\nwill do some additional queries to set these parameters.
\nLike PostgreSQL itself, Django defaults to the READ COMMITTED isolation\nlevel. If you need a higher isolation level such as REPEATABLE READ or\nSERIALIZABLE, set it in the OPTIONS part of your database\nconfiguration in DATABASES:
import psycopg2.extensions\n\nDATABASES = {\n # ...\n 'OPTIONS': {\n 'isolation_level': psycopg2.extensions.ISOLATION_LEVEL_SERIALIZABLE,\n },\n}\nvarchar and text columnsLink to this heading#When specifying db_index=True on your model fields, Django typically\noutputs a single CREATE INDEX statement. However, if the database type\nfor the field is either varchar or text (e.g., used by CharField,\nFileField, and TextField), then Django will create\nan additional index that uses an appropriate PostgreSQL operator class\nfor the column. The extra index is necessary to correctly perform\nlookups that use the LIKE operator in their SQL, as is done with the\ncontains and startswith lookup types.
If you need to add a PostgreSQL extension (like hstore, postgis, etc.)\nusing a migration, use the\nCreateExtension operation.
When using QuerySet.iterator(), Django opens a server-side\ncursor. By default, PostgreSQL assumes that\nonly the first 10% of the results of cursor queries will be fetched. The query\nplanner spends less time planning the query and starts returning results\nfaster, but this could diminish performance if more than 10% of the results are\nretrieved. PostgreSQL's assumptions on the number of rows retrieved for a\ncursor query is controlled with the cursor_tuple_fraction option.
Using a connection pooler in transaction pooling mode (e.g. pgBouncer)\nrequires disabling server-side cursors for that connection.
\nServer-side cursors are local to a connection and remain open at the end of a\ntransaction when AUTOCOMMIT is True. A\nsubsequent transaction may attempt to fetch more results from a server-side\ncursor. In transaction pooling mode, there's no guarantee that subsequent\ntransactions will use the same connection. If a different connection is used,\nan error is raised when the transaction references the server-side cursor,\nbecause server-side cursors are only accessible in the connection in which they\nwere created.
One solution is to disable server-side cursors for a connection in\nDATABASES by setting DISABLE_SERVER_SIDE_CURSORS to True.
To benefit from server-side cursors in transaction pooling mode, you could set\nup another connection to the database in order to\nperform queries that use server-side cursors. This connection needs to either\nbe directly to the database or to a connection pooler in session pooling mode.
\nAnother option is to wrap each QuerySet using server-side cursors in an\natomic() block, because it disables autocommit\nfor the duration of the transaction. This way, the server-side cursor will only\nlive for the duration of the transaction.
Django uses PostgreSQL's SERIAL data type to store auto-incrementing primary\nkeys. A SERIAL column is populated with values from a sequence that\nkeeps track of the next available value. Manually assigning a value to an\nauto-incrementing field doesn't update the field's sequence, which might later\ncause a conflict. For example:
>>> from django.contrib.auth.models import User\n>>> User.objects.create(username='alice', pk=1)\n<User: alice>\n>>> # The sequence hasn't been updated; its next value is 1.\n>>> User.objects.create(username='bob')\n...\nIntegrityError: duplicate key value violates unique constraint\n"auth_user_pkey" DETAIL: Key (id)=(1) already exists.\nIf you need to specify such values, reset the sequence afterwards to avoid\nreusing a value that's already in the table. The sqlsequencereset\nmanagement command generates the SQL statements to do that.
You can use the TEST['TEMPLATE'] setting to specify\na template (e.g. 'template0') from which to create a test database.
You can speed up test execution times by configuring PostgreSQL to be\nnon-durable.
\n\nDjango supports MySQL 5.6 and higher.
\nDjango's inspectdb feature uses the information_schema database, which\ncontains detailed data on all database schemas.
Django expects the database to support Unicode (UTF-8 encoding) and delegates to\nit the task of enforcing transactions and referential integrity. It is important\nto be aware of the fact that the two latter ones aren't actually enforced by\nMySQL when using the MyISAM storage engine, see the next section.
\nMySQL has several storage engines. You can change the default storage engine\nin the server configuration.
\nMySQL's default storage engine is InnoDB. This engine is fully transactional\nand supports foreign key references. It's the recommended choice. However, the\nInnoDB autoincrement counter is lost on a MySQL restart because it does not\nremember the AUTO_INCREMENT value, instead recreating it as "max(id)+1".\nThis may result in an inadvertent reuse of AutoField\nvalues.
The main drawbacks of MyISAM are that it doesn't support transactions or\nenforce foreign-key constraints.
\nMySQL has a couple drivers that implement the Python Database API described in\nPEP 249:
\nmysqlclient is a native driver. It's the recommended choice.
MySQL Connector/Python is a pure Python driver from Oracle that does not\nrequire the MySQL client library or any Python modules outside the standard\nlibrary.
These drivers are thread-safe and provide connection pooling.
\nIn addition to a DB API driver, Django needs an adapter to access the database\ndrivers from its ORM. Django provides an adapter for mysqlclient while MySQL\nConnector/Python includes its own.
\nDjango requires mysqlclient 1.3.7 or later.
\nMySQL Connector/Python is available from the download page.\nThe Django adapter is available in versions 1.1.X and later. It may not\nsupport the most recent releases of Django.
\nIf you plan on using Django's timezone support,\nuse mysql_tzinfo_to_sql to load time zone tables into the MySQL database.\nThis needs to be done just once for your MySQL server, not per database.
\nYou can create your database using the command-line tools and this SQL:
\nCREATE DATABASE <dbname> CHARACTER SET utf8;\nThis ensures all tables and columns will use UTF-8 by default.
\nThe collation setting for a column controls the order in which data is sorted\nas well as what strings compare as equal. It can be set on a database-wide\nlevel and also per-table and per-column. This is documented thoroughly in\nthe MySQL documentation. In all cases, you set the collation by directly\nmanipulating the database tables; Django doesn't provide a way to set this on\nthe model definition.
\nBy default, with a UTF-8 database, MySQL will use the\nutf8_general_ci collation. This results in all string equality\ncomparisons being done in a case-insensitive manner. That is, "Fred" and\n"freD" are considered equal at the database level. If you have a unique\nconstraint on a field, it would be illegal to try to insert both "aa" and\n"AA" into the same column, since they compare as equal (and, hence,\nnon-unique) with the default collation. If you want case-sensitive comparisons\non a particular column or table, change the column or table to use the\nutf8_bin collation.
Please note that according to MySQL Unicode Character Sets, comparisons for\nthe utf8_general_ci collation are faster, but slightly less correct, than\ncomparisons for utf8_unicode_ci. If this is acceptable for your application,\nyou should use utf8_general_ci because it is faster. If this is not acceptable\n(for example, if you require German dictionary order), use utf8_unicode_ci\nbecause it is more accurate.
Refer to the settings documentation.
\nConnection settings are used in this order:
\n\nIn other words, if you set the name of the database in OPTIONS,\nthis will take precedence over NAME, which would override\nanything in a MySQL option file.
Here's a sample configuration which uses a MySQL option file:
\n# settings.py\nDATABASES = {\n 'default': {\n 'ENGINE': 'django.db.backends.mysql',\n 'OPTIONS': {\n 'read_default_file': '/path/to/my.cnf',\n },\n }\n}\n\n\n# my.cnf\n[client]\ndatabase = NAME\nuser = USER\npassword = PASSWORD\ndefault-character-set = utf8\nSeveral other MySQLdb connection options may be useful, such as ssl,\ninit_command, and sql_mode.
sql_modeLink to this heading#From MySQL 5.7 onwards and on fresh installs of MySQL 5.6, the default value of\nthe sql_mode option contains STRICT_TRANS_TABLES. That option escalates\nwarnings into errors when data are truncated upon insertion, so Django highly\nrecommends activating a strict mode for MySQL to prevent data loss (either\nSTRICT_TRANS_TABLES or STRICT_ALL_TABLES).
If you need to customize the SQL mode, you can set the sql_mode variable\nlike other MySQL options: either in a config file or with the entry\n'init_command': "SET sql_mode='STRICT_TRANS_TABLES'" in the\nOPTIONS part of your database configuration in DATABASES.
When running concurrent loads, database transactions from different sessions\n(say, separate threads handling different requests) may interact with each\nother. These interactions are affected by each session's transaction isolation\nlevel. You can set a connection's isolation level with an\n'isolation_level' entry in the OPTIONS part of your database\nconfiguration in DATABASES. Valid values for\nthis entry are the four standard isolation levels:
'read uncommitted'
'read committed'
'repeatable read'
'serializable'
or None to use the server's configured isolation level. However, Django\nworks best with and defaults to read committed rather than MySQL's default,\nrepeatable read. Data loss is possible with repeatable read.
When Django generates the schema, it doesn't specify a storage engine, so\ntables will be created with whatever default storage engine your database\nserver is configured for. The easiest solution is to set your database server's\ndefault storage engine to the desired engine.
\nIf you're using a hosting service and can't change your server's default\nstorage engine, you have a couple of options.
\nAfter the tables are created, execute an ALTER TABLE statement to\nconvert a table to a new storage engine (such as InnoDB):
ALTER TABLE <tablename> ENGINE=INNODB;\nThis can be tedious if you have a lot of tables.
\nAnother option is to use the init_command option for MySQLdb prior to\ncreating your tables:
'OPTIONS': {\n 'init_command': 'SET default_storage_engine=INNODB',\n}\nThis sets the default storage engine upon connecting to the database.\nAfter your tables have been created, you should remove this option as it\nadds a query that is only needed during table creation to each database\nconnection.
\nThere are known issues in even the latest versions of MySQL that can cause the\ncase of a table name to be altered when certain SQL statements are executed\nunder certain conditions. It is recommended that you use lowercase table\nnames, if possible, to avoid any problems that might arise from this behavior.\nDjango uses lowercase table names when it auto-generates table names from\nmodels, so this is mainly a consideration if you are overriding the table name\nvia the db_table parameter.
Both the Django ORM and MySQL (when using the InnoDB storage engine) support database savepoints.
\nIf you use the MyISAM storage engine please be aware of the fact that you will\nreceive database-generated errors if you try to use the savepoint-related\nmethods of the transactions API. The reason\nfor this is that detecting the storage engine of a MySQL database/table is an\nexpensive operation so it was decided it isn't worth to dynamically convert\nthese methods in no-op's based in the results of such detection.
\nAny fields that are stored with VARCHAR column types have their\nmax_length restricted to 255 characters if you are using unique=True\nfor the field. This affects CharField,\nSlugField.
TextField limitationsLink to this heading#MySQL can index only the first N chars of a BLOB or TEXT column. Since\nTextField doesn't have a defined length, you can't mark it as\nunique=True. MySQL will report: "BLOB/TEXT column '<db_column>' used in key\nspecification without a key length".
MySQL 5.6.4 and later can store fractional seconds, provided that the\ncolumn definition includes a fractional indication (e.g. DATETIME(6)).\nEarlier versions do not support them at all.
Django will not upgrade existing columns to include fractional seconds if the\ndatabase server supports it. If you want to enable them on an existing database,\nit's up to you to either manually update the column on the target database, by\nexecuting a command like:
\nALTER TABLE `your_table` MODIFY `your_datetime_column` DATETIME(6)\nor using a RunSQL operation in a\ndata migration.
TIMESTAMP columnsLink to this heading#If you are using a legacy database that contains TIMESTAMP columns, you must\nset USE_TZ = False to avoid data corruption.\ninspectdb maps these columns to\nDateTimeField and if you enable timezone support,\nboth MySQL and Django will attempt to convert the values from UTC to local time.
QuerySet.select_for_update()Link to this heading#MySQL does not support the NOWAIT, SKIP LOCKED, and OF options to\nthe SELECT ... FOR UPDATE statement. If select_for_update() is used\nwith nowait=True, skip_locked=True, or of then a\nNotSupportedError is raised.
When performing a query on a string type, but with an integer value, MySQL will\ncoerce the types of all values in the table to an integer before performing the\ncomparison. If your table contains the values 'abc', 'def' and you\nquery for WHERE mycolumn=0, both rows will match. Similarly, WHERE mycolumn=1\nwill match the value 'abc1'. Therefore, string type fields included in Django\nwill always cast the value to a string before using it in a query.
If you implement custom model fields that inherit from\nField directly, are overriding\nget_prep_value(), or use\nRawSQL,\nextra(), or\nraw(), you should ensure that you perform\nappropriate typecasting.
Django supports SQLite 3.7.15 and later.
\nSQLite provides an excellent development alternative for applications that\nare predominantly read-only or require a smaller installation footprint. As\nwith all database servers, though, there are some differences that are\nspecific to SQLite that you should be aware of.
\nFor all SQLite versions, there is some slightly counter-intuitive behavior when\nattempting to match some types of strings. These are triggered when using the\niexact or contains filters in Querysets. The behavior\nsplits into two cases:
1. For substring matching, all matches are done case-insensitively. That is a\nfilter such as filter(name__contains="aa") will match a name of "Aabb".
2. For strings containing characters outside the ASCII range, all exact string\nmatches are performed case-sensitively, even when the case-insensitive options\nare passed into the query. So the iexact filter will behave exactly\nthe same as the exact filter in these cases.
Some possible workarounds for this are documented at sqlite.org, but they\naren't utilized by the default SQLite backend in Django, as incorporating them\nwould be fairly difficult to do robustly. Thus, Django exposes the default\nSQLite behavior and you should be aware of this when doing case-insensitive or\nsubstring filtering.
\nSQLite is meant to be a lightweight database, and thus can't support a high\nlevel of concurrency. OperationalError: database is locked errors indicate\nthat your application is experiencing more concurrency than sqlite can\nhandle in default configuration. This error means that one thread or process has\nan exclusive lock on the database connection and another thread timed out\nwaiting for the lock the be released.
Python's SQLite wrapper has\na default timeout value that determines how long the second thread is allowed to\nwait on the lock before it times out and raises the OperationalError: database\nis locked error.
If you're getting this error, you can solve it by:
\nSwitching to another database backend. At a certain point SQLite becomes\ntoo "lite" for real-world applications, and these sorts of concurrency\nerrors indicate you've reached that point.
Rewriting your code to reduce concurrency and ensure that database\ntransactions are short-lived.
Increase the default timeout value by setting the timeout database\noption:
'OPTIONS': {\n # ...\n 'timeout': 20,\n # ...\n}\nThis will simply make SQLite wait a bit longer before throwing "database\nis locked" errors; it won't really do anything to solve them.
\nQuerySet.select_for_update() not supportedLink to this heading#SQLite does not support the SELECT ... FOR UPDATE syntax. Calling it will\nhave no effect.
For most backends, raw queries (Manager.raw() or cursor.execute())\ncan use the "pyformat" parameter style, where placeholders in the query\nare given as '%(name)s' and the parameters are passed as a dictionary\nrather than a list. SQLite does not support this.
Django supports Oracle Database Server versions 12.1 and higher. Version\n5.2 or higher of the cx_Oracle Python driver is required.
\nIn order for the python manage.py migrate command to work, your Oracle\ndatabase user must have privileges to run the following commands:
CREATE TABLE
CREATE SEQUENCE
CREATE PROCEDURE
CREATE TRIGGER
To run a project's test suite, the user usually needs these additional\nprivileges:
\nCREATE USER
ALTER USER
DROP USER
CREATE TABLESPACE
DROP TABLESPACE
CREATE SESSION WITH ADMIN OPTION
CREATE TABLE WITH ADMIN OPTION
CREATE SEQUENCE WITH ADMIN OPTION
CREATE PROCEDURE WITH ADMIN OPTION
CREATE TRIGGER WITH ADMIN OPTION
While the RESOURCE role has the required CREATE TABLE,\nCREATE SEQUENCE, CREATE PROCEDURE, and CREATE TRIGGER privileges,\nand a user granted RESOURCE WITH ADMIN OPTION can grant RESOURCE, such\na user cannot grant the individual privileges (e.g. CREATE TABLE), and thus\nRESOURCE WITH ADMIN OPTION is not usually sufficient for running tests.
Some test suites also create views; to run these, the user also needs\nthe CREATE VIEW WITH ADMIN OPTION privilege. In particular, this is needed\nfor Django's own test suite.
All of these privileges are included in the DBA role, which is appropriate\nfor use on a private developer's database.
\nThe Oracle database backend uses the SYS.DBMS_LOB and SYS.DBMS_RANDOM\npackages, so your user will require execute permissions on it. It's normally\naccessible to all users by default, but in case it is not, you'll need to grant\npermissions like so:
GRANT EXECUTE ON SYS.DBMS_LOB TO user;\nGRANT EXECUTE ON SYS.DBMS_RANDOM TO user;\nTo connect using the service name of your Oracle database, your settings.py\nfile should look something like this:
DATABASES = {\n 'default': {\n 'ENGINE': 'django.db.backends.oracle',\n 'NAME': 'xe',\n 'USER': 'a_user',\n 'PASSWORD': 'a_password',\n 'HOST': '',\n 'PORT': '',\n }\n}\nIn this case, you should leave both HOST and PORT empty.\nHowever, if you don't use a tnsnames.ora file or a similar naming method\nand want to connect using the SID ("xe" in this example), then fill in both\nHOST and PORT like so:
DATABASES = {\n 'default': {\n 'ENGINE': 'django.db.backends.oracle',\n 'NAME': 'xe',\n 'USER': 'a_user',\n 'PASSWORD': 'a_password',\n 'HOST': 'dbprod01ned.mycompany.com',\n 'PORT': '1540',\n }\n}\nYou should either supply both HOST and PORT, or leave\nboth as empty strings. Django will use a different connect descriptor depending\non that choice.
A Full DSN or Easy Connect string can be used in NAME if both\nHOST and PORT are empty. This format is required when\nusing RAC or pluggable databases without tnsnames.ora, for example.
Example of an Easy Connect string:
\n'NAME': 'localhost:1521/orclpdb1',\nExample of a full DSN string:
\n'NAME': (\n '(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=localhost)(PORT=1521))'\n '(CONNECT_DATA=(SERVICE_NAME=orclpdb1)))'\n),\nIf you plan to run Django in a multithreaded environment (e.g. Apache using the\ndefault MPM module on any modern operating system), then you must set\nthe threaded option of your Oracle database configuration to True:
'OPTIONS': {\n 'threaded': True,\n},\nFailure to do this may result in crashes and other odd behavior.
\nBy default, the Oracle backend uses a RETURNING INTO clause to efficiently\nretrieve the value of an AutoField when inserting new rows. This behavior\nmay result in a DatabaseError in certain unusual setups, such as when\ninserting into a remote table, or into a view with an INSTEAD OF trigger.\nThe RETURNING INTO clause can be disabled by setting the\nuse_returning_into option of the database configuration to False:
'OPTIONS': {\n 'use_returning_into': False,\n},\nIn this case, the Oracle backend will use a separate SELECT query to\nretrieve AutoField values.
Oracle imposes a name length limit of 30 characters. To accommodate this, the\nbackend truncates database identifiers to fit, replacing the final four\ncharacters of the truncated name with a repeatable MD5 hash value.\nAdditionally, the backend turns database identifiers to all-uppercase.
\nTo prevent these transformations (this is usually required only when dealing\nwith legacy databases or accessing tables which belong to other users), use\na quoted name as the value for db_table:
class LegacyModel(models.Model):\n class Meta:\n db_table = '"name_left_in_lowercase"'\n\nclass ForeignModel(models.Model):\n class Meta:\n db_table = '"OTHER_USER"."NAME_ONLY_SEEMS_OVER_30"'\nQuoted names can also be used with Django's other supported database\nbackends; except for Oracle, however, the quotes have no effect.
\nWhen running migrate, an ORA-06552 error may be encountered if\ncertain Oracle keywords are used as the name of a model field or the\nvalue of a db_column option. Django quotes all identifiers used\nin queries to prevent most such problems, but this error can still\noccur when an Oracle datatype is used as a column name. In\nparticular, take care to avoid using the names date,\ntimestamp, number or float as a field name.
Django generally prefers to use the empty string ('') rather than\nNULL, but Oracle treats both identically. To get around this, the\nOracle backend ignores an explicit null option on fields that\nhave the empty string as a possible value and generates DDL as if\nnull=True. When fetching from the database, it is assumed that\na NULL value in one of these fields really means the empty\nstring, and the data is silently converted to reflect this assumption.
TextField limitationsLink to this heading#The Oracle backend stores TextFields as NCLOB columns. Oracle imposes\nsome limitations on the usage of such LOB columns in general:
LOB columns may not be used as primary keys.
LOB columns may not be used in indexes.
LOB columns may not be used in a SELECT DISTINCT list. This means that\nattempting to use the QuerySet.distinct method on a model that\nincludes TextField columns will result in an ORA-00932 error when\nrun against Oracle. As a workaround, use the QuerySet.defer method in\nconjunction with distinct() to prevent TextField columns from being\nincluded in the SELECT DISTINCT list.
In addition to the officially supported databases, there are backends provided\nby 3rd parties that allow you to use other databases with Django:
\n\nThe Django versions and ORM features supported by these unofficial backends\nvary considerably. Queries regarding the specific capabilities of these\nunofficial backends, along with any support queries, should be directed to\nthe support channels provided by each 3rd party project.
\n