Django natively supports Unicode data everywhere. Providing your database can\nsomehow store the data, you can safely pass around Unicode strings to\ntemplates, models and the database.
\nThis document tells you what you need to know if you’re writing applications\nthat use data or templates that are encoded in something other than ASCII.
\nMake sure your database is configured to be able to store arbitrary string\ndata. Normally, this means giving it an encoding of UTF-8 or UTF-16. If you use\na more restrictive encoding – for example, latin1 (iso8859-1) – you won’t be\nable to store certain characters in the database, and information will be lost.
\nMySQL users, refer to the MySQL manual for details on how to set or alter\nthe database character set encoding.
PostgreSQL users, refer to the PostgreSQL manual (section 22.3.2 in\nPostgreSQL 9) for details on creating databases with the correct encoding.
Oracle users, refer to the Oracle manual for details on how to set\n(section 2) or alter (section 11) the database character set encoding.
SQLite users, there is nothing you need to do. SQLite always uses UTF-8\nfor internal encoding.
All of Django’s database backends automatically convert Unicode strings into\nthe appropriate encoding for talking to the database. They also automatically\nconvert strings retrieved from the database into Python Unicode strings. You\ndon’t even need to tell Django what encoding your database uses: that is\nhandled transparently.
\nFor more, see the section “The database API” below.
\nWhenever you use strings with Django – e.g., in database lookups, template\nrendering or anywhere else – you have two choices for encoding those strings.\nYou can use Unicode strings, or you can use normal strings (sometimes called\n“bytestrings”) that are encoded using UTF-8.
\nIn Python 3, the logic is reversed, that is normal strings are Unicode, and\nwhen you want to specifically create a bytestring, you have to prefix the\nstring with a ‘b’. As we are doing in Django code from version 1.5,\nwe recommend that you import unicode_literals from the __future__ library\nin your code. Then, when you specifically want to create a bytestring literal,\nprefix the string with ‘b’.
Python 2 legacy:
\nmy_string = "This is a bytestring"\nmy_unicode = u"This is an Unicode string"\nPython 2 with unicode literals or Python 3:
\nfrom __future__ import unicode_literals\n\nmy_string = b"This is a bytestring"\nmy_unicode = "This is an Unicode string"\nSee also Python 3 compatibility.
\n\nIf your code only uses ASCII data, it’s safe to use your normal strings,\npassing them around at will, because ASCII is a subset of UTF-8.
\nDon’t be fooled into thinking that if your DEFAULT_CHARSET setting is set\nto something other than 'utf-8' you can use that other encoding in your\nbytestrings! DEFAULT_CHARSET only applies to the strings generated as\nthe result of template rendering (and email). Django will always assume UTF-8\nencoding for internal bytestrings. The reason for this is that the\nDEFAULT_CHARSET setting is not actually under your control (if you are the\napplication developer). It’s under the control of the person installing and\nusing your application – and if that person chooses a different setting, your\ncode must still continue to work. Ergo, it cannot rely on that setting.
In most cases when Django is dealing with strings, it will convert them to\nUnicode strings before doing anything else. So, as a general rule, if you pass\nin a bytestring, be prepared to receive a Unicode string back in the result.
\nAside from Unicode strings and bytestrings, there’s a third type of string-like\nobject you may encounter when using Django. The framework’s\ninternationalization features introduce the concept of a “lazy translation” –\na string that has been marked as translated but whose actual translation result\nisn’t determined until the object is used in a string. This feature is useful\nin cases where the translation locale is unknown until the string is used, even\nthough the string might have originally been created when the code was first\nimported.
\nNormally, you won’t have to worry about lazy translations. Just be aware that\nif you examine an object and it claims to be a\ndjango.utils.functional.__proxy__ object, it is a lazy translation.\nCalling unicode() with the lazy translation as the argument will generate a\nUnicode string in the current locale.
For more details about lazy translation objects, refer to the\ninternationalization documentation.
\nBecause some string operations come up again and again, Django ships with a few\nuseful functions that should make working with Unicode and bytestring objects\na bit easier.
\nThe django.utils.encoding module contains a few functions that are handy\nfor converting back and forth between Unicode and bytestrings.
smart_text(s, encoding='utf-8', strings_only=False, errors='strict')\nconverts its input to a Unicode string. The encoding parameter\nspecifies the input encoding. (For example, Django uses this internally\nwhen processing form input data, which might not be UTF-8 encoded.) The\nstrings_only parameter, if set to True, will result in Python\nnumbers, booleans and None not being converted to a string (they keep\ntheir original types). The errors parameter takes any of the values\nthat are accepted by Python’s unicode() function for its error\nhandling.
If you pass smart_text() an object that has a __unicode__\nmethod, it will use that method to do the conversion.
force_text(s, encoding='utf-8', strings_only=False,\nerrors='strict') is identical to smart_text() in almost all\ncases. The difference is when the first argument is a lazy\ntranslation instance. While smart_text()\npreserves lazy translations, force_text() forces those objects to a\nUnicode string (causing the translation to occur). Normally, you’ll want\nto use smart_text(). However, force_text() is useful in\ntemplate tags and filters that absolutely must have a string to work\nwith, not just something that can be converted to a string.
smart_bytes(s, encoding='utf-8', strings_only=False, errors='strict')\nis essentially the opposite of smart_text(). It forces the first\nargument to a bytestring. The strings_only parameter has the same\nbehavior as for smart_text() and force_text(). This is\nslightly different semantics from Python’s builtin str() function,\nbut the difference is needed in a few places within Django’s internals.
Normally, you’ll only need to use force_text(). Call it as early as\npossible on any input data that might be either Unicode or a bytestring, and\nfrom then on, you can treat the result as always being Unicode.
Web frameworks have to deal with URLs (which are a type of IRI). One\nrequirement of URLs is that they are encoded using only ASCII characters.\nHowever, in an international environment, you might need to construct a\nURL from an IRI – very loosely speaking, a URI that can contain Unicode\ncharacters. Quoting and converting an IRI to URI can be a little tricky, so\nDjango provides some assistance.
\nThe function django.utils.encoding.iri_to_uri() implements the\nconversion from IRI to URI as required by the specification (RFC 3987 Section 3.1).
The functions django.utils.http.urlquote() and\ndjango.utils.http.urlquote_plus() are versions of Python’s standard\nurllib.quote() and urllib.quote_plus() that work with non-ASCII\ncharacters. (The data is converted to UTF-8 prior to encoding.)
These two groups of functions have slightly different purposes, and it’s\nimportant to keep them straight. Normally, you would use urlquote() on the\nindividual portions of the IRI or URI path so that any reserved characters\nsuch as ‘&’ or ‘%’ are correctly encoded. Then, you apply iri_to_uri() to\nthe full IRI and it converts any non-ASCII characters to the correct encoded\nvalues.
The iri_to_uri() function will not change ASCII characters that are\notherwise permitted in a URL. So, for example, the character ‘%’ is not\nfurther encoded when passed to iri_to_uri(). This means you can pass a\nfull URL to this function and it will not mess up the query string or anything\nlike that.
An example might clarify things here:
\n>>> urlquote('Paris & Orléans')\n'Paris%20%26%20Orl%C3%A9ans'\n>>> iri_to_uri('/favorites/François/%s' % urlquote('Paris & Orléans'))\n'/favorites/Fran%C3%A7ois/Paris%20%26%20Orl%C3%A9ans'\nIf you look carefully, you can see that the portion that was generated by\nurlquote() in the second example was not double-quoted when passed to\niri_to_uri(). This is a very important and useful feature. It means that\nyou can construct your IRI without worrying about whether it contains\nnon-ASCII characters and then, right at the end, call iri_to_uri() on the\nresult.
Similarly, Django provides django.utils.encoding.uri_to_iri() which\nimplements the conversion from URI to IRI as per RFC 3987 Section 3.2.\nIt decodes all percent-encodings except those that don’t represent a valid\nUTF-8 sequence.
An example to demonstrate:
\n>>> uri_to_iri('/%E2%99%A5%E2%99%A5/?utf8=%E2%9C%93')\n'/♥♥/?utf8=✓'\n>>> uri_to_iri('%A9helloworld')\n'%A9helloworld'\nIn the first example, the UTF-8 characters and reserved characters are\nunquoted. In the second, the percent-encoding remains unchanged because it\nlies outside the valid UTF-8 range.
\nBoth iri_to_uri() and uri_to_iri() functions are idempotent, which means the\nfollowing is always true:
iri_to_uri(iri_to_uri(some_string)) == iri_to_uri(some_string)\nuri_to_iri(uri_to_iri(some_string)) == uri_to_iri(some_string)\nSo you can safely call it multiple times on the same URI/IRI without risking\ndouble-quoting problems.
\nBecause all strings are returned from the database as Unicode strings, model\nfields that are character based (CharField, TextField, URLField, etc.) will\ncontain Unicode values when Django retrieves data from the database. This\nis always the case, even if the data could fit into an ASCII bytestring.
\nYou can pass in bytestrings when creating a model or populating a field, and\nDjango will convert it to Unicode when it needs to.
\n__str__() and __unicode__()Link para este cabeçalho#One consequence of using Unicode by default is that you have to take some care\nwhen printing data from the model.
\nIn particular, rather than giving your model a __str__() method, we\nrecommended you implement a __unicode__() method. In the __unicode__()\nmethod, you can quite safely return the values of all your fields without\nhaving to worry about whether they fit into a bytestring or not. (The way\nPython works, the result of __str__() is always a bytestring, even if you\naccidentally try to return a Unicode object).
You can still create a __str__() method on your models if you want, of\ncourse, but you shouldn’t need to do this unless you have a good reason.\nDjango’s Model base class automatically provides a __str__()\nimplementation that calls __unicode__() and encodes the result into UTF-8.\nThis means you’ll normally only need to implement a __unicode__() method\nand let Django handle the coercion to a bytestring when required.
get_absolute_url()Link para este cabeçalho#URLs can only contain ASCII characters. If you’re constructing a URL from\npieces of data that might be non-ASCII, be careful to encode the results in a\nway that is suitable for a URL. The reverse() function\nhandles this for you automatically.
If you’re constructing a URL manually (i.e., not using the reverse()\nfunction), you’ll need to take care of the encoding yourself. In this case,\nuse the iri_to_uri() and urlquote() functions that were documented\nabove. For example:
from django.utils.encoding import iri_to_uri\nfrom django.utils.http import urlquote\n\ndef get_absolute_url(self):\n url = '/person/%s/?x=0&y=0' % urlquote(self.location)\n return iri_to_uri(url)\nThis function returns a correctly encoded URL even if self.location is\nsomething like “Jack visited Paris & Orléans”. (In fact, the iri_to_uri()\ncall isn’t strictly necessary in the above example, because all the\nnon-ASCII characters would have been removed in quoting in the first line.)
You can pass either Unicode strings or UTF-8 bytestrings as arguments to\nfilter() methods and the like in the database API. The following two\nquerysets are identical:
from __future__ import unicode_literals\n\nqs = People.objects.filter(name__contains='Å')\nqs = People.objects.filter(name__contains=b'\\xc3\\x85') # UTF-8 encoding of Å\nYou can use either Unicode or bytestrings when creating templates manually:
\nfrom __future__ import unicode_literals\nfrom django.template import Template\nt1 = Template(b'This is a bytestring template.')\nt2 = Template('This is a Unicode template.')\nBut the common case is to read templates from the filesystem, and this creates\na slight complication: not all filesystems store their data encoded as UTF-8.\nIf your template files are not stored with a UTF-8 encoding, set the FILE_CHARSET\nsetting to the encoding of the files on disk. When Django reads in a template\nfile, it will convert the data from this encoding to Unicode. (FILE_CHARSET\nis set to 'utf-8' by default.)
The DEFAULT_CHARSET setting controls the encoding of rendered templates.\nThis is set to UTF-8 by default.
If you intend to allow users to upload files, you must ensure that the\nenvironment used to run Django is configured to work with non-ASCII file names.\nIf your environment isn’t configured correctly, you’ll encounter\nUnicodeEncodeError exceptions when saving files with file names that\ncontain non-ASCII characters.
Filesystem support for UTF-8 file names varies and might depend on the\nenvironment. Check your current configuration in an interactive Python shell by\nrunning:
\nimport sys\nsys.getfilesystemencoding()\nThis should output “UTF-8”.
\nThe LANG environment variable is responsible for setting the expected\nencoding on Unix platforms. Consult the documentation for your operating system\nand application server for the appropriate syntax and location to set this\nvariable.
In your development environment, you might need to add a setting to your\n~.bashrc analogous to::
export LANG="en_US.UTF-8"\nDjango’s email framework (in django.core.mail) supports Unicode\ntransparently. You can use Unicode data in the message bodies and any headers.\nHowever, you’re still obligated to respect the requirements of the email\nspecifications, so, for example, email addresses should use only ASCII\ncharacters.
The following code example demonstrates that everything except email addresses\ncan be non-ASCII:
\nfrom __future__ import unicode_literals\nfrom django.core.mail import EmailMessage\n\nsubject = 'My visit to Sør-Trøndelag'\nsender = 'Arnbjörg Ráðormsdóttir <arnbjorg@example.com>'\nrecipients = ['Fred <fred@example.com']\nbody = '...'\nmsg = EmailMessage(subject, body, sender, recipients)\nmsg.attach("Une pièce jointe.pdf", "%PDF-1.4.%...", mimetype="application/pdf")\nmsg.send()\nHTML form submission is a tricky area. There’s no guarantee that the\nsubmission will include encoding information, which means the framework might\nhave to guess at the encoding of submitted data.
\nDjango adopts a “lazy” approach to decoding form data. The data in an\nHttpRequest object is only decoded when you access it. In fact, most of\nthe data is not decoded at all. Only the HttpRequest.GET and\nHttpRequest.POST data structures have any decoding applied to them. Those\ntwo fields will return their members as Unicode data. All other attributes and\nmethods of HttpRequest return data exactly as it was submitted by the\nclient.
By default, the DEFAULT_CHARSET setting is used as the assumed encoding\nfor form data. If you need to change this for a particular form, you can set\nthe encoding attribute on an HttpRequest instance. For example:
def some_view(request):\n # We know that the data must be encoded as KOI8-R (for some reason).\n request.encoding = 'koi8-r'\n ...\nYou can even change the encoding after having accessed request.GET or\nrequest.POST, and all subsequent accesses will use the new encoding.
Most developers won’t need to worry about changing form encoding, but this is\na useful feature for applications that talk to legacy systems whose encoding\nyou cannot control.
\nDjango does not decode the data of file uploads, because that data is normally\ntreated as collections of bytes, rather than strings. Any automatic decoding\nthere would alter the meaning of the stream of bytes.
\n