https://blog.miguelgrinberg.com/post/it-s-time-for-a-change-datetime-utcnow-is-now-deprecated miguelgrinberg.com * Home * My Courses and Books * Consulting * About Me * RSS Feed Twitter Mastodon GitHub YouTube LinkedIn Patreon It's Time For A Change: datetime.utcnow() Is Now Deprecated Posted by [Miguel Grinberg] on 2023-11-18T12:00:19Z under [Programming] [Python] I was going through the release notes of the new Python 3.12 version the other day, and one item caught my attention in the deprecations section: datetime.datetime's utcnow() and utcfromtimestamp() are deprecated and will be removed in a future version. If you have followed my web development tutorials you must have seen me use utcnow() a lot, so I will clearly need to re-train myself to use an alternative, in preparation for the eventual removal of this function (likely a few years out, so no need to panic!). In this short article I'll tell you more about why these functions are getting the axe, and what to replace them with. What's Wrong with utcnow() and utcfromtimestamp()? The problem that the Python maintainers have found comes from the fact that these functions return "naive" datetime objects. A naive datetime object is one that does not have a timezone, which means that it can only be used in a context where the timezone does not matter or is already known in advance. This is in contrast to "aware" datetime objects, which do have a timezone attached to them explicitly. If you ask me, I think the names of these functions are misleading. A function that is called utcnow() should be expected to return UTC datetimes, as implied by the name. I would have made it more clear that these functions work with naive time, maybe by calling them naive_utcnow() and naive_utcfromtimestamp(). But their names are not the problem here. The specific issue is that some Python date and time functions accept naive timestamps and assume that they represent local time, according to the timezone that is configured on the computer running the code. There is a GitHub issue from 2019 that provides some background into this, with the following example: >>> from datetime import datetime >>> dt = datetime.utcfromtimestamp(0) >>> dt datetime.datetime(1970, 1, 1, 0, 0) >>> dt.timestamp() 18000 The example above was executed on a computer that was configured for Eastern Standard Time (EST). First, dt is assigned a naive datetime that is converted from the "zero" time or UNIX epoch, which is January 1st, 1970 at midnight. When this object is converted back to a timestamp, the dt.timestamp() method finds that it does not have a timezone to use in the conversion, so it uses the computer's own timezone, which in this example was EST (note that the EST timezone is 5 hours, or 18,000 seconds behind UTC). So we have a UNIX timestamp that originated as midnight on January 1st, 1970, and after being converted to a datetime and back ends up as 5 am. If you read the issue linked above, they suggest that this ambiguity did not exist in Python 2 and for that reason this was not a problem for a long time, but it now is and needs to be addressed. This sounded strange, so I had to go and check, and sure enough, the timestamp() method that returns the incorrect UNIX time in the example was introduced in Python 3.3 and nothing similar appears to have existed back in Python 2 times. So basically, at some point they've added a datetime.timestamp() method (and possibly others as well) that accept both aware and naive datetimes and this was a mistake, because these methods must have a timezone to work. These methods should have been designed to fail when a naive datetime object is passed to them, but for some strange reason they decided that when a timezone is not provided the timezone from the system should be used. This is really the bug, but instead of fixing the broken implementations of these methods they are now trying to force people to move to aware datetimes by deprecating the two main functions that generate naive ones. They think that because a few functions assume that naive timestamps represent local times, all naive uses that are not in local time should be discouraged. I may be missing something here, but I don't really follow this logic. Do We Need Naive Datetimes Anyway? To me it is clear that the Python maintainers behind this deprecation have a problem with naive datetimes and are using this supposed problem as an excuse to cripple them. So why would you want to work with naive datetimes in the first place? An application may be designed in such a way that all dates and times are in a single timezone that is known in advance. In this case there is no need for individual datetime instances to carry their own timezones, since this uses more memory and processing power for no benefit, since all these timezones would be the same and it would never be necessary to perform timezone math or conversions. This is actually very common in web applications or other types of networking servers, which are configured with UTC time and normalize all dates and times to this timezone when they enter the system. It is also a best practice to store naive datetimes representing UTC in databases. The DateTime type in SQLAlchemy represents a naive datetime object by default, for example. This is such a common database pattern that SQLAlchemy provides a recipe for applications that use aware datetime objects to convert these to and from naive ones on the fly as they are saved to or loaded from the database. So yes, I expect naive datetime objects will continue to be used, in spite of these deprecations. Updating Your Code Even though the deprecations are disappointing, it is important to keep in mind that it may take a few years for the functions to actually be removed. The problem is that once you switch to Python 3.12 or newer you will start seeing deprecation messages on your console and your logs, and these can get annoying. Here is an example of what you can expect to see: $ python Python 3.12.0 (main, Oct 5 2023, 10:46:39) [GCC 11.4.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from datetime import datetime >>> datetime.utcnow() :1: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC). datetime.datetime(2023, 11, 18, 11, 22, 54, 263206) I'm only using Python 3.12 in a small number of projects, and I'm already tired of seeing these warnings. So let's go ahead and look at how these two functions can be replaced. The advice from the Python maintainers is to switch to aware datetime objects. The deprecation warning provides a hint of what they think we should use, and the deprecation notices included in the documentation are even more specific. Here is what the notice for the utcnow() function says: Deprecated since version 3.12: Use datetime.now() with UTC instead. Below you can see the one for utcfromtimestamp(): Deprecated since version 3.12: Use datetime.fromtimestamp() with UTC instead. So this gives us an idea of what can be done. Here are my custom versions of the deprecated functions, with the additional option to choose between aware or naive implementations: from datetime import datetime, timezone def aware_utcnow(): return datetime.now(timezone.utc) def aware_utcfromtimestamp(timestamp): return datetime.fromtimestamp(timestamp, timezone.utc) def naive_utcnow(): return aware_utcnow().replace(tzinfo=None) def naive_utcfromtimestamp(timestamp): return aware_utcfromtimestamp(timestamp).replace(tzinfo=None) print(aware_utcnow()) print(aware_utcfromtimestamp(0)) print(naive_utcnow()) print(naive_utcfromtimestamp(0)) Note that if you are using Python 3.11 or newer, you can replace datetime.timezone.utc with a shorter datetime.UTC. Running this script I get the following results: 2023-11-18 11:36:35.137639+00:00 1970-01-01 00:00:00+00:00 2023-11-18 11:36:35.137672 1970-01-01 00:00:00 You can tell that the first and second lines show aware datetime instances from the +00:00 suffix that indicates that the timezone is 00:00 or UTC. The third and fourth lines show abstract timestamps without a timezone, fully compatible with those returned by the deprecated functions. What I like about these implementations is that they give you the choice to work with or without timezones, removing any ambiguity. Explicit is better than implicit, as the old adage says. Become a Patron! Hello, and thank you for visiting my blog! If you enjoyed this article, please consider supporting my work on this blog on Patreon! [patreon-bu] Share this post: Hacker News Reddit Twitter LinkedIn Facebook E-Mail 4 comments * [5fa7995da0] #1 Alex said 2023-11-19T13:43:31Z This is actually very common in web applications or other types of networking servers, which are configured with UTC time and normalize all dates and times to this timezone when they enter the system. It is also a best practice to store naive datetimes representing UTC in databases. If you store "naive datetimes representing UTC" in your DB then doesn't that clash with Python's naive datetimes? Python's naive datetimes are usually assumed to represent local time rather than UTC, unless they come from these two funny functions. If all the times in your DB are in UTC then the right type to use in python is a datetime with datetime.UTC attached, not a naive one. Similarly, if you want to convert all times to UTC on the way into your web application as you say is best practice, then once you've done so the result should be a datetime with datetime.UTC attached not a naive one. (It may be the case that the whole webserver is configured to be in UTC time so these are more or less the same, but it's still better to be explicit just like it's better to encode all your text in UTF-8 instead of using the system locale.) * [729e26a2a2] #2 Miguel Grinberg said 2023-11-19T15:07:37Z @Alex: I get what you are saying, but the idea that naive datetimes represent local time is fairly recent, and only put to practice in a handful of functions. The Python 3.12 documentation has what I consider the correct definition for naive datetime objects: A naive object does not contain enough information to unambiguously locate itself relative to other date/time objects. Whether a naive object represents Coordinated Universal Time (UTC), local time, or time in some other timezone is purely up to the program, just like it is up to the program whether a particular number represents metres, miles, or mass. Naive objects are easy to understand and to work with, at the cost of ignoring some aspects of reality. So there is really no clash. As long as you know that you are working with naive datetimes and only perform operations that do not require knowing the timezone (which in my opinion is a pre-requisite to working with naive datetimes), there is absolutely no problem. I can turn the argument around and say that it really makes no sense to use a naive datetime to represent a local time, since you can explicitly state that you want local time by storing the local timezone in the object. The footgun exists from both sides of the argument. * [3fe6077712] #3 Jeffry Babb said 2023-11-19T15:44:16Z Hmmm, I don't know, the very purpose of UTC is to create a reference, so a "naive" datetime should have always defaulted to UTC + 0. * [729e26a2a2] #4 Miguel Grinberg said 2023-11-19T15:49:37Z @Jeffry: No, that was never the intention. There is already a way to represent UTC timestamps as aware datetimes, and that works well. A naive datetime does not specify a timezone. This works for applications that do not care about timezones, or for applications that know what timezone they want to use already. * <<<< * << * >> * >>>> Leave a Comment Name [ ] Email [ ] Comment [ ] Captcha [Submit] The New and Improved Flask Mega-Tutorial The New Flask Mega-Tutorial If you would you like to support my work on my Flask Mega-Tutorial series on this blog and as a reward have access to the complete tutorial nicely structured as a book and/or a set of videos, you can now order it from my Courses site or from Amazon. Click here to get the Book! Click here to get the Video Course! About Miguel [miguel] Welcome to my blog! I'm a software engineer and technical writer, currently living in Drogheda, Ireland. You can also find me on Twitter, Mastodon, Github, LinkedIn, YouTube, Facebook and Patreon. Thank you for visiting! Categories AWS RSS Feed [AWS] 1 Arduino RSS Feed [Arduino] 7 Authentication RSS Feed [Authentication] 9 Blog RSS Feed [Blog] 1 C++ RSS Feed [C++] 5 Cloud RSS Feed [Cloud] 8 Database RSS Feed [Database] 18 Docker RSS Feed [Docker] 4 Filmmaking RSS Feed [Filmmaking] 6 Flask RSS Feed [Flask] 102 Games RSS Feed [Games] 1 HTML5 RSS Feed [HTML5] 1 Heroku RSS Feed [Heroku] 1 IoT RSS Feed [IoT] 8 JavaScript RSS Feed [JavaScript] 30 MicroPython RSS Feed [MicroPython] 8 Microservices RSS Feed [Microservices] 2 Movie Reviews RSS Feed [Movie Reviews] 5 OpenStack RSS Feed [OpenStack] 1 Personal RSS Feed [Personal] 3 Photography RSS Feed [Photography] 7 Product Reviews RSS Feed [Product Reviews] 2 Programming RSS Feed [Programming] 160 Project Management RSS Feed [Project Management] 1 Python RSS Feed [Python] 143 REST RSS Feed [REST] 6 Rackspace RSS Feed [Rackspace] 1 Raspberry Pi RSS Feed [Raspberry Pi] 7 React RSS Feed [React] 18 Robotics RSS Feed [Robotics] 6 Security RSS Feed [Security] 11 Video RSS Feed [Video] 22 Webcast RSS Feed [Webcast] 3 Windows RSS Feed [Windows] 1 (c) 2012- by Miguel Grinberg. All rights reserved. Questions?