[HN Gopher] Python datetime pitfalls, and what libraries are (no...
___________________________________________________________________
Python datetime pitfalls, and what libraries are (not) doing about
it
Author : marban
Score : 162 points
Date : 2024-02-18 08:26 UTC (14 hours ago)
(HTM) web link (dev.arie.bovenberg.net)
(TXT) w3m dump (dev.arie.bovenberg.net)
| WesolyKubeczek wrote:
| I wish such an overview also existed for Go. Yes I could do it
| but I'm lazy.
| smitty1e wrote:
| I don't think any of the DST complaints are legitimate.
|
| If doing work involved with timestamps, not keeping everything in
| GMT is questionable. GMT is the UTF-8 of timestamps: the low
| common-denominator rules.
|
| The datetime library is mechanism. DST is policy. Policy is
| wildly variable and your system library would be unmaintainable
| if it had to chase policy. Just say no.
| lifthrasiir wrote:
| As long as you can't predict the future, some future date since
| a specific cut-off can't be reliably converted to UTC (not GMT,
| which _does_ observe DST) anyway.
| graemep wrote:
| GMT does not have DST. It is only slightly different from
| UTC. That said, you should really use UTC.
|
| I think you are confusing it with British Summer Time which
| is GMT +1
| pasc1878 wrote:
| Being a pedant here.
|
| What is being confused is GMT/BST/(BDST) with the time in a
| location Greenwich.
|
| Currently the time in London is BST between end March to
| end of October and GMT the rest of the year. Historically
| the dates have changed and also had period where it was BST
| in the winter and BDST (GNT+2) in the summer, and also
| years with BST all year round.
|
| There are still people who suggest BST all year round (I
| think if Scotland when independent it could happen) Also
| now UK is not in the EU it is more free to change things.
| jasode wrote:
| _> That said, you should really use UTC._
|
| No, that "store UTC everywhere" is often repeated but it's
| incorrect advice when applied to _future human-constructed
| datetimes_ such as appointments or social events. Future
| "human-interpreted wall-clock" datetimes cannot be
| unambiguously stored as future UTC values with perfect
| roundtrip fidelity.
|
| I've tried to explain the difference in previous comment:
|
| https://news.ycombinator.com/item?id=19534843
| Glyptodon wrote:
| The number of times very smart people seem to think
| having to alter a pile of UTC times because their
| associated region has changed their DST behavior is
| better than having local zoneless and just changing TZDB
| always weirds me out.
| graemep wrote:
| That is a good point, but fortunately a rare case.
|
| The other problem is that when you are using human inputs
| such as appointments you may not know which is
| appropriate. If I arrange a meeting with someone in
| another timezone, and there is such a change, how can a
| system know whether I want the UTC time, my timezone, or
| the other person's timezone.
|
| I am not sure there is a good way to do this in many
| cases. As you say some culturally defined times are
| definitely tied to the local timezone, but a lot of
| things will not be. There is a loss of information, but
| that information may not be useful on many cases.
|
| I think I agree with you in that there is no universal
| solution. However, store UTC is probably good enough a
| lot of the time. In general, timezone changes would be
| announced well ahead of time so UTC would be fine except
| for times fixed far ahead.
|
| I have actually lived through timezone changes that were
| decided on in emergency circumstances (Sri Lanka faced
| with a severe electricity shortage) and the result was
| chaos. People genuinely did not know whether an
| appointment would take place in "old time" or "new time"!
| No system devised earlier could have coped with that
| because intent was no unambiguous.
| Hackbraten wrote:
| > how can a system know whether I want the UTC time, my
| timezone, or the other person's timezone
|
| A decent system will allow the user to make that choice
| on a case-by-case basis but assume the common case as a
| default. I've seen apps pull that off successfully, e.g.
| iCal.
| praash wrote:
| A true observation, especially since many European countries
| (or just some citizens) wish to get rid of DST.
|
| However, +-1 hour is not much of an error when talking about
| a time delay of at least a year.
| autarch wrote:
| > However, +-1 hour is not much of an error when talking
| about a time delay of at least a year.
|
| It is if this a doctor's appointment or a meeting with a
| friend or ... you get my point.
| macintux wrote:
| I missed most of my last chance to see my cousin play
| college basketball because I trusted her college's
| website, and it was off by an hour due to timezone
| confusion.
| praash wrote:
| This is the only sane policy. Timezone conversion must be
| delayed to the very last moment before displaying time to
| users.
|
| If I needed to define a daily alarm in a local wall clock time,
| I'd happily deal with a zoneless dateless hour-minute tuple and
| determine the alarm time by combining a date and a time using a
| timezone. Right after I'd convert that and handle the rest in
| UTC.
| autarch wrote:
| > This is the only sane policy. Timezone conversion must be
| delayed to the very last moment before displaying time to
| users.
|
| No, this only works for datetimes in the _past_. For future
| datetimes you often must store the datetime in the user's
| (IANA) timezone. Otherwise your application could be off. And
| your app could be off by more than one hour. In 2011 Samoa
| moved across the international date line. See
| https://www.theguardian.com/world/2011/dec/30/samoa-loses-
| da....
| Spivak wrote:
| Thank god someone in the thread understands the problem.
| Everyone saying "just make everything UTC and convert" is
| both throwing away information and making it impossible to
| do future calculations.
|
| "Schedule a 1-1 with my boss every two weeks at 1 PM"
|
| If you take my local time zone, convert it to UTC which
| puts my meeting at say 9AM UTC, schedule every two weeks at
| 9AM UTC and convert back to my local time zone it will be
| wrong when I hit change DST.
|
| But if I instead store the (datetime, zone) pair when I say
| "1PM fourteen calendar days from now in zone" it will
| always be correct.
|
| The UTC "trick" only works when your problem domain is
| representing specific moments in time.
| smitty1e wrote:
| And how can you predict timezones policy changes in the
| future?
| pasc1878 wrote:
| You need to know more than that. Is the appoint meant 2pm in
| London or in New York. So for forward times you need a
| location as well as the date and time.
|
| Yes when it happens you store in UTC.
|
| In both cases you can display to user with local timezone.
| (Although if the London user is going top fly to NY they will
| still want to see the NY time and not their local time)
| Kwpolska wrote:
| DST and local time are relevant in calculations when humans are
| involved. A user in Japan shouldn't be told it's still Sunday
| just because it's still Sunday somewhere in western Europe. If
| I set up a recurring meeting every Monday at 10:00, it
| shouldn't suddenly shift to 09:00 or 11:00 just because of a
| DST switchover.
| kroolik wrote:
| What if the meeting is between people in EU and USA, where
| DST transitions occur at different moments?
| rightbyte wrote:
| Hmm ... if a problem is hard enough maybe just don't try to
| solve it?
|
| Treat the meeting time as a string, rather than a time,
| since it is ambiguous anyway what the organizer meant, and
| let the user figure out what the string means.
| airstrike wrote:
| Tell those people not to meet at 2am on the night DST
| changes
| demondemidi wrote:
| All the meetings are hosed for about two weeks. This is a
| known issue for anyone working with time zones that don't
| change together.
| Kwpolska wrote:
| Calendars like Outlook let you specify a timezone for the
| event. If it's set to Europe/Warsaw, the US goes out of
| sync for a few weeks. If America/New_York, the EU does.
| This still isn't a problem for a meeting between LA and NY,
| for example (unless the meeting is in the middle of the
| night).
| demondemidi wrote:
| Yep! I hate those few weeks between PST and CET.
| smitty1e wrote:
| I didn't say "irrelevant". I questioned keeping that in the
| library.
| Hackbraten wrote:
| If the library isn't supposed to do all the heavy lifting,
| then who is?
| Glyptodon wrote:
| I tend to prefer:
|
| - Things happen/happened at UTC times
|
| - Things will happen at zoneless time + IANA DB label. Though
| there are some edges. Obviously it's possible to pre-apply and
| convert to UTC, which is okay, but whether that makes sense to
| do in practice varies some. Future UTC time of often worthless
| for understanding human centric dates and times.
| Kwpolska wrote:
| For past events, knowing the local time might be useful
| context. I'd store local time with UTC offset, or with IANA
| time zone name (which might change the UTC meaning if there's
| some change affecting past dates in the IANA database).
| tremon wrote:
| _If doing work involved with timestamps_
|
| Of course, any argument becomes much easier if you're allowed
| to just discard half the purpose of the thing you're arguing
| about. The Python datetime library doesn't just exist to record
| timestamps. From the actual documentation:
|
| > The datetime module supplies classes for manipulating dates
| and times.
|
| > [..] the focus of the implementation is on efficient
| attribute extraction for output formatting and manipulation.
|
| Datetime manipulation is part of the explicit focus of the
| module. So why would you argue that we shouldn't expect said
| manipulations to be DST-aware?
| TrackerFF wrote:
| ISO 8601 is the only way to go.
|
| With that said, it's hard to control what systems others use. I
| frequently work with datasets that have 4-5 different timestamp
| formats.
|
| One of the most annoying bugs I've worked on, was in a table
| where _almost_ everything followed a DDMMYYYY format - sans a 2-3
| month period where the dates were flipped to MMDDYYYY
| Thorrez wrote:
| Does ISO 8601 fix any of the problems mentioned in the article?
| It doesn't really seem relevant to me.
| sitkack wrote:
| > doesn't really seem relevant to me
|
| Isn't this overly 'you' centric? Wouldn't this require access
| to your inner monologue? Doesn't seem like a referentially
| invariant supporting argument, to _me_.
| falserum wrote:
| Charitable interpretation of previous comment would be "I
| think it does not seem relevant to problems specified in
| the article".
|
| Iso 8601 is about date formatting, while article is about a
| bit orthogonal to that. (But I agree/hope that ISO 8601 is
| the future that is less ambiguous than current state)
| sitkack wrote:
| > I think it does not seem relevant to problems specified
| in the article
|
| Is a low effort way to make someone form a rebuttal in
| response to no argument. It is pseudo intellectual way to
| sound smart while offering nothing in return. As if that
| persons opinion has weight because "they don't see the
| relevance", I know lots of people that "don't see
| relevance". Is our responsibility to educate their
| feigned confusion?
| LikesPwsh wrote:
| That bug is probably because of someone opening a CSV in Excel.
|
| It's a common one, sadly. Anyone using the US date format for
| transfer or storage (debatably also display) should consider it
| a bug.
| praash wrote:
| After all these years I still feel Excel cannot handle a
| fully compliant datetime string.
| teddyh wrote:
| I actually prefer RFC 3339 over an not-freely-available ISO
| standard.
| falserum wrote:
| https://ijmacd.github.io/rfc3339-iso8601/
|
| (Maybe useful, comparison)
| teddyh wrote:
| Useful, but one thing immediately jumps out as incorrect:
| RFC 3339 _does_ specify a syntax for periods; it's defined
| in Appendix A as ABNF token "duration".
| praash wrote:
| ISO 8601, ironically, also defined acceptable a (now
| deprecated) set of truncated date formats such as YY-MM-DD and
| YYMMDD.
| pepoluan wrote:
| As long as it is now deprecated then it's no longer ironic.
| rini17 wrote:
| Okay but it does not standardize location. To reliably manage
| local time conversions, including location together with
| date/time is the only way.
| frumiousirc wrote:
| Best code strategy here is twofold.
|
| 1. Maximize the code context where date/time is represented as a
| simple count of Unix epoch seconds.
|
| 2. Never pass calendar/clock representations through API methods.
|
| 3. Always pass a timezone to methods that require it.
|
| 4. Counting, like time, is hard.
|
| The only contexts I see where calendar/clock representation are
| needed are:
|
| - Accept calendar/clock info from the user or external data
| sources.
|
| - Present calendar/clock info to the user or external data
| sources.
|
| - Internal "calendar aware" operations (find the Unix epoch
| second count "one month from now").
|
| These all require a timezone to be specified. They may still be a
| mess internally but their operation is unambiguous. They may use
| imperfect data types like datetime but the issues of these data
| times are confined to their internal code context.
| pepoluan wrote:
| The problem with Unix Epoch is that it began in 1970.
|
| There are applications where you need dates before 1970 and
| that starts to get hairy.
| magicalhippo wrote:
| Signed integers exist. What is hairy about using them?
| zokier wrote:
| > 1. Maximize the code context where date/time is represented
| as a simple count of Unix epoch seconds.
|
| Either your values will be different from UNIX timestamp (and
| you need to write your own conversion functions), or they are
| not simple count of seconds and will have ambiguities. Neither
| is not great situation.
| Kwpolska wrote:
| I wouldn't use an int. Some things use milliseconds since 1970
| instead of seconds to get sub-second precision without
| involving double, and you need to read the docs to understand
| what is expected. Or you can just pass a DateTime object
| standard for the language if running in a single binary, or
| something like ISO 8601 when there's a network in between.
| Databases can often store DateTime objects sanely.
| targafarian wrote:
| 64 bit unsigned integer nanoseconds gets you out to 584 years
| (that's the year 2554 if you're using the Unix epoch). That's
| good enough for me to use universally for passing times
| around in the internals of my code. User input and output are
| going to and from that representation.
|
| Half as many, of course, if you use a signed integer. If you
| don't need nanoseconds, then use microseconds and you get 292
| thousand years to work with.
|
| Integers are just a bit easier than floats for timestamps in
| my experience (e.g., comparing floats to one another is
| fraught and you'll be fighting this at every turn in your
| code).
| zokier wrote:
| tbh using signed 64 bit microseconds with ISO-8601
| 0000-01-01 as epoch has certain elegance to it and should
| cover fairly wide range of use-cases.
| Kwpolska wrote:
| You have standardized on int64 = nanoseconds. Libraries you
| use might have standardized on int64 = milliseconds, int64
| = seconds, double = seconds, or the preferred DateTime
| class/struct of your programming language -- even the C
| standard library has `struct tm` [0].
|
| If you've wrapped your int64 in some struct/class/type-
| alias-without-automatic-downcasting, it might be fine. But
| if you haven't, you might end up mixing the different
| scales, or littering the code with pointless conversions to
| and from the standard DateTime class/struct.
|
| [0] https://www.gnu.org/software/libc/manual/html_node/Brok
| en_00...
| zokier wrote:
| > Some things use milliseconds since 1970 instead of seconds
| to get sub-second precision without involving double
|
| Or be like JS which uses both floats and milliseconds as its
| date format. That to me feels like taking worst parts of
| every option. But then, making fun of JS is like shooting
| fish in the barrel
| Kwpolska wrote:
| (This is a side effect of JS not having a real integer
| type.)
| winternewt wrote:
| The problem with Unix timestamps is that it doesn't take leap
| seconds into account, which makes it ambiguous during a leap
| second and makes it messy to compute the duration between two
| timestamps.
|
| I think you're on the right track but a timestamp should only
| count the actual number of seconds that have passed, not some
| half-measure that tries to align with the rotation of the
| earth.
| guitarnick wrote:
| The author provides this example to illustrate inconsistent
| handling of non-existent datetimes: # This time
| doesn't exist on this date d = datetime(2023, 3, 26, 2,
| 30, tzinfo=paris) # No timestamp exists, so it just
| makes one up t = d.timestamp()
| datetime.fromtimestamp(t) == d # False
|
| Criticisms:
|
| 1) The example would fail just as well for any datetime with
| given tzinfo. Because fromtimestamp returns native datetimes.
|
| 2) The timestamp isn't just "made up". Its behaviour is clearly
| documented in PEP 495, as linked by the author [0]. In this case
| it consistently corresponds to datetime(2023, 3, 26, 3, 30,
| tzinfo=paris).
|
| [0] https://peps.python.org/pep-0495/
|
| Finally if we disallow creating non-existent datetimes in the
| proposed library, how do we represent the 2am in "clock changes
| forwards at 2am"? Use 3am? There are tradeoffs.
| falserum wrote:
| > how do we represent the 2am in "clock changes forwards at
| 2am".
|
| If we are willing to adjust a tiny bit, there are options:
|
| Option1: "01:59 is followed by 03:00".
|
| Option2: "clock changes forwards at 2am GMT+2" (GMT+2 is offset
| before the hour jump)
| ariebovenberg wrote:
| Author of the blog post here--
|
| 1) You're absolutely right, I'll fix this mistake in the
| example.
|
| 2) "Made up" was perhaps stirring the pot too much ;), but the
| fact remains that a timestamp is created when one essentially
| doesn't exist. That this is meticulously documented in PEP495
| doesn't change this fact.
|
| Note that PEP495 also explicitly describes some of the other
| pitfalls. Documenting a 'suprise' is not the same as
| eliminating it.
|
| > Finally if we disallow creating non-existent datetimes in the
| proposed library, how do we represent the 2am in "clock changes
| forwards at 2am"? Use 3am? There are tradeoffs.
|
| Indeed, there are always tradeoffs. Often, only time can tell
| what was the 'least bad' decision. It is telling though that
| modern datetime libraries (Noda Time, Chrono, Temporal) in
| other languages choose to not represent these 'missing' local
| times.
| dwattttt wrote:
| > Documenting a 'surprise' is not the same as eliminating it.
|
| I'm going to have to remember that one
| Hackbraten wrote:
| > only time can tell
|
| I see what you did there.
| amluto wrote:
| > # The local system timezone > LocalDateTime,
|
| Here's my feedback: one should essentially never use "local"
| time. I would even argue that the existence of systemwide local
| time is a historical mistake.
|
| Why would you ever want to use the "local" timezone of the system
| on which you are running? If the user is literally sitting in
| front of a monitor plugged in to the machine, it might be
| appropriate (but it should be a per-user setting). But if the
| user is using X forwarding or ssh or a remote desktop, you have
| no idea what timezone the user is in. (And this was common in the
| mainframe era!) If you're servicing a request over the network,
| you don't want local time.
|
| The only sort of legitimate use for "local" time I can think of
| is compatibility with libc and it's horrible time API.
|
| You can't even tell what local time means without getenv, which
| is a footgun.
|
| So maybe LocalDateTime should be called LibcCompatLocalDateTime.
| jncfhnb wrote:
| Imagine having the audacity to run software on the machine in
| front of you
| ahartmetz wrote:
| You shouldn't do that, it's bad for some business models.
| CJefferson wrote:
| What is your alternative? I suspect 99.999% of computer and
| phone users want the time of their local device. Do you want
| them all to have to explicitly set their time zone?
| adrian_b wrote:
| Since the appearance of computer networks, all operating
| systems require during installation an explicit setting of
| the time zone.
|
| This includes Windows and any operating system that chooses
| to keep the time as local time, because even those need to be
| able to convert the local time to other time zones.
|
| The users who buy a computer with a preinstalled operating
| system may not see this, but someone has explicitly set their
| time zone.
|
| Because the time zone must be set anyway, it is always better
| to use inside programs a time that is either UTC or TAI and
| convert it only for presentation purpose.
|
| For purposes like storing the time at which future events are
| scheduled, if they are established in local time, so the
| future correct conversion is yet unknown, a distinct type
| must be used and that must be a structure with 3 members,
| date, time of the day and time zone. It must not be a time
| expressed in seconds or other time units, as it may be used
| for UTC or TAI.
| CJefferson wrote:
| Windows 11 does not require setting timezone. I just did it
| in a VM to check I wasn't confused. Neither do android or
| iPhones.
| jraph wrote:
| Any Android install I did myself in VMs or on a phone
| asked me for the timezone. (lineage and derivative).
| Maybe it could guess from the GSM network though.
|
| My dumb phone also does, though in terms of UTC offset +
| DST offset.
| cm2187 wrote:
| Then how do you express that something must happen at a date in
| the future at 4pm local time? If you use UTC based on today's
| expectation of daylight savings, they may have changed by the
| time this date happens.
| amluto wrote:
| You start by figuring out what you're even trying to do.
|
| For example, I read your comment three times, and _I don't
| know what you mean by "4 pm local time"_. I don't even have
| what I would consider to be a credible guess.
| jcranmer wrote:
| Schedule an event to occur at 4pm every day, say in
| crontab.
| amluto wrote:
| Works great until you have an office in a different time
| zone.
|
| The whole concept of a computer belonging to a timezone
| works to some extent when that computer has a single
| human tenant, but it falls apart pretty quickly when the
| scope broadens.
| jcranmer wrote:
| If I'm scheduling a meeting, then yes, I am going to want
| to schedule the meeting using a specific time zone, which
| may or may not be my local time zone at that moment.
|
| But if I'm scheduling when I want my computer to reboot,
| I want that to be a fixed in time zone in my local time
| zone. And if I subsequently change my local time zone, I
| want it to remain at that time in the _new_ local time
| zone, not the old one.
|
| Yes, local time zones don't always make sense. But that
| is a far cry from saying they _never_ make sense!
| amluto wrote:
| I don't know about you, but I have computers that aren't
| in the same time zone I'm it. I probably want them to
| reboot at night where _I_ am, not where they are.
|
| But yes, "reboot a laptop at 2am" is in the general
| category of alarm clock use cases, where a time (but not
| a date!) with a separately determined time zone makes
| sense.
|
| The OP mentioned LocalDateTime, which doesn't make sense
| for this use case.
| falserum wrote:
| Instant in history -> GMT+0/UTC
|
| Future shedule for humans to coordinate (contracts,
| meetings) -> ZonedDateTime
|
| Personal alarm that should be kept in sync with the sun
| (e.g. go to sleep) -> LocalDateTime
|
| What's good example/purpose for OffsetDateTime?
| jcranmer wrote:
| OffsetDateTime would be useful for a lot of historical
| records. For example, if I'm chronicling a WW2 battle in
| the Pacific, I need to know what the local times of the
| events were, but I don't necessarily care about knowing
| the DST rules.
|
| Additionally, most date/time serialization formats (e.g.,
| RFC 822, ISO 3601) only allow you to specify up to
| OffsetDateTime and don't provide a full timezone
| reference like US/Eastern.
| zodiac wrote:
| Something like "schedule a reminder to stop using the
| computer at 10pm so I can sleep by 11pm"? Since I want to
| sleep at 11pm local time every day
| amluto wrote:
| This is about the only use case for local time that makes
| sense to me. But this is a UI issue, and it seems to me
| that that should be something that comes from a UI
| library or framework, not from libc and or /etc or
| environment variables.
|
| (I'm using "UI" broadly. A user session on a UNIXy system
| could have an associated timezone, although such a design
| should allow changing the timezone without logging on and
| off. And this "10pm local" thing is quite different from
| an ordinary zoned time in that "10pm local in 10 days"
| does not correspond to an actual known UTC time because
| the local time zone may change.)
| falserum wrote:
| 1. Contract says that goods must be delivered at 4pm.
|
| 2. Government suddenly changes the law and moves the offset
| of whole country.
|
| 3. Contract is still valid, goods have to be delivered
| still at 4pm, but with new offset (so at different utc
| time).
|
| (I guess timezoned time would still be better than local
| time, but UTC would not work here)
| amluto wrote:
| > (I guess timezoned time would still be better than
| local time, but UTC would not work here)
|
| Exactly.
| MereInterest wrote:
| You and I would like to meet at a coffee shop, on a
| specific date about six months from now, when the clock on
| the wall of the coffee shop reads "4:00 PM".
|
| * The coffee shop may not be located within UTC+00:00. If
| you show up at UTC time 16:00+00:00, I won't be there.
|
| * The coffee shop may not be located in your current time
| zone. If you show up when it is "4:00 PM" in your local
| time zone, I won't be there.
|
| * The coffee shop may be located somewhere that follows
| Daylight Saving Time. If you show up when it is "4:00 PM"
| in the current UTC offset of the coffee shop, I won't be
| there.
|
| * The coffee shop may currently be located somewhere that
| follows Daylight Saving Time, but the legislature stops
| following Daylight Saving Time between now and our meeting.
| If you show up when it is "4:00 PM" according to the
| predicted UTC offset of the coffee shop for the day of the
| meeting, rather than the actual UTC offset of the coffee
| shop for the day of the meeting, I won't be there.
|
| Saying "local time" implies all of the above. It means that
| something is being specified according to the local
| convention of timekeeping.
| amluto wrote:
| > Saying "local time" implies all of the above. It means
| that something is being specified according to the local
| convention of timekeeping.
|
| No, it really doesn't. Saying "4pm _at the coffee shop_ "
| means that, and this is a big distinction.
|
| If I'm in a different time zone two days before the
| meeting, looking at my calendar, I sure hope my calendar
| understands the difference. If I move my entire home
| outside the timezone, the appointment doesn't magically
| shift an hour as a result.
|
| And any decent time library can handle this just fine as
| long as you don't use "local" time. Just specify the
| actual timezone!
| lijok wrote:
| Unfortunately timezones have the nasty ability to change.
|
| See: https://codeblog.jonskeet.uk/2019/03/27/storing-utc-
| is-not-a...
| Hackbraten wrote:
| That's why they wrote "state the actual timezone" and not
| "state the actual UTC offset."
|
| Actual timezone would mean "2025-01-01 16:00:00
| Europe/Berlin," not just "2025-01-01 16:00:00 +01:00."
| lijok wrote:
| Even those change
| Hackbraten wrote:
| I agree. The city where the cafe is located might be
| annexed or usurped by another country, and have another
| timezone imposed on it.
|
| Handling that kind of change would be beyond impractical
| though?
| MereInterest wrote:
| > No, it really doesn't. Saying "4pm at the coffee shop"
| means that, and this is a big distinction.
|
| Some words and phrases, such as "4 pm local time",
| require some of their semantic meaning to be inferred
| from context. This doesn't mean that they are
| meaningless. It means that the relevant context must be
| provided in order to interpret the phrase. If you and I
| have been scheduling the coffee shop meetup, then the
| phrases "4pm", "4pm local time", and "4pm as measured by
| the applicable standards in the area of the coffee shop
| on the day of the meeting" are all equivalent.
|
| > Just specify the actual timezone!
|
| This is the problem that I pointed out. The appropriate
| context for "4pm local time" is "at the coffee shop".
| However, most calendar programs will only let you provide
| context such as "Eastern Standard Time". This is the
| wrong context altogether, and choosing it assumes that
| the area will have extended the duration of DST between
| time of scheduling and time of meeting.
| amluto wrote:
| > Some words and phrases, such as "4 pm local time",
| require some of their semantic meaning to be inferred
| from context. This doesn't mean that they are
| meaningless. It means that the relevant context must be
| provided in order to interpret the phrase.
|
| This is not what the proposed LocalDateTime would do.
| This is a "naive" time, and those are only really useful
| for calendar math or when you want to apply a concrete
| timezone.
| lelandbatey wrote:
| Nah, you need a location; it's right there in the name
| "local" which implies time at a place. Saying "local
| time" alone doesn't always give sufficient detail in your
| example, because you want the device to be able to change
| it's location (its timekeeping context) while maintaining
| awareness of the difference between the device's local
| time and the local time of the thing you've recorded.
|
| "Local time" is basically never "good enough", actually.
| At a minimum, a time as you've described should include:
|
| - a location where you're tracking the local time
|
| - the destination local time at that location, from which
| you can derive most of the timekeeping context (time
| zone), usually
|
| - the current time of the thing doing the computation, in
| order to accurately compare device to destination time
|
| You then use those to get a UTC time that you can use to
| do things like a countdown. Or you can just make
| assumptions about those things and accept they'll be
| wrong; that's allowed but it's not usually what users
| want given we KNOW how to give them what they want.
| joshribakoff wrote:
| > don't know what you mean by "4 pm local time"
|
| It means "whenever it is 4pm wherever you are".
|
| As in, i want my alarm to go off at 4pm everyday regardless
| of whether i am traveling.
|
| Python calls it a "naive" time, it has no timezone or
| offset. Its just a data structure that says "4pm" rather
| than "4pm eastern time"
|
| An example ive used is when a user had to specify schedules
| for a digital menu board system. When the user says
| breakfast ends at 10am, that means wherever the digital
| menu board is located, not "10am where i was when i set the
| schedule". It also doesn't mean "end breakfast an hour
| early/late when daylight savings occurs"
|
| It makes sense in any context where you need to do
| something at a certain _time_ each day, and you
| specifically don't mean "do this thing exactly every 24hrs"
| amluto wrote:
| > As in, i want my alarm to go off at 4pm everyday
| regardless of whether i am traveling.
|
| I think this may be the exception that proves the rule.
| Sure, this is a valid, if specialized, semantic use case.
| But you only barely need a LocalDateTime type for this.
| If you stick that 4pm into a local database, you want to
| make sure that everything, including the database, knows
| you're storing hours, minutes, and seconds relative to an
| unspecified "local time".
|
| So IMO this is more of an UnspecifiedZoneTime than a
| LocalDateTime. (Also, it has no date portion.) It makes
| no sense to compare it to a UTC or other date+time.
| coldtea wrote:
| > _So IMO this is more of an UnspecifiedZoneTime than a
| LocalDateTime. (Also, it has no date portion._
|
| It very well could also have a date portion.
|
| E.g. "I want this alarm/cleanup/whatever to happen on 9am
| on the 1st Monday of each month" based on the timezone
| the user happens to be at that time - i.e. not tied to
| any specific timezone.
| amluto wrote:
| That's still a different thing: it's a recurring event,
| which is an almost entirely separate can of worms from a
| datetime.
|
| And a recurrence library probably wants to avoid libc-
| style "localtime" here -- plumbing the concept of users
| in places to it seems awkward. A genuinely naive time may
| be better -- the interface could be "calculate the next
| recurrence of this recurring event in this timezone," not
| "here's a recurring event with associated 'local' time --
| calculate the next occurrence given my /etc/localtime and
| TZ environment variable".
| MereInterest wrote:
| Because the local time is the thing that you have immediate
| access to, with no additional conversions applied. Every
| conversion requires an external source of information.
| Converting from local time to time zone requires measuring the
| system clock drift. Converting from time zone to UTC requires
| knowing which time zone you are in. Converting from UNIX
| timestamp to UTC requires measurements of the earth's rotation
| to know how many leap seconds to add.
|
| I'd propose going in the opposite direction: All timestamps
| should indicate which clock they were produced from. There is
| no such thing as a global clock, only conversions between your
| local clock and the UTC clock.
| amluto wrote:
| > Because the local time is the thing that you have immediate
| access to, with no additional conversions applied.
|
| Only in the sense that BIOS systems historically programmed
| non-DST-aware local time into the RTC.
|
| Other than that, you actually have no-conversion-needed
| access to UTC or something UTC-like. Linux mostly tracks the
| conversion from the hardware clock (TSC, for example) to UTC.
| NTP gives UTC, and GPS gives something that is a lot closer
| to UTC than to local time.
| MereInterest wrote:
| > Only in the sense that BIOS systems historically
| programmed non-DST-aware local time into the RTC
|
| I would love to hear proposals on how, at a hardware level,
| you would track Daylight Saving Time. Not just how a
| Daylight Saving Time correction would be applied, but also
| how the hardware would receive updates to dates at which
| DST is observed, within which geographic borders, and how
| to handle conflicting updates for locations where the
| geographic borders are contested.
|
| > you actually have no-conversion-needed access to UTC or
| something UTC-like
|
| No, you don't. You may have access to an API that has
| already applied conversions when by the time a result it
| returned to you, but that is significantly different than
| having direct access to a universal clock. (And assumes
| that such a universal clock even exists.)
|
| Your clock is a hardware-based oscillator, with a counter
| for the integer number of oscillations that has occurred.
| You do not have a UTC clock. You have an experimentally-
| determined and infrequently-updated conversion between your
| clock and the UTC clock.
|
| > NTP gives UTC
|
| No, it doesn't. NTP gives an approximation to UTC, and is
| only available when you have a network connection
| available. The accuracy of that approximation depends on
| quality of that network connection, and whether it has
| asymmetric delays.
|
| > GPS gives something that is a lot closer to UTC than to
| local time
|
| While true, this is only available for devices that have
| GPS capabilities, in a location without sharp elevation
| changes that would delay GPS signals.
|
| Everything has a conversion factor. Everything. If you
| record the original measurement and the conversion factor
| used, you can recover from errors in the conversion. If you
| only record the measurement after conversion factors have
| been applied, you cannot.
| o11c wrote:
| >I would love to hear proposals on how, at a hardware
| level, you would track Daylight Saving Time.
|
| Traditionally the BIOS ignores it, and Windows on reboot
| detects if the clock needs to be adjusted according to
| the latest rules that it knows about. This failed quite
| often in practice.
|
| Linux systems using tzdata are much more reliable
| (Windows is _still_ very buggy with historical data), but
| UTC time support on non-Linux systems was flaky for a
| long time, leading to bad API design in languages
| historically.
|
| `timegm` despite being a critical and irreplaceable
| function is still not in POSIX, for example (though it is
| widely supported in practice)
| bongodongobob wrote:
| Settle down. Most people aren't running or accessing things
| globally.
|
| Local time is perfectly acceptable for a list of obvious
| reasons.
| make3 wrote:
| ... a huge fraction of Python code runs on servers everywhere
| azinman2 wrote:
| And a bunch isn't.
|
| Time is hard no matter what. Having a variety of types and
| options is a good thing, as long as everything is explicit
| and convertible.
|
| The wonder if obj-c/swift do well here with NSDate/Date
| representing a moment in time separate from any
| calendar/time zone... although the name is confusing for
| what it represents.
| bongodongobob wrote:
| Which is exactly what you use time zones for.
| coldtea wrote:
| > _But if the user is using X forwarding or ssh or a remote
| desktop, you have no idea what timezone the user is in._
|
| Perhaps I don't care, and I want to present things from the
| point of view of the server's location?
| falserum wrote:
| Across all languages, what are the best libraries, with least
| amount if pitfalls?
|
| (java.time seems good, anything even better elsewhere?)
| assbuttbuttass wrote:
| I really like Go's "time" package. The only pitfall I can think
| of is that it doesn't really handle overflow at all.
|
| https://pkg.go.dev/time
| rad_gruchalski wrote:
| And date time parsing sucks with its predefined mask
| elements.
| pgwhalen wrote:
| Go's time package seems like one of the bad ones, since it
| makes the classic mistake of combining zoned and naive times
| into the same type (arguably it doesn't support naive times
| at all).
| zokier wrote:
| java.time doesn't handle leap seconds very well :(
| jshell> ChronoUnit.SECONDS.between(Instant.parse("1974-01-01T00
| :00:00Z"), Instant.parse("2024-01-01T00:00:00Z")) % 86400
| $22 ==> 0
|
| astropy gets you the correct answer: >>> from
| astropy.time import Time >>>
| (Time('2024-01-01T00:00:00Z') -
| Time('1974-01-01T00:00:00Z')).sec % 86400 24.0
|
| of course astropy is bit specialized and not really something
| I'd use as general purpose date/time library. But this is one
| good litmus test for datetimes, feel free to try it with your
| favorites.
| zokier wrote:
| I tested this with the library from TFA, and it fails too :(
| >>> from whenever import UTCDateTime >>>
| (UTCDateTime(2024, 1, 1) - UTCDateTime(1974, 1,
| 1)).total_seconds() % 86400 0.0
| DandyDev wrote:
| What does TFA mean here? Trifluoroacetic Acid?
| zokier wrote:
| The Fucking/Fine/... Article
| ariebovenberg wrote:
| Author here--
|
| This is a conscious decision, I'll explicitly address this
| in the FAQ soon.
|
| As I commented elsewhere, this is completely in line with
| industry standards (iCal, Unix time) and other modern
| libraries. I don't (yet) see any reason to deviate.
| ariebovenberg wrote:
| I wouldn't consider 'not handling leap seconds' as a pitfall.
| If you need to account for leap seconds, you're probably not
| going to use a general-purpose datetime library.
|
| Unix time ignores leap seconds, so does RFC 5545 (iCal), and
| most modern libraries (Chrono, Temporal, NodaTime).
| zokier wrote:
| You need to account for leap seconds literally any time you
| are mixing durations/timedeltas with instants/datetimes,
| otherwise you will get wrong answer. Also needed whenever
| you parse ISO8601/RFC3339 style strings. For example this
| (correct) timestamp fails to parse: >>>
| UTCDateTime.from_rfc3339('1998-12-31T23:59:60Z')
| Traceback (most recent call last): File
| "<stdin>", line 1, in <module> File
| "/tmp/tmp.4kBsq6IYyF/venv/lib/python3.11/site-
| packages/whenever/__init__.py", line 797, in from_rfc3339
| return cls._from_py_unchecked(_parse_utc_rfc3339(s))
| ^^^^^^^^^^^^^^^^^^^^^ File
| "/tmp/tmp.4kBsq6IYyF/venv/lib/python3.11/site-
| packages/whenever/__init__.py", line 2420, in
| _parse_utc_rfc3339 return
| _fromisoformat(s.upper())
| ^^^^^^^^^^^^^^^^^^^^^^^^^ ValueError: second must
| be in 0..59
| zokier wrote:
| I do point out that if your internal representation would
| be simple naive epoch+duration(/timedelta) then you don't
| need any special accounting for leap seconds when doing
| time arithmetic like this. The problem arises only if you
| are trying to be somehow clever with your internal
| representation (and failing).
| cutler wrote:
| Why doesn't someone just port Luxon from Javascript which seems
| to have solved most of these problems. Why are we still
| reinventing the wheel in 2024?
| darkwater wrote:
| How do you solve "give me the epoch for 2:30AM" the night
| daylight changes?
| assbuttbuttass wrote:
| from whenever import ( # For the "UTC everywhere"
| case UTCDateTime, # Simple localization
| sans DST OffsetDateTime, # Full-featured
| IANA timezones ZonedDateTime, # The local
| system timezone LocalDateTime, # Detached
| from any timezones NaiveDateTime, )
|
| Why so many different types? Doesn't ZonedDateTime cover all of
| the other use cases?
| jcranmer wrote:
| No.
|
| First, there's a fundamental difference between naive datetimes
| and timezone-aware datetimes. Try to combine them in one
| representation, and you are guaranteed to end up with a broken
| datetime library that spawns endless blog posts attacking it.
|
| Second, there's a more subtle distinction between a datetime
| which has a known timezone and one which merely has a known
| instantaneous offset. A timezone is essentially a naive date
| time -> tzoffset, albeit one whose values for future dates
| tends to be surprisingly unrobust. Among the offset-only date
| times, UTC is a very common case that deserves its own alias.
|
| Finally, separating the local-tz datetime from other tz-aware
| datetimes can be very useful, for if the user changes their
| local timezone, then a local-tz datetime usually wants to
| switch the timezone it is using as well.
| assbuttbuttass wrote:
| IMO, any program using naive timestamps is probably doing
| something wrong. Same with fixed offsets (except UTC
| obviously).
|
| For local time zone, how often does the local time zone
| change, and you want to modify all timestamps in the system?
| That actually seems like a huge pitfall
| Hackbraten wrote:
| > IMO, any program using naive timestamps is probably doing
| something wrong.
|
| Here's an example where you'd legitimately want to process
| and store a naive (wall clock) time object:
|
| https://news.ycombinator.com/item?id=39419845
| demondemidi wrote:
| Excellent research. Clearly explains the problem, shows how the
| industry addresses it, and provides a solution. You're hired!
|
| And I never want to write time code ever again. I wonder how many
| DST bugs I have in the wild after 40 years of coding...
| rhelz wrote:
| Really, there is no good technical solution for this, because in
| part, it is not even a technical problem. The laws about
| timezones, etc, very from country to country, from year to year
| etc. And some parts of the world are claimed by two or more
| different countries, each one of which has an opinion about what
| time it should be there.
|
| One might say "well lets skip all this political b.s. and do
| everything in UTC" but really, viewing time as running according
| to a single universal clock is a relic of Newtonian Physics.
| There is no single, global answer to what time it is.
|
| And we are getting to the point where special relativity is
| kicking in. High-frequency trading is already in the
| 50-microsecond range. Not so far off from the relativistic time
| dilations experienced even in flying in a jet airplane: flying
| around the world eastward results in about a 180 ns slowdown,
| flying around the world westward give about an 80 ns speedup.
|
| Even concepts like simultaneity become relative at the speeds
| which we will soon be computing at. Imagine two corporate raiders
| flying in their private jets, biding on the same block of
| controlling stock: who bought it first would be different
| depending upon which frame of reference you were in.
|
| If you have to write programs for when time really matters (as I
| did writing stock-trading software) my advice is to just accept
| the realities, and _glory_ in the complexities. Being able to
| write software which can keep track of time, and obey all the
| rules and regulations of various exchanges and countries, is a
| competitive advantage for you.
| make3 wrote:
| I think they could improve the quality of life of the users a
| lot without addressing the "high frequency trading inside a
| jetplane flying against the rotation of the earth" edge case
| lol
|
| (not to diminish having an accurate understanding of time ofc,
| GPS is the main example of an application, etc)
| heavenlyblue wrote:
| > Imagine two corporate raiders flying in their private jets,
| biding on the same block of controlling stock: who bought it
| first would be different depending upon which frame of
| reference you were in.
|
| Market maker decides that, not them.
| svilen_dobrev wrote:
| what special font/characters is this article using?
|
| in that scorecard table i got all same-looking boxes. yes/no Y/N
| T/F x/o would have been much more readable. And also in all
| bullet points also see boxes..
| ariebovenberg wrote:
| Author of the post here--
|
| They are emoji. I've added plain text to the table now, so it
| is readable without them.
|
| edit: I've used a workaround to display emoji as images now
| the__alchemist wrote:
| I will state this briefly as a tangent. This article mentions
| third party libraries; the top are Arrow and Pendulum. These are
| a wellspring of subtle bugs due to a decision they made that I
| suspect is based off the Javascript library Moment: They conflate
| Dates, Times, and Datetimes, as a single type. If you are using
| their Arrow etc type to represent a Date or Time, the resulting
| code will be fragile, and fail in surprising ways. Additionally,
| if my function signature accepts a time or date, you have no way
| of validating the parameter, and can do things like ask for
| "seconds" on a date. The moment/arrow/pendulum community
| advocates that the concept of a date or time individually is
| meaningless; I disagree. It is application specific, and there
| are applications that call for any of the 3 variants.
|
| Python's builtin `datetime` library has flaws as pointed out in
| the article, but is safer than Arrow or Moment. I think Rust's
| Chrono lib is the best datetime library I've seen, in terms of
| not surprising you with errors.
| ariebovenberg wrote:
| Author of the article here--
|
| I actually did start out basically porting Chrono to Python,
| but decided against it for two reasons:
|
| 1. While I really liked handling local/UTC conversions with the
| "LocalResult" enum in Rust, it translated poorly to Python.
| Somewhere along the line you'd need to transition from Rust's
| "errors as values" to Python's exception-oriented world. I
| actually wish Python would go Rust's way, but I'm not going to
| swim against the current here...
|
| 2. Most modern libraries in other languages choose to
| disambiguate local datetimes. I felt it was safer to go the
| road more traveled here.
| timmaxw wrote:
| Well done. It's horrifying that the Python standard library "+/-"
| operators don't handle DST correctly.
| aimor wrote:
| I don't really understand why OffsetDateTime doesn't allow adding
| time deltas (reason provided is that it might have it's offset
| change relative to UTC). Is the offset supposed to be relative to
| UTC or is it supposed to be relative to ANY TimeZone (including
| those with time changes)? I thought it was relative to UTC, but
| even the other way I just don't understand why that's a problem.
| I might be making some assumptions about the behavior I expect.
| ariebovenberg wrote:
| Author here. "Offset" is indeed relative to UTC. The problem
| with allowing adding timedeltas is that you give users the
| impression they are doing valid arithmetic, while they may not
| be.
|
| Example: You receive a timestamp of an event in Paris:
| 2024-03-31 01:00:00+01:00. If you allow addition, you may be
| tempted to think you can "just add three hours" and get
| 04:00:00+01:00. However, because Paris has a DST transition,
| you'd actually want to have 05:00:00+02:00
|
| In the end, the choice is between "users know what they're
| doing, just let them" and "users need to be prevented from
| making common mistakes". I choose the latter.
|
| Note that also NodaTime doesn't support arithmetic on their
| OffsetDateTime, for the same reason.
| aimor wrote:
| Thank you for answering, I understand the dilemma now and why
| you chose to force users to explicitly convert to UTC.
|
| When I see the example my thinking is: Well, if you want a
| time offset relative to Europe/Paris then define your time
| zone that way instead. Of course I haven't seen other
| libraries give that option and maybe for good reasons.
| ariebovenberg wrote:
| Yeah--the perfect solution doesn't exist. The problem is
| that fixed-offset datetimes are very common since RFC3339
| and ISO8061 don't support timezone names. The result is
| that you often end up with fixed-offset datetimes that
| should instead be linked to a IANA tz ID.
| magicalhippo wrote:
| > If you allow addition, you may be tempted to think you can
| "just add three hours" and get 04:00:00+01:00. However,
| because Paris has a DST transition, you'd actually want to
| have 05:00:00+02:00
|
| Since you know the offset then adding three hours should be
| unambiguous no? Convert to UTC, add three hours, convert back
| to target timezone, which would net you the correct
| 05:00:00+02:00.
| Hackbraten wrote:
| > Convert to UTC, add three hours, convert back to target
| timezone
|
| What if you don't even know the target offset?
|
| You may happen to know it for an event three hours in the
| future, but what if you're adding three months? With that
| time frame, the relevant laws even may not have been passed
| yet.
___________________________________________________________________
(page generated 2024-02-18 23:01 UTC)