[HN Gopher] Store epoch times as 64-bit floats
___________________________________________________________________
Store epoch times as 64-bit floats
Author : iudqnolq
Score : 37 points
Date : 2023-06-09 16:56 UTC (6 hours ago)
(HTM) web link (www.leebutterman.com)
(TXT) w3m dump (www.leebutterman.com)
| ClumsyPilot wrote:
| Use 128 bit integer in femptoseconds, with epoch being the
| begining of the universe. And it should be signed, because what
| if there was a previous universe.
|
| Jokes aside, Unix Epoch shouls be replaced by Tai time on every
| computer.
|
| https://en.m.wikipedia.org/wiki/International_Atomic_Time
| mihaic wrote:
| The person that pushed for the leap second should be flogged
| very hard!
| halosghost wrote:
| If we're going to go this far, why not go all the way? [1]
|
| I.e., use a 536-bit unsigned integer to store Planck time with
| one Planck time following the big bang as the epoch, and use
| the top three bits as a tetration height instead of an
| exponent, and you can get _real_ far.
|
| All the best,
|
| [1] https://halosgho.st/blog/planck/
| cyann wrote:
| At first glance I thought you were talking about "Thai" time,
| which is a surprising calendar when you have dates with years
| in the 2500 range. Looking at you Symantec AntiVirus, storing
| AV definitions date using the user's locale...
|
| https://en.wikipedia.org/wiki/Buddhist_calendar
| well_actulily wrote:
| That gets us to ~5.392 * 10^15 years after the Big Bang, which
| doesn't nearly get you to the heat death of the universe ~1.7 *
| 10^106 years from now, but is probably good enough.
| jjoonathan wrote:
| Black hole farmers hate this.
| [deleted]
| evil-olive wrote:
| yeah...no. don't use floats for this.
|
| int64 "nanoseconds since the epoch" seems to be emerging as the
| de facto standard.
|
| used in Golang [0] and adopted by Python starting in 3.7 [1]
|
| that PEP also spells out some of the problems with using a
| float64:
|
| > The Python time.time() function returns the current time as a
| floating-point number which is usually a 64-bit binary floating-
| point number (in the IEEE 754 format).
|
| > The problem is that the float type starts to lose nanoseconds
| after 104 days. Converting from nanoseconds (int) to seconds
| (float) and then back to nanoseconds (int) to check if
| conversions lose precision:
|
| 0: https://pkg.go.dev/time#Time
|
| 1: https://peps.python.org/pep-0564/
| snailtrail wrote:
| Posting blog like this how you get rejected from your dream job.
| cassepipe wrote:
| Care to elaborate ?
| KaiserPro wrote:
| I think they are implying that storing time as a float is
| riddled with problems (like arithmetic needs to think about
| hardware bugs and implementations)
|
| The problem being that OP kinda glosses over them.
| lordnacho wrote:
| Think he's saying the advice in the blog post is bad.
| bjornsing wrote:
| > Even through the 90s, long after many system calls became
| formalized, floating point math was much more expensive than
| integer math.
|
| IIRC the main reason floating point numbers weren't used in
| system calls back in the 90-ties was that kernel code couldn't
| use the CPUs floating point registers, or they would have to be
| saved at the entry point of each system call and restored before
| returning to user space. This was deemed too expensive.
| johnea wrote:
| The article fails to mention the earliest standard addressing
| this issue, DJB's taitime:
|
| https://cr.yp.to/proto/utctai.html
| wood_spirit wrote:
| UUIDv7 is a good way to store time in UUIDs. See
| https://www.ietf.org/archive/id/draft-peabody-dispatch-new-u...
| theandrewbailey wrote:
| This _would_ have been a good April Fools joke.
| pphysch wrote:
| They meant to post it at 04-01, but posted it at 02-01 instead
| to prove that precise timing doesn't matter
| johnklos wrote:
| Nah. I think we should move to 500 bit times. That'll give us the
| Planck time resolution from the Big Bang through the heat death
| of the universe.
| loxias wrote:
| I might be missing something but is this actually arguing for
| blindly and naively taking time-since-epoch as fp value and then
| think of the smallest (which would change over time!) possible
| epsilon??
|
| Disappointing to me how visually appealing this site is for how
| useless the words/idea themselves are.
| Szpadel wrote:
| on February 25, 1991 American Patriot Missile battery in Dhara
| failed to track incoming missile because they used floating point
| time instead of integers. 28 dead 100 injured
|
| time precision was decreasing with time since system was booted
| and after about 100h of operation this accumulated enough to
| system believe that incoming missile is already unreachable using
| existing calculations and caused it to not react at all
|
| I think that is great example why changing precision depending on
| time might be very bad idea.
|
| imagine that afer Y2038 that slight drop in precision would cause
| some critical system to fail in unexpected way
| CamperBob2 wrote:
| The point is, double precision fixes this for timescales within
| reach of human existence.
|
| A 24 bit mantissa isn't enough for anybody. A 53-bit mantissa
| is enough unless you need +/- one nanosecond precision at 100
| days. If you need _that_ -- and some people do -- your
| application is too exotic to influence the standards that apply
| to everyone else.
| deafpolygon wrote:
| Real chad developers use string.
| not2b wrote:
| 64 bit integers are just as cheap, and that's where everyone is
| going.
| shpx wrote:
| Let's store time as integer Planck times since the Big Bang. 256
| bits ought to be enough for anybody.
| gumby wrote:
| From the post:
|
| > Why am I not using float64 time already??? [fp used ot be slow
| and was not standardized, but it got faster and after a while you
| could count on having IEEE-754]
|
| This is not the reason.
|
| Until IEEE-754 fp was _terrible_. As in buggy terrible. It took
| years of pain to learn how to get it right. And you can see this
| in the standard: it 's a very short standard, super easy to
| understand, trivial, really...if you ignore the _enormous_ amount
| explanations, corner cases, special cases, surprising symmetries,
| and other copious testament to the well-meaning Mistakes that
| Were Made in the years before. Intel provided a ton of trade
| secret understanding from the mistakes in the 8087, and UCB maths
| prof was intimately involved. It 's an incredible achievement.
|
| The history of the standard is quite exciting but has been
| covered on HN several times before.
| sposeray wrote:
| [dead]
| mungoman2 wrote:
| Yeah this doesn't hold water. If you need the precision, float
| isn't suitable since he precision drops of. If you don't need the
| precision, no need to use float.
|
| Just use int64.
| Gibbon1 wrote:
| As an aside dang would probably ban my account if I let rip
| about what I think about hardware engineers designing real time
| clocks that count time in HH:MM:SS + day/month/two digit year.
| And 100 or 128 hz subseconds.
|
| But yeah started using int64 30 years ago and all the
| fuckedness went away.
| marcosdumay wrote:
| Yeah, with 64 bits we can go into 1/32768 of second precision
| and still not get any problem.
| fredoralive wrote:
| I suspect the hardware engineer would explain the great
| favour he's doing you by handling all the handling of all the
| weird splits between things like hours, minutes and days for
| you in hardware logic, rather than you having to waste
| previous cycles of your MCU's core to do that sort of thing.
| But it is indeed a real pain when you want dates in more
| abstract than human forms...
| JohnFen wrote:
| Yeah, that hardware engineer isn't really doing anyone any
| favors.
| dragontamer wrote:
| > int64
|
| I'll go one step further. Time should be represented as
| unsigned integers, while time-differences should be represented
| as signed integers.
|
| "10 minutes ago" is int64. June 9th 2023 should be unsigned
| integer64.
| NoMoreNicksLeft wrote:
| There are reasons to want pre-1070 dates. I need signed.
| layer8 wrote:
| If you use signed int64 to represent differences, then in
| order to be able to represent all possible differences,
| timestamps need to be restricted to a 63-bit range. Might as
| well use a signed int64 restricted to nonnegative numbers as
| an uint63, or restricted to 62 bits of each negative and
| nonnegative numbers as an int63.
|
| Alternatively, you'd need an int65.
| dontlaugh wrote:
| That doesn't work, since it can't represent dates before the
| epoch.
| Kilenaitor wrote:
| Wait if you just need dates, why not store dates?
|
| I was under impression unixtime is best for timestamps. But
| no reason to store something like a birthday using them;
| just do a string YYYY-MM-DD?
| NoMoreNicksLeft wrote:
| One might want to store legislation as a git repo. But to
| do that, you'd need pre-1970 timestamps. Git doesn't use
| string dates.
| dheera wrote:
| For lots of use cases you want to record precise events
| in a universal sense, and be able to add and subtract
| them or compare whether one timestamp is the same exact
| time as another timestamp regardless of timezones.
|
| UNIX timestamps have no timezones. They are just the
| number of actual seconds that have elapsed since a
| certain commonly-agreed time. You measure the number of
| seconds between two events just by subtracting their
| timestamps arithmetically, and don't need to do any
| parsing or interpreting of timezones. You can just say
| "this hacker in Russia breached the firewall 35 seconds
| after this dev committed code in the US" just by
| subtracting two numbers.
|
| The time an airplane takes off, the time your firewall
| detected a security event, the time a cross-timezone
| meeting starts, these are all good to store as UNIX
| timestamps.
|
| Birthdays are different because they are inherently
| imprecise; humans tend to celebrate their birthdays when
| it's a certain day of the month in their local time zone,
| with little to no regard for the time zone they were born
| in. Someone born in Singapore (UTC+8) and lives in Alaska
| (UTC-8) might still celebrate their 30th birthday on the
| month and day of their birthday in Alaska time even
| though their body might need another 16 hours before it
| has technically experienced 30 years of life.
| zokier wrote:
| > UNIX timestamps have no timezones. They are just the
| number of actual seconds that have elapsed since a
| certain commonly-agreed time. You measure the number of
| seconds between two events just by subtracting their
| timestamps arithmetically, and don't need to do any
| parsing or interpreting of timezones
|
| If only. UNIX mishandling of leap seconds makes this all
| a illusion, in reality you need to have logic to handle
| them
| JohnFen wrote:
| I don't think that UNIX mishandles leap seconds. It
| properly ignores them because they aren't relevant to it.
| The UNIX timestamp is literally the count of units past
| the epoch. The existence of leap seconds don't affect
| that count, it only affects the timekeeping of a
| completely different scheme.
|
| So yes, you have to handle leap seconds when you're
| converting from the timestamp to/from the time scheme you
| are interested in, just as you have to handle leap years,
| etc. The UNIX timestamp is just a different way of
| marking time, and is blessed with not worrying at all
| about how that time relates to the actual motion of the
| Earth. That makes it easy and consistent for common
| operations. It is not mishandling anything. It just can't
| shield you from the complexity of the forms of
| timekeeping that make it a point to be synchronized with
| the Earth.
| zokier wrote:
| > The UNIX timestamp is literally the count of seconds
| past the epoch
|
| Nope. UNIX timestamp is days since epoch x 86400 +
| seconds since midnight. And that is the crux of the
| issue.
| JohnFen wrote:
| That's not true. Unix time is specifically defined as "A
| value that approximates the number of seconds that have
| elapsed since the Epoch."
|
| Are you perhaps confusing that with the standard
| algorithms for converting between Unix time and "human"
| time? Because what you say isn't wrong for that case, and
| this is a classic trap for new players:
|
| "A Coordinated Universal Time name (specified in terms of
| seconds (tm_sec), minutes (tm_min), hours (tm_hour), days
| since January 1 of the year (tm_yday), and calendar year
| minus 1900 (tm_year)) is related to a time represented as
| seconds since the Epoch, according to the expression
| below.
|
| If the year is <1970 or the value is negative, the
| relationship is undefined. If the year is >=1970 and the
| value is non-negative, the value is related to a
| Coordinated Universal Time name according to the
| C-language expression, where tm_sec, tm_min, tm_hour,
| tm_yday, and tm_year are all integer types:
|
| tm_sec + tm_min _60 + tm_hour_ 3600 + tm_yday _86400 +
| (tm_year-70)_ 31536000 + ((tm_year-69)/4) _86400 -
| ((tm_year-1) /100)_86400 + ((tm_year+299)/400)*86400"
|
| (Quotes from https://pubs.opengroup.org/onlinepubs/969991
| 9799/basedefs/V1...)
| NovemberWhiskey wrote:
| Do you think UNIX epoch offset is different for "the
| second before the leap second", "the leap second" and the
| "second after the leap second"? If so, how? If not, then
| we can't rely on differences between UNIX epoch offsets
| for measuring intervals can we?
|
| Moreover, if the units you are referring to are "seconds"
| then the statement "[t]he UNIX timestamp is literally the
| count of units past the epoch" is also wrong. If not,
| what are the units here?
| toast0 wrote:
| I think you two are talking past each other.
|
| > That's not true. Unix time is specifically defined as
| "A value that approximates the number of seconds that
| have elapsed since the Epoch."
|
| It's missing 27 seconds that elapsed since the epoch.
| (Plus whatever happened with rubber seconds before 1972).
| Maybe that fits within the definition of approximate.
|
| But if you need to record events that happen on a leap
| second or the second before a leap second, unixtime is
| unhelpful.
|
| If you want to record a duration that includes a
| leapseconds, unixtime is unhelpful, although if the
| duration is long enough, the difference may be of no
| consequence.
| dontlaugh wrote:
| That's not the point. How do you store a moment before
| 1970 if your timestamp is an unsigned integer?
| WalterBright wrote:
| Because of daylight savings time and time zones.
| dragontamer wrote:
| Postgres's time handling types are elaborate, and
| probably the best I've personally worked with.
|
| Timestamp with TimeZone (including DST + timezone offset)
| is a different type from timestamp without TimeZone (ie:
| normalized to GMT). Timestamp - Timestamp results in
| "interval".
|
| Perhaps my method I was discussing doesn't work. But the
| overall idea that "Timestamp" needs to be a different
| type than "Interval", is pretty key IMO to handling
| date/times appropriately.
| hfkwer wrote:
| Same in C#. DateTime, DateTimeOffset, TimeSpan.
| WalterBright wrote:
| Everybody has this painful urge to reinvent the trivial
| in an incompatible way.
| WalterBright wrote:
| > is pretty key IMO to
|
| A time_t can handle intervals just fine.
| time_t interval = t1 - t0;
| dragontamer wrote:
| time_t will not handle daylight savings time conversions
| like Postgresql does.
|
| Unambiguously resolving "Timestamp at Eastern Time -
| Timestamp at Arizona Time" is not as simple as you might
| think. Postgresql handles this case.
|
| Trying to make "time_t" handle all cases is a mistake. We
| need like 3 to 4 types at a minimum to handle all of the
| edge cases that occur in the real world.
| duskwuff wrote:
| > Time should be represented as unsigned integers
|
| Why artificially limit yourself to only representing times
| after the epoch? It's not like an extra bit of range is going
| to make or break anything.
| uoaei wrote:
| Adding one extra bit doubles the time horizon...
| zamadatix wrote:
| Sure but 1970 (or 1899 or 1582 or whatever) is a hell of
| a lot closer than 2^63 or what have you so why double the
| end already ridiculously far away instead of the one
| extremely close?
| uoaei wrote:
| i32 is used for Unix time, resulting in overflow sometime
| in 2038. If that was changed to u32, overflow would
| happen instead in 2106. 15 years is close enough that
| such a change would actually "make or break" timekeeping
| on human timescales.
|
| Using i64 for nanoseconds with the same origin
| (1970-01-01), we could cover roughly 1677-2262. Using u64
| for nanoseconds we could cover roughly 1970-2554, or more
| generally about 585 years of timestamps from wherever a
| new proposed origin would be. Likely we don't need
| nanoscale precision for times far enough in the past, so
| I guess "far enough" is the definition worth arguing over
| here.
| Tuna-Fish wrote:
| No, because times before 1970 exist. With signed timestamps,
| you have to figure out weird ways to represent when you need
| to talk about times in the past, with signed 64-bit
| timestamps you can represent any moment since well before big
| bang.
| booi wrote:
| > times before 1970 exist
|
| mmm I don't think that's true.
|
| - your computer
| benmmurphy wrote:
| swift has TimeInterval which is a double and can hold an epoch
| offset.
| https://developer.apple.com/documentation/foundation/timeint...
| happytoexplain wrote:
| Yes, but it's just a typealias intended to represent a span of
| time in APIs. As the docs say, "On its own, a time interval
| does not specify a unique point in time, or even a span between
| specific times."
| Someone wrote:
| That type wasn't invented by the swift standard library
| writers; it's a 1:1 copy of Objective-C's NSTimeInterval (https
| ://developer.apple.com/documentation/foundation/nstimei...), so
| it's not necessarily what a modern language designer would
| pick.
| jbay808 wrote:
| I strongly disagree with this. It only makes sense to use floats
| if you want time to have increasing precision the closer you get
| to 1970. From a symmetry group perspective, that makes no sense;
| the amount you care about the incremental precision of time
| measurement is invariant with respect to the present date.
| (Translation invariance). This means you want uniformly
| distributed time samples, hence an integer format.
|
| If you think microsecond precision is good enough for anybody,
| then you could consider using an integer where the LSB represents
| 1 microsecond. But don't use floating point time, unless your
| specific application cares more about time scale invariance than
| time translation invariance.
| dheera wrote:
| You can even go to nanosecond. The largest uint64 is
| 18446744073709551615
|
| and epoch time now in nanoseconds is less than a tenth of that
| i.e. 1686340765688397824
|
| You can go for another 500+ years safely on nanoseconds-as-
| uint64.
|
| I fully disagree with using float64. It wastes bits on an
| exponent and you cannot safely use == to compare timestamps
| that have math done on them or have been converted to/from a
| string (including e.g. sent as part of a JSON).
| johnea wrote:
| The article fails to mention the earliest standard addressing
| this issue, DJB's taitime:
|
| https://cr.yp.to/proto/utctai.html
| deadmutex wrote:
| int64 to represent times before 1970, and to make it easier
| to do time diffs (since subtracting two uint64s in C++ can
| cause overflow)
| lordnacho wrote:
| Me too. I thought it was normal already to just store the
| unix nanos. At least that's what I do and it seems to work.
| Yes I have stuff with nano granularity.
|
| You don't want to get into floating point, it gets all fuzzy
| and then when you need to change it, there's a bunch of code
| to change.
| binkHN wrote:
| FWIW, Java does this with a long and an int:
| https://docs.oracle.com/javase/8/docs/api/java/time/Instant....
| KMag wrote:
| I remember one colleague concerned with casting a Java 64-bit
| millisecond timestamp to double seconds when crossing languages
| (over to a somewhat TypeScript-like financial modeling DSL
| developed in the mid-late 1990s, where it's a bit clunky to deal
| with numbers in any format other than doubles).
|
| I added a comment apologizing to software archaeologists on 12
| October 287,396 C.E., when the precision of double timestamps
| (with 1970-01-01T00:00:00 UTC epoch) drops to 2 milliseconds, and
| the reviewer was fine after that.
| WalterBright wrote:
| I think it was Microsoft BASIC that stored time as a floating
| point number. What a bad idea. Converting from it to an integer
| and then converting back to float would cause the time to "drift"
| due to rounding errors.
|
| It's the same reason why you should never use floating point
| numbers for account balances.
| adzm wrote:
| The VARIANT date type uses a floating point number; the whole
| number portion is the day (days past 1900 January 1) and the
| fractional part is the time.
|
| Not sure about classic BASIC; I don't really recall there being
| a specific time type, though I know later things like Visual
| Basic and etc would just use the VARIANT date type above with a
| 0 day component.
|
| I like Windows' FILETIME which stores time as 100-nanosecond
| intervals since the beginning of the year 1601. C#'s DateTime
| is similar but uses an epoch of 0001 January 1.
|
| I know JavaScript and others simply use the usual unix Epoch
| but with milliseconds rather than seconds, so time() * 1000.
| gumby wrote:
| Great choice! Far away points in time will be less precise, but
| really: how good is anyone at predicting the future anyway?
|
| (in other words a terrible choice, though not for the reason
| above).
| junon wrote:
| No thanks. For one, hashing becomes ambiguous. Under no
| circumstances are NaN and NaN equal to each other. That will
| cause mayhem in any system looking for an iota of resiliency.
___________________________________________________________________
(page generated 2023-06-09 23:01 UTC)