TDateTime and assoc. functions - valid for negative TDateTime? lazarus / 21-General ---------------------------------------------------------------------- AlanTheBeast · 2022-08-31 12:12 TDateTime = 0 gives me: 1899-12-30::00:00:00.000. I need to calculate days from 1601-01-01 until a particular recent date. Do functions such as DaysBetween (D0, D1) where D0 is negative and D1 is positive work correctly? (Accounting for leap years, of course). Thanks. ---------------------------------------------------------------------- MarkMLl · 2022-08-31 12:20 Quote from: AlanTheBeast on August 31, 2022, 02:12:42 pmI need to calculate days from 1601-01-01 until a particular recent date. In what country? https://fpc-devel.freepascal.narkive.com/foHpHSEU/definition-of-the- time-fration-in-a-tdatetime-before-30-12-1899 appears relevant, but this is likely to be deeply messy, on account of the shift from the Julian to Gregorian calendar. MarkMLl ---------------------------------------------------------------------- wp · 2022-08-31 12:46 Run this and convince yourself that the fractional (time) part is handled correctly (like Delphi), and that leap years are detected according to the usual rules. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program Project1; uses  SysUtils; const  YEARS: array[0..6] of Integer = (2004, 2000, 1900, 1896, 1800, 1700, 1600);var  dt: TDateTime;  y, m, d, h, n, s: Integer;  i: Integer;  str1, str2: String;begin  DefaultFormatsettings.ShortDateFormat := 'yyyy/mm/dd';  DefaultFormatsettings.LongTimeFormat := 'hh:nn:ss';  DefaultFormatsettings.DateSeparator := '-';  DefaultFormatsettings.TimeSeparator := ':';  m := 1;  d := 1;  h := 3;  n := 0;  s := 0;  for i := 1 to 10 do  begin    str1 := Format('%.4d-%.2d-%.2d %.2d:%.2d:%.2d', [y, m, d, h, n, s]);    dt := StrToDateTime(str1);    str2 := DateTimeToStr(dt);     WriteLn('Year=', y, ' Month=', m, ' Day=', d,      ' --> str1=', str1,      ' --> StrToDateTime=', dt:12:6,      ' --> str2=', str2);  dec(y);  end;   for i := Low(YEARS) to High(YEARS) do    WriteLn('Year=', YEARS[i], ' Leap year=', boolToStr(IsLeapYear(YEARS[i]), true));   WriteLn('Press ENTER to close...');  ReadLn;end. BUT: TDateTime does not know about the introduction of the Gegorian calendar and thus may be very inaccurate before some date in the past, depending on when your country introduced it. The other thing to be aware of is that time calculation changes it sign in a mathematical sense. Suppose you want to calculate some time 3 hours later than a reference time. After 1899/31/12 you add 3/24 (24, because the day has 24 hours). Before this date (when DateTime is negative) you must subtract it! ---------------------------------------------------------------------- MarkMLl · 2022-08-31 12:55 Quote from: wp on August 31, 2022, 02:46:26 pminaccurate before some date in the past, depending on when your country introduced it. Country? Referring to https://en.wikipedia.org/wiki/Gregorian_calendar#Adoption_by_country as an abridged summary, note that Strasbourg and Alsace changed calendars several years apart. It's almost as bad as the regional time situation in even a small country like the UK before the arrival of the railways, or the original Jewish calendar being based on the sighting of the New Moon :-) MarkMLl ---------------------------------------------------------------------- AlanTheBeast · 2022-08-31 13:16 Quote from: MarkMLl on August 31, 2022, 02:20:01 pmQuote from: AlanTheBeast on August 31, 2022, 02:12:42 pmI need to calculate days from 1601-01-01 until a particular recent date. In what country? Non-specific.  I'm using UTC time and the country is irrelevant to the function. To be sure, Ultibo does have "locale" specific settings and maybe my issue lies therein. (Edit: seems more oriented to presentation of date formats, keyboard settings, code page, etc.  No indication that it may affect how TDateTime related functions will behave according to Locale.  This statement is about as firm as Donald Trump's mind, however). I get the same bizarre result as in that link: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i           RD :=0;          writeln (formatdatetime('YYYY-MM- DD::HH:mm:ss.zzz',RD));          RD := RD -1.25;          writeln (formatdatetime('YYYY-MM-DD::HH:mm:ss.zzz',RD));          RD := -0.25;          writeln (formatdatetime('YYYY-MM- DD::HH:mm:ss.zzz',RD)); Result: 1899-12-30::00:00:00.000 1899-12-29::06:00:00.000 ... I would have expected 12:28::18:00:00.000. 1899-12-30::06:00:00.000 ... I would have expected 12:29::18:00:00.000. So this may be why I need to add 1 day to my computation (which is the "kludge" I have in there now, but your linked page may explain it). Question is: portability... ---------------------------------------------------------------------- AlanTheBeast · 2022-08-31 13:23 Quote from: wp on August 31, 2022, 02:46:26 pmRun this <<>> BUT: TDateTime does not know about the introduction of the Gegorian calendar and thus may be very inaccurate before some date in the past, depending on when your country introduced it. The other thing to be aware of is that time calculation changes it sign in a mathematical sense. Suppose you want to calculate some time 3 hours later than a reference time. After 1899/31/12 you add 3/24 (24, because the day has 24 hours). Before this date (when DateTime is negative) you must subtract it! Ran your code (FPC, Mac, command line) And get: An unhandled exception occurred at $00000001065C7BCC: EConvertError: Invalid date   $00000001065C7BCC   $00000001065C8EEE IAC ... See my example surrounding TDr = 0, -1.25, -0.25 ... Bizarre result.  But explains my issue.  Probably. As Mark says wrt country, I may need to look into how Ultibo handles regional issues.  I'm in Canada but Ultibo may default to Australia. They're already on the wrong side of the planet in 2 ways that don't cancel out ... IAC, seems to work with the extra day thrown in so avoids a messier solution of counting days per year accounting for leap years. That's okay, but when correcting to the beginning of the GPS week, it's uglier:  last week? Okay, but was it also previous month?  Okay and previous year?... ---------------------------------------------------------------------- MarkMLl · 2022-08-31 13:43 "TDateTime = 0 gives me: 1899-12-30::00:00:00.000" ...has always struck me as a bit odd since December is a 31-day month. If I were doing this, the first thing I'd do is work from +2.0 to -2.0 in one-hour (later perhaps one minute and then one second) decrements looking for breaks in monotonicity etc. You might possibly do better using the astronomers' Julian calendar, with the caveat that they're 12 hours out compared with civil time. See CalcJD() etc. in https://github.com/MarkMLl/NOAA_AstroDemo/blob/main/sunposition.pas MarkMLl ---------------------------------------------------------------------- AlanTheBeast · 2022-08-31 13:53 Quote from: MarkMLl on August 31, 2022, 03:43:49 pm"TDateTime = 0 gives me: 1899-12-30::00:00:00.000" ...has always struck me as a bit odd since December is a 31-day month. If I were doing this, the first thing I'd do is work from +2.0 to -2.0 in one-hour (later perhaps one minute and then one second) decrements looking for breaks in monotonicity etc. You might possibly do better using the astronomers' Julian calendar, with the caveat that they're 12 hours out compared with civil time. See CalcJD() etc. in https://github.com/MarkMLl/NOAA_AstroDemo/blob/main/sunposition.pas MarkMLl The Dec 31 day thing struck me as odd too. As to the time reference, Ultibo's "finest" time reference is in the 100ns since 1601-01-01 "form".  Once you set the time, it converts that to that time system. Since I'm testing the accuracy of NTP, and NTP "sets" the system time it will be in that time system.  Stuck with it. So, for this project (NTP v. GPS testshttps://forum.lazarus.freepascal .org/index.php/topic,60420.0.html), the "kludge" day added in will probably survive and I'm done with it. I just need to verify that it all works correctly this coming Sunday (new "GPS week"). The GPS system has its own peculiarities such as a separate Leap Second that is not in the GPS raw time1, but the GPS satellites do broadcast the GPS Leap Sec. When my program gets a first valid PVT message from the receiver, it polls the receiver for the Leap second. Once that's received, then I can compute the number of 100 ns ticks since 1601-01-01 up until morning of the most recent Sunday. That value remains the same until the next Sunday (where it's just a matter of adjusting it by +(7 * 86400 * 10000000) when the GPS ms of the week resets). Edit: more boring stuff. ____ 1. Never mind ~19.7 year week number rollovers ... (10 bit week number) .. but the receivers usually take care of that detail.  Usually. ---------------------------------------------------------------------- wp · 2022-08-31 14:05 I remember it like this: The time part runs in the opposite direction for negative datetime values, and the date part is always the integer of the datetime. So, when the fractional part of the datetime number is, say, 0.25, this means: 1 quarter of this day (6 hours) into the day given as the integer part. This rule makes a problem when the 0-date is crossed. The difference between DateTime=0.25 and DateTime=-0.25 is not half a day but a full day (in order words: 0.25 is a quarter day on day 0, and -0.25 is a quarter day on day -1 --> there is a full day difference). But better to forget all this math in this case and use the Inc* functions in DateUtils instead: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program Project1; uses  SysUtils, DateUtils;var  dt: TDateTime;  i: Integer;begin  WriteLn('"Normal" Math:');  dt := 1.25;  for i := 1 to 5 do  begin    WriteLn('  ', dt:0:2, ' ', DateTimeToStr(dt));    dt := dt - 1.0;  end;   WriteLn('IncDay');  dt := 1.25;  for i := 1 to 5 do  begin    WriteLn('  ', dt:0:2, ' ', DateTimeToStr(dt));    dt := IncDay(dt, -1);  end;   ReadLn;end. [EDIT] Found an issue in the DaysBetween function (and probably other *Between functions) when the magic zero date is crossed: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i var dt1, dt2: TDateTime;...  dt1 := 1.0;  dt2 := IncDay(dt1, -3);   WriteLn(dt1:0:6, ' ', DateTimeToStr(dt1));  WriteLn(dt2:0:6, ' ', DateTimeToStr(dt2));  WriteLn(DaysBetween(dt1, dt2));The datetime dt1 was decremented by 3 days, but DaysBetween reports a difference of only 2 days (unlike delphi, which reports the correct value, 3). ---------------------------------------------------------------------- MarkMLl · 2022-08-31 15:21 Hmm. "There are no TDateTime values from –1 through 0."... allegedly from Delphi documentation. https://stackoverflow.com/questions/49201389/delphi-displays-strange- results-for-operations-with-negative-values-to-tdatetime If I wanted to "make a better unit" I'd use the longstanding IBM mainframe convention of an integer where the LSB represents some unreasonably small timescale (e.g. 1 nSec) with the lowest bit actually guaranteed to increment (e.g. every 1024 x 1024 nSec) being implementation-defined. I once thrashed out a workable 64-bit (?) representation which gave both high resolution and a limit several 10's of thousands of years in the future, only to be told by my boss that the coverage wasn't sufficient for his plans for World domination. MarkMLl ---------------------------------------------------------------------- AlanTheBeast · 2022-08-31 16:04 Quote from: MarkMLl on August 31, 2022, 05:21:08 pmHmm. "There are no TDateTime values from –1 through 0."... allegedly from Delphi documentation. https://stackoverflow.com/questions/49201389/delphi- displays-strange-results-for-operations-with-negative-values-to- tdatetime I better make sure my time machine doesn't attempt to go there, I may never get back. Quote from: MarkMLl on August 31, 2022, 05:21:08 pmI once thrashed out a workable 64-bit (?) representation which gave both high resolution and a limit several 10's of thousands of years in the future, only to be told by my boss that the coverage wasn't sufficient for his plans for World domination. I can't find the reference offhand, but the updated NTP protocol (proposed?) by Mills (Pater NTP) will have both higher resolution and greater breadth of time than reasonably usable by anyone, anytime, anywhere over that breadth of time. If humanity lasts that long, it will be a good problem to solve. ---------------------------------------------------------------------- AlanTheBeast · 2022-08-31 16:11 Quote from: wp on August 31, 2022, 04:05:48 pmI remember it like this: The time part runs in the opposite direction for negative datetime values, and the date part is always the integer of the datetime. So, when the fractional part of the datetime number is, say, 0.25, this means: 1 quarter of this day (6 hours) into the day given as the integer part. This rule makes a problem when the 0-date is crossed. The difference between DateTime=0.25 and DateTime=-0.25 is not half a day but a full day (in order words: 0.25 is a quarter day on day 0, and -0.25 is a quarter day on day -1 --> there is a full day difference). But better to forget all this math in this case and use the Inc* functions in DateUtils instead: <> Date functions for me usually are in the relatively here and now of ±50 years, so I'm not beating this horse any more.  (Adding 1 day to my result for the NTP/GPS tests does the job). I'll re-verify them Sunday because of the "end of GPS week" issue - just to be sure. (Actually Sat late evening, as I'm in Canada so UTC midnight comes a little early). I'm glad to have found out I'm not crazy, though. ---------------------------------------------------------------------- MarkMLl · 2022-08-31 18:20 Quote from: AlanTheBeast on August 31, 2022, 06:04:00 pmI can't find the reference offhand, but the updated NTP protocol (proposed?) by Mills (Pater NTP) will have both higher resolution and greater breadth of time than reasonably usable by anyone, anytime, anywhere over that breadth of time. If humanity lasts that long, it will be a good problem to solve. What the NTP timestamp doesn't do though is say "from this bit towards the MSB, the value is guaranteed to count monotonically at such-and- such a rate". OK, you could /assume/ that that applied to seconds (leaving out consideration of leap-seconds) but it's not guaranteed. The IBM people- who in their day were very good indeed- /did/ make such a guarantee for anything faster than (something like) 100 mSec which was represented by a fixed bit position, with the behaviour of the less-significant bits being rigidly defined for any particular machine plus combination of "facilities". I think that both NTP and the IBM systems agreed that the less- significant bits of a timestamp were a fraction of a second rather than being a nSec (etc.) count, which is of course also what TDateTime is doing. MarkMLl ---------------------------------------------------------------------- AlanTheBeast · 2022-09-01 13:26 Quote from: MarkMLl on August 31, 2022, 08:20:59 pmQuote from: AlanTheBeast on August 31, 2022, 06:04:00 pmI can't find the reference offhand, but the updated NTP protocol (proposed?) by Mills (Pater NTP) will have both higher resolution and greater breadth of time than reasonably usable by anyone, anytime, anywhere over that breadth of time. If humanity lasts that long, it will be a good problem to solve. What the NTP timestamp doesn't do though is say "from this bit towards the MSB, the value is guaranteed to count monotonically at such-and- such a rate". OK, you could /assume/ that that applied to seconds (leaving out consideration of leap-seconds) but it's not guaranteed. The IBM people- who in their day were very good indeed- /did/ make such a guarantee for anything faster than (something like) 100 mSec which was represented by a fixed bit position, with the behaviour of the less-significant bits being rigidly defined for any particular machine plus combination of "facilities". I think that both NTP and the IBM systems agreed that the less- significant bits of a timestamp were a fraction of a second rather than being a nSec (etc.) count, which is of course also what TDateTime is doing. Given that NTP from poll to poll can (and will) reverse system time slightly ...