Network Time Protocol (NTP) on Ultibo Pi lazarus / 79-Embedded - ARM ---------------------------------------------------------------------- AlanTheBeast · 2022-08-29 20:55 In case anyone was wondering. In the basic Ultibo package, depending on what units are selected, NTP1 is automatically activated; for example if Webstatus is included, then it is on.  (Not sure what other units might invoke it - but if they include the Services unit, it's a good bet).  Thus if you use "Now" or "ClockGetTime" you're using UTC time (Assuming you're connected to the internet, of course).  (There are also some time zone functions in "there" somewhere which I don't use). (It can be set off using a initunit:  NTP_AUTOSTART := false; statement ). I tested it against a GPS 1 PPS output2 using the interrupt and found a large error (700ms if memory serves).  Repeat tests had random errors in [0..999] ms. I mentioned this on the Ultibo forum.  It turns out that in versions of Ultibo before 2.5.105 that Ultibo weren't confident in the accuracy of their NTP implementation and so rounded (truncated) off to 1s.  (No clue why - Ultibo folks do a fine job with everything). Knowing that I was doing a GPS nav project and a bit obsessed with time accuracy3, Ultibo asked me to test v. the GPS if they let the NTP go full accuracy.  Of course! It was clear from the very first test that it works very well.  I'm now wrapping up other tests on various servers just to confirm some behaviours, but if you need NTP on Ultibo Rasp Pi, fear not - it works very well with typical ±10 ms level error v. UTC out of the gate.  (Though it may start out higher (20ms) and take a while to settle).  Along the way learned some things I didn't expect and tripped over my own assumptions and biases several times.  What else is new?  IAC, Ultibo's voice, big G, is quite patient with me and I got some great suggestions from others on that forum. (For the curious I use the 1 PPS off of the GPS receiver and timestamp with ClockGetTotal4 in an interrupt and then wait for the 1 Hz nav message which includes ms since the beginning of the week.  Convert the GPS time to whatever  number of 100ns ticks since 1601-01-01 is called5 (plus time since the interrupt) to compare against NTP in the Pi. My test is accurate to about 10 µs on the Pi 4 - way sharper than NTP.  (I'd need an oscilloscope to characterize the test time error better than that....)). ____________________________________________________ 1. https://www.eecis.udel.edu/~mills/database/papers/history/history.pdf 2. About ±30 to ±50 ns v. UTC top of second. 3. I'm not much concerned about UTC but rather events at higher rates (25 and 1000 Hz). 4. ClockGetTotal in the Pi 4 is 54,000,000 ticks/s - ~18.5 ns.  GetTickCount64?  Hardly knew ye6. 5. Some standard that MS use and that Ultibo uses for file times.  Why MS couldn't be happy with 1 µs is one of those mysteries ... 6. Don't worry - still very useful. ---------------------------------------------------------------------- AlanTheBeast · 2022-08-30 23:04 A graph of Ultibo-NTP v. GPS over a 1 hour period.  YMMV. ---------------------------------------------------------------------- ccrause · 2022-09-02 12:08 Quote from: AlanTheBeast on August 29, 2022, 10:55:24 pmMy test is accurate to about 10 µs on the Pi 4 - way sharper than NTP.  (I'd need an oscilloscope to characterize the test time error better than that....)). I'm interrested in your measurement methodology - are you reading the GPS pulses with a pin on the PI?  Is your code called via pin change interrupt, or running in a busy loop? Also, is both the GPS monitoring code and the NTP/internal timer code running on the same core? ---------------------------------------------------------------------- MarkMLl · 2022-09-02 13:19 Quote from: ccrause on September 02, 2022, 02:08:43 pmI'm interrested in your measurement methodology - are you reading the GPS pulses with a pin on the PI?  Is your code called via pin change interrupt, or running in a busy loop? And if using a GPIO pin, what API? The obvious questions are (a) how much jitter is being seen and (b) the overall drift. My experience is that there is a slowly-varying systematic error between the various satellite constellations, a much larger variation between that and a PPS signal, and forget anything to do with a radiocode clock using consumer electronics. I don't want to say too much more about what I was doing there, but it involved a decent logic analyser attached to (among other things) either a PC or an RPi dedicated to generating a PPS signal. Link below is relevant. https://www.eevblog.com/forum/metrology/pps-output-using-linux-on-a- pc/msg3558208/#msg3558208 MarkMLl ---------------------------------------------------------------------- AlanTheBeast · 2022-09-02 19:50 Quote from: ccrause on September 02, 2022, 02:08:43 pmQuote from: AlanTheBeast on August 29, 2022, 10:55:24 pmMy test is accurate to about 10 µs on the Pi 4 - way sharper than NTP.  (I'd need an oscilloscope to characterize the test time error better than that....)). I'm interrested in your measurement methodology - are you reading the GPS pulses with a pin on the PI?  Is your code called via pin change interrupt, or running in a busy loop? Also, is both the GPS monitoring code and the NTP/internal timer code running on the same core? 0. RTL:  Ultibo Pascal RTL (32 bit presently, 64 bit v. imminent (I'm told)). 1. Pi 4. 2. GPS UART @ 921600 bps (not that it matters much - if it were at the default 38400 bps it would work just as well) 1 PPS pulse programmed (by setting to the GPS via UART) for TOS on rising edge. The GPS is "told" to make measurements at 1 Hz. The GPS is told to send the NAV-PVT message at the rate of 1 per measurement.  (in this case 1 Hz) NAV-PVT contains, amongst many things, the number of ms since the beginning of the GPS week (00:00:00.000000000 of Sunday morning, UTC). The GPS is told to send its LeapSecond value since the inception of GPS (1980 ish).  Current Leap second is 18. GPS manufacturer (ublox) states the 1 PPS error as ±30 ns RMS, ±60 ns (99%) Antenna (active) is on the roof with a good view of the sky.  At 1 Hz, the GPS tracks about 30 satellites (GPS, Galileo, Beidou and *&^%&*^%).  For UTC, the time is off of the US GPS system - but the over determined position is from all sources (and SBAS) which helps the time be more accurate (ns level - not very germane here.  I've also programmed the GPS for a 76ns delay for the RF cable (15.24m), but again, not really germane to these results). 3. GPIO Interrupt programmed as follows.  This was with a lot of help from big G at Ultibo.   It's a FIQ (fastest) and is the fastest possible interrupt for users of the RTL. There are some h/w details I don't fully understand, but, at least since the Pi4, interrupts can be finely "caught" on a pin-by-pin basis and dispatched to an interrupt handler.  That is handled by the RTL.  I've used this interrupt at up to 2000 Hz with no issues (but it's 1 Hz for the NTP checking program). Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i Procedure GPS_Pulse_Interrupt (P:pointer; PI, TI:Longword);  // H/W interrupt trapped by GPIO unit with callback to here.BEGIN   GPS_PPS_InterruptTime := ClockGetTotal;  // On Pi4: 1/54000000s resolution   Inc(GPS_PPS_InterruptCount);END; Procedure Config_PPS_Interrupt;CONST     PPS_intline = GPIO_PIN_4;BEGIN  If Interrupt_PPS = nil then    begin      GPS_PPS_InterruptCount := 0;   //this is an int64      Interrupt_PPS := GPIODeviceFindByDescription('BCM2838 GPIO');      if Interrupt_PPS = nil then begin ConsoleWindowWriteln(Con1, 'Set up GPS pulse interrupt - received NIL pointer'); exit end;      GPS_Int_Result := GPIODeviceInputEvent(Interrupt_PPS, PPS_intline,                      GPIO_TRIGGER_RISING, GPIO_EVENT_FLAG_REPEAT or GPIO_EVENT_FLAG_INTERRUPT,                        INFINITE, @GPS_Pulse_Interrupt, nil);    end;END;  Method: 1. Notice in the interrupt that there is "GPS_PPS_InterruptTime".  This tells me the time-since-start-of-the-computer of the top of the UTC second to sub µs level.  Not sure "how" sub µsec as I don't have a scope to set up measurements. .. this is number of ticks since the computer started. .. each tick has a resolution of 1/54,000,000s.  (about 18.5 ns). What "time"? A. System time: When NTP is active on the computer the computer system time is set by the NTP function in the Services unit.  Thus, "Now" and "ClockGetTime" return the system time set by NTP.  This program doesn't need to "do" anything with respect to that.  (But does have a function to load a list of NTP sources and use them via a client function within the Services package).  Or simply set the default NTP to a particular service and re-start. B: GPSTime.  Pretty trivial to take the GPS time and set the system time and that can be done in a few lines of code.  Of course if NTP is running, when it polls and updates the system time there is no "notice given".  So for this program GPS time is always kept separate from system time so that comparison can be made. Just using a UART one can set their system time to well less than 1 second away from UTC.  Depending on the maker of the device, the offset from top-of-second (TOS) will vary.  To know when that "was" the 1 PPS pulse can be used to move the result into the µsec (or even 100 nsec) realm.  An interrupt is the best way to do this but more complicated.  OTOH, if one wanted to "watch" the pulse once per 5 minutes or 1 per hour, the end result would be very sharp.  Some smart programming would have the "watch" period begin close to TOS.  Note that there's also the possibility to characterize the drift rate of the computer's clock and that in turn can be used to decide on the polling rate.  Once per day would suffice for many uses. 2. Elsewhere in the code is the stuff that reads in messages from the GPS receiver, identifies the message and passes to the correct decoder. 3. One of those decoders is the NAV-PVT decode thread.   It has various "states" and begins with initialization states which (amongst other things): - on receipt of the first valid nav message (state -1). .... sends a request to the GPS to send the Leap Second message  .... waits in the next state (-2) (could take 2 or 3 seconds) for the GPS  to reach at least a 2D fix _and_ ,,,,,for the LeapSecondValid flag (set by the LeapSecond decoder thread). .... computes the beginning of the week in the RTL's time frame reference, to wit:    100ns ticks since 1601-01-01.    MINUS: the GPS LeapSecond. THIS IS THE KEY:  That value represents exactly the beginning of the GPS week in the RTL's time system of 100ns ticks since 1601-01-01. NOTICE: a) The "real time" clock of the system is 1/10,000,000s (100ns) ticks origin 0 at 1601-01-01::00:00:00.000... b) The "measurement" ticks are 1/54,000,000s (~18.5ns) since the computer started (h/w level tick counter of the Pi 4). 4. Meanwhile, the 1 PPS (from GPS PPS pin) interrupt "stamps" the Pi's H/W "tick counts" into GPS_PPS_InterruptTime.  That is precisely when it happened to a resolution of ~18.5ns and to an accuracy somewhat better than 1 µsec. The GPS NAV message also contains an error estimate for the time.  This is usually around 35ns or so.  I don't know if this applies to the 1PPS or not, but it is negligible for these tests.  (The doc says: "Time accuracy estimate (UTC)" w/o further elaboration). 5. At 1 Hz, the GPS message comes in about 30 - 45 ms after the interrupt.  This varies for too many reasons to get into, but it's in that range. 6. Once the message is in, and the ToWms value extracted from the message ("when the interrupt happened"), then the number of 100ns ticks since the beginning of 1601-01-01 can be computed (note, at 1 Hz, the ToWMS portion of the nav time is always exactly 0 as it represents "top of the second".  The message arrives 30 - 45ms later, however). Point of compare Sample both the RTL "Now" [ClockGetTime] (which is set to 100ns ticks since 1601-01-01 - and with NTP running represents NTP time(UTC)) and the [ClockGetTotal] (ticks since "start of computer @ 1/54,000,000s per tick).  It doesn't matter "when" we sample these values, but they must be close together.  Since pre-emption is possible, we'll do this inside a 2 1 µsec gate.   (Per Rt results it's usually around 35 and has never gone past 84 ticks or ~1.5 µs in a passed gate.   (It's tempting to narrow the gate to 1 µs and I'll try that today and add a counter for times the loop has to repeat.  Update: the 1 µs gate has been running for an hour week.  It had to repeat at least once a few times until I removed the checking code.  Theoretically improves the result by some tiny amount (½ µs?  Not sure.)). While the gate (Repeat .. Until) is 1 µs "wide", the time between NOWCGT := ... and NOWCGTot is much smaller - and the "real" time difference beteen the 2 at the "point of compare" is less than ½ µs.  (And when Ultibo finally releases as 64 bit, probably less than ¼ µs). (Also removed a "Rt := NowCGTot - Rt" before the Until test, saves about 5 ARM instructions.  Will improve moreso with 64 bit). Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i                    Repeat                             //in case preempted between samples.                         Rt:= ClockGetTotal;          //Gate ref: 18.5 ns resolution                         NowCGT   := ClockGetTime;      //sample (100 ns resolution)  <<<< Is = NTP time but in the 100ns ticks since 1601-01-01 format                         NowCGTot := ClockGetTotal;     //sample ( 18.5 ns resolution)                   Until (NowCGTot - Rt) < 54;       //gate for close sampling : 1 µsec window. The above can be improved by disabling interrupts but care needs to be taken to ensure the system time has been set to some "real" value before the call to ClockGetTime (some deeper issue with potential deadlock if the time has not yet been set).  I take care to do so with the GPS time before I start the thread that contains this code.  So, with that caveat in mind, replace the above section with: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i                    Mask     := SaveIRQFIQ;        //Disable IRQ/FIQ and return the previous state                     NowCGT   := ClockGetTime;   //sample (100 ns resolution)  <<<< Is = NTP time but in the 100ns ticks since 1601-01-01 format                     NowCGTot := ClockGetTotal;   //sample ( 18.5 ns resolution)                     RestoreIRQFIQ (Mask);          //Restore the previous state of IRQ/FIQ Where Mask is a variable of type TIRQFIQMask .  This is, to me, more satisfactory as driving the machine goes than the "gate" Repeat/Until loop above.  But the prior works quite well and there is little time cost to it.  The latter is slightly less friendly, but as long as the time is correctly set earlier in the run, then it is no issue.   Further of course, the difference in time between the two samples is guaranteed to be very close to the bottom end of a µsec rather than probably close to the bottom of a µs.  I'd venture that in 64 bit code it would be sub µsec which will be very useful on other projects.  Is this "bare metal"?  No. But the coating is very thin.... So, onward .... .. compute the exact time of the interrupt since 1601-01-01 in 100 ns ticks. ToWms is "Top of week milliseconds"  -- ms since sun morning UTC.  X 10000 = 100 ns ticks. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i Function GPSTimeInCGT (ToWms:longword):int64; inline;BEGIN     Result := WeekTickRef + int64(Towms) * 10000;END;  Then add the number of 100ns ticks between the interrupt and the moment of comparison ("sampling "instant"") - and convert from 1/54,000,000 (~18.5ns) ticks to 1/10,000,000 (100ns) ticks: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i                    NowGPS := GPSTimeInCGT(ToWms) + ((NowCGTot - GPS_PPS_InterruptTime) * 10 ) div 54;  Then we can compute the error between the NTP and GPS as simply: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i                    dCG := NowGPS - NowCGT;  Where dCG is in 100 ns tick units.  +result == NTP "lateness". In summary it comes down to three "chunks" of time. -- A really big chunk of 100ns ticks since 1601-01-01 to the beginning of the GPS week - computed once per program execution (and updated if ToWMS < Previous_ToWMs) by adding 7 days of 100ns ticks to the WeekTickRef value.  This happens once per week. -- a chunk of 100ns ticks since top-of-the GPS week until the instant of the interrupt. -- a tiny chunk of 100ns ticks since the interrupt until instant of sampling the "Now" (in NTP 100ns ticks) v the GPS (in 100ns ticks). As to core run location. The NTP tests use some of my NAV code which is divided: ..serial GPS input thread: Core 0. ..message dispatcher: Core 1.     ___ Frankly the message dispatcher and serial input can be on the same core with no issues at all. ..decoders: Core 2 ____ these could also run on the same core as the above. ..EXCEPT NAV-PVT messages: thread is on Core 3.  it's a big (92 byte) message (as far as GPS goes) with about 35 parameters (I use about a dozen) and in the Nav app is at 25 messages per second.  A lot of stuff happens with it. ..NTP code: no idea what core it's on.  The Ultibo thread thrower is pretty well balanced in my experience.  NTP does not "cycle" often.  It's a slow machine that goes to the web (typically) once every 5 minutes.  The process is not all that intensive. Edit: typos, clarification and 1 PPS error range per manuf. Edit: 1 PPS error Edit: added stuff about core/NTP. Edit: 2022-09-03 - added some details about time references (clarification) and fixed some typos. Edit: 2022-09-05 - more massage... Edit: 2022-09-07 - finicky detail around the sampling gate. Edit: 2022-09-09 - alternate sampling scheme that avoids using a gate ---------------------------------------------------------------------- AlanTheBeast · 2022-09-02 20:18 Quote from: MarkMLl on September 02, 2022, 03:19:49 pmQuote from: ccrause on September 02, 2022, 02:08:43 pmI'm interrested in your measurement methodology - are you reading the GPS pulses with a pin on the PI?  Is your code called via pin change interrupt, or running in a busy loop? And if using a GPIO pin, what API? The obvious questions are (a) how much jitter is being seen and (b) the overall drift. - Don't need no stinkin' API.   See my reply to M. Crause for specfics. a) Which jitter? interrupt?  Past experiments (extensive sampling) suggest sub 18.5ns (99.99% - literally - it's actually a test I want to run again - I suspect it's closer to 99.999% sub 18.5ns) And, of course, for these tests if it were up in the several µs it would still be fine. b) i: Drift of UTC is taken as 0.  On a day to day / week to week basis it oscillates at the 1 or 2 ns level.  An anomaly event of ±3 ns in 2016 had the community in an uproar for a while. ii: The drift of the Pi (well my Pi4) is about 10 .. 12 PPM.  And doesn't matter at all for these tests - though as the graph above shows, it's consistent. Get my drift?  Any others? (Well, of course every few years they throw in a leap second to account for the rotation rate of the rock, but that's waaaaaaaaaayyyyyyyy outside this test - but accounted for at the gross scale of seconds in the reference timemark.  (No choice). Quote from: MarkMLl on September 02, 2022, 03:19:49 pm My experience is that there is a slowly-varying systematic error between the various satellite constellations, a much larger variation between that and a PPS signal, and forget anything to do with a radiocode clock using consumer electronics. Not esp. relevant as I'm using one constellation for the 1 PPS (US GPS aligned to the US Navy observatory's definition of UTC).  The rate of drift of their UTC mark is: So system UTC error:  ±2 ns GPS receiver PPS error: ±30 ns RMS, ±60 ns (99%) (per manufacturer data sheet). So: irrelevant as my test "truth" is up in the 1 .. 10 µsec range to test NTP which is reliably in the ±20 ms range - most of the time... ---------------------------------------------------------------------- MarkMLl · 2022-09-02 20:54 Quote from: AlanTheBeast on September 02, 2022, 10:18:37 pm- Don't need no stinkin' API.   See my reply to M. Crause for specfics. Good point: you're effectively bare-metal. I'd caution anybody getting here via Google to be extremely careful with the Raspberry Pi (or for that matter PC) GPIO APIs, which are either imprecise, non-portable, or both. Quotea) Which jitter? interrupt?  Past experiments (extensive sampling) suggest sub 18.5ns (99.99% - literally - it's actually a test I want to run again - I suspect it's closer to 99.999% sub 18.5ns) And, of course, for these tests if it were up in the several µs it would still be fine. Sub-5nSec in my case, comparing the various GNSS constellations using multiple receivers. However at present I don't want to put a time constraint on that, in all probability there's some fairly major relative shifts due either to different underlying maths or to different explicit corrections. QuoteNot esp. relevant as I'm using one constellation for the 1 PPS (US GPS aligned to the US Navy observatory's definition of UTC).  The rate of drift of their UTC mark is: Noting your other comments, but I thought you were in the Commonwealth so WTF does the US have to do with UTC? I /do/ have a problem here, since in the UK we don't have an official high-precision clock other than the phase of 60kHz MSF... unless one can afford to pay Big Money to get a fibre from NPL. MarkMLl ---------------------------------------------------------------------- AlanTheBeast · 2022-09-02 21:17 Quote from: MarkMLl on September 02, 2022, 10:54:53 pmQuote from: AlanTheBeast on September 02, 2022, 10:18:37 pma) Which jitter? interrupt?  Past experiments (extensive sampling) suggest sub 18.5ns (99.99% - literally - it's actually a test I want to run again - I suspect it's closer to 99.999% sub 18.5ns) And, of course, for these tests if it were up in the several µs it would still be fine. Quote from: MarkMLl on September 02, 2022, 10:54:53 pmSub-5nSec in my case, comparing the various GNSS constellations using multiple receivers. However at present I don't want to put a time constraint on that, in all probability there's some fairly major relative shifts due either to different underlying maths or to different explicit corrections. Absent the eqt. my former employer had for us, I'm delighted I can measure anything this accurately for a few bucks and a few lines of code. In the end this test is in 4 time realms: - ns level    - resolution of finest available timer (~18.5 ns) - µsec level - resolution accuracy of finest available "truth" with the equipment on hand (1..10µs conservatively - I'm pretty sure it's sub µs). - ms level.  - the expected accuracy of a web delivered NTP time (±10ms generally, but can be worse, can be better ... even over long distances). - centuries  - it still irks me that, for this test, I had to resort to a time 0 when Shakespeare was still working on Hamlet. Quote author=William Shakespeare date=1601-01-01 QuoteI'll watch to-night. Upon the platform, 'twixt eleven and twelve I'll visit you. /Hamlet Quote from: MarkMLl on September 02, 2022, 10:54:53 pmQuoteNot esp. relevant as I'm using one constellation for the 1 PPS (US GPS aligned to the US Navy observatory's definition of UTC).  The rate of drift of their UTC mark is: Noting your other comments, but I thought you were in the Commonwealth so WTF does the US have to do with UTC? Their GPS... their time mark, kind of hard to dance away from it. Quote from: MarkMLl on September 02, 2022, 10:54:53 pmI /do/ have a problem here, since in the UK we don't have an official high-precision clock other than the phase of 60kHz MSF... unless one can afford to pay Big Money to get a fibre from NPL. Brit inferiority complex would be quaint if it weren't so frequent.  At least you didn't bemoan GMT's loss of relevance.  There's that. I'd say have a labour free labour day weekend, but I guess that's not British either ... ---------------------------------------------------------------------- MarkMLl · 2022-09-03 08:05 Quote from: AlanTheBeast on September 02, 2022, 11:17:52 pmTheir GPS... their time mark, kind of hard to dance away from it. But /not/ their UTC. QuoteBrit inferiority complex would be quaint if it weren't so frequent.  At least you didn't bemoan GMT's loss of relevance.  There's that. If you don't mind, we /never/ refer to ourselves as Brits and find it vaguely offensive. And I can assure you that there are people with far worse NIH syndrome than ours. The bottom line is that in any jurisdiction it's highly desirable to be able to adhere to the standards that that jurisdiction implements. In the EU a good timing datum can be extracted from Galileo, but the UK's leaving the EU was so ill-considered (and, in my opinion, ill- advised) that it is still unclear where our metrology stands. I agree that the loss of GMT is unfortunate, after all we invented that stuff. But in any sane World the Prime Meridian would run through Florence. MarkMLl ---------------------------------------------------------------------- ccrause · 2022-09-03 10:39 Thank you for the detail Alan. ---------------------------------------------------------------------- AlanTheBeast · 2022-09-03 10:57 Quote from: MarkMLl on September 03, 2022, 10:05:47 amQuote from: AlanTheBeast on September 02, 2022, 11:17:52 pmTheir GPS... their time mark, kind of hard to dance away from it. But /not/ their UTC. QuoteBrit inferiority complex would be quaint if it weren't so frequent.  At least you didn't bemoan GMT's loss of relevance.  There's that. If you don't mind, we /never/ refer to ourselves as Brits and find it vaguely offensive. And I can assure you that there are people with far worse NIH syndrome than ours. The bottom line is that in any jurisdiction it's highly desirable to be able to adhere to the standards that that jurisdiction implements. In the EU a good timing datum can be extracted from Galileo, but the UK's leaving the EU was so ill-considered (and, in my opinion, ill- advised) that it is still unclear where our metrology stands. I agree that the loss of GMT is unfortunate, after all we invented that stuff. But in any sane World the Prime Meridian would run through Florence. MarkMLl 1. /You/ don't refer to yourself as Brits.  What the rest of us label you is not much in your control.  Engage stiff upper lip, wot. 2. UTC is an international standard. The US are part of that standards body. (bodies?).     Heck, they legally adopted the metric system in 1895 ish, though it's hard to tell. 3. Last time I "ran through Florence" her husband came after me with a shotgun. ---------------------------------------------------------------------- AlanTheBeast · 2022-09-03 10:58 Quote from: ccrause on September 03, 2022, 12:39:27 pmThank you for the detail Alan. You're very welcome.  I'll probably tidy that post up a little over time. ---------------------------------------------------------------------- beria · 2022-09-03 13:32 Quote from: MarkMLl on September 03, 2022, 10:05:47 am If you don't mind, we /never/ refer to ourselves as Brits and find it vaguely offensive. And I can assure you that there are people with far worse NIH syndrome than ours. Sorry to intrude, but what do you call yourself? ---------------------------------------------------------------------- MarkMLl · 2022-09-03 13:50 Quote from: beria on September 03, 2022, 03:32:13 pmQuote from: MarkMLl on September 03, 2022, 10:05:47 amIf you don't mind, we /never/ refer to ourselves as Brits and find it vaguely offensive. And I can assure you that there are people with far worse NIH syndrome than ours. Sorry to intrude, but what do you call yourself? Well, the country is the United Kingdom of Great Britain and Northern Ireland, and while we might be formally described as UK citizens that's hardly ever used informally even in discussion of documents such as "British Passport" or "UK passport" (never "UKish"). "British" is common, but equally common are "English", Scottish" and "Welsh"... these are a combination of toponym and ethnicity. Plus of course "Northern-Irish Catholic" and "Loyalist": that one's complex and unpleasant. Plus the extremely informal and unofficial "Scouse", "Geordie", "Kernow" and so on. So as somebody born in Wales of mostly Welsh ethnicity, I'm a British Welshman or (loosely) a Welsh Briton. Unfortunately, I can no longer claim to be European except by inclination. But "Brit" is quite simply never used. MarkMLl ---------------------------------------------------------------------- ccrause · 2022-09-03 15:56 Quote from: MarkMLl on September 03, 2022, 03:50:43 pmQuote from: beria on September 03, 2022, 03:32:13 pmQuote from: MarkMLl on September 03, 2022, 10:05:47 amIf you don't mind, we /never/ refer to ourselves as Brits and find it vaguely offensive. And I can assure you that there are people with far worse NIH syndrome than ours. Sorry to intrude, but what do you call yourself? Well, the country is the United Kingdom of Great Britain and Northern Ireland, and while we might be formally described as UK citizens that's hardly ever used informally even in discussion of documents such as "British Passport" or "UK passport" (never "UKish"). "British" is common, but equally common are "English", Scottish" and "Welsh"... these are a combination of toponym and ethnicity. Plus of course "Northern-Irish Catholic" and "Loyalist": that one's complex and unpleasant. Plus the extremely informal and unofficial "Scouse", "Geordie", "Kernow" and so on. So as somebody born in Wales of mostly Welsh ethnicity, I'm a British Welshman or (loosely) a Welsh Briton. Unfortunately, I can no longer claim to be European except by inclination. But "Brit" is quite simply never used. MarkMLl Some of this complexity can be illustrated by a Venn diagram, see e.g. this diagram. ---------------------------------------------------------------------- MarkMLl · 2022-09-03 16:36 Quote from: ccrause on September 03, 2022, 05:56:04 pmSome of this complexity can be illustrated by a Venn diagram, see e.g. this diagram. Although that does rather skimp on the "what do you call yourself?" and "with which neighbouring village do you engage in ritual combat at weekends?" fronts :-) MarkMLl ---------------------------------------------------------------------- mas steindorff · 2022-09-05 21:24 Quote from: MarkMLl on September 02, 2022, 03:19:49 pmMy experience is that there is a slowly-varying systematic error between the various satellite constellations, a much larger variation between that and a PPS signal, and forget anything to do with a radiocode clock using consumer electronics. Dido!  but try and convince anyone you have a measurement better than "GPS".  Still have management folks that figure I'm full of it.  My testing was over 16 hour windows (at night) and a saw slow drifts up and down.  I was thinking it had something to do with cloud cover somehow creating a delay but different constellations make more sense as the drifts were somewhat repeatable on a 24 hour scale. (but so were the clouds to some degree). ---------------------------------------------------------------------- MarkMLl · 2022-09-06 07:09 Quote from: mas steindorff on September 05, 2022, 11:24:39 pmDido!  but try and convince anyone you have a measurement better than "GPS".  Still have management folks that figure I'm full of it.  My testing was over 16 hour windows (at night) and a saw slow drifts up and down.  I was thinking it had something to do with cloud cover somehow creating a delay but different constellations make more sense as the drifts were somewhat repeatable on a 24 hour scale. (but so were the clouds to some degree). It's fair to assume that the maths embedded in each type of GNSS satellite is different, that the update methodology is different, and that the time that corrections are implemented for different constellations differs. As such, it is a wonder that they agree as well as they do (sub-microsecond). So the problem boils down to "I'm in Europe, to what extent should I correct Galileo by the input from other constellations?". Slightly later: https://galmon.eu/ Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i 08:10 UTC: Galileo-UTC offset: -1.84 ns, Galileo-GPS offset: -0.32 ns, GPS-UTC offset: -3.21 ns, BeiDou-UTC offset: -4.77 ns, GLONASS-UTC offset: 2.79 ns, GLONASS-GPS offset: 45.63 ns, 18 leap seconds (GPS/Galileo) 08:20 UTC: Galileo-UTC offset: -1.84 ns, Galileo-GPS offset: -0.32 ns, GPS-UTC offset: -2.28 ns, BeiDou-UTC offset: -4.77 ns, GLONASS-UTC offset: 3.73 ns, GLONASS-GPS offset: 42.84 ns, 18 leap seconds (GPS/Galileo)  Note the drift above, even over ten minutes or so. MarkMLl ---------------------------------------------------------------------- AlanTheBeast · 2022-09-06 21:33 Quote from: mas steindorff on September 05, 2022, 11:24:39 pmQuote from: MarkMLl on September 02, 2022, 03:19:49 pmMy experience is that there is a slowly-varying systematic error between the various satellite constellations, a much larger variation between that and a PPS signal, and forget anything to do with a radiocode clock using consumer electronics. Dido!  but try and convince anyone you have a measurement better than "GPS".  Still have management folks that figure I'm full of it.  My testing was over 16 hour windows (at night) and a saw slow drifts up and down.  I was thinking it had something to do with cloud cover somehow creating a delay but different constellations make more sense as the drifts were somewhat repeatable on a 24 hour scale. (but so were the clouds to some degree). What were you testing it against? Please describe your test in some detail and what was the scale of the "slow drifts up and down". Clouds won't bother it, but ionospheric and tropospheric changes will certainly challenge it. ---------------------------------------------------------------------- AlanTheBeast · 2022-09-06 21:41 Quote from: MarkMLl on September 06, 2022, 09:09:52 amSlightly later: https://galmon.eu/ Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i 08:10 UTC: Galileo-UTC offset: -1.84 ns, Galileo-GPS offset: -0.32 ns, GPS-UTC offset: -3.21 ns, BeiDou-UTC offset: -4.77 ns, GLONASS-UTC offset: 2.79 ns, GLONASS-GPS offset: 45.63 ns, 18 leap seconds (GPS/Galileo) 08:20 UTC: Galileo-UTC offset: -1.84 ns, Galileo-GPS offset: -0.32 ns, GPS-UTC offset: -2.28 ns, BeiDou-UTC offset: -4.77 ns, GLONASS-UTC offset: 3.73 ns, GLONASS-GPS offset: 42.84 ns, 18 leap seconds (GPS/Galileo)  Note the drift above, even over ten minutes or so. MarkMLl From your post: Gal-UTC is 0 change over 10 minutes. Gal-GPS is 0 change over 10 minutes. GPS-UTC is 0.93 ns over 10 minutes. Doesn't make much sense unless the data is presented at 10 min intervals but the sampling is not in sync. ---------------------------------------------------------------------- mas steindorff · 2022-09-06 22:08 It's been a while but from what I remember: I was testing a (Temperature controlled Osc) Clock's calibration in a circuit of our own design.  I could capture the difference between the 1PPS from the GPS and the counter I had running off the TCO at 16MHz.  I setup a program that would collect/log this data every few seconds.  I was also monitoring the temperature of the air and the board and the TCO as part of the data set but these did not change as I had the system in a Temperature controlled box for this testing.  I was after the long term drift difference between the GPS and our clock and expected a liner drift.  But what I found was the differences I was logging would slowly grow for about an hour or two and then change. the rate of change sometimes would be reversed or would get worst but the error would return to 0 in the long scale. in other words if I average the differences over a longer term, it would go to the same value as it started with. since this 0 drift was what I was after, I only ran the test 4 or 5 times while the hardware was not being used.  all of my testing on different boards came to nearly the same results (± 10 parts in 16e6 per test). ---------------------------------------------------------------------- MarkMLl · 2022-09-07 07:12 Quote from: AlanTheBeast on September 06, 2022, 11:41:33 pmDoesn't make much sense unless the data is presented at 10 min intervals but the sampling is not in sync. Go read their code. It's open source. MarkMLl ---------------------------------------------------------------------- MarkMLl · 2022-09-07 07:23 Quote from: mas steindorff on September 07, 2022, 12:08:08 amsince this 0 drift was what I was after, I only ran the test 4 or 5 times while the hardware was not being used.  all of my testing on different boards came to nearly the same results (± 10 parts in 16e6 per test). I think the first thing one has to assume is that none of the GNSS designers and operators are idiots, so their underlying maths is probably sound. Following on from that, one needs to look at the available signals and make a "best guess" of a datum by looking at some weighting of the available signals that results in minimal overall discontinuities. One approach which would probably be valid would be to start off by assuming that your oscillator runs at the same rate as UTC, and over (say) 24 hours optimise weighting to minimise the GNSS errors relative to that. Then keep the same weighting and see if it continues to minimise errors over say a month: if it does then there's a reasonable chance that you've accounted for the various constellations inherent differences. People such as drone operators make a great fuss over the fact that they can get a stable position from GPS etc., but are decidedly coy about whether the position remains stable over hours or days. My experiments, even using differential correction relative to a comparatively local fixed site, suggests that they're brushing a great deal under the carpet. MarkMLl ---------------------------------------------------------------------- AlanTheBeast · 2022-09-07 21:47 Quote from: MarkMLl on September 07, 2022, 09:12:23 amQuote from: AlanTheBeast on September 06, 2022, 11:41:33 pmDoesn't make much sense unless the data is presented at 10 min intervals but the sampling is not in sync. Go read their code. It's open source. I spot a simple arithmetic discrepancy in data you post, and you want me to go looking for the source of the discrepancy?  Your issue, your mystery to solve. As previously stated:   I'm testing an NTP client at the user end which is in the low ms "realm" of errors using a reference method that is in the very low µs realm of error with reference to a receiver with a manufacturer declared error of ±30ns RMS, ±60 ns / 99%.  Even if the receiver were "off" by full microseconds it wouldn't mean much for this test.  And obviously the receiver can't account for the couple ns that are a burr under your saddle blanket... ---------------------------------------------------------------------- AlanTheBeast · 2022-09-07 21:59 Quote from: MarkMLl on September 07, 2022, 09:23:21 amQuote from: mas steindorff on September 07, 2022, 12:08:08 amsince this 0 drift was what I was after, I only ran the test 4 or 5 times while the hardware was not being used.  all of my testing on different boards came to nearly the same results (± 10 parts in 16e6 per test). I think the first thing one has to assume is that none of the GNSS designers and operators are idiots, so their underlying maths is probably sound. Following on from that, one needs to look at the available signals and make a "best guess" of a datum by looking at some weighting of the available signals that results in minimal overall discontinuities. One approach which would probably be valid would be to start off by assuming that your oscillator runs at the same rate as UTC, and over (say) 24 hours optimise weighting to minimise the GNSS errors relative to that. Then keep the same weighting and see if it continues to minimise errors over say a month: if it does then there's a reasonable chance that you've accounted for the various constellations inherent differences. People such as drone operators make a great fuss over the fact that they can get a stable position from GPS etc., but are decidedly coy about whether the position remains stable over hours or days. My experiments, even using differential correction relative to a comparatively local fixed site, suggests that they're brushing a great deal under the carpet. GPS isn't that stable, position wise, so there are things there in plain site and no carpeting is required, but perhaps their eyesight needs checking: - first off they have the benefit of a very over determined solution - secondly they have the benefit of SBAS in many places, esp. North America (WAAS) and Europe (EGNOS). further... - stability though smart use of accelerometers (aiding) - stability in the very nature of the control system of the drones: the control 'loops' tell the drone much about its real movement in the air - possible stability through vertical pointing cameras (DJI drones for example will auto land with remarkable accuracy - if the operator gives the drone the opportunity to take the reference image(s) in the several seconds after take off (ie: climb vertically to about 10m and pause.  This does not work all that well in strong winds as the drone is "pushed off centre" before it's high enough to make the reference image(s)). - appearance of stability in a few minutes or even 10's has little to do with hours, days or more. ---------------------------------------------------------------------- AlanTheBeast · 2022-09-07 22:11 Quote from: mas steindorff on September 07, 2022, 12:08:08 amIt's been a while but from what I remember: I was testing a (Temperature controlled Osc) Clock's calibration in a circuit of our own design.  I could capture the difference between the 1PPS from the GPS and the counter I had running off the TCO at 16MHz.  I setup a program that would collect/log this data every few seconds.  I was also monitoring the temperature of the air and the board and the TCO as part of the data set but these did not change as I had the system in a Temperature controlled box for this testing.  I was after the long term drift difference between the GPS and our clock and expected a liner drift.  But what I found was the differences I was logging would slowly grow for about an hour or two and then change. the rate of change sometimes would be reversed or would get worst but the error would return to 0 in the long scale. in other words if I average the differences over a longer term, it would go to the same value as it started with. since this 0 drift was what I was after, I only ran the test 4 or 5 times while the hardware was not being used.  all of my testing on different boards came to nearly the same results (± 10 parts in 16e6 per test). Hard to evaluate that, but one could wonder about temperature hysteresis of the setup as well (despite the care of the setup [ qui custodit custodes]).  But - as said - the ionosphere and (to a lesser degree) the troposphere do change from day to night and that alone could account for some of what you observed.  A big part of SBAS (WAAS here, EGNOS in Europe, other systems in Japan QZSS (very, very clever) and India refer) as things "taming" these effects. ---------------------------------------------------------------------- MarkMLl · 2022-09-08 07:51 Quote from: AlanTheBeast on September 07, 2022, 11:47:44 pmI spot a simple arithmetic discrepancy in data you post, and you want me to go looking for the source of the discrepancy?  Your issue, your mystery to solve. No mystery at all. I cherry-picked a couple of readings a few minutes apart, purely to illustrate that the relative timing /does/ vary and that there are people interested in it. I'd add that in my case I've got identical receivers, a shared antenna and splitter, and matched final cables... and I still see relative drift easily detectable on a logic analyser. You make the point that decent drones use multiple mechanisms to achieve stability. I'm making the point that for precision timing you can't ignore the fact that there are multiple GNSS constellations each reporting a slightly different time relative to UTC: even if one doesn't attempt to correct for it one should be aware of it: and that's /particularly/ the case in somewhere like the UK which doesn't have its own easily-accessible system (or a clear relationship with some jurisdiction which does). MarkMLl ---------------------------------------------------------------------- MarkMLl · 2022-09-08 07:54 Quote from: AlanTheBeast on September 07, 2022, 11:59:32 pm- appearance of stability in a few minutes or even 10's has little to do with hours, days or more. Yes, I agree. The point I was making was that people who say "my drone is stable therefore it demonstrates that you /can/ get stable nSec timing from GPS" are brushing stuff under the carpet. MarkMLl ---------------------------------------------------------------------- AlanTheBeast · 2022-09-08 12:33 Quote from: MarkMLl on September 08, 2022, 09:51:16 amQuote from: AlanTheBeast on September 07, 2022, 11:47:44 pmI spot a simple arithmetic discrepancy in data you post, and you want me to go looking for the source of the discrepancy?  Your issue, your mystery to solve. No mystery at all. I cherry-picked a couple of readings a few minutes apart, purely to illustrate that the relative timing /does/ vary and that there are people interested in it. You cherry picked and failed to notice that the information didn't pass its own checksum. Quote from: MarkMLl on September 08, 2022, 09:51:16 am I'd add that in my case I've got identical receivers, a shared antenna and splitter, and matched final cables... and I still see relative drift easily detectable on a logic analyser. You make the point that decent drones use multiple mechanisms to achieve stability. I'm making the point that for precision timing you can't ignore the fact that there are multiple GNSS constellations each reporting a slightly different time relative to UTC: even if one doesn't attempt to correct for it one should be aware of it: and that's /particularly/ the case in somewhere like the UK which doesn't have its own easily-accessible system (or a clear relationship with some jurisdiction which does). Yes, so many people need UTC±1 ns in the world.  Do you suggest an advertising campaign?  Raise funds? ---------------------------------------------------------------------- MarkMLl · 2022-09-08 12:38 Quote from: AlanTheBeast on September 08, 2022, 02:33:29 pmYes, so many people need UTC±1 ns in the world.  Do you suggest an advertising campaign?  Raise funds? Our use cases and interests differ. MarkMLl