[HN Gopher] AWS releases their version of Google Spanner's TrueT...
___________________________________________________________________
AWS releases their version of Google Spanner's TrueTime API
Author : sizediterable
Score : 36 points
Date : 2021-11-03 20:43 UTC (2 hours ago)
(HTM) web link (aws.amazon.com)
(TXT) w3m dump (aws.amazon.com)
| babelfish wrote:
| Using atomic clocks too, nice. From an older post:
|
| > It uses a fleet of redundant satellite-connected and atomic
| clocks in each Region to deliver time derived from these highly
| accurate reference clocks.
|
| https://aws.amazon.com/blogs/mt/manage-amazon-ec2-instance-c...
| drewda wrote:
| Perhaps the biggest "fake-out" in 21st century computing: Google
| publicly released its MapReduce paper -- directing most of the
| rest of the industry toward loosely coupled, overly complex
| distributed data processing systems like Hadoop for the following
| decade -- but internally they just bought a bunch of atomic
| clocks and built a distributed RDBMS.
|
| I know this is a somewhat simplified story, but it does make me
| chuckle.
| [deleted]
| skj wrote:
| Google uses map reduce extensively... where it's appropriate.
|
| True time helps with things like spanner transactions. It's
| just a totally different use case.
| drewda wrote:
| Yes, which is why it's amusing in hindsight that for a decade
| everyone* outside Google was forcing all* their distributed
| data tasks into the MapReduce paradigm, without considering
| alternative approaches like the one used by Spanner.
|
| * slight exaggerations, I know
| mrep wrote:
| I'm not sure how you think a distributed data processing
| technology would "fake-out" other companies when
| building/choosing database technology. They are totally
| different problem sets.
|
| MapReduce does not have a set in stone data source/sink and
| can use multiple things like bigtable and spanner so they
| are complementary technologies.
| quin3 wrote:
| It's not even related. No one was running OLTP workloads as
| MapReduce jobs at any point.
| d_watt wrote:
| Those aren't really equivalent, are they? Hadoop is for
| analytics, spanner is transactional.
|
| In terms of popular nosql vs google sql products, it's more
| Hadoop : BigQuery :: Mongo? : Spanner
|
| You're pretty explicitly not supposed to run OLAP queries on
| spanner.
| adamfeldman wrote:
| Google and GCP offer their own NTP endpoints. Does anyone know if
| GCP also exposes the TrueTime API to customers, or if it's only
| internal to Spanner?
|
| I can't find any instance of the word "bound" in the GCP or
| Google NTP docs.
|
| [1]:
| https://developers.google.com/time/guides#google_compute_eng...
|
| [2]: https://cloud.google.com/compute/docs/instances/managing-
| ins...
| londons_explore wrote:
| I'm pretty sure if you use the GCP dataflow stuff, the dataflow
| worker VM's will be given access to an RPC server which has the
| necessary endpoints for TrueTime.
|
| Obviously it's all a matter of reverse engineering rather than
| documented API's.
| tyingq wrote:
| I'm curious about the title on this submission. I thought
| TrueTime had unusually strong guarantees about accuracy that
| don't seem to be called out in what I'm reading on the linked
| article.
| morei wrote:
| Not quite: Truetime has guarantees about ordering, rather than
| accuracy.
|
| If you ask for the time and get A, and then ask for the time
| again and get B, then Truetime guarantees that A is less than
| B.
|
| Obviously, in a distributed setting this is much easier to do
| if you have accurately sync'ed clocks, but that accuracy goes
| to reducing the uncertainty in the time (and hence making
| truetime faster) rather than providing accuracy.
| sizediterable wrote:
| Sorry if it's a bit sensationalized. I wanted to give some
| color as to why this release might be interesting and took this
| tweet I saw at face value
| https://twitter.com/rbranson/status/1455923426359578631
| tyingq wrote:
| Oh, I wasn't criticizing, just genuinely curious. If the AWS
| service has similar accuracy, that's interesting to me. AWS
| doesn't seem to say much about the details.
| londons_explore wrote:
| It's _really_ hard to use these API 's correctly.
|
| Remember... your CPU can halt at any time for any number of
| milliseconds. That means simple things like:
| upperBound, lowerBound = readTime() if
| (upperBound<deadline) do_stuff(x, y, z)
|
| Are incorrect... There is no guarantee that the 'if' statement
| didn't take many milliseconds, and that the stuff didn't end up
| happening after the deadline.
|
| It's also very easy to write code that works, but is
| theoretically wrong. You will leave a hidden bug that may only
| rear its head years down the line.
| karmakaze wrote:
| Is that really the API? I'm sure I'd be writing:
| lowerBound, upperBound = readTime()
|
| often without noticing my error.
| jeffbee wrote:
| One of the key aspects of the TrueTime system is any device with
| serious clock error is simply murdered. It seems like offering
| that would significantly benefit users of this AWS API.
___________________________________________________________________
(page generated 2021-11-03 23:00 UTC)