[HN Gopher] Neon - Serverless Postgres
       ___________________________________________________________________
        
       Neon - Serverless Postgres
        
       Author : nikolay
       Score  : 575 points
       Date   : 2022-05-28 01:37 UTC (21 hours ago)
        
 (HTM) web link (neon.tech)
 (TXT) w3m dump (neon.tech)
        
       | rattray wrote:
       | FYI, the image at the bottom says "Zenith's prices" instead of
       | "Neon's prices"
       | 
       | https://neon.tech/static/saas-illustration-lg-410ada378df755...
        
         | nikita wrote:
         | Fixed. Thank you for catching in. Renaming a company is hard
         | even pre launch :)
        
           | rattray wrote:
           | I know the pain :)
           | 
           | Good luck, this is a very exciting project. I'm extremely
           | curious to see how it unfolds...
        
       | ushakov wrote:
       | that's one of the best lookin landing pages i've seen this year!
       | 
       | very well executed! congrats!
        
         | nikita wrote:
         | Thank you. PixelPoint (https://pixelpoint.io/) designed the
         | landing page. Copy stuff may change we want to make the story
         | clearer and highlight more features.
        
       | Aeolun wrote:
       | I need some kind of pricing for me to consider this. Did I miss
       | the page describing that?
        
         | nikita wrote:
         | We haven't launched pricing yet. In fact our official launch is
         | in a few weeks. This HN post "leaked". We will launch, allow
         | people to use it for free and introduce pricing later this
         | year.
        
       | jasonhansel wrote:
       | Does this handle sharding? Or is it like ordinary Postgres, which
       | only performs replication?
        
         | mattashii wrote:
         | It is ordinary PostgreSQL, in which only the storage interface
         | with the file system is replaced with our own storage
         | interface.
        
       | talkingtab wrote:
       | Why is this a good idea? In my experience, getting Postgres up
       | and running is trivial. Docker anyone? And in many cases your
       | data is your business so why hand it off? And if you are going to
       | offer this product why not just call it what it is, "Postgres as
       | service", instead of serverless which seems a bit misleading.
       | Really it is simply Postgres running on _your_ server.
        
         | chimen wrote:
         | Not everyone can manage a database properly and, sorry to say
         | this to you but, Docker is a terrible idea for a database in
         | general. Setting up your own database somewhere still puts your
         | trivial data on someone else's server more or less.
         | 
         | All these "Serverless" keywords pretty much mean you don't have
         | to be spinning up servers (cloud) or setting up & maintaining
         | one. Nothing is "Serverless" per-se so it's time to move on
         | from picking on this, I agree, bad choice of words.
        
           | smokey_circles wrote:
           | > Docker is a terrible idea for a database in general
           | 
           | Why? Genuine question, my gut feel is there's something wrong
           | about it too but I can't put words to it nor have I found a
           | benchmark that convinced me, but it's worth noting I'm not
           | sure what I'm looking for
        
             | chimen wrote:
             | I manage about 200 servers and docker crashing accounts for
             | 20%+ of my issues so far. The servers are brought up easily
             | on crash and that's not an issue for my services. For a
             | database docker is nothing more than an extra layer of
             | complications on top with iptables, volume system and all
             | the layers it brings. It's just a bad wrapper for a
             | production database which needs stability.
        
       | Komodai wrote:
       | Typo in footer 'Maid in SF and the World' should be 'Made in SF
       | and the World'
        
         | nikita wrote:
         | Fixed - sorry, sleepless night.
        
       | cja wrote:
       | In what way is this serverless? Surely it's running on a server?
        
         | ttfkam wrote:
         | Refers to management and autoscaling, not whether physical
         | hardware exists somewhere. From your point of view it has DNS
         | entries, you've defined how high/low it's allowed to scale and
         | some other configs, you've defined network access rules, and
         | allows client connections. That's it. You don't manage the
         | server instances. Hence: "serverless"
        
           | cja wrote:
           | Thank you for explaining. There must be a better term than
           | serverless though. "Auto-scaled cloud database" would be more
           | accurate, for example
        
       | AndrewDucker wrote:
       | Having a relational database where you're charged purely for the
       | calls you make is a game-changer.
       | 
       | All of the relational databases I looked at in the past required
       | you to have a gateway node on at all times, which is far too
       | expensive for a simple hobby project.
        
       | rapind wrote:
       | I wonder how well this will work with PostgREST. Looks very
       | interesting.
        
       | lewisl9029 wrote:
       | Seems like this might implement database branching in the way
       | most people would assume: branching both the data and schema? I
       | remember being a bit disappointed to learn that PlanetScale's
       | database "branching" was only for the schema [1], which is still
       | quite useful, but this would be so much cooler!
       | 
       | I couldn't find much info about the replication models
       | available/planned however. I would consider this to be table
       | stakes at this point for a serverless database with the recent
       | trend of pushing compute to the edge. This is much more
       | interesting to me than scaling to 0, which is only really useful
       | during the prototyping phase.
       | 
       | PlanetScale is single primary with eventually consistent read
       | replicas, Fauna has strongly consistent global writes (or
       | regional if you choose, but no option for replication between
       | regions if you do) with a write latency cost, Dynamo/Cosmos are
       | active-active eventual consistently replicated with fast writes
       | globally. All useful in different scenarios, but I'd love to have
       | one DB tech that can operate in all of these modes for different
       | use cases within the same app, using the same programming model
       | to interact with data across the board.
       | 
       | I think the decoupled storage engine here would open up some
       | really interesting strategies around replication. What are the
       | team's plans here?
       | 
       | [1] https://docs.planetscale.com/concepts/branching
        
         | nikita wrote:
         | Great questions!
         | 
         | 1. Yes schema and data via "copy on write". This will let you
         | instantly create test environments, backups, and run CI/CD.
         | There is a long video here that shows a prototype with GitLab:
         | https://www.youtube.com/watch?v=JVCN9X-vO1g&t=1s.
         | 
         | 2. We don't have this feature at the launch, but Matthias van
         | de Meent is already working on it. We will publish and RFC and
         | solicit comments from the community.
         | 
         | 3. We are working on two: regional read replicas and consistent
         | multi-region writes (together with Dan Abadi who helped design
         | FaunaDB). Former is much, MUCH easier.
         | 
         | 4. An obvious one is a time machine - we want to allow you
         | query at LSN (or timestamp). A less obvious one is templates:
         | you can start your project with a pre-populated database. We
         | will allow you to create and publish such "templates".
         | Disclaimer - it might not be called templated when we ship it.
        
           | rattray wrote:
           | For those unfamiliar, LSN is "Log Sequence Number", a pointer
           | to a location in the WAL (Write-Ahead Log).
           | 
           | https://www.postgresql.org/docs/current/datatype-pg-lsn.html
        
       | sudhirj wrote:
       | Does this storage layer still require VACUUMing? The Aurora
       | system that AWS has uses a log based storage system, so along
       | with point in time restore there's no longer a need to go free up
       | old garbage rows. This system says it's copy on write, wondering
       | if vacuuming is still necessary.
        
         | petergeoghegan wrote:
         | The difficulty comes from the fact that VACUUM requires most of
         | the same state that is required when processing a user
         | transaction. For example, VACUUM needs the commit log to check
         | whether a given transaction ID is for a transaction that
         | committed or aborted (VACUUM also truncates the commit log to
         | remove information about older XIDs when it is no longer
         | required).
         | 
         | Aurora Postgres does still have VACUUM, which seems to work in
         | the same way -- which is the same in Neon. AWS have in the past
         | promoted Aurora as having significantly better performance
         | characteristics when VACUUM is run. That may well be true, but
         | the benefits seem to come from not having to generate full page
         | writes in WAL, which are a way of preventing a low-level
         | problem called torn pages. Theoretically you could just turn
         | off full-page writes in standard Postgres if you had hardware
         | that offered atomic page writes, though I don't think that it's
         | a widespread practice.
         | 
         | I spend most of my time working on problems with VACUUM in
         | Postgres itself (I'm one of the committers). An approach to
         | organizing storage within transactional constraints seems
         | necessary to push vacuuming down to storage, and that would
         | more than likely need plenty of work in Postgres itself to be
         | practical -- since it would cross a few layers of abstraction.
         | Currently heapam doesn't specifically try to keep tuples
         | inserted around the same time together, so it's hard to make
         | anything that VACUUM does work implicitly or logically.
         | 
         | Disclaimer: I work for Neon
        
           | sudhirj wrote:
           | Thanks for the explanation, I'm nowhere near your experience
           | level with this, but from what I understand Aurora is an
           | append only log with rolling snapshots (this might be wrong).
           | In an append only log storage system there would be no
           | concept of page writes, right? The VACUUM process might still
           | be necessary to find and mark dead tuples, but this seems
           | very minor compared to filesystem operations to reclaim
           | (sometimes gigabytes) of storage. The reclamation of storage
           | would simply left to the background snapshotting process and
           | would not affect DB performance in any way.
        
             | petergeoghegan wrote:
             | The most important job of VACUUM is usually to make sure
             | that queries remain responsive -- it's roughly comparable
             | to a "full scan" within generational garbage collectors
             | (for managed language runtimes). VACUUM does reclaim space
             | in various ways, but that's often considered to be of
             | secondary importance.
             | 
             | In Postgres, the on-disk representation is virtually the
             | same thing as the in-memory representation used by pages
             | stored in the buffer cache. In a system like Aurora or
             | Neon, the representation of pages in the buffer cache is
             | identical to that of Postgres (or has very minimal
             | divergence to deal with one or two isolated problems). That
             | part doesn't really change, which makes it possible to have
             | a very deep level of compatibility without much effort. So
             | it's radically different in one narrow, scoped way, but
             | otherwise very similar.
             | 
             | While the storage knows how to materialize pages on demand,
             | these are not transactionally consistent pages -- they
             | often need to be interpreted by using metadata about
             | transaction commit status, just like in Postgres.
        
         | hlinnaka wrote:
         | Yes, it still requires VACUUM. I believe Aurora does too.
         | 
         | It would be cool to push down at least parts of the VACUUM down
         | to the storage layer, but it would require more invasive
         | changes Postgres code, which we try to avoid. Maybe in the
         | future. Ideally though, I'd like to improve PostgreSQL itself,
         | to reduce the need for VACUUM in the first place.
         | 
         | Disclaimer: Neon co-founder
        
           | canadiantim wrote:
           | What are your thoughts on orioleDB? Which is a slightly
           | modified Postgres that does away with the need for vacuum,
           | amongst other changes. Ostensibly done by a dev who is
           | "solving Postgres' wicked problems". Eventually the team
           | behind orioleDB plan to upstream their minor changes to
           | vanilla Postgres such that orioleDB becomes simply just an
           | extension of Postgres itself.
           | 
           | Would you ever consider supporting the orioleDB extension in
           | the future?
        
       | captnObvious wrote:
       | I hope y'all have a plan for when AWS decides to pick up your
       | open source project and turn it into a managed cloud solution.
       | It's a pattern of theirs. And with the way egress charges are
       | structured they're likely to snap up any clients straddling their
       | cloud and yours.
        
         | oxfordmale wrote:
         | AWS already has this, Google for Aurora Serverless. It is not
         | cheap though, and this might well be cheaper.
        
         | mattashii wrote:
         | AWS already has Aurora, which is their own in-house closed-
         | source variant that does very similar things.
         | 
         | We think we'll be able to provide a better experience at lower
         | cost for smaller developers, while having some very useful
         | quality-of-life features like zero-cost branching and instant
         | PITR.
        
         | onphonenow wrote:
         | AWS has Aurora Serverless v2 out already for postgresql along
         | with RDS for postgresql. Not scale to zero though / bottom is
         | 40/month or so
        
       | thomasfromcdnjs wrote:
       | Anyone got a spare invite code? =D
        
         | felixmeziere wrote:
         | Same here please!
        
           | oknoorap wrote:
           | same here ^_^
        
             | nikita wrote:
             | We got a lot of "join waitlist" submission. Thank you
             | HackerNews for this!
             | 
             | Please email beta@neon.tech for the invite code. We will
             | ask for feedback in return.
        
       | tomatowurst wrote:
       | how does it differ from fly.io?
       | 
       | https://fly.io/blog/globally-distributed-postgres/
        
         | tptacek wrote:
         | Our Postgres is just a cluster of Postgres servers. It's not
         | "serverless"; it's server-y. :)
        
       | XorNot wrote:
       | Well this seems like a big deal. "Needs a database" has been an
       | ongoing pain point for working with serverless apps. Staying in
       | the postgres command library, if it works, means you can now just
       | prototype and assume "postgres will be there".
       | 
       | I have a couple of apps I would've used this exact service for in
       | the recent past. Looking forward to trying it in the future.
        
       | kosolam wrote:
       | Cool! Can be used in production already?
        
         | mattashii wrote:
         | We cannot yet recommend users to run their critical workloads
         | on Neon.
        
         | kelvich wrote:
         | We are planning to open public tech preview next month. We'll
         | go to general availability when we see a decent SLO for several
         | months. Tech preview should be okay for hobby projects and
         | staging workloads, though.
        
       | rad_gruchalski wrote:
       | What's the licensing story? Can I run this in my org and expose
       | this as a service to users outside of my org?
       | 
       | Or does it have strings attached, like CockroachDB?
       | 
       | Asking because github says Apache 2 but the devil is in details.
        
         | kelvich wrote:
         | You can, no strings attached
        
           | rad_gruchalski wrote:
           | Awesome, will take it for a spin.
        
         | mattashii wrote:
         | Yep, for now the storage engine and the modifications on
         | PostgreSQL are licensed under APLv2. Our cloud offering which
         | orchestrates the scale-to-0 and other glue between the
         | components are closed-source and not covered by that license.
         | 
         | Do note however that the licensing story is not entirely
         | fleshed out yet, as the product we're building is still in
         | closed beta. As we work towards a paid cloud offering, we'll
         | further flesh out the license model for the code, but for now
         | we're planning on keeping this license.
        
       | igammarays wrote:
       | Are they going to stay up-to-date with the latest version of
       | Postgres? One problem with Yugabyte, TimescaleDB, Aurora etc. is
       | that they are stuck on older versions of Postgres, which makes it
       | feel like an entirely different product after a few years.
        
         | mattashii wrote:
         | We are planning on supporting the latest stable version of
         | PostgreSQL. Right now, we're a bit behind (we're at 14.1,
         | latest PosgreSQL is 14.3) but that shouldn't be much of an
         | issue.
         | 
         | We don't yet know how we're going to do major version
         | migrations, as the product is still not even out of private
         | beta.
        
           | rattray wrote:
           | "how will this handle major version upgrades" was one of my
           | first/biggest questions when reading through the homepage
           | fwiw
        
       | infogulch wrote:
       | How fast can a "scaled to zero" database start up? Does Neon use
       | a "uninitialized hot spare" strategy to reduce startup latency
       | like crdb?
       | 
       | How much memory do they expect a typical single postgresql
       | compute instance to take? I saw that Neon is targeting
       | 'thousands' of postgresql processes per server, though with giant
       | multi-TB servers these days that doesn't really narrow it down.
       | 
       | Are the postgresql processes multi-tenant as well, or is
       | multitenancy isolated to the storage layer?
       | 
       | ---
       | 
       | Heikki from the Neon team presented a talk about why they chose
       | to develop Neon in Rust and what their experience was in Rust
       | Finland 2022. https://www.youtube.com/watch?v=kAQeout-mh8
        
         | Shorn wrote:
         | https://youtu.be/kAQeout-mh8?t=700
         | 
         | Nobody knew Rust, so they started out by hiring someone who
         | did. Good move.
         | 
         | Business idea: consultancy that hires out competent Rust devs
         | to new projects.
        
         | nikita wrote:
         | Postgres process in single tenant. Right now both provisioning
         | a new Postgres instance (we call it project) and cold start is
         | 2 seconds. We will be improving on that.
        
       | peterkelly wrote:
       | Please stop this abuse of language.
       | 
       | Here's serverless sqlite: https://www.sqlite.org/serverless.html
        
         | tuukkah wrote:
         | Don't say server-"less" if you mean in-process (which may well
         | run on a server).
         | 
         | Here's Sqlite without a server, running in the client and
         | backed by static file hosting:
         | https://phiresky.github.io/blog/2021/hosting-sqlite-database...
        
         | dandanua wrote:
         | Modern marketing makes me mad sometimes. They call something
         | "serverless" while in fact it runs on their servers somewhere
         | in the world, where you have even less control than on your
         | cloud. Though, I believe it was popularized by Amazon, which
         | doesn't care much about being honest in its expansionism.
        
         | derefr wrote:
         | People don't say "serverless" to mean "there are no servers
         | involved." They say "serverless" to mean that the data
         | architecture is such that:
         | 
         | 1. scaling does not require thinking about servers; and
         | 
         | 2. you don't have to pay for committed capacity measured in
         | servers.
        
           | hestefisk wrote:
           | Then why not call it "scalable", "elastic"? All these
           | neologisms are tiring.
        
             | ignoramous wrote:
             | Marketing doesn't care about your personal preferences :)
             | 
             | And Marketing is King, for bottom-up SaaS businesses
             | (businesses selling to individual developers), which most
             | companies open-sourcing their core projects most often are.
        
         | jjeaff wrote:
         | From your link:
         | 
         | Classic Serverless: The database engine runs within the same
         | process, thread, and address space as the application. There is
         | no message passing or network activity.
         | 
         | Neo-Serverless: The database engine runs in a separate
         | namespace from the application, probably on a separate machine,
         | but the database is provided as a turn-key service by the
         | hosting provider, requires no management or administration by
         | the application owners, and is so easy to use that the
         | developers can think of the database as being serverless even
         | if it really does use a server under the covers.
         | 
         | It sounds like this is neo-serverless but serverless
         | nonetheless.
        
           | eyelidlessness wrote:
           | It "is" in the generous sense that the quoted section
           | interprets generously for the purpose of interpreting it
           | generously.
        
         | tomatowurst wrote:
         | you are really nitpicking and taking offense over something you
         | made up in your head. The context of serverless has changed to
         | mean cloud offering.
        
         | infogulch wrote:
         | This ship is so sailed already that the stories of its voyages
         | are legendary classics that were told to our great
         | grandparents.
        
           | CharlesW wrote:
           | > _This ship is so sailed already that the stories of its
           | voyages are legendary classics that were told to our great
           | grandparents._
           | 
           | Specifically, the term is 10 years old this October:
           | http://readwrite.com/2012/10/15/why-the-future-of-
           | software-a...
           | 
           | It has never, _ever_ meant that the software wasn 't running
           | on a server. From TFA:
           | 
           |  _" The phrase 'serverless' doesn't mean servers are no
           | longer involved. It simply means that developers no longer
           | have to think that much about them. Computing resources get
           | used as services without having to manage around physical
           | capacities or limits."_
        
           | ahartmetz wrote:
           | Ah, the captainless ship (where the captain is behind a
           | locked door).
        
           | eyelidlessness wrote:
           | It's a tale so old that "server" lost meaning before we could
           | even be in a state of "serverless"ness. First there was a
           | cloud, and then there was no clarity of what conditions a
           | process even runs under. And it wasn't good, but it became
           | stable. And on the next day, there were databases from on
           | high, as if they'd spontaneously burned a bush.
        
       | zwily wrote:
       | How exciting!
       | 
       | What version of Postgres are they targeting? And do they have a
       | strategy for keeping up to date with new versions?
        
         | mattashii wrote:
         | Right now, we're based on PG14 (14.1 to be precise).
         | 
         | We're looking into supporting PG15 when that comes out too, but
         | I wouldn't hold my breath on that as we still have a lot to do.
        
         | nikita wrote:
         | One idea we have is to allow running future unreleased and beta
         | versions of Postgres. Do people want this?
        
         | jadbox wrote:
         | Says 14 in the docs
        
       | tuukkah wrote:
       | In case anyone else is wondering, it was called Zenith / ZenithDB
       | before launch: https://github.com/neondatabase/zenith
        
         | nikita wrote:
         | Neon is a better name!
        
       | coder543 wrote:
       | > Branches are virtually free and implemented using the "copy of
       | write" technique.
       | 
       | Copy _on_ write, presumably.
        
         | kelvich wrote:
         | Fixed, thank you!
        
       | riedel wrote:
       | This sounds really interesting! I wonder what kind of scaling use
       | cases neon is good for. Is it e.g. good for custom scenarios like
       | a geospatial timeseries database on top of postgres?
       | 
       | We have admittedly not really a clue about current database
       | cluster tech as we are IoT/ML researchers, but we are running a
       | custom timescaledb cluster that receives constant nonchunked
       | write load from a lot of devices and may encounter some long
       | running queries on an around 500GB DB filled with geolo (even
       | timing out if users are too creative), why we splitted into a
       | single ingress master and multiple outgres WAL readonly
       | replicated query clients to relax the consistency and sync, that
       | seemed to be killing us (we need postgres because of postgis and
       | have no capacity to rewrite the front-end). I wonder if neon
       | would be good for such a use case and if it easily supports
       | postgres extension like timescaledb hypertables and postgis).
       | Most of the time our system just measurements, but sometimes we
       | really need to scale up for PoCs, which makes dimensioning really
       | hard (for us).
        
         | mattashii wrote:
         | Neon has the ability to make a Copy-on-write replica of a
         | database, which for you would allow you to create an instant
         | read-only copy of the data that has been ingested up to that
         | point, without significant storage overhead. The new data would
         | still be written to the primary database, and long running
         | queries would only see the snapshot that the instance was
         | started with (using their own pool of CPU and memory
         | resources).
         | 
         | Assuming that the extensions that you use are compatible (that
         | is, they don't access database files in a way that PostgreSQL
         | doesn't, and the licence is compatible) then Neon could be a
         | good solution to that issue.
        
       | cultofmetatron wrote:
       | don't know too much about this yet but i've been desperately
       | wanting there to exist a postgres equivalent to planetscale.
       | Bravo guys! I'll keep you guys in mind for my next mvp.
       | 
       | PS: btw why call it Neon? thats already the name of the
       | rust/nodejs interop library which I assume you know about since
       | your storage layer is written in rust.
       | 
       | PS: imagine a colab between this and fly.io. I wish thsi stuff
       | was available when I was starting Blinq
        
       | IgorPartola wrote:
       | I am trying to understand how it works without digging into the
       | code. It sounds like the disk-backed storage here uses S3 which
       | would introduce some severe latency as well as orders of
       | magnitude more access errors (S3 is not going to be more reliable
       | than EBS, let alone physical disk arrays on a day to day basis).
       | Also how do they mitigate latency from their network to mine? In
       | other words why would I run this over a local install if
       | performance mattered at all to me?
        
         | mattashii wrote:
         | How it works is:
         | 
         | PostgreSQL WAL is sent to 3 'Safekeeper' nodes, which provide
         | temporary persistence of WAL on their local disks. This allows
         | us to provide low commit latencies.
         | 
         | After Safekeepers acknowledge the WAL, a PageServer will
         | receive the WAL from these Safekeepers and transform it into
         | LSM-tree "Layers" - blocks of lookup-optimized changelogs,
         | which (when complete) are sent to S3. At that point, the data
         | is considered fully persisted against most, if not all,
         | outages.
         | 
         | The PageServer (which serves as the long-term data server for
         | the running compute nodes) maintains a local cache of Layers.
         | Still, by design, that is only a cache -- it allows for fast
         | responses but is not strictly necessary for the persistence
         | model.
        
           | cpursley wrote:
           | Can we set our own logical replication? I'm listening to the
           | WAL the same way Supabase does it:
           | https://github.com/cpursley/walex#logical-replication
        
             | mattashii wrote:
             | In our cloud offering logical replication is not yet
             | supported. There's an open epic to support receiving a
             | logical replication stream [0], but we've not yet checked
             | on sending logical or physical replication streams.
             | 
             | [0] https://github.com/neondatabase/neon/issues/1632
        
               | netcraft wrote:
               | Everything you've described sounds unbelievably awesome -
               | this is the only thing I can think of that would make it
               | better. If we can get all of these features (I've been
               | dreaming of branching for years, scale to zero!,
               | unlimited storage - just not having to think ahead about
               | it!) plus easy, even on by default logical replication,
               | you will have found the holy grail of cloud databases
               | imo.
        
           | IgorPartola wrote:
           | Again I am super impressed with the technology involved but
           | do want to clarify: in order to have D in ACID the update
           | must be sent to S3, right? Is there a mode which makes it so
           | that an INSERT, UODATE, and DELETE do it return until this
           | happens? What kind of latency does that introduce and is that
           | latency affected by throughput at all?
        
             | manigandham wrote:
             | Many distributed systems offer ACID by using distribution +
             | replication for the initial write commit.
             | 
             | It's much faster and cheaper to just have your data on
             | multiple nodes (RAM or local disk) and provides better
             | reliability against crashes. Data can then be compacted and
             | streamed out in an async fashion to more durable storage.
        
             | mattashii wrote:
             | Kind of. S3 is the long-term low-cost durability guarantee,
             | while our Safekeepers (3, each in a different zone) provide
             | a high-cost short-term durability guarantee with their
             | local persistent disks.
             | 
             | Latency from PostgreSQL WAL to S3 depends on WAL throughput
             | and the configured pageserver checkpoint distance (default
             | 256MB, and this config field is not equal to that of
             | PostgreSQL).
        
               | IgorPartola wrote:
               | When you say short term do you mean for hot data or that
               | the guarantee is short term? As in, once it is written to
               | the Safekeeprs is there any chance that the data will
               | disappear?
        
               | mattashii wrote:
               | We keep it there for a short duration, until the changes
               | are confirmed to also be written to S3.
               | 
               | Writing to 3 instances in 3 availability zones is
               | considered persistent enough while also maintaining a
               | high performance, and even though it does not provide the
               | 11 9s of durability that S3 has, 3 availability zones
               | dropping out with loss of all instance-local storage is
               | considered rare enough that we do not think that it will
               | impact our availability and durability guarantees.
        
               | IgorPartola wrote:
               | That makes sense, thank you! Sounds pretty damn robust.
        
         | tuukkah wrote:
         | Well, you can also run this as a local install if you want to
         | eliminate network latency. :-)
        
           | nikita wrote:
           | Short answer: page servers and cache on compute nodes hide S3
           | latency. An architecture description is here:
           | https://neon.tech/docs/storage-engine/architecture-overview/.
           | We will be publishing more on that.
        
       | SonOfLilit wrote:
       | > Neon allows to instantly branch your Postgres database to
       | support a modern development workflow. You can create a branch
       | for your test environments for every code deployment in your
       | CI/CD pipeline.
       | 
       | > Branches are virtually free and implemented using the "copy on
       | write" technique.
       | 
       | Unless I missed that everyone supports this, this here could be a
       | killer feature and should be advertised higher.
        
         | jhgb wrote:
         | It sounds like something you might be able to accomplish with a
         | copy-on-write VFS on top of a Firebird database file. (Not sure
         | about PostgreSQL, but with Firebird, you only deal with one
         | file, so with Firebird, this should definitely work.)
        
           | nikita wrote:
           | There is an enterprise company called Delphix that does it on
           | top of Zfs - so the idea was in the air.
           | 
           | Instead of duct taping this together with a filesystem we
           | purpose built database storage. The advantage to this is that
           | we can much tighter control execution paths and can profile
           | them end-to-end. Additionally this allows us to integrate
           | with S3 and make it much much cheaper to run.
        
             | jhgb wrote:
             | Technically Firebird just requires a block device, so you
             | might not even need a filesystem.
        
         | zxspectrum1982 wrote:
         | You can get that feature on any Postgres server by installing
         | Citus
        
           | mattashii wrote:
           | Does Citus provide any such storage-level multi-cluster
           | features? I can't seem to find any documentation on that...
        
         | rkwz wrote:
         | What are the intended usecases for "branching" a database?
         | Currently, I use separate databases for different environments,
         | are branches better?
        
           | thejosh wrote:
           | It's a great feature on heroku for branches, it shares data
           | between review apps. Quite nice.
        
           | ukd1 wrote:
           | Have a staging / qa env, then fork it for a branch for
           | testing. Much faster than reseeding / restoring.
        
           | kelvich wrote:
           | Now the most common setup is to copy the production database
           | to the staging once in a while and test migration against
           | staging. With branching, you can test each PR against its own
           | production database branch -- just put branch creation in
           | your CI config. Hence, it has fewer moving parts, is a bit
           | easier to set up, and reduces the lag between prod and
           | staging.
        
         | jvolkman wrote:
         | AWS Aurora Postgres supports this to an extent with "clones".
         | You can even clone cross-account. The same copy-on-write stuff
         | applies, so they're relatively cheap and fast. I hope that
         | Google's new AlloyDB will also support it.
         | 
         | https://aws.amazon.com/about-aws/whats-new/2019/07/amazon_au...
         | 
         | There are some annoying restrictions, though. You can only have
         | a single cross-account clone of a particular db per account.
        
           | samokhvalov wrote:
           | The problem with Aurora's thin clones is extra cost each
           | clone adds
           | 
           | For CI/CD, you want multiple clones running on the same
           | compute power, in a shared environment, to keep the budget
           | constant
        
         | samokhvalov wrote:
         | Agreed, this direction is underestimated and should be
         | developed better -- we (Postgres.ai) do it for any Postgres
         | with our Database Lab Engine [1], and Neon would bring even
         | more power if it's installed on production
         | 
         | [1] https://github.com/postgres-ai/database-lab-engine
        
       | [deleted]
        
       | iknownothow wrote:
       | I knew my bet to sticking with Postgres would pay off! This looks
       | super exciting.
       | 
       | I thought of doing something similar for our data warehouse with
       | AWS Fargate and Postgres but the cold starts and limited disk
       | space required too much engineering on top to make it work.
       | 
       | Moving to Snowflake comes at the cost of losing so many Posgtres
       | features in exchange for speed. Things like foreign keys,
       | constraints, extensions etc which requires so much engineering to
       | replace in Snowflake. I would be happy to pay 25x the price for a
       | 10x speed increase for a specific query.
        
         | manigandham wrote:
         | Snowflake is an OLAP system. It's an entirely different kind of
         | "speed" designed for analyzing vast amounts of data through
         | scans and aggregations.
        
         | thejosh wrote:
         | Snowflake is a data warehouse though. Completely different use
         | case.
         | 
         | If your data can be done via PG, highly recommend that over SF.
         | Especially with this concept.
         | 
         | Snowflake is great when you use a tool like dbt, their modern
         | SQL approach and functions are fantastic. Downsides is it's
         | pretty pricey, and can catch you out.
        
         | nikita wrote:
         | Thank you!
         | 
         | Snowflake is a better cloud data warehouse than Postgres, but
         | of course Postgres is so versatile. Neon will give you some of
         | the Snowflake features: time machine, cloning - we call this
         | branching, data sharing.
        
       | nikita wrote:
       | Nikita - CEO of Neon here. We intended to post this at the launch
       | next month, but since it here, I'm happy to answer any questions.
       | 
       | We have been hard at work and looking to open the service to the
       | public soon.
        
         | zeusly wrote:
         | Hey Nikita, could you maybe put some more legal information on
         | the webpage?
         | 
         | I'm trying to find out if you're a company and where you are
         | located. Is there no legal entity behind this? Do you have a
         | privacy policy?
        
           | mattashii wrote:
           | The company is Neon, Inc., which is registered in the USA.
           | We're a remote company, with a significant portion of the
           | developers being located in Europe.
           | 
           | Privacy policy and related stuff will be ready when we
           | publish the public beta, which we expect to happen soon.
        
             | nikita wrote:
             | Yep, Delaware corp with top tier US investors. I'm in the
             | Bay Area. Heikki is in Finland. Stas is in Cyprus. Majority
             | of engineering is in Europe, some in the US and Canada.
             | 
             | Postgres is a global phenomenon.
        
         | timmg wrote:
         | How "cheap" is it to create new db instances?
         | 
         | I can imagine a world where it might be practical to have one
         | master db for all of your customers/accounts. But a separate db
         | instance for each customer's data.
         | 
         | Is that the kind of architecture you think might be workable
         | with your system?
        
           | nikita wrote:
           | It's cheap. Storage footprint is 15Mb and will be shrunk
           | further. Min compute footprint is a 1 core container that
           | shuts down when not used.
           | 
           | We are already working with customers that do that. This is
           | for sure a great use case for Neon.
        
             | unraveller wrote:
             | won't the tiny compute units on AWS have relatively slow
             | storage? (no NVMe allowed for them I think) fine for small
             | datasets that fit in ram but benchmarks are needed to show
             | the bigger picture.
        
         | httgp wrote:
         | Do you plan to solve for global data-at-the-edge availability?
         | That to me is the killer feature for databases and one I'm
         | direly in need of at work.
        
           | nikita wrote:
           | Yes, we are discussing either simply using Postgres
           | replication to move data to other regions and use our proxy
           | to rout reads to the datacenter closer to the user (like
           | fly.io). This will have issues with supporting more than ~5
           | regions.
           | 
           | OR we can separate storage from replication and purpose build
           | a multi-tenant replication service. This will support as many
           | regions as you want (over 200) but it's more work. We will
           | publish an RFC for that.
        
         | code_biologist wrote:
         | Cool stuff! Is PostGIS support difficult?
        
           | nikita wrote:
           | It's supported. The beauty of the architecture is that it
           | doesn't break plugins.
        
             | tuukkah wrote:
             | Could you include it in the tech preview?
             | https://neon.tech/docs/cloud/compatibility/
        
               | kelvich wrote:
               | Sure, we will
        
         | avinassh wrote:
         | This is really exciting and thank you for making it open
         | source. I am still trying to wrap my head around the Neon, but
         | is there any design document or architecture description? I
         | want to learn more about the Neon storage engine and how it all
         | fits together.
         | 
         | Also, how do I get an invite code to try?
         | 
         | edit: found this to get started -
         | https://neon.tech/docs/storage-engine/architecture-overview/
        
           | nikita wrote:
           | We will send you an invite code soon. This is a good start
           | and also RFCs on github. We will be publishing more and more
        
         | akmodi wrote:
         | Hey Nikita! I was just looking at the docs but I was a bit
         | confused about what the various compute instances were doing.
         | Do they all serve reads and writes? If so, is there data
         | partitioning or does this support distributed transactions?
        
           | nikita wrote:
           | Various compute instances are different endpoints to separate
           | databases. So for now it's single writer system. You can get
           | a lot of power out of a 128 core compute node. In the future
           | will will also spin up extra compute to scale reads.
           | 
           | In the future after that future we will introduce data
           | partitioning - we have a cool design for it, but one step at
           | a time.
        
             | akmodi wrote:
             | Ah got it thanks! And what's the consistency on the
             | instances that serve reads?
             | 
             | Super interested in this space since we're always looking
             | for ways to evolve our pg!
        
               | akmodi wrote:
               | Ignore this. I misread your previous reply (--;
        
       | pgt wrote:
       | Incredibly good sales copy by someone who understands their
       | target market.
        
       | devit wrote:
       | Why not use normal networked storage (nbd/drbd, AWS EBS, etc.)
       | instead of the ad-hoc Pageserver + Safekeeper architecture?
       | 
       | Or even better, use simple local SSD/HDD storage where data is
       | small enough.
        
         | mattashii wrote:
         | Because all of those don't or can't have both low-latency reads
         | and regional persistence guarantees, which is what we're after.
        
       | anilgulecha wrote:
       | This is the missing piece on cloud for masses                 *
       | we already have compute scale-to-zero (cloudrun, lambda, fly.io).
       | * Network is default pay for use. Storage (S3) is default pay for
       | use.            * The only piece in the stack that was always-on
       | was the database (only serverless db thus far was firestore, or
       | something like sqlite+litestream)
       | 
       | With something like this we get a solid RDBMS engineered to be
       | scale-to-zero, and with good developer experience.
       | 
       | This opens up a world of try-out mini applications that cost
       | cents to host. serverless db (postgres) + serverless compute
       | (cloud-run) + use as you go storage+network. This is a paradigm-
       | shift stack. Exciting days ahead.
        
         | hamandcheese wrote:
         | If you can scale your app to zero, couldn't you also just scale
         | your database to zero once no more app servers are running?
         | 
         | Or for try-out apps, as you mention, you could just run
         | Postgres next to your app in the same container.
         | 
         | This might be possible with fly.io, or will soon, I think.
         | 
         | I'm not sure how comfortable I am using a custom flavor of
         | Postgres (even if it's just the storage layer).
        
         | rektide wrote:
         | > _This is the missing piece on cloud for masses_
         | 
         | I like this perspective a lot & think it's absolutely key here.
         | 
         | We- the world- still pick single-node writer postgres & read
         | replicas when we have to store & query data. There's great
         | Kubernetes postgres operators, but it's still a distinctly pre-
         | cloud pre-scale type of technology, & this decoupling & shared-
         | storage sounds ultra promising, allows independent & radicaly
         | scale up & scale down, sounds principally much more managable.
        
         | jokethrowaway wrote:
         | I don't see the benefits over a 5$ VPS. Even admitting I'll
         | save a few cents over a VPS (which is absolutely not
         | guaranteed), the cost saving is so minimal I won't bother
         | rewriting everything under the serverless paradigm just for it.
         | Of course if your cloud doesn't cost an eye and an arm. I can
         | understand people excited to save on expensive Aws instances
         | but maybe you should just consider dumping Aws.
         | 
         | Scale doesn't matter for mini applications and scaling
         | vertically (=throw money for a bigger server) will work for 99%
         | of the companies. The 1% who need horizontal scaling will have
         | custom everything regardless and will need to hire experts, not
         | a good niche to release a product.
        
           | jeroenhd wrote:
           | I do. There are real benefits to using a hosted server for
           | large projects (think hundreds of gigabytes to terabytes of
           | data) because getting sharding, fallbacks, and downtimeless
           | maintenance right is difficult, risky, and expensive.
           | 
           | These are reasons why one might go for a hosted database at
           | AWS/GCE/Azure. There are tons of good servers out there for
           | small to medium projects and I don't think this service is
           | right for those, unless they can make do with the free tier.
           | The real benefit is in the larger cloud application space.
           | 
           | A system that does this type of scaling automatically while
           | also reducing the dependency on a single cloud provider's
           | service can be a gamechanger for some companies with huge
           | database servers that risk getting locked in.
           | 
           | I think using this service for most existing applications
           | will introduce a performance drop, a rise in expenses, and a
           | complicated migration path, but on the other hand I think
           | that developing against this system for new projects that are
           | very likely to grow in scale will end up with some major data
           | control benefits.
           | 
           | The open source nature also allows for competitors in markets
           | like the EU to start serving databases that don't break
           | privacy laws (although most companies don't care until they
           | receive a fine).
           | 
           | I'm not sure how this company will become profitable while
           | giving away its special sauce that can be modified to run at
           | competing companies relatively easily, but that's a whole
           | different story.
        
         | derefr wrote:
         | Snowflake is probably the closest comparison.
        
           | nikita wrote:
           | Some users called us Snowflake for OLTP, some others
           | Snowflake for Postgres. Obv b/c of separation of storage and
           | compute.
        
         | logifail wrote:
         | > This opens up a world of try-out mini applications that cost
         | cents to host
         | 
         | Given how much performance you can squeeze out of a $5/month
         | VPS (I've been spinning them up and indeed down regularly over
         | the last couple of years), is this really a paradigm shift?
        
           | nikita wrote:
           | We don't know. But we built it anyway because it may be that.
        
           | anilgulecha wrote:
           | Millions of students and enthusiasts around the world would
           | find that cost sufficiently friction-ful to not try out
           | things.
        
             | kortilla wrote:
             | Who absolutely would not want to give a credit card to AWS
             | where a bill is dynamic. If $5/mo is bad they definitely
             | can't handle the screw up that scales up and runs overnight
             | for $500.
        
             | logifail wrote:
             | > Millions of students and enthusiasts around the world
             | would find that cost sufficiently friction-ful to not try
             | out things
             | 
             | I appreciate there are indeed billions of people for whom
             | $5 is a lot of money, but just how many of them are
             | "students and enthusiasts" itching to get started with
             | Postgres?
             | 
             | I realise that - perhaps particularly here - a $5/month VPS
             | is a deeply unsexy thing. You can, however, achieve (and
             | learn) an awful lot with one.
        
             | vidarh wrote:
             | Free tier instances on various providers provides an
             | option. And if you can't afford $5 a month you really
             | shouldn't be playing with services where there's a risk of
             | huge overages if you face a sudden spike in users.
        
               | avinassh wrote:
               | You cannot sign up on AWS Free Tier without the credit
               | card info and many students end up with a huge bill. This
               | sub reddit.com/r/aws/ has many such posts.
               | 
               | Some links and discussions:
               | 
               | - https://cloudirregular.substack.com/p/please-fix-the-
               | aws-fre...
               | 
               | -
               | https://twitter.com/alexwlchan/status/1399095011178958851
               | 
               | - https://news.ycombinator.com/item?id=27044371
        
               | vidarh wrote:
               | Well, and that is just as much of an issue if you use
               | most serverless offerings. But you're not prepared to
               | carefully manage your use, cloud services should not even
               | be on your radar.
        
               | ptman wrote:
               | Comparison of free tiers:
               | https://paul.totterman.name/posts/free-clouds/
        
           | sofixa wrote:
           | On that $5/month VPS there's management overhead - you need
           | to have at least basic Linux knowledge, and ideally more than
           | that to know not to do stupid things like chmod 777 and
           | database exposed on the public internet. You also need to do
           | your updates, etc.
           | 
           | I'm an (former) SRE and run my own Kubernetes cluster for
           | fun, but still use serverless (containers as a service,
           | static website hosting) depending on the project.
        
             | vidarh wrote:
             | With every cloud service there's a management overhead too.
             | Just different skills you need to learn.
             | 
             | I've done SRE/devops work in various capacities including
             | consulting longer than cloud services have existed, and my
             | experience is that I've consistently earned more from
             | clients who insisted on cloud services because they
             | consistently need more help. Nothing is driving more demand
             | for devops consulting services than cloud providers.
        
               | elondaits wrote:
               | The issue with a full Linux system's overhead is that if
               | there are any new security vulnerabilities the situation
               | could blow up in your face (e.g. the system is used to
               | send spam, or host malware), so you need to maintain it
               | at least minimally. With a serverless cloud architecture
               | at worst it'll stop working.
        
               | vidarh wrote:
               | Or you just use Flatcar (a derivative of CoreOS), and
               | don't worry about anything more than rebooting once a new
               | image has been (auto-)installed, and run everything else
               | in app containers where you have to worry about nothing
               | more than what you would in your regular cloud setups.
               | 
               | This is not hard to get right. Yes, you need to learn how
               | to do it, but the amount of money I've made from clients
               | who thought cloud was simple and proceeded to create
               | massive security holes for themselves is fairly
               | substantial. People who think they're reducing their
               | attack surface by using these services need to reevaluate
               | - they're large, complex architectures that very few
               | users understand properly. You need to learn the skills
               | either way.
        
             | lelanthran wrote:
             | > On that $5/month VPS there's management overhead - you
             | need to have at least basic Linux knowledge
             | 
             | Don't you need that _as well as_ cloud-specific knowledge
             | if you go serverless?
             | 
             | > and ideally more than that to know not to do stupid
             | things like chmod 777 and database exposed on the public
             | internet.
             | 
             | You still need some arcane knowledge to make sure your
             | serverless doesn't experience cost overruns, right?
             | 
             | IME (your's obviously differs), the amount of cloud-
             | specific + vendor-specific knowledge needed to avoid using
             | a $5/m VM is a lot more in volume and a lot less in
             | stability[1] than learning basic Linux once and using VMs
             | everywhere[2].
             | 
             | [1] How the different cloud providers bill, when they bill,
             | how to control your limits, etc changes much more often
             | than knowing how to keep your server patched. Knowing how
             | to get your serverless DB going on AWS doesn't help when
             | you want to use Azure. And each cloud vendor regularly
             | requires you to update your knowledge. Knowing how to keep
             | a PostgreSQL-on-Linux up-to-date can be learned once and
             | used for years. Even if running a managed DB, you'll still
             | need to gain some of that knowledge anyway.
             | 
             | [2] Once you get to a scale where treating your machines
             | like cattle rather than pets, you'll obviously have the
             | team required to use cloud stuff optimally.
        
               | vidarh wrote:
               | > You still need some arcane knowledge to make sure your
               | serverless doesn't experience cost overruns, right?
               | 
               | I know people who have literally written their own
               | analysis tooling just to figure out what's impacting
               | their AWS spend. It's gotten better, but I could have
               | retired many times over on what I've seen clients overpay
               | to cloud providers because they didn't understand what
               | would drive cost.
               | 
               | > [2] Once you get to a scale where treating your
               | machines like cattle rather than pets, you'll obviously
               | have the team required to use cloud stuff optimally.
               | 
               | The "problem" with the cloud story is that at that point
               | you also have a team that could save you a fortune with a
               | hybrid setup. Cloud providers get their margins off those
               | who don't understand how much they're overpaying or who
               | are too small to a) care or b) have leverage. Those big
               | enough to have leverage who understand either negotiate
               | hefty discounts or build out cheaper setups (basically at
               | the point you're spending 7 figures a year, if you're
               | paying anywhere near list prices for cloud services
               | you're a chump; below that it's hit and miss)
               | 
               | I'm not at all against using cloud services, but I wish
               | more people actually understood their costs and picked
               | based on merits rather than cargo-culting. Some teams
               | benefit greatly from cloud services, but usually if
               | they're not cost sensitive. In my current job we have
               | everything on AWS because we're never going to scale to
               | somewhere where it'll get expensive and it's convenient.
               | We'd save money if I moved it to, say, Hetzner, but the
               | hosting bill is too small to matter. For that use it's
               | fine.
               | 
               | The moment the bill starts to bite people ought to at
               | least price out alternatives, and consider hybrid setups.
               | E.g. I've had setups where even just putting a caching
               | proxy in front of AWS to cache images to cut the egress
               | bill would have paid for a _team_ to keep it running.
               | Their egress cost is still bad, but at the time it was
               | just pure highway robbery.
        
         | 8organicbits wrote:
         | What's the cold start time for something using
         | sqlite+lightstream on scale-to-zero compute? I think you'd need
         | to pull the db out of storage, so I would be slow to go from
         | 0->1 instance. Anyone know if that's right?
         | 
         | Is there any cold start delay for neon?
        
           | nikita wrote:
           | Right now it is 2 seconds. We are working on improving it.
        
         | TruthWillHurt wrote:
         | uh... AWS Aurora? Azure CosmosDB? GCP BigQuery?
         | 
         | All serverless, scale-to-zero or pay for demand...
        
         | dragonwriter wrote:
         | AWS Aurora Serverless v1 (in MySQL and Postgres flavors) has
         | had serverless, scale-to-zero for quite a while.
        
           | shaicoleman wrote:
           | Aurora Serverless v1 has cold boot times of ~30 seconds when
           | scaling up to zero, which precludes it from being a viable
           | option for most usecases
        
           | pid-1 wrote:
           | Unfortunately V1 is getting very little love from AWS and the
           | new one, V2, does not scale to zero.
        
         | pid-1 wrote:
         | Fly.io does not scale to zero.
         | 
         | Lambda has many limitations.
         | 
         | In particular, for some reason AWS is allergic to providing a
         | container deployment service that actually scales to zero.
        
           | hamandcheese wrote:
           | Not yet, but soon.
        
           | russellendicott wrote:
           | > AWS is allergic to providing a container deployment service
           | that actually scales to zero
           | 
           | Isn't this what Fargate is?
        
             | pid-1 wrote:
             | No.
        
         | manigandham wrote:
         | There are plenty of serverless database options already:
         | Firestore, DynamoDB, CosmosDB, FaunaDB, even MongoDB, and there
         | are "newsql" distributed relational systems like CockroachDB
         | and Planetscale with serverless plans.
        
           | jillesvangurp wrote:
           | Many teams would prefer a postgresql compatible database with
           | full sql support without compromises, missing features, etc.
           | So, this could break that market open a little. Both AWS and
           | Google are unreasonably expensive for this stuff. Most teams
           | don't need a huge database and would be able to run
           | postgresql on a tiny instance and get away with it. Have 2 of
           | those and failover and backups and it's good enough for a lot
           | of small shops.
           | 
           | Most managed/serverless options begin at hundreds of dollars
           | per month. So, you get lots of companies either just handing
           | over the cash or jumping through hoops to get something more
           | reasonable. The latter is a stupid waste of time if you can
           | afford the former. That's how Google and Amazon make money:
           | they make the expensive option more tempting and the cheap
           | option needlessly hard. They are not interesting in
           | supporting frugal teams. The whole point is squeezing their
           | customers hard.
           | 
           | So, this is potentially very nice if it offers some
           | competition on the cost front. I'd certainly consider using
           | this if it proves reliable. In fact, the whole reason I opted
           | out of a relational database is the above. What I'd need is
           | something that is reasonable in cost relative to the modest
           | data I store and retrieve.
        
             | davidzweig wrote:
             | We have a single big bare-metal machine. We run Postgres
             | with a ~1TB DB, moderate load, on a Hetzner AX101
             | (16C/128GB ram). It has 2* 3.84TB nvme drives (zfs mirror
             | with hourly snapshots) used for postgres storage only, and
             | a seperate pair of mirrored sata drives for system/boot
             | (had to request the extra drive, ask support to change boot
             | option in BIOS, and reinstall OS using rescue system). It's
             | about 100 EUR/mo with unlimited data. We bounce all
             | incoming requests from clients (the machine also runs a
             | node backend) through a digital ocean machine (NGINX
             | proxy), as their peering agreements are better, without
             | this some users in Brazil, Turkey etc. have very slow
             | access. OVH I think would be even better for this use
             | (better? peering and IIRC cheaper data). ZFS snapshots are
             | backed up with sanoid to a machine under my desk with
             | spinning disks. AX101 can be fitted with up six 3.84TB
             | drives, that's almost 12TB of mirrored storage, we should
             | be good for a while. You can (should) use at least lz4
             | compression on zfs.. can consider zstd-1, bit slower, that
             | could double the effective space. The compression also
             | applies to the in-ram ZFS cache, that can be 100GB+.
             | 
             | We used firestore before.. got a bit tired of some of the
             | limitations (latency, indexing). Cost-wise I don't think
             | it's that different actually, but we aren't using much
             | bandwidth, then self-hosted can be dramatically cheaper.
             | Have to manage some details of course (zfs filesystem
             | parameters, set up backups, config postgres etc.), but I
             | found that stuff quite interesting and it's knowledge that
             | will always be useful.
        
               | ed25519FUUU wrote:
               | Sounds neat. What's the weak link here though?
        
               | szundi wrote:
               | Maybe that zfs snaphot under a running database? Is that
               | ok?
        
               | alduin32 wrote:
               | Yes, ZFS snapshots are atomic, so as long as the database
               | use sane semantics to do disk I/O, you're fine (AFAIK,
               | PostgreSQL and MySQL/InnoDB are ok, MySQL/MyISAM is not
               | though).
               | 
               | A snapshot is effectively identical to a killed database
               | server (due to OOM, or power outage), and database
               | servers should be crash-resistant.
               | 
               | With PostgreSQL, checkpointing the database before taking
               | the snapshot may make the crash recovery quicker when
               | loading the snapshotted database.
        
               | skrtskrt wrote:
               | No read replicas that can be promoted to master in an
               | outage?
        
               | davidzweig wrote:
               | Ah, nop. If it goes down, we'll figure it out.. 24hrs
               | outage wouldn't be the end of the world. Would have been
               | nice to have a second (read replica) machine in East
               | Asia, to reduce latency for users there, but didn't find
               | a provider like Hetzner.. maybe could take a server in a
               | suitcase and install it in a collocation center there.
               | Bit of a hassle though.
        
               | alduin32 wrote:
               | I have a similar setup as OP, and I use ZFS snapshots to
               | be able to quickly rebuild the whole machine on another
               | host. Of course, it still requires manual intervention in
               | an outage (it could be automated, theoretically), and I
               | may lose up to 5 minutes of data due to my snapshot
               | schedule.
        
           | pid-1 wrote:
           | Most stuff you mentioned:
           | 
           | 1 - Is not an actual relational DB
           | 
           | 2- Doesn't really scale to zero
           | 
           | Planetscale does scale to zero, but has a ridiculous billing
           | model.
        
             | acjohnson55 wrote:
             | What's the billing model / how is it ridiculous?
        
               | pid-1 wrote:
               | Per row read / write billing -> how do you estimate that?
        
             | manigandham wrote:
             | Both CRDB and Planetscale "scale" to zero - but neither
             | expose any concept of individual instances so I'm still not
             | sure what difference it makes.
        
           | geysersam wrote:
           | Yes, but they are not "scale-to-zero"
        
             | manigandham wrote:
             | Both CRDB and Planetscale are.
        
           | nikita wrote:
           | Hi Mani! For sure there are many serverless options - fewer
           | that separated storage and compute and fewer that are open
           | source end-to-end. Neon is also 100% compatible with Postgres
           | (unlike CockroachDB) because compute is Postgres.
           | 
           | Our intention is to standardize the separation of storage and
           | compute cloud architecture - that's why it's open source
           | under the Apache 2.0 license.
        
             | boomskats wrote:
             | You should update your HN profile :). Neon looks really
             | great. Far more interesting to me and my team than S2.
             | 
             | I noticed you mention Azure BS in your RFCs as a potential
             | backend. Have you done much work towards that yet?
        
               | mattashii wrote:
               | For now we've focused on making the product production-
               | ready on AWS, so that we can get users to try it out.
               | Once the model has been proven we'll likely branch out to
               | other clouds.
               | 
               | However, if you really can't wait to run Neon on Azure,
               | you could contribute the integration yourself: the code
               | is available under Apache 2 at
               | https://github.com/neondatabase/neon/
        
               | boomskats wrote:
               | Exact reason why I asked :) Thanks for the response & all
               | the work you're putting in, it looks really promising.
        
               | hlinnaka wrote:
               | Haven't done anything with Azure yet. Shouldn't be hard
               | to add though, we don't rely on any special cloud storage
               | features, just simple upload/download of files.
        
           | rektide wrote:
           | MongoDB & CockroachDB are the only open source ones, the only
           | ones we can hack on & improve & grow.
           | 
           | Neon seems like a vast vast improvement & great & desperately
           | needed potential leap for mankind.
        
             | avinassh wrote:
             | Minor correction, both of them are source available, but
             | not open source
             | 
             | Cockroach DB license - https://github.com/cockroachdb/cockr
             | oach/blob/2c4e2c6/LICENS...
             | 
             | Mongo license -
             | https://github.com/mongodb/mongo/blob/39e4b70/LICENSE-
             | Commun...
        
               | lawik wrote:
               | Cockroach is eventually open source which I rather
               | appreciate. Any code that is 3 years old goes Apache if I
               | recall it correctly.
        
               | orra wrote:
               | That only applies to the core. The "enterprise" features
               | are proprietary until the copyright expires.
               | 
               | IIRC, backup was enterprise but is now part of the core.
               | However, restore is still enterprise.
        
             | tluyben2 wrote:
             | There is also gigabyte as open source. In our tests (which
             | means nothing in general as it's specific to our business
             | case), it outperforms cockroachdb.
        
               | tluyben2 wrote:
               | Sorry, cannot edit; Yugabyte (ios spellcheck).
        
             | manigandham wrote:
             | Planetscale is Vitess which is also opensource:
             | https://vitess.io/
             | 
             | > _" great & desperately needed potential leap for
             | mankind"_
             | 
             | Are you being serious? That's very hyperbolic if so.
        
               | tomatowurst wrote:
        
               | [deleted]
        
               | sigstoat wrote:
        
               | tomatowurst wrote:
               | I felt suspicious of these accounts flooding here writing
               | overtly zealous praises I didn't claim anything.
               | 
               | I just find your response and other comments in this
               | thread, very suspicious, to shut down and censor my
               | opinion.
        
               | nikita wrote:
               | rektide is not Neon employee, nor is he or she was asked
               | to post this.
               | 
               | Thank you for the comment rektide. We of course would
               | love to see constructive comments and criticism.
               | 
               | - Neon CEO
        
               | tomatowurst wrote:
               | Really bizarre that you would reply here, I only
               | mentioned it was odd that there were so many bizarrely
               | zealous praises from what I saw were relatively inactive
               | new accounts which only lead to me be suspicious.
               | 
               | > We of course would love to see constructive comments
               | and criticism
               | 
               | but you got my criticism flagged and censored, so again
               | it makes me even more suspicious.
        
               | dang wrote:
               | Can you please stop posting these off-topic comments
               | (which are against the site guidelines, as you'll see if
               | you review
               | https://news.ycombinator.com/newsguidelines.html)? Also,
               | can you please stop posting unsubstantive comments
               | generally? You've been doing that in other threads as
               | well.
               | 
               | I appreciate your concern for the integrity of HN
               | discussions, but the thing to do if you suspect abuse or
               | manipulation is to email hn@ycombinator.com so we can
               | look into it. Posting about it in the threads themselves
               | is explicitly against the rules.
        
               | bryanrasmussen wrote:
               | I've met my share of developers without perspective in
               | time, and hey I was probably one of them for my first 5-6
               | years as a developer, that I don't think one needs to
               | assume corporate toadying for hyperbolic importance for
               | mankind to be ascribed to a corporate offering.
        
               | psnehanshu wrote:
               | Doesn't seem like an employee due to another comment they
               | posted. https://news.ycombinator.com/item?id=31537613
        
               | rektide wrote:
               | > _Are you being serious? That 's very hyperbolic if so._
               | 
               | Yes Im serious. This is one of the most foundational &
               | key levels of computing: storing & quering data. Without
               | this, computing isnt good for much.
               | 
               | Getting good at this is a huge task for humanity. Right
               | now that task is almost entirely being fulfilled by far
               | off hyperscalers. Aurora, BigTable, Firebase, DynamoDB,
               | CosmoDB, more special works like DataDog Husky... the
               | world is running off vast super-awesome dataengines. But
               | ones that are not ours, that we cant hack on, that we
               | cant run ourselves. It might as well be the Martian's
               | (little green men's) databases as far as I'm concerned:
               | these are not humanity's heritdge & humanity is cut off
               | from active participation with them.
               | 
               | Right now there are so so so few scalable data systems
               | available for the world, that we have. This seems like a
               | great & novel effort to radically open up the range of
               | human capabilities, in one of the most important sectors
               | of computing: handling data.
               | 
               | We have lots of other cloudware for humankind but data
               | has seemingly been much slower & un-scaled. I agree with
               | this @anilgulecha comment[1]:
               | 
               | > _This is the missing piece on cloud for masses_
               | 
               | [1] https://news.ycombinator.com/item?id=31537313
        
               | manigandham wrote:
               | There are tons of databases, I just named several and
               | there are hundreds more for every possible niche.
               | 
               | > _" But ones that are not ours, that we cant hack on,
               | that we cant run ourselves."_
               | 
               | Who is "we"? Is your entire issue that not everything is
               | open-source?
               | 
               | > _" these are not humanity's heritdge & humanity is cut
               | off from active participation with them."_
               | 
               | This is still incredibly exaggerated. Everything humanity
               | does is humanity's heritage. There are plenty of open-
               | source databases of all types and sizes if you look:
               | https://db-engines.com/en/ranking
        
               | rektide wrote:
               | Your free to your opinion, but I respectfully do not
               | share your values & sense of collective ownership over
               | remote, far off software I can't see, shape, control or
               | rework.
               | 
               | The most notable feature of Neon, that makes it so
               | exceptionally smart, is that it takes the humanity's
               | existing first best choice default-go-to database, by a
               | country mile, and layers in really smart decoupling of
               | storage to help it scale. Reinventing things from scratch
               | can be a win, but that this is already a well known
               | quantity, much loved & cherished & used by all, hacked by
               | all, grown by all is a planet-sized huge plus mark.
               | 
               | I think you vastly overrate the broad availability of
               | alternatives to this, of potential starting places others
               | might explore. Most of your list is dominated by
               | proprietary whatever. Even skipping that, yes, there is a
               | ton of novelty & promise, the potential for
               | breakthroughs. But this is starting with the best, and
               | pushing it into the cloud-layer, into the troposphere.
        
           | anilgulecha wrote:
           | (Some of these are options I've not looked deeper into -
           | Fauna, Planetscale)
           | 
           | This sentiment is perhaps right, but I was careful about
           | calling out scale-to-zero. We do have options that are zero
           | cost (or pay as you use), but there's a fundamental
           | difference in something that may be zero cost because a cloud
           | provider is using it as a customer-acquisition ploy.
           | 
           | Options like litestream+sqlite+s3, or what Neon seems to be,
           | are verifiable you-pay-for-when-db-is-booted up, else the
           | verifiable cost is storage only.
           | 
           | So the trifecta that will be very productive for masses is 1)
           | database where compute is scale-to-zero, 2)open source or
           | commoditised, and 3) is RDBMS.
        
             | manigandham wrote:
             | What is the effective difference to you? Technically all
             | compute can enter hibernation by dumping RAM (or just using
             | virtualized memory backed only by SSDs).
             | 
             | CockroachDB already does true scale-to-zero if that's your
             | requirement: https://www.cockroachlabs.com/blog/how-we-
             | built-cockroachdb-...
        
               | anilgulecha wrote:
               | The effective difference is commoditization of the
               | paradigm - there's confidence in a commodity technology
               | that a proprietary cannot give (standardized use and wide
               | support and community, multiple providers, self-hosting).
               | 
               | Eg: of innovation -> commodity. EC2 was innovative, but
               | is today common-place. S3 was the same. DBs that scale-
               | to-zero have not reached that state yet.
               | 
               | Thanks for the link on cockroachdb - it sounds promising.
               | I wonder what's the minimum self-deployable unit of
               | cockroachdb - will google around a bit.
        
               | ignoramous wrote:
               | I think it is more "market leadership" than
               | commoditization that's working in favour of the MySQL-
               | Postgres duopoly.
        
               | manigandham wrote:
               | > _" minimum self-deployable unit of cockroachdb"_
               | 
               | If you host it yourself, or use their dedicated
               | enterprise clusters then you still have instances which
               | are individual (virtual) servers.
               | 
               | The serverless model has no instances exposed to you and
               | instead is a multitenant architecture that uses a pool of
               | shared compute nodes with a routing layer that intercepts
               | and introspects your connection to load up your specific
               | database context for query processing.
        
             | nikita wrote:
             | There is a big difference between in architecture between
             | Neon and PlanetScale, CockroachDb, and Yugabyte. Neon is
             | shared storage (storage is distributed but shared) and the
             | others are shared nothing. Shared nothing systems are hard
             | to build with supporting all the features of the base
             | system. E.g.
             | https://vitess.io/docs/13.0/reference/compatibility/mysql-
             | co....
             | 
             | Neon is 100% compatible from Postgres b/c we didn't (or
             | almost didn't) change the Postgres engine.
        
         | guggleet wrote:
         | https://aws.amazon.com/blogs/aws/amazon-aurora-serverless-v2...
        
           | ldoughty wrote:
           | I virtually never self-promote, but that exact article got me
           | to investigate the offering:
           | 
           | https://ldoughty.com/2022/05/exploring-aws-aurora-
           | serverless...
           | 
           | Short answer if you don't want to read my post: it constantly
           | uses CPU, it's always on. After creation, waiting 2 days,
           | never logged into it, never ran a script against it, never
           | gave it access to any networks, minimum cost is $43/month
           | because it can't actually scale down to 0.5 units unless you
           | CAP it at 0.5, which makes it unusable, because it consumes
           | all of that capacity just to exist.
           | 
           | It sounds like this Neon offering is exactly what I hoped AWS
           | was offering... Or they are using Language to suggest it and
           | mislead the customer just the same ... If it's the former, if
           | probably sign up and try it out. If it's the latter, I'll
           | probably never touch it for the false hope.
           | 
           | Edit: lots of typos from phone keyboard
        
             | taspeotis wrote:
             | I am using Serverless v2 with min/max ACU of 0.5/8 and it
             | spends most nights at 0.5.
        
           | fitzoh wrote:
           | No scale to zero unfortunately
        
             | res0nat0r wrote:
             | It does scale to zero no?
             | 
             | > It automatically starts up, shuts down, and scales
             | capacity up or down based on your application's needs.
             | 
             | The only tradeoff is the additional latency someone will
             | have when connecting to the db after it has shutdown and
             | waiting for it to spin back up and become ready.
        
               | ldoughty wrote:
               | AWS aurora serverless says:
               | 
               | > You pay only for the capacity your application
               | consumes.
               | 
               | > Scales down to 0.5
               | 
               | But it actually can't scale down to 0.5 or the DB falls
               | over just existing.. auto scaling won't let you go down
               | that low unless you set 0.5 as the max, which literally
               | makes it not scale up, and it's dead, because the DB
               | can't run with that little CPU.
               | 
               | So it's fair to ask if neon can scale to 0, both in
               | marketing, and in practice.
        
               | ololobus wrote:
               | We do scale compute part down to zero after 5 mins of
               | inactivity now (no active transactions). This 5 mins
               | threshold is a random pick, it could be 1 min or 30 mins
               | later, or even customizable by the end-user. Storage part
               | is heavily multi-tenant, so it's always running and our
               | main objective is to make resource utilization as
               | effective as possible.
               | 
               | It still has a significant latency on the first
               | connection attempt after suspend (1-2 seconds), but we
               | are working on that and it seems to be realistic to put
               | the startup time under 1 sec.
               | 
               | Pricing model is still work-in-progress, so cannot say
               | much about it. Yet, my personal intention is to make it
               | cost-effective for both end-user and us. I'd prefer to
               | don't build a service with claims like 'here is your
               | free-tier serverless Postgres with zero-latency on
               | connect', which actually means that under the hood there
               | is an always-running compute burning the investors money.
               | Hope it's realistic to achieve :)
               | 
               | -- Cloud engineer @ Neon
        
               | tpetry wrote:
               | How do you plan to start a PostgreSQL instance in less
               | than 1 sec? Sounds interesting.
               | 
               | I tried fast booting of PostgreSQL instances and it
               | always took multiple seconds. So i am really curious!
        
               | ololobus wrote:
               | That's where the separation of storage and compute kicks
               | in, I guess. Startup process of our Postgres instance
               | (compute node) is a bit different from vanilla Postgres.
               | We need to go to the network storage service (pageserver
               | and safekeepers) to get the last known commit LSN, but we
               | don't need to perform any sort of recovery on the compute
               | node side. That way, compute is mostly stateless.
               | 
               | Basically, to start we need to know this LSN and to
               | bootstrap the Postgres processes. This is really that
               | quick. After that compute is ready to accept connections
               | and serve requests, as it's able to get any missing pages
               | from pageserver with GetPage@LSN request.
               | 
               | We do have the whole bunch of problems to solve: queries
               | latency after cold start; startup after the unexpected
               | exit of the heavily loaded Postgres instance could be
               | slower; etc.
        
               | [deleted]
        
               | mattashii wrote:
               | Some parts of the PostgreSQL start-up sequence take a
               | long time:
               | 
               | - Initializing shared memory -> We, for now, have only
               | small instances, so that doesn't hit us as hard
               | 
               | - Reading data directories -> We don't have to do that at
               | all
               | 
               | - Replaying WAL from a previous unclean shutdown -> We
               | don't need to do that, PageServer is responsible for that
               | 
               | - When initializing a whole new database: Initializing
               | the data directory -> We have a copy that each instance
               | gets initialized from, which makes the process "copy
               | those ~16MB in the background", which saves us from
               | having to do the costly initialization process.
               | 
               | And there's several more infrastructural optimizations,
               | such as pre-loading the docker images onto the hosts.
        
               | ldoughty wrote:
               | That's interesting to hear. That probably works great for
               | my use cases, which is typically wake up to refresh a CDN
               | for guests, but ready to work for a bit if a content
               | creator logs in (e.g. a WordPress instance without
               | comments or non-author logins).
               | 
               | Looking forward to seeing how this works out. I have no
               | issues paying for services, I just hate that the minimum
               | entry level cost is $20... I can't imagine why, at scale,
               | it can't be more affordable for hobby/fun level projects.
        
               | tuukkah wrote:
               | It seems Aurora v1 used to scale to zero but v2 has a
               | minimum of 0.5 ACU.
        
       | funstuff007 wrote:
       | How do I keep my application servers physically close to Neon?
        
         | mattashii wrote:
         | Right now, we have our hosting in AWS (us-west-2).
         | 
         | We're planning on expanding to other regions and cloud
         | providers eventually, though.
        
           | GordonS wrote:
           | Having data in Europe will be important for many users, hope
           | you expand to EU soonish.
        
       | ngcc_hk wrote:
       | The site just invite to trial run. Any basic doc, tutorial etc.
       | come from db2 ...
        
         | mattashii wrote:
         | The docs are available on https://neon.tech/docs/cloud/about/ .
         | It contains some information, but we're looking to expand that
         | while working towards a public beta that's arriving soon.
        
       | nojvek wrote:
       | Nikita, I am a huge fan of SingleStore - amazing what you've
       | accomplished there and now what you're building at Neon.
       | 
       | Do you plans to build a columnstore index on top of postgres that
       | supports insert/update/delete?
       | 
       | Love how MSSQL has a columnstore index for a subset of columns on
       | a row store table.
       | 
       | Always wondered why nobody has built something like that for
       | postgres yet.
       | 
       | Citus is nice but it's append only, which is a huge restriction.
        
         | nikita wrote:
         | We are not building it in house right now - we are focusing on
         | OLTP. But have plenty ideas of how to do it in the future. In
         | the first iteration we are thinking about an integration with
         | Singlestore, Snowflake, and BigQuery. Singlestore will be best
         | due to low latency updates, but the other two are very popular
         | so integration with Snowflake and Big Query just makes sense.
         | 
         | In the future we can integrate a columnstore right into the
         | engine to make a smooth on system experience. There are some
         | awesome open source implementations: arrow and duckdb.
         | Updatability is tricky but doable as proven by Singlestore and
         | SQL Server (I'm ex SQL Server and a huge fan of this feature).
         | Not this year.
        
           | nojvek wrote:
           | Makes sense. I agree.
           | 
           | DuckDB is pretty phenomenal. I enjoyed reading its source
           | code and playing around with it.
           | 
           | Another open source nice C++ codebase is Typesense for
           | Hyperfast text search (algolia competitor).
           | 
           | It's been on my mind for many months how to build indexing
           | like this as postgres extensions.
           | 
           | I love how versatile postgres is with so many indexing
           | datastructures.
        
             | nikita wrote:
             | Again it is a big commitment to do a good job on
             | implementing columnstores. If you don't do it all the way
             | it not very usable and you are confusing your users by
             | giving them too many options. The performance expectation
             | is now set by great columstore implementations and you just
             | can't afford a half ass job here.
             | 
             | Here is an example from Google AlloyDB:
             | https://twitter.com/mim_djo/status/1527900193626025984. My
             | understanding that DuckDB is even faster on the TPCH
             | benchmark. TPCDS is much harder and I doubt AllowDB can
             | even run it at any reasonable scale.
        
       | gorgoiler wrote:
       | Postgres is mind boggling, coming from sqlite. In a good way, and
       | both are amazing tools.                  with ordinal
       | jsonb_*             '3 minutes'::interval             create
       | index on my_json ->> 'a key'
       | 
       | It's amazing how much stuff there is available. All the toys!
        
         | CGamesPlay wrote:
         | Just a quick point in defense of SQLite: that last one is
         | almost verbatim possible in SQLite, and it is possible to
         | calculate ordinals, although the syntax is with standard SQL
         | rather than a custom syntax. The SQLite docs mention that they
         | never found a use case for jsonb that ended up being faster or
         | more efficient than json, so they left it out, although they do
         | reserve the BLOB data type for jsonb if such a use case is
         | discovered.
        
           | gorgoiler wrote:
           | Well this is a doozy: so you're saying they are both equally
           | awesome as opposed to being individually awesome in different
           | ways.
           | 
           | What a time to be a developer.
        
       | ranguna wrote:
       | Just yesterday I was comparing managed serverless postgres offers
       | and was sad to temporarily end my investigation with a compromise
       | of using managed aws RDS for development, hoping that a fully
       | serverless postgres with a nice free tier would pop up before
       | going to production, and here we are!
       | 
       | Congrats to the team for what feels like an amazing product.
       | Signed up for the early access, can't wait to get my hands on
       | this!
       | 
       | For anyone interested, these ere the DB offers I looked into:
       | 
       | * DO managed postgres, no free tier but price scaling was not too
       | aggressive, the issue is that it's not natively serverless and
       | we're gonna get 100s of ephemeral connections.
       | 
       | * Cockroach, was the best option for our use case but it doesn't
       | support triggers and stored procedures, so we can't use it right
       | now (closely following
       | https://github.com/cockroachdb/cockroach/issues/28296)
       | 
       | * Fly.io price scaling is too aggressive 6$ -> 33 -> 154 -> 1000s
       | a month and no free tier that I could find.
       | 
       | * Aurora serverless v2 is only for aws internal access and we are
       | using gcp.
       | 
       | * Aurora v1 was what we were gonna go with, but a lot of people
       | online have showed their negative opinion around slow scaling. I
       | didn't investigate enough but I'm thinking we'd need to setup RDS
       | proxy for it handle all our connections, which would've bumped up
       | the price by a good amount. Also no free tier.
       | 
       | * Alloydb looked promising but also no free tier and starting
       | price is a bit much for our current phase of development, but it
       | was definitely something we'd look into in the future.
       | 
       | And now Neon, natively serverless with a (hopefully) good free
       | tier to test things out and some hints about cross region data
       | replication, amazing stuff!
        
         | rattray wrote:
         | Did you look at Crunchy Bridge? Not sure if they support that
         | use case.
        
         | rad_gruchalski wrote:
         | If CockroachDB was fitting your use case the best, you should
         | have a look at YugabyteDB. It does triggers, stored procedures,
         | extensions, almost everything. Some alter table features aren't
         | working yet but it's getting there.
         | 
         | Not associated with the company but a very happy user.
         | 
         | Bonus point: YugabyteDB is full Apache 2-licensed so you can
         | roll your own.
        
         | spiffytech wrote:
         | > Fly.io price scaling is too aggressive 6$ -> 33 -> 154 ->
         | 1000s a month and no free tier that I could find.
         | 
         | Fly has a general purpose free tier of 3 of their smallest
         | instances. You can use that to run their 2-node Postgres
         | cluster plus an app server.
         | 
         | The pricing you pulled is examples of various compute + storage
         | configurations, not the exhaustive list of options. It should
         | look like $4 (or free tier) -> $11 -> $21 -> $62 -> $82 ... +
         | storage, since it's just 2x their VM price (for the two nodes)
         | + any storage above free tier.
        
           | nwienert wrote:
           | Last I used them (last year) their postgres offering, even
           | scaled up to larger nodes, was significantly slower than the
           | cheapest DO offering. I filed a few issues but haven't
           | checked back since.
        
         | lysecret wrote:
         | "Aurora serverless v2 is only for aws internal access and we
         | are using gcp." You can have public access to serverless v2.
         | I'm using it with retool for example. That said I moved a
         | Postgres DB to Aurora, the process was hilarious in how crazy
         | it was. Also they haven't implemented scaling to 0 yet!!!! And
         | the minimum 0.5 Compute unites are actually pretty expensive.
        
         | sitkack wrote:
         | Curious why a free tier is so important?
         | 
         | I think a FT encourages bad behaviors on both sides. I don't
         | think pricing should be linear at all. But even for
         | development, one is using resources, but most of the time they
         | can be minuscule for individual devs.
         | 
         | Aside from production reliability, Postgres is one of the
         | easiest things to get running on a VM and runs fine on a 5$ a
         | month instance.
        
         | jvolkman wrote:
         | AlloyDB is free during its preview phase (not sure how long
         | that is).
         | 
         | https://cloud.google.com/alloydb/pricing#fair-usage-limits
        
       | vira28 wrote:
       | Amazing work by the Team. Congrats y'all. It was one of the best
       | presentations in the PGcon22.
       | 
       | I did email Heikki the following questions, in case if someone
       | from Neon is around here.
       | 
       | a) How does Neon compare to polardb
       | https://github.com/ApsaraDB/PolarDB-for-PostgreSQL.
       | 
       | b) The readme mentions a component "Repository - Neon storage
       | implementation". Does it use any special FileSystem? Any links to
       | read more about it?
       | 
       | c) Heard the cold start is a second (IIRC), how does that value
       | differ if one runs Neon on bare metal instead of k8s?
        
         | ololobus wrote:
         | > c) Heard the cold start is a second (IIRC), how does that
         | value differ if one runs Neon on bare metal instead of k8s?
         | 
         | Yeah, as Nikita mentioned it's 2 seconds now. We did some tests
         | and measurements and on bare metal, it's sub 500 ms usually, so
         | the remaining part is the k8s (+ our own control plane)
         | orchestration overhead. For example, with plain Docker (which
         | we use in CI in addition to k8s) it's around 1 second already.
         | 
         | K8s provides a convenient abstraction layer, though. So I think
         | that we'll continue using it and optimization will come with
         | pods pool / over-provisioning and it'll be realistic to bring
         | the startup time closer to bare-metal.
         | 
         | -- Cloud engineer @ Neon
        
         | avinassh wrote:
         | > It was one of the best presentations in the PGcon22.
         | 
         | I can't find it on Youtube, do you have the link?
         | 
         | edit: I found the link, seems it is not on the Youtube yet:
         | https://www.pgcon.org/events/pgcon_2022/schedule/session/236...
        
           | nikita wrote:
           | I can't recommend this presentation enough!
        
         | nikita wrote:
         | Thank you!
         | 
         | a. PolarDB is based on a similar idea. https://www.cs.utah.edu/
         | ~lifeifei/papers/polardbserverless-s.... This paper describes
         | it. The biggest difference that I see glancing through the
         | paper is that we really integrated S3 into the storage. In Neon
         | architecture branches, backups, checkpoint are all the same
         | thing and instant to run. This simplifies a good amount of
         | database management AND deliver on better costs. S3 is cheap.
         | 
         | b. Neon doesn't need a special filesystem. Neon storage is in a
         | way a filesystem, however it doesn't expose filesystem API.
         | It's a key value store - serves 8k pages to Postgres and a
         | consensus - update API to the key value store. Pages are
         | organized in LSM trees and background processes put layers of
         | the LSM trees to S3.
         | 
         | c. The cols start is 2sec right now. There is a dependency on
         | K8S. Bare metal implementation will require new code to
         | orchestrate starts and stops.
        
           | ignoramous wrote:
           | > _S3 is cheap._
           | 
           | S3 has its limitations though, like too many small files and
           | the get/delete/list ops get very expensive. There's also an
           | upper-limit on throughput per S3-bucket partition. I guess,
           | sstables that pageserver flushes periodically help work
           | around these issues?
           | 
           | > _Neon storage is in a way a filesystem, however it doesn 't
           | expose filesystem API._
           | 
           | Genuinely curious: When would anyone consider using
           | filesystems like Amazon FSx for Lustre instead which is
           | backed by S3 anyway over implementing a filesystem-esque
           | abstraction of their own (like neon.tech does, and other
           | solutions like rocketset.com, tiledb.com, xata.io, and
           | quickwit.io do).
           | 
           | > _Pages are organized in LSM trees and background processes
           | put layers of the LSM trees to S3._
           | 
           | Curious how merges are handled? Also, are you using RocksDB /
           | some other engine underneath?
           | 
           | > _Bare metal implementation will require new code to
           | orchestrate starts and stops._
           | 
           | Speaking of new code... SingleStore started as a very high-
           | throughput OLTP database and eventually evolved to into a
           | HTAP (?) database. Do you see Neon evolving in a similar
           | manner, too?
           | 
           | Thanks!
        
             | nikita wrote:
             | 1. Yes. Our first attempt at storage implementation had a
             | problem with many small file. Then the team rearchitected
             | it around LSM trees and it got a LOT better. Our benchmarks
             | show that we are very close in performance with vanilla
             | Postgres and Aurora. There are some "worst case" scenarios
             | where Neon is worse than vanilla Postgres. Aurora has
             | similar problems too.
             | 
             | 2. It's best to custom build a storage system here.
             | External distributed filesystems introduce complexity,
             | cost, and bottlenecks that you don't control.
             | 
             | 3. Purpose built. LSM trees also have a temporal dimension
             | - LSN. You can fetch a page by pageId and LSN. This is what
             | allows time machine and branching.
             | 
             | 4. I call it convergence when OLTP and OLAP is one system -
             | ultimate dream for a database systems engineer. Since I
             | spent 10 years building it I have both scars and
             | aspirations. I think it will come, but this will take a
             | long time. HTAP is in a way a subset of convergence - most
             | systems will have some HTAP. Neon will have some too, but
             | for now it squarely focused on OLTP and helping developers
             | build apps.
        
           | unraveller wrote:
           | >serves 8k pages to Postgres
           | 
           | will page size be tunable on neon cloud for larger datasets?
        
           | timmg wrote:
           | The way you describe it, to me, is one of those "this sounds
           | obvious in retrospect". Sounds completely elegant and
           | "right". Congratulations on a great idea. I really hope you
           | pull it off!
        
             | nikita wrote:
             | Thank you! We are super hard at work. You can see our
             | velocity here: https://github.com/neondatabase/neon
        
           | 1500100900 wrote:
           | > we really integrated S3 into the storage
           | 
           | Will it be possible to use something else in place of S3? I'm
           | thinking on-premise or what some would call a private cloud.
        
             | mattashii wrote:
             | Right now, it should be possible to use anything that is
             | compatible with the S3 API, as our current focus is on
             | getting the product to the market. Once the business model
             | is proven, we'll likely branch out to other clouds, with
             | their storage providers.
             | 
             | If you can't wait that long to run Neon on your own cloud,
             | feel free to contribute an integration to your persistent
             | blob storage: the code is available under APLv2 here:
             | https://github.com/neondatabase/neon/
        
       | TruthWillHurt wrote:
       | "We separated storage and compute"
       | 
       | No. just no.
       | 
       | Compute where the data lives, else you incur traffic cost and
       | latency.
       | 
       | A lesson learned in life, known from the dawn of time..
        
         | dunkeylim wrote:
         | What? This isn't controversial, Snowflake did this years ago
         | and this is the main reason it became a $40B business.
         | 
         | Amazon came out with Redshift, a cloud OLAP database, but it
         | tied compute with data so teams couldn't scale compute and data
         | separately and thus had to pay disproportional costs to their
         | required workloads.
         | 
         | Sure, there's good reasons to keep compute and data together.
         | But there is obviously a massive market for technologies that
         | keep them separate...
        
           | TruthWillHurt wrote:
           | Snowflake is a BI data platform, which doesn't require low-
           | latency and fast response like systems powered by RDBs do.
           | 
           | Apples and oranges.
        
             | ranguna wrote:
             | I believe you were the one who implicitly started the
             | apples and oranges comparison with your original post
             | 
             | > "no. Just no."
             | 
             | Conveys the feeling that there is no scenario where doing
             | this is feasibly and you yourself just said it's acceptable
             | for BI but not AI training.
             | 
             | So the "no. Just no." actually means, "my use case does not
             | allow for this, so I believe no one else should use this"
             | which is a fallacy on its own.
             | 
             | In conclusion, this is fairly usable, but like everything
             | else, it's not for all use cases.
        
         | wnolens wrote:
         | Easy trade-off to get scalability. Writing log off-host is fast
         | with 10G network and nvme on the other side (see Amazon Aurora,
         | Microsoft hyper scale, etc)
        
         | mattashii wrote:
         | Do note that many installations of PostgreSQL already have some
         | form of "separation of storage and compute" through a networked
         | storage solution like NFS, EBS, or other SAN-like systems.
         | 
         | The major part of what Neon does is remove the file system
         | abstraction that is between that Storage and Compute, so that
         | we can better utilize the available resources because we can
         | better select what information is or isn't being lost.
         | 
         | A good example of what removing the file system abstraction
         | enables for us is effectively free PITR, (lagging) replicas,
         | and data branching. This is because PostgreSQL's file-system-
         | based storage engine expects to be the only one working on the
         | data directory, which means that any FS attached to a replica
         | cannot be shared. If you remove that file-system based storage
         | engine and plug in a different storage engine, those
         | expectations are removed too, and after some effort integrating
         | into the smgr-APIs, we're now able to provide a storage layer
         | that only needs to contain one copy of the data for N physical
         | replicas, instead of N copies.
        
           | TruthWillHurt wrote:
           | The fact some systems do it, doesn't mean it is correct or
           | optimals.
           | 
           | The latency of NFS & EBS or EFS is actually the reason many
           | businesses *do not* use them for their databases.
           | 
           | I've seen deployments that had to go bare-metal because the
           | tiny latency of EBS caused their compute times to rise
           | exponentialy (doing AI training).
        
             | mattashii wrote:
             | I agree that it does have its limitations, but databases
             | are all about trade-offs.
             | 
             | Talking about the "correctness" of a choice between
             | tradeoffs is weird though. Running your database on hard
             | drives nowadays is not a great choice - yet people still
             | did that because the cost of hard drives was way cheaper
             | than that of memory.
             | 
             | Running your database in a way that doesn't guarantee that
             | a 'committed' response actually retains the changes that it
             | was responding on - yet people still ran their database in
             | such configuration to scrape the barrel on performance.
             | 
             | All Neon does is put up another option: If you don't mind
             | the implications of networked storage, then here's one
             | database system that has zero-cost cloning and does not
             | lose data on single-node failure.
        
         | userbinator wrote:
         | The goal of the "cloud" is to extract maximum $$$ from its
         | users. Once you realise that, it's not hard to see through the
         | marketing propaganda and the rest of the decisions following.
         | 
         |  _else you incur traffic cost and latency_
         | 
         | Thus, they are separated so they can charge you (more) for them
         | separately.
        
           | panarky wrote:
           | Zero bandwidth cost within the same zone.
        
       | Sytten wrote:
       | I am glad someone is taking on this challenge! Its been a few
       | years since I was saying that the last piece of the serverless
       | puzzle is a good postgres serverless database.
       | 
       | If I was not working on my startup I would apply for sure. It
       | would be nice to present the project on the CMU database group
       | youtube channel at some point to dive into the implementation.
        
         | nikita wrote:
         | I'll work with Andy Pavlo who is a friend to set up a
         | presentation.
         | 
         | And yes, we are hiring! So if and when you are ready let us
         | know.
        
       | rubenfiszel wrote:
       | This is exactly what I was looking for windmill, an OSS tool for
       | multi-step automation from scripts. By any chance, are
       | documentation to host it on top of nomad be on the Roadmap ? If
       | yes, I would try it immediately to replace the postgres server we
       | currently use.
        
         | nikita wrote:
         | No documentation of how to self host yet. It's not hard, it
         | requires K8S and S3 compatible object store. We want to harden
         | Neon operationally before documenting and supporting on prem
         | production installs.
        
           | ccakes wrote:
           | _Requires_ k8s as in - uses k8s APIs? Or is supported on k8s?
           | 
           | Apologies haven't read the docs but wanted to highlight this.
           | A hard requirement on k8s to the exclusion of other
           | schedulers would be a shame and an odd choice
        
             | mattashii wrote:
             | We've built our current cloud offering on k8s, and that is
             | our only scale-to-0 implementation.
             | 
             | That does not mean that you cannot run Neon outside k8s,
             | but we are not actively maintaining nor supporting other
             | hosting options.
        
           | rubenfiszel wrote:
           | Got it, makes sense. I think we are pursuing similar
           | strategies with OSS/self-hostable products. I would love to
           | get in touch with you about it. Could you drop me an email at
           | ruben@windmill.dev ?
        
             | nikita wrote:
             | Replied
        
       | TobyTheDog123 wrote:
       | I'm a bit concerned that the free trial mentions "compute up to 1
       | vCPU / 256 MB"
       | 
       | Why would I need to worry about this for a serverless database
       | provider?
        
       | edude03 wrote:
       | This is really amazing, super excited to try it out.
       | 
       | I read the docs and I noticed you can run it locally, but has the
       | kubernetes bits been made available? I see
       | https://github.com/neondatabase/helm-charts and
       | https://github.com/neondatabase/neon/tree/main/.circleci/hel...
       | but I think there is some charts missing?
        
         | mattashii wrote:
         | Correct: we do not yet use k8s for provisioning the Safekeepers
         | and PageServers for our closed-beta cloud offering, and the
         | PostgreSQL instances are managed in k8s by our closed-source
         | console. As such, there's little we can open-source at the
         | orchestration level at this point in time.
        
           | chwzr wrote:
           | Is there a timeline on releasing orchestration things?
        
       | smallerfish wrote:
       | Could you document some of your differentiation against aurora,
       | both on price and architecture? I don't care about scale to 0. I
       | care about more scale to NNN, efficiently & reliably, with
       | minimal devops needs.
        
       | rektide wrote:
       | _Really_ interesting. I 've seen so much disagregated database
       | work, and so so so much of that exposes postgres interfaces. But
       | all the good stuff has been closed source!
       | 
       | I'm very very excited to hear about a team taking this effort to
       | postgres itself, in an open source fashion! From the
       | Architecture[1] section of the README:
       | 
       | > _A Neon installation consists of compute nodes and Neon storage
       | engine._
       | 
       | > _Compute nodes are stateless PostgreSQL nodes, backed by Neon
       | storage engine._
       | 
       | > *Neon storage engine consists of two major components: A)
       | Pageserver. Scalable storage backend for compute nodes. B) WAL
       | service. The service that receives WAL from compute node and
       | ensures that it is stored durably.
       | 
       | Sounds like a very reasonable disaggregation strategy. Really
       | hope to hear about this wonderful effort for many more years.
       | Ticks the boxes: open-source with a great service offering: nice.
       | Rust: nice.
        
         | rattray wrote:
         | [1] https://github.com/neondatabase/neon#architecture-overview
        
           | rektide wrote:
           | Oops thanks!
        
         | nikita wrote:
         | We are committed to building a durable company and we are well
         | funded. So yes, you will hear from us for years to come as we
         | will be shipping more and more features.
        
           | avinassh wrote:
           | I could not find funding information on the Neon site. Is
           | that information not public?
           | 
           | edit: I found the info here:
           | https://boards.greenhouse.io/neondatabase/jobs/4506003004
        
             | nikita wrote:
             | We will announce in a few weeks. Top tier Silicon Valley
             | investors.
        
       | madelyn wrote:
       | This is really interesting. Are there restrictions or limitations
       | on the PageServer + Safekeeper design when running OLAP queries
       | on larger datasets?
       | 
       | Phrased another way, would a query that needs to access a
       | relatively large amount of data (10-100 GB) ever need to read
       | from s3, incurring extra latency?
        
         | mattashii wrote:
         | In general, all data of live clusters will also be stored
         | locally at a PageServer.
         | 
         | Only in recovery scenarios will a PageServer not hold the data
         | that is needed to serve the requests of a compute node - but
         | that would recover quickly as the local cache of the PageServer
         | is repopulated with data from S3.
        
           | madelyn wrote:
           | Awesome, thank you!
        
       | manigandham wrote:
       | From the teams page, the CEO of Neon is the cofounder of
       | MemSQL/Singlestore which is one of the best database products
       | I've used. Looks like a solid team to get this done. Very similar
       | approach to Yugabyte (real postgres compute layer + custom scale
       | out data layer) and many others in the OLAP space.
        
         | nikita wrote:
         | Thank you for the kind words Mani! Singlestore is indeed an
         | amazing product and company. I'm really proud of it!
        
         | ignoramous wrote:
         | Manish Jain of dgraph.io noted that building on top of Postgres
         | or betting on Postgres seems like a necessary condition for
         | database startups to be successful.
         | 
         | Some are commodotizing Postgres' wire format but implementing
         | their own query and storage layers (like CockroachDB / Aurora /
         | AlloyDB), while others are modifying parts of Postgres (like
         | Timescale / EdgeDB / YugaByte), and others still are building
         | atop it (Supabase).
         | 
         | https://twitter.com/manishrjain/status/1496174276474732544
        
       | canadiantim wrote:
       | Can you use extensions with it like normal postgres?
        
         | takeda wrote:
         | I'm guessing no, or at most just have set of extensions that
         | you can use (kind of like it's done in AWS RDS or AWS Aurora).
         | 
         | The claim is "serverless" i.e. you don't have access to the
         | server and if you could install any extension you essentially
         | have full access to the server as there's no restriction what
         | you could do in an extension. I don't think that would be
         | allowed.
        
         | kelvich wrote:
         | Sure. So far, we just precompiled a few popular extensions, and
         | they are available for installation. Ultimately we want to
         | provide an option to bring your own extensions e.g., by
         | specifying a docker image based on our base image. But that is
         | some work on a security front: with a custom extension, you
         | have access to the corresponding Unix user and can construct
         | malicious WAL, send requests to the control plane, etc.
         | 
         | Disclaimer: Neon co-founder
        
         | mpartel wrote:
         | https://neon.tech/docs/cloud/compatibility/ says "During
         | technical preview Neon has restrictions on user ability to
         | install PostgreSQL extensions. Following PostgreSQL extensions
         | come pre-installed: [..]"
        
       | rattray wrote:
       | This sounds awesome, but one of my first reactions to the notion
       | of separated compute/storage Postgres with copy-on-write is
       | "okay, so... slow Postgres?"
       | 
       | Is there anything the dev team can share on read/write
       | performance compared to RDS, NVMe EC2 instances, EBS-backed EC2,
       | etc? In what situations would this setup perform poorly, and in
       | what situations would it excel?
        
       | hestefisk wrote:
       | What is "bottom-less" storage? This just seems like a managed
       | hosting of PostgreSQL, nothing else...
        
         | kelvich wrote:
         | We wrote a custom storage layer for postgres, so in our setup,
         | postgres node (k8s pod actually) doesn't store any data, and it
         | is easy to start/stop/reschedule it. So while we actually are
         | DBaaS, the closest analogy here is Aurora or Alloy, not RDS-
         | like setups.
         | 
         | A bit more details are on https://github.com/neondatabase/neon
        
         | excuses_ wrote:
         | Typically it means S3 and alike. Not being bounded by disk size
         | of your server.
         | 
         | Or in their case they might refer to horizontal scalability of
         | their storage layer which is independent from computing.
        
           | nikita wrote:
           | Yes, you will never run out of storage with Neon. The reason
           | to it that in our tiered storage implementation S3 is the
           | cheapest tier and we will offload data into S3 if it gets too
           | big.
        
       ___________________________________________________________________
       (page generated 2022-05-28 23:01 UTC)