[HN Gopher] Things I learned from building a production database
       ___________________________________________________________________
        
       Things I learned from building a production database
        
       Author : dangoldin
       Score  : 85 points
       Date   : 2021-11-23 19:55 UTC (3 hours ago)
        
 (HTM) web link (maheshba.bitbucket.io)
 (TXT) w3m dump (maheshba.bitbucket.io)
        
       | turbocon wrote:
       | I really think this is a great list however I think one of the
       | major factors why this project succeeded isn't listed but is
       | mentioned in the intro.
       | 
       | > We hit production with a 3-person team in less than a year; and
       | subsequently scaled the team to 30+ engineers spanning multiple
       | sub-teams
       | 
       | Starting with a small skilled team to build the core and actively
       | plan for a larger team at some point is absolutely critical for
       | project success in my experience, especially with complex
       | projects like this. That small team can have a much tighter
       | communication loop and all keep a single unified picture of what
       | the end goal is. When the team grows this single unified picture
       | is no longer feasible so it has to be broken down into sections,
       | which is a good thing, but if the core isn't setup so that the
       | sections are sufficiently decoupled things fall apart quickly.
        
       | nzgrover wrote:
       | Nowhere does it say what an "IC" is. Is it so hard to provided
       | the Full Name(FN) and the abbrevation the first time you use it?
        
         | otras wrote:
         | IC is a commonly used acronym for Individual Contributor. For
         | instance, when the author says "customer ICs", I imagine that
         | means the internal engineers that would be using the database
         | they created (individual contributors on other teams who would
         | be customers/clients of this service).
         | 
         | (Vouched for this flagged comment after seeing the other
         | comment)
        
       | tyingq wrote:
       | There was a comment asking what an "IC" was, but the comment was
       | flagged and now dead.
       | 
       | Guessing by the context, I'm pretty sure it means "individual
       | contributor", or "person that is not a manager".
        
       | nicoburns wrote:
       | This is a really excellent list. Some of it is specific to the
       | kind of low level work this person was doing, but a lot of it
       | isn't. I would love to work on a team with the author of this
       | blog post.
        
       | sjansen wrote:
       | Although the author says his advice is for people "building new
       | infra at large companies", there's certainly a number of points
       | that apply to a larger audience.
       | 
       | For example:
       | 
       | > Measurement is a means, not an end.
       | 
       | Reminds me of a lesson I learned early in my career. Bad things
       | happen when you lose sight of your purpose and focus just on
       | measurement.
       | 
       | https://en.wikipedia.org/wiki/Goodhart%27s_law
        
       | krisrm wrote:
       | Very good list.
       | 
       | [22] Avoid using real-time for correctness guarantees or
       | comparing clocks across machines unless you have (and understand)
       | error bounds on the clock.
       | 
       | I shudder to think of the hours spent head-scratching that went
       | into this Lesson Learned :)
        
         | dmoy wrote:
         | For [22] taken to an extreme level, read the original Spanner
         | paper from 2012 (?):
         | 
         | https://static.googleusercontent.com/media/research.google.c...
         | 
         | It goes into a lot of detail on this
        
       ___________________________________________________________________
       (page generated 2021-11-23 23:00 UTC)