[HN Gopher] Ask HN: What is the best code base you ever worked on?
___________________________________________________________________
Ask HN: What is the best code base you ever worked on?
And what made it so good? Was there someone enforcing good
practices top down? Just being in a group of great engineers? Or
something else?
Author : pcatach
Score : 244 points
Date : 2024-06-28 08:40 UTC (14 hours ago)
| cqqxo4zV46cp wrote:
| A blank slate, so I can make the first mistake :)
| escapecharacter wrote:
| I agree, the best code base is always a zero-line code base.
| alex_suzuki wrote:
| After that first line, it's all downhill from there... :-)
| azangru wrote:
| So true!
|
| I spend so much time obsessing over how what I am about to
| write ties in with what has already been written; or fuming
| over the stupidity of earlier decisions (usually made by
| myself). A blank slate is incredibly refreshing.
| mulholio wrote:
| Although I'm only on job three and have not had that much
| involvement with open source, I think my current employer (Attio)
| has one of the best codebases I've seen.
|
| Qualitatively, I experience this in a few ways: * Codebase
| quality improves over time, even as codebase and team size
| rapidly increase * Everything is easy to find. Sub-packages are
| well-organised. Files are easy to search for * Scaling is now
| essentially solved and engineers can put 90% of their time into
| feature-focused work instead of load concerns
|
| I think there are a few reasons for this:
|
| * We have standard patterns for our common use cases * Our hiring
| bar is high and everyone is expected to improve code quality over
| time * Critical engineering decisions have been consistently
| well-made. For example, we are very happy to have chosen our
| current DB architecture, avoided GraphQL and used Rust for some
| performance-critical areas * A TypeScript monorepo means code
| quality spreads across web/mobile/backend * Doing good migrations
| has become a core competency. Old systems get migrated out and
| replaced by better, newer ones * GCP makes infra easy * All the
| standard best practices: code review, appropriate unit testing,
| feature flagging, ...
|
| Of course, there are still some holes. We have one or two dark
| forest features that will eventually need refactoring/rebuilding;
| testing needs a little more work. But overall, I'm confident
| these things will get fixed and the trajectory is very good.
| kleton wrote:
| google3, all the devex tooling was taken care of by other teams.
| Tons of useful library functions available to import, accumulated
| over decades.
| chrisvenum wrote:
| My favourite projects are small, with very focused goals and
| features.
|
| I have a Laravel project that I have maintained for a customer
| for seven years. The app is straightforward and allows users to
| create portals that list files and metadata, such as expiration
| dates and tags.
|
| Every other year, they ask me to add a new batch of features or
| update the UI to reflect the business's branding. As the app is
| so small, I have the opportunity to review every part of the app
| and refactor or completely rewrite parts I am not happy with.
|
| It is a joy to work on and I always welcome new requests.
| 2Pacalypse- wrote:
| Any codebase that I had complete control over.
|
| No, but more seriously, I've found that familiarity with the
| codebase is more important than having it be perfectly
| engineered. Once you're really familiar with the codebase, you
| know where dragons be, and you can make changes more easily. And
| God (PM) forbid, if you ever find yourself with some extra free
| time you might even reduce the size of dragons over time.
|
| This brings me to my final point. Any codebase that I really
| enjoyed working with was the one that was constantly evolving. I
| don't mean rewriting everything from scratch every few months,
| but as long as I have permission (and time) to refactor the
| things that have been bothering me for months as patterns emerge,
| I'm a happy bee.
| sam_lowry_ wrote:
| > Any codebase that I had complete control over.
|
| No excuses. Code ownership is important. Sometimes it works for
| a team, sometimes only for individuals.
|
| But not having to submit to core teams, architects and self-
| proclaimed experts of all kinds is a blessing.
|
| I now work for an organization that discourages code ownership,
| and it struggles on many fronts: 1. core
| teams are dysfunctional 2. people find niches and
| stick to them 3. top talent is leaving, although
| pay is good and business creates real value for citizens
| 4. there is virtually no horizontal communication
| 5. mediocre ones rise to the level of their incompetence and
| infest the lives of others 6. and so on and so
| forth...
|
| And I think the root cause of all this is lack of individual
| (code) ownership.
| Aeolun wrote:
| I've had exactly the same issues but because I couldn't
| change anything without getting approval from 16.5 code
| owners on every PR submitted. It's a real pain if you start
| modifying your coding for 'least code owners hit' instead of
| 'best architecture'.
| rsanek wrote:
| I like how my workplace does it -- there are rigorous
| codeowners and usually you only need approval from 1-2. if
| you do need approval from 5+, you can request a select
| 'super' codeowner review which will approve it for all.
| ownagefool wrote:
| I think core teams being helpful or harmful really comes down
| to the individuals.
|
| The problem is at this level, most orgs don't have anyone to
| really judge or contest competency, so they hire the salesmen
| rather than the doers and when they don't, they tend to cheap
| out and just get inexperienced people.
|
| Logically it makes a bunch of sense, though.
|
| Why rebuild yet another platform? Why is your central
| platform bad? Usually it's not self-service, sometimes it's
| because it's built in cumbersome ways, other times its
| because it actually enforces good standards on you rather
| than just giving app your apps admin.
|
| It's difficult for the person who hires the core team to
| differentiate between those complaints, unless they
| themselves both have the technical competency and the empathy
| to really understand the problem. They usually don't.
|
| Point being, done well, it's great, but most folks can't do
| it well.
| threatofrain wrote:
| Also nice is when an entire community has agreed to architect a
| codebase more or less the same. You're basically psychic when
| that happens.
| blitzar wrote:
| > Any codebase that I had complete control over.
|
| Anyone other than myself would instantly observe it as the the
| worst codebase they have ever seen.
| sam_lowry_ wrote:
| For that, there is a saying: "code as if your kids will
| maintain it".
|
| But I think it does not convey the right meaning. When I code
| something I will have to maintain for a long time, I try to
| make it as simple as possible for my future, older, less
| motivated and weary self.
|
| The worst codebases are written by people who landed the gig
| a few months before and do not expect to stay around longer
| than a year or two.
| null_glyph wrote:
| This is very much true. Initially when I joined the industry I
| used to work for a product from its inception. So I was aware
| of what section of the code affects what part of the product.
| Although I hadn't worked on all of those, I kept an eye for the
| all the changes that were coming in. I knew where I should look
| immediately a bug is reported even if it is not something
| related to my line of work.
|
| Recently I switched teams and now I find myself taking up bugs
| that are only related to my line of work. Not being familiar
| with the codebase decreases productivity and wants you to rely
| on other people in the team for most of the time.
| sgbeal wrote:
| > Once you're really familiar with the codebase, you know where
| dragons be, and you can make changes more easily.
|
| This is an interesting, and often overlooked, point. A month or
| two ago someone asked us, the Fossil SCM maintainers, if we'd
| be open to them refactoring the tree to something which better
| matches modern sensibilities (i'm paraphrasing here). Despite
| its many proverbial dragons, the long-time maintainers are
| comfortable with it and know where those dragons are and how to
| defeat them (or, in some cases, sneak around them), so, despite
| our collective geek tendencies to prefer "perfect code," we're
| happier with the dragons we know than those we don't. (That's
| not to say that fossil's code is awful, but its structure
| varies significantly from what one might write today if one was
| to start such a project from the ground up.)
| nevi-me wrote:
| I also echo the code ownership part.
|
| I have a part-time gig where I maintain accounting software for
| a former client of mine. It takes up a few months' weekends a
| year.
|
| I wrote about 60-70% of it when I was working for the owner of
| the software. It's something where as long as the client's
| happy, and they get new integrations and updates on time, they
| could keep using it for a decade longer.
|
| I had almost complete ownership of the architecting of the
| software. It's broken down into a few microservices (think
| database, core business logic, reporting, auth, logging etc).
| The best thing I did at the time was pushing to use gRPC even
| though management felt it was too new tech.
|
| The UI is in Angular, pain-free periodic upgrades. I've even
| rewritten some perf-sensitive code in Rust, and everyone's
| happy with snappier calculations.
|
| The code hygiene is relatively good.
|
| The only downside's that if someone else were to take over the
| code, they'd struggle (it's one of those things where I'm
| wearing many hats). I've been fortunate to be a professional
| accountant who moved into software engineering, so everything
| makes sense to me.
| KronisLV wrote:
| > Any codebase that I had complete control over.
|
| This is probably one of the pillars of good codebases, or at
| least decoupling the bits that you don't control as well as you
| can (this includes external services). I remember needing to
| write a wrapper around another JWT library, but because it was
| quite important, I aimed for >95% test coverage, some of the
| tests acted as documentation for how to use the code, there was
| also a nice README, there was CI configuration for pushing the
| build artifacts to Maven and suddenly even managing dependency
| updates became easy because of the test suite. Years later,
| it's been integrated in a bunch of services and "just works".
|
| Come to think of it, things always get less pleasant once you
| add a bunch of complex dependencies and libraries/frameworks.
| Need to make a few RESTful Web APIs in Java? Something like
| Dropwizard will probably give you fewer headaches than Spring
| (or Spring Boot) and all of its inherent complexity, in the
| case of the former you might even need to do configuration in
| XML and that has honestly never been pleasant. If you need to
| integrate with a bunch of other stuff and want something
| opinionated, going for Spring Boot will make sense then, but
| for simple use cases it's overkill. Same for ASP.NET, Ruby on
| Rails, Laravel and many others, while they might be easier to
| use, updates (especially across major versions) and breaking
| changes will give you a headache and just add a bunch of churn
| to the project.
|
| Similarly, if you need a message queue, externalizing that into
| RabbitMQ might make a lot of sense, same with storing files in
| an S3 compatible store, using something like Redis or Valkey
| for key-value storage, as well as not trying to shove too much
| logic into your RDBMS. Just pick whatever tool feels best for
| the given task at hand, instead of shaping things into what
| they're not (using the database for blob storage, for example),
| unless you have a whole bunch of constraints to contend with.
| Otherwise, sometimes you just get the worst of both worlds,
| like needing to use Oracle for a project, not having easily
| launchable local environments (because Oracle XE doesn't
| support some features), having to share DB instances with
| others during development and also running into weird obscure
| issues like DATABASE LINK queries taking 100 times longer in
| some cases, even when executing the same SELECT query on the
| remote DB works without issues.
|
| To not go into a rant, I'd sum it up like this: be in control,
| isolate the things that you cannot control, pick the correct
| technologies, do the simplest thing that you can get away with
| without overengineering and think about the person who'll have
| to maintain, debug and grow the system in the following months
| and years (which might also be yourself, sans knowledge about
| what the code did, if you don't make it explain itself or don't
| comment the non-trivial stuff).
| idrios wrote:
| > And God (PM) forbid, if you ever find yourself with some
| extra free time you might even reduce the size of dragons over
| time.
|
| Honest question, what is the company like where you can do
| that? Everywhere I've worked (only been working in industry for
| 6 years) has had such rigid agile development that even when I
| do find myself with free time, there's no flexibility to work
| on things that haven't been assigned to you and the best I can
| do is work on profiling/debugging tools.
| arp242 wrote:
| Usually at smaller companies with just a handful of
| developers. Can be a "start-up", or just a smaller business
| running fine with small teams. Or sometimes in a larger
| company where there's a small team doing its own thing.
|
| There's up-and-downsides to everything. The pay is usually
| less, and "fewer processes" often also means fewer processes
| surrounding HR and such. This can mean you're subject to the
| whims and mood swings of one person. I was once fired over a
| pretty far-fetched misunderstanding, but 1) they didn't
| mention it because "I should know myself", so I didn't really
| have a chance to clear it up, and 2) when I got fired they
| agreed it was a complete misunderstanding and that they made
| a huge mistake, but fired me anyway because that was the road
| already taken now. Great ... thanks ... It's hard to imagine
| that happening at a larger company with an actual HR
| department and stuff.
|
| Also: a Wikipedia-esque "be bold" attitude helps. Obviously
| refactoring things just for the craic isn't a good idea, but
| if there's something that can really make an impact then I'd
| just do it and send a PR, regardless of what I've been
| assigned. Obviously "refactor everything" on your own
| initiative isn't a good idea, but usually there's tons of
| smaller stuff that really helps: "introduce new
| function/package that helps reduce boiler plate", "split up
| this difficult to understand function", things like that.
|
| Most PMs will realize they don't have a 100% in-depth
| understanding of the code, and will trust you when you say it
| helps you do your work better. Usually it's fine as long as
| you apply proper judgement and don't spend too much time on
| it. That said, there are some places where this really isn't
| appreciated regardless. IMHO that's pretty dysfunctional and
| when I start looking for somewhere else to work.
| lanstin wrote:
| Your successors just hope you can only remember 3 things at
| once so you end up with elegant designs rather than one of
| those monsters who can remember "to add a new wrinkle to this
| feature, just edit these 25 files."
| ysris wrote:
| Considering the biases, the one I wrote for the company I
| created.
|
| When we have the opportunity to be in this context, keeping in
| mind what bothered us in the codebases with which we were able to
| work in the past, we can force ourselves not to reproduce the
| same errors. Like the unmaintained unit and integration tests,
| the lack of refactoring, other developers that use fancy
| technologies instead of simpler concepts more for the opportunity
| to play with technologies than real need..
|
| And also, I guess, because we are more aware that the code is a
| reflection of the company that we want to have, that the simpler
| the better is a key point when we need to debug.
| anonzzzies wrote:
| Generally code that hasn't been tested commercially. Unencumbered
| by pesky client driven features, just code for dreamt up features
| that are fun to code but perhaps will never be used.
| jmartin2683 wrote:
| Hopefully the last one you were responsible for.
| jmartin2683 wrote:
| Hopefully the most recent one you were responsible for.
| bittermandel wrote:
| Any code base that doesn't use the advanced features of it's
| language(s) are always better in my experience. Heavy usage of
| e.g. meta-programming in Python or perhaps uber's fx (dependency
| injection) in Go makes projects infinitely harder to get into.
| jamestimmins wrote:
| Such a great point. I audibly groan when I come across Python
| meta-programming.
|
| While not an advanced feature, I have a similar response when I
| see lots of decorators in Python. They quickly become a
| debugging nightmare.
| sebstefan wrote:
| I can see a few cases where that depends...
|
| Really simple languages: Ruling out meta-programming is really
| going to limit you in Lua for example. Just being able to do
| `mySocket:close()` instead of `Socket.close(mySocket)` involves
| meta-programming.
|
| Older languages: For C++ the "simple" features are going to
| include raw pointers and macros. Maybe it's not so bad to allow
| smart pointers and templates to avoid those
| wruza wrote:
| Both of these examples are examples of an under-programmed
| core though. Lua is notorious for lacking batteries, so
| everyone has to reinvent their own. There's literally no
| serious Lua program without some sort of classes, but they
| still resist adding them into lauxlib.
| BiteCode_dev wrote:
| Depends who is providing it.
|
| Django and pydantic meta programming usually make the code
| easier to deal with.
|
| In shop written meta programming usually sucks.
| tommypalm wrote:
| I worked at GOV.UK for a few years on what was effectively
| specialised CMSs all written in Rails. Mostly basic CRUD stuff.
| A contractor came along and built the most insane CMS I've ever
| seen. I found out later it was flavor of the Java Repository
| Pattern using dependency injection. It became so difficult to
| work with that the last thing I worked on there was to delete
| it and rebuild it using standard Rails patterns.
|
| The KISS philosophy exists for a reason and that includes over-
| using advanced language features just to show off.
| nijave wrote:
| Besides just KISS, a lot of messes I've seen have been
| implementing patterns outside the framework or implementing
| complex patterns that didn't add value.
|
| Besides KISS (or maybe as an extensive), try to keep
| framework-based codebases as close to the official documented
| setup as possible. You automatically get to s of free, high-
| quality documentation available on the Internet.
| andrei_says_ wrote:
| Props for gov.uk! I've looked at its documentation and design
| system and see both as peak user experience and clarity.
| RangerScience wrote:
| I've had a consistent experience in Rails where I think for a
| day or two I've got a legitimate use case for one the whacky
| things you can do... and then as I work the problem more, it
| turns out: nope, the simple stuff is still the Right Way.
|
| Someday, Ruby shenanigans, someday...
|
| PS - Being able to pull the shenanigans _is_ super useful
| during the dev process, usually to skip some yak shaving
| during exploration, so it 's nice to have anyway.
| Mikhail_Edoshin wrote:
| Having just removed a metaclass from my Python code I totally
| agree.
| rvdginste wrote:
| I think one should not use advanced language features just
| because, but I also think one should not avoid using advanced
| language features where it is useful.
|
| Why would the code base be worse when advanced language
| features are used?
| peelle wrote:
| I disagree with this, but I understand where it's coming from.
| I think you have a form of whiplash from things like: - Novices
| overusing the new shiny. - Java/C++/etc Jr programmers
| overusing design patterns. - Perl programmers solving
| everything with regexes. - Small startups with GraphQL, or any
| other large enterprise tool. - Metaprogramming, Maco's,
| Dependency injection, Recursion, etc when a simpler solution is
| a better fit.
|
| IMHO, a "best codebase" will be just a bit more advanced than I
| am, with good resources for me to grok it. I want to be able to
| learn from it. I also don't want to be so far out of my depth
| that I can't make a reasonable contribution.
| arp242 wrote:
| A pinch of salt can really liven up a dish. Not every dish
| needs it, but when used appropriately it's almost magic in
| how much difference it can make.
|
| A lot of salt always make everything disgusting.
| tracerbulletx wrote:
| Huh? Salt is one of the most important foundational
| elements of basically all cooking.
| pphysch wrote:
| It would be cool if newer (imperative) languages had a clear
| protocol to delimit the simple core of the language from the
| advanced, library-building features. Like Rust has `unsafe`
| blocks, imagine a `advanced` or `metaprogramming` block that is
| required in order to use advanced metaprogramming features.
| Then you could tell a junior to implement $MUNDANE_INTEGRATION
| without use of `advanced`, and that constraint could be
| statically verified in PRs, etc.
|
| It seems like it would vastly simplify language evolution. The
| core language can have rigid limits on changes and follow the
| Zen of Python or whatever, while the `advanced` language
| extensions can be a bit looser. Features could graduate from
| advanced to core without breaking anything. You get a single
| powerful+expressive language with a "porcelain interface"
| without necessarily giving juniors ammo to shoot themselves in
| the foot.
| burutthrow1234 wrote:
| I worked with someone who insisted on using fx for DI in go.
| It's so antithetical to the entire Go philosophy, I don't even
| think it's an "advanced feature". It's just bringing Java cruft
| to a language where it isn't necessary and making everything
| worse.
| ethegwo wrote:
| A Python version about core features of the Hindley-Milner type
| inferencer: https://github.com/ethe/typer
| donatj wrote:
| The latest Go micro-service I have built.
|
| About once a year roughly, for the last couple years, the
| opportunity has arisen to greenfield a Go micro-service with
| pretty loose deadlines.
|
| Each time I have come into it with more knowledge about what went
| well and what I wasn't particularly happy with the last time.
| Each one has been better than the last.
|
| I've been building software professionally for twenty years, and
| these micro-services have been one of the few projects in that
| time that have had clear unified vision and time to build with
| constant adjustments in the name of code quality.
| lanstin wrote:
| Sounds fun. I have a similar policy but only every other year.
| It means I know what the hell I am talking about when in
| meetings with people who have slow or inefficient services. It
| also means my job of mostly telling other people how to solve
| problems is still a lot of fun for me.
| intelVISA wrote:
| Go micro-service with pretty loose deadlines
|
| Sounds like A Money Printer, congrats.
| Traubenfuchs wrote:
| Every code base I have ever worked on was a legacy nightmare.
| Every "greenfield" project I joined turned into a legacy
| nightmare within weeks. I have never encountered enjoyable code.
| I had the displeasure of wading through Spring, Hibernate and
| Apache HTTP client code before and they were all an
| incomprehensible mess.
|
| My conclusion: You know the claim "any medication that really has
| an effect must also have side effects". I would like to adapt
| that for code: Any code that does a lot of useful and complex
| things must be an arcane, barely maintainable mess that can only
| be understood by deep study.
| alex_suzuki wrote:
| IMHO library code is especially challenging, as cruft has a
| tendency to accumulate, historical behavior needs to be
| preserved, and APIs are set in stone once they're built.
| lukan wrote:
| "Any code that does a lot of useful and complex things must be
| an arcane, barely maintainable mess that can only be understood
| by deep study."
|
| "Every code base I have ever worked on was a legacy nightmare.
| Every "greenfield" project I joined turned into a legacy
| nightmare within weeks"
|
| I am sorry to say this, but it really sounds like you were
| either really, really unlucky, or part of the reason why it
| became a mess.
|
| Complicated things are complicated. Nothing can ever change
| that and it requires study to understand it.
|
| But it still does matter a lot, how one organizes the whole
| thing. How it is structured, documented(!), refactored. Are
| there competent people in charge who understand it all and kick
| peoples asses if they make even a temporary mess or forget to
| document, or do random people make changes wherever they see
| fit, because a deadline is ticking?
|
| Modularisation is usually the key. Small modules do one thing
| and are as seperated as much as possible with as little side
| effects as possible.
|
| And if one has to ship things, it is not always possible to
| keep it pure and if the code is not intended to live forever
| then this is often fine. But if the codebase is supposed to
| stay, then there needs to be the time to clean up the hacks. Or
| don't and then you end up scared touching anything.
|
| That being said, the technologies you mentioned, I would not
| like to touch either..
| t43562 wrote:
| I think that's a bit cruel. Usually mess happens because lots
| of people with different ideas about the future needs and
| best structure meet up in the code - and it's hard to develop
| a consistent culture.
|
| Also codebases get too large for any one person to refactor
| them into shape in the time they have each day. So you end up
| needing people who are responsible just for keeping things in
| shape.
| lukan wrote:
| Well, that is why I said with those words it _sounds_ like
| this to me, not that he or she _was_ in fact responsible.
| (at least this is what I meant)
|
| "Also codebases get too large for any one person to
| refactor them into shape in the time they have each day."
|
| Which is why modules, or subprojects were invented. Or
| however you want to call it, if one person is only
| responsible for a small part and not everyone for
| everything. And yes, there is also the non trivial problem
| of time management.
| t43562 wrote:
| yup, but the style of the modules starts to diverge as
| different people maintain them.
|
| IMO one does want people whose fulltime job is to looking
| at a codebase orthogonally to those who are just
| implementing a feature. People who make sure it builds
| fast, is secure tries to be consistent to some degree
| etc.
|
| Many companies call teams/indivuduals like this a "cost
| centre" and disparage it because they are dolts.
| lukan wrote:
| If those code checkers are experienced programmers, who
| do not annoy people with arbitary guidelines and
| standards, then yes, this might also work. But I do not
| think they are always necessary as a seperate full time
| role, if everything works normal otherwise. And with
| working normal, I mean there is enough time to refactor
| and document and clean up and is actually done.
| RangerScience wrote:
| > Modularisation is usually the key. Small modules do one
| thing and are as seperated as much as possible with as little
| side effects as possible.
|
| 100% agree. The nuance I find is that a lot times, people
| want to draw the lines of the module along something that's
| more immediately intuitive, but results in longer-term
| complexity.
| lukan wrote:
| Yes, the hard part is to define what goes inside one
| module. There are rules of thumb, but lots of exeptions.
| Taking any advice too literally can also make things
| awkward.
| arp242 wrote:
| > Modularisation is usually the key. Small modules do one
| thing and are as seperated as much as possible with as little
| side effects as possible.
|
| I think the key aspect is a bit different: understandability.
| That is, if you open two pages of random code, then can you
| roughly understand and follow what it roughly does?
|
| Everything in a single package/module/directory can be quite
| understandable. It doesn't necessarily _need_ to be modular,
| but it often does help. "Understandability" is more broad
| though, and also includes things such as not having weird
| hard to understand functions nested 8 levels deep, and things
| like that.
|
| It also means not over-engineering things. "Our code is hard
| to understand, therefore, we need more architecture" is a
| fallacy I've seen a few times. The code wasn't hard to
| understand due to lack of modularity or architecture, but
| because it was just badly written unclear code.
| lukan wrote:
| "It also means not over-engineering things. "Our code is
| hard to understand, therefore, we need more architecture"
| is a fallacy I've seen a few times. The code wasn't hard to
| understand due to lack of modularity or architecture, but
| because it was just badly written unclear code."
|
| For sure, if I have 10 tightly separated micro modules,
| just for the sake of it, then this is not helping clarity.
|
| But if everything is an entangled mess of spaghetti code,
| then modularisation helps, so I do not have to look into 10
| places to get what a certain function is doing.
| mrweasel wrote:
| Something in your answer triggered a flash back to when I
| worked for a phone company, 20 years ago. One team, under the
| leadership of our questionable chief architect, had produced
| our new, all encompassing backend system. This was to be the
| corner stone of all future development and integration, dog
| slow and complicated as it where. I worked as a .Net developer
| and had the misfortune to be among the first to integrate with
| this monster. Try as I might, I could not get .Net to
| interoperate with these services. Finally I figured out that
| the SOAP services was using some old deprecated versions. Going
| back to the architect, I ask "did you build this on Apache
| Axis, and please say Axis2", but no, it was just Axis, a
| deprecated version, that would generate webservices not
| supported by newer .Net version. That wasn't an problem,
| because: "None of our project have upgraded to those .Net
| versions yet"... DUDE, we've launched a brand new system based
| on that .Net version a year ago, and what was you f-ing plan
| for the future, to redo every single service using Axis2?
|
| This guy had based a brand new system on a framework/library
| that was no longer maintained, even before the system was
| launched.
| noelwelsh wrote:
| My open source projects. One in particular I've been working on
| for about 10 years. The code is consistent and always getting
| better, even though there is a lot more work that could be done.
| structural wrote:
| Worked on a codebase for a large safety-critical system where
| everything was 100% documented, and the development guide for the
| project was followed so closely that you couldn't tell, across
| millions of lines of code, that the whole thing wasn't written by
| one person. Absolutely impressive levels of attention to detail
| everywhere, down to not even being able to find typographical
| errors in comments or documentation (a typo in a comment was
| treated just as seriously as any other bug).
| n_ary wrote:
| Let me guess, it was very well funded and there were no fake
| deadlines and cross-team dependencies, am I correct or am I
| very correct?
| contingencies wrote:
| Cancel the rocket launch people, we have a semantic
| incongruity! Correct is a binary state.
| unboxedtype wrote:
| I am almost sure this is because the system would have to pass
| a certification procedure somewhere, and for that they would
| need this level of clarity. Am I right?
| pavlov wrote:
| For me, the most eye-opening codebase of my career was Cocotron,
| around 2007:
|
| https://github.com/cjwl/cocotron
|
| I was looking for a way to port my native Mac Cocoa apps to
| Windows. I had been already disappointed by the aimless sprawl of
| GNUstep.
|
| This one-person project implemented all the essential APIs for
| both Foundation and AppKit. Reading the code was a revelation:
| can it really be this simple and at the same time this
| effortlessly modular for cross-platform support?
|
| I contributed a few missing classes, and successfully used
| Cocotron for some complex custom GUI apps that needed the dual-
| platform support.
|
| Cocotron showed me that one person with a vision can build
| something that will rival or even outgun large teams at the big
| tech companies. But focus is essential. Architecture astronauts
| usually never get down from their high orbits to ship something.
| tlarkworthy wrote:
| Google3 codebase. It's just so vast and it works. It's a miracle.
| I feel lucky to have seen it. Everytime you change it, it reruns
| the dependencies. Everyone has different view concurrently.
| Commits are efficient immutable snapshots. It's just incredible
| multiplayer. So massively beyond what can be done with GitHub.
| Really I feel it's peak codebase. I've not seen what the other
| big techs do but it blew my mind
| asymmetric wrote:
| What is Google3?
| kqr wrote:
| I would assume Google's monorepo.
| Daub wrote:
| This link might help...
| https://dl.acm.org/doi/10.1145/2854146
|
| The linked pdf has lots of details.
| bushbaba wrote:
| The replacement of google2
| arcade79 wrote:
| It was wonderful to work with. :-) It's one of the things I
| truly miss from Google.
| VirusNewbie wrote:
| Google3 codebase very consistently has _clean_ code, but some
| of the architecture there is very much not great.
|
| Some is great, some not so much.
|
| Some of Verizon's code was _much_ more elegant (though much
| smaller scope) from an API perspective, and really leaned into
| advanced type systems in a way Google has not.
| dilippkumar wrote:
| +1
|
| I can not understate how much I agree with parent comment.
|
| The opposite of move fast, build a shitty prototype and iterate
| is a deliberate problem solving approach undertaken by the
| highest caliber of engineers. The actual challenges to be
| addressed are effectively addressed right at the design stage.
|
| The result is a thing of immense beauty and elegance.
|
| I will forever be grateful for the opportunity I had to see
| this magnificent piece of engineering in action.
| robertsdionne wrote:
| Cannot overstate.
| semitones wrote:
| I, like many others here, echo this sentiment. While I
| disliked working in ads, the experience of working with that
| repo+tooling is unmatched by anything else in my career.
| inoffensivename wrote:
| I honestly can't tell if this comment is sarcastic...
|
| google3 is a quagmire and it's getting worse by the day
| ignoramous wrote:
| Well, first impressions: It is part of our
| human condition to long for hard lines and clear concepts.
| When we have them we have to either face the fact that some
| realities elude them, or else bind ourselves to the
| inadequacy of the concepts.
|
| https://fs.blog/impressions-are-schematically-determined /
| https://archive.is/F8uPE
| cpojer wrote:
| https://github.com/nkzw-tech/athena-crisis
| joeshachaf wrote:
| I worked with a very experienced engineer who created his own WAF
| (software-based) using Java. From a purely architectural
| perspective, it might not be the best (I might have used an ISAPI
| filter back then), but the code itself was very efficient, well-
| written and documented. I used it several times as a teaching
| example.
| Agingcoder wrote:
| I haven't contributed code to it, but I've read lots of code in
| it ( for work reasons ) - I really like the golang compiler and
| library codebase.
|
| About codebases I've written code for, the best one strived for
| simplicity, and was driven by very strong engineers who actively
| considered code hygiene ( in the broadest possible sense ) a
| first class citizen.
| prinny_ wrote:
| The one in my previous job, which was an admin board for a market
| intelligence application. Ultimately, the reason it was good was
| because the engineers had zero ego on top of having excellent
| skills. The team that set the codebase were basically 4 seniors
| and 3 principals (the client actually did pay for top talent in
| this case) so not only everything was based on industry
| standards, written elegantly and organized perfectly, but every
| time some new requirement came up, these senior / principal
| engineers would discuss it in the most civilized matter I have
| ever seen.
|
| E.g, "we need to come up with a way to implement X". Person A
| gives their idea, person B gives another idea and so on until
| everybody shared their thoughts. Then someone would say "I think
| what person C said makes the most sense" and everybody would
| agree and that was it. 30 minutes to hear everybody out, 3
| minutes to discuss who will do it and when and the meeting was
| over.
|
| I think the biggest testament to this code base was that when
| junior members joined the team, they were able to follow the
| existing code for adding new features. It was that easy to
| navigate and understand the big picture of.
| brailsafe wrote:
| This is the approach I tried to take as an IC frontend on a
| building management system dashboard about a year ago. It was
| basically me and one other frontend, one full-stack, and a QA
| or 2, plus a manager. My manager I guess had written a lot of
| the code we'd be interacting with, and I guess was somewhat
| protective and reluctant to delegate authority over decisions
| around it. It was a big refactoring project, and I just
| encouraged my colleagues to take the things they wanted, like
| rebuilding the state management system. We'd discuss the
| requirements and why it was necessary, and I'd look for reasons
| to agree rather than disagree, then we'd be off. Something
| burnout has taught me is that the marginal differences between
| one implementation detail or another are not worth getting hung
| up on unless they pose a real problem, especially when someone
| else gets to decide that doing it fast is priority (sprints).
| BiteCode_dev wrote:
| Last year I worked for a client that gave me a lot of time, money
| and autonomy to lead dev on a critical software rewrite.
|
| We got a small team of competent people, with domain experts to
| peer code with the devs.
|
| It was wonderful. We could test, document and clean up. Having
| people who knew the trade and users at hand removed second
| guessing.
|
| The result was so good we found bugs even in competitors'
| implementations.
|
| We also got x5 in perfs compared to the system it was replacing
| and more features.
| leetrout wrote:
| Good for you. Magical moments in careers are hard to find in my
| experience but they are so satisfying when you get there.
|
| Glad whomever was over this didnt just drop the "dont rewrite"
| joel spolsky article and fight making it happen.
| BiteCode_dev wrote:
| I actually was the one telling them not to rewrite, lol.
|
| But the original code was a mess of matlab spaghetti, they
| couldn't find a way to hire for that. Not to mention turning
| it into a web service was already a big hack of java parsing
| a raw dump of matlab datastructures that nobody dared to
| touch.
|
| I had to read the matlab code, and it tooks hours to decypher
| a few lines. Plus the language doesn't have great debugging
| and error handling capabilities and the tooling is quite
| terrible.
|
| So rewriting to python won, and for once, I must say it was a
| good call.
| junon wrote:
| Joel has since said that that he doesn't really agree with
| that advice anymore, at least not in the same way. Super
| annoying that it gets parroted over and over again as though
| it's the word of the lord.
| thecupisblue wrote:
| Similar thing.
|
| Had time and autonomy from a client, so took sweet time
| examining the domain, the existing systems et al. Spent a few
| months writing the basis and the framework around what will be
| done, based on years and years of experience I had with bad
| frameworks and codebases, combined with working on the same
| domain for their parent company years ago.
|
| And it worked. We delivered features insanely fast, hundreds of
| forms were migrated, feature generators would create 90% of the
| boilerplate code and the code was small, readable and neatly
| clustered. Maintaining it was a piece of cake, leading to us
| not having enough work after a while so I negotiated our time
| to half a week for the same money.
|
| After a while, client deemed us too expensive to pay for only
| 2.5 days of work - after all, how does it make sense - if we
| are paying them that much, they should work 5 days!
|
| So they cut us out. Two things happened:
|
| 1. Devs that got moved to other projects in the company told me
| they didn't know development could be so smooth and tried to
| replicate it in future projects, even tho they say it failed a
| lot of lessons they picked up from the framework were highly
| relevant in their future careers.
|
| 2. The company found a cheaper developer, he said "this is
| unusable and has to be rewritten" and rewrote it into "clean
| code", taking longer than the original project took. At least
| he works 5 days a week now.
| simfoo wrote:
| My past three employers code bases: mono-repos, Bazel, lots ot
| C++ and Python, thousands of libraries and tools, code generation
| and modeling tools that are fully integrated into the build, easy
| cross compilation, large integration tests just one bazel test
| invocation away, hermetic and uniform dependencies...
| fire_lake wrote:
| Bazel Python does not cross compile IIRC. How was this
| achieved?
| Claudus wrote:
| Laravel, once you get the hang of it everything just works, and
| using a debug bar to optimize database calls is very satisfying.
| neilv wrote:
| One of them stands out, due to being super-productive, over
| years, and then decades.
|
| A large system that was originally written by only two super-
| productive engineers (I mean real engineers, both with PhDs in an
| area of Engineering). And a comparably capable and essential IT
| person.
|
| The reasons for the super-productivity include one of the
| developers choosing great technology and using it really well, to
| build a foundation with "force multiplier" effects, and the other
| developer able to build out bulk with that, while understanding
| the application domain.
|
| Another reason was understanding and being pretty fully in
| control of the code base, so that, as needs grew and changed,
| over years, someone could figure out how to do whatever was
| needed.
|
| One of the costs was that most things had to be built from
| scratch. Over time that also proved to be an advantage, because
| whenever they needed (put loosely) a "framework" to something it
| couldn't do, they effectively owned the framework, and could make
| dramatic changes.
|
| When I said "costs", I mean things like, many times they needed
| to make a component from scratch that would be an off-the-shelf
| component in some other ecosystem. So if someone looked closely
| at how time was sometimes spent, without really understanding it
| or knowing how that panned out, it would look like a cost that
| they could optimize away. But if they looked at the bigger
| picture, they'd see a few people consistently, again and again,
| accomplishing what you'd think would take a lot more people to
| do.
|
| It helped that the first programmer also became the director for
| that area of business, and made sure that smart engineering kept
| happening.
|
| Someone might look for a reason this couldn't work, and think of
| bus factor. What I think helped there was the fact that the work
| involved one of those niche languages that attract way more super
| programmers than there are jobs. "Gosh, if only we had access to
| a secret hiring pool of super programmers who were capable of
| figuring out how to take up where the other person left off, and
| we had a way to get them to talk with us...")
|
| It was easy to imagine a competitor with 100 developers, not able
| to keep up, and at many points getting stuck with a problem that
| none of them were able to solve.
| klibertp wrote:
| I assume you avoided identifying (or even hinting at) this
| "great technology" on purpose, but could you persuaded to
| divulge what it was?
| neilv wrote:
| It started as a bespoke structured data and Web architecture,
| which did somewhat complex things that would've been a huge
| headache to build and maintain in the dominant languages of
| the time, but were viable for one person to figure out and
| implement in Scheme and PostgreSQL.
|
| That bespoke architecture and implementation language lent
| itself to a lot of rapid domain-specific functionality, as
| well as doable architecture changes over time.
| klibertp wrote:
| Thanks! :) I hoped it'd be a Lisp or Smalltalk - great to
| hear I almost guessed right, though I never imagined an
| RDBMS to be in the mix :)
| neilv wrote:
| There's actually a bespoke object metamodel mapping atop
| the RDBMS, and it permits customers who want to use SQL
| direct to the database to do so.
|
| And, IMHO, PostgreSQL is the default choice for most
| storage backend purposes, unless one has figured out a
| good reason something else is better. :)
|
| There were also multiple kinds of large blob storage
| needs, and changing architecture needs over time (e.g.,
| multi-tenant, changing authn implications of things like
| browser plugins, scalability, move to cloud), so systems
| programming skills and controlling your framework comes
| in handy, for not getting stuck or blocking on vendors,
| but just solving the problem.
| bruce343434 wrote:
| What is this niche language attracting super programmers?
| neilv wrote:
| Scheme, Common Lisp, Haskell, [edit] Smalltalk, probably more
| obscure ones...
|
| Also, Erlang, Rust, and Clojure, though those have been
| rumored to be employable, so no longer get as much of the
| filter as you get when it was just people caring strongly
| enough to want to use a particular language _despite_ the
| unemployability.
|
| Whether that caring happened because of skill (to identify
| something good), or skill happened because of caring (to
| invest the time in exploration and thinking and practice),
| these communities seem to get more than their share of great
| programmers.
|
| And so you know where to look for them, and you have a reason
| that they might want to talk to you.
| nonameiguess wrote:
| No name because the project code name changes with each new
| contract, but the first-level ground processing system for the
| NRO that turns raw satellite downlinks into some kind of human-
| intelligible files.
|
| I think a lot was just the restrictions built into the way
| development happens required a high level of discipline, care,
| and planning. Also, requirements were pretty tightly coupled to
| sensor platform capabilities, which are known well in advance and
| don't change unexpectedly, so waterfall development actually
| works and you don't have to deal with the chaos of not really
| knowing what will and won't work and customers constantly
| changing their minds.
|
| Code base was overwhelmingly C++, some Fortran, a lot of it was
| very old. It was all developed on airgapped networks, and the
| difficulty of bringing in external dependencies meant there
| largely were not any. All of the library functionality we
| required was mostly developed in-house, so we had extremely well-
| documented and stable functions available to do damn near
| anything you could want, with a good chance that whoever first
| wrote it was still there. All development had always been tied to
| a ticketing system of some sort that included all of the original
| discussion, design documents, and that kind of thing might add
| process overhead upfront, but it means that forever new
| developers can simply read the history and learn exactly why
| everything works the way it works.
|
| The system itself was very Unixy. In production, it was meant to
| be run as a server with many instances striped across high-
| performance compute nodes, but it did not have to be run that
| way. Every individual product flow could also be built as its own
| transient executable, so that working on a single component could
| easily be done locally. You didn't have to rebuild the world or
| spin up your own server. Performance requirements were enough
| that we had our own customized networking stack and filesystem in
| production, but nothing depended on this for function, so you
| could still develop and test on plain Linux with ext4.
|
| The culture was terrific. We were part of one of the big five
| defense contractors, but an acquisition and this program was
| largely still staffed by employees of the original company that
| had been acquired. We were de facto exempted from most of the
| bullshit any other program had to deal with. I don't know if that
| was part of the original terms of being acquired or just a
| consequence of having so many long-time developers that couldn't
| afford to be lost if you subjected them to nonsense. This was the
| kind of project that people intentionally stayed on and retired
| from because the experience was so much better than any other
| project you could get assigned to.
|
| Ironically, it had none of the characteristics that high-
| performing companies often tout. You work in private. The rest of
| the company, including your own management chain, doesn't even
| know what you're working on. You'll never get any recognition or
| publicity. The pay is mediocre. We weren't attracting the best
| and brightest from all of the world. You had to be American, have
| a top-secret clearance, and be geographically close enough to the
| facility to get there every day, so this was a pretty constrained
| hiring pool. I still worked with some of the smartest people and
| best engineers I've ever known. The upside of this kind of
| environment is you have no mercenaries or publicity hounds.
| Everyone who sticks around is a person who really loves and cares
| about what they're working on, and a lot of people did stick
| around. The sanity and organization of the code was heavily
| facilitated by having a whole lot of people working on it who'd
| been working on it for 30+ years.
| rthnbgrredf wrote:
| The best codebase is the one you fully understand. I prefer
| codebases that are small enough to understand within a week. This
| is why I like Microservices. Large codebases can be overwhelming
| and even senior developers working a decade in the company of
| might not fully understand them. Instead, I prefer maintaining a
| few Microservices that our team fully comprehends, where the
| entire codebase fits into a clear mental model. We then interact
| with other codebases, that have active mental models in other
| teams, via APIs.
| _kb wrote:
| I don't think the 'code' side of code base can be considered in
| isolation.
|
| What makes a _project_ objectively good (from subjective
| experience) is a combination of code, design, documentation, and
| often the humans involved.
| fer wrote:
| One that had a sort of improvised facade/adapter pattern (it
| didn't really follow either) in a clearly cut multilayered and
| pipelined structure, with actor model bits where it made sense.
|
| The code wasn't simple, at all. It took active training of new
| arrivals for them to understand it. But it was very well thought
| out, with very few warts given the complexity, and extremely easy
| to extend (that was the main requirement, given constant changes
| in APIs and clients).
|
| We had an API, with multiple concurrent versions, that
| transformed requests into an intermediate model, on which our
| business logic operated, later targetted external APIs (dozens of
| them, some REST, some SOAP, some under NDAs, some also with
| multiple versions), whose responses turned again into the
| intermediate model, with more business logic on our end, and a
| final response through our API. Each transaction got its context
| serialized so we could effectively have what was an, again
| improvised, "async/await"-like syntax in what was (trigger
| warning) C++03 code.
|
| The person who engineered it didn't have formal CS background.
| Maledictus wrote:
| Ruby on Rails.
|
| It is the only framework I have read top to bottom.
|
| Also the FreeBSD kernel, if you want to see a C code base that's
| quite beautiful (for C).
| sgt wrote:
| In this category I would nominate Django as well. It's very
| well designed (opinionated, but usually for good reasons).
|
| In terms of large C code bases I enjoy reading the PostgreSQL
| source code.
| rc_mob wrote:
| whenever I'm stuck on how to structure some code, I ask
| myself how would Laravel do it? and look up their code and
| structure mine similarly
| t43562 wrote:
| I just cannot face the thought of calling any of them "best".
|
| Every one had good and bad features though. One or two were OS-
| sized and I think a codebase that compiles and links to 85GB of
| output for 20+ devices without being a total disaster inside is
| harder to do than a neat small python module or whatever.
|
| GOOD FEATURES:
|
| Maintenance of the build and test: I worked on tools that helped
| builds go faster so I saw a lot of codebases where people were
| not maintaining the build partly because nobody had that a s a
| responsibility. There was bad management of dependencies leading
| to build failures, poor performance, incorrect build output.
| Android would be a counter example to that - I don't know if
| people like developing in it but it was always hard to accelerate
| it as the maintainers fixed performance problems regularly
| leaving our tools with little to improve.
|
| Using appropriate languages. Writing everything in C++ was a fad
| at one time. All projects work better, port better, have faster
| build times, are easier to test etc if they use memory safe
| "build once" languages to a maximum (e.g. java) and unsafe ones
| (e.g. C/C++ which have to be rebuilt and tested for each
| device/os) to a minimum. IMO Android beat Symbian amongst other
| reasons because it wasn't all C/C++ and that meant a lot of code
| didn't have to be rebuilt for different devices. This made builds
| faster and fast builds lead to better quality because of a short
| dev-test cycle.
|
| Use of faster compilers over "better" compilers. Ultimate code
| performance and quality depends on a fast development cycle more
| (IMO) than on having the most optimizing compiler. GCC versus the
| older ARM compilers for example. Now the ARM compiler is based on
| LLVM and I know that happened indirectly from a suggestion I made
| to someone who then made it to ARM who then did it.
|
| The setup and build of one codebase I worked on was as easy as
| one could expect, the build errored out if you tried to use the
| wrong tools so you never ended up debugging weird failures
| because of an incorrect tool in your PATH somewhere. I made this
| feature happen :-D. With big codebases the tools could be
| included in the version control system so you knew you had the
| right compiler, right object dumper etc. This is another strength
| of Android and yet I was in a project for Symbian to do the
| opposite because of some utter bonehead who never touched a build
| in his life who was trying to make a name for himself with his
| slimy bosses as a "doer" and "reformer."
|
| Codebases (especially big ones) benefit a lot from some kind of
| codesearch/index where you could find out where some
| function/class/variable was defined and what version of the
| source base it was introduced in.
|
| BAD FEATURES:
|
| Exclusively Owned code - we need to know who understands code
| best and who is best to review it but I don't think anyone should
| have totally exclusive control. It was a nightmare for me at one
| job - trying to get another team to make some needed change (like
| fixing their stupid makefiles to work properly in parallel). We
| (build team) should have been able to do it ourselves - maybe
| including them in the PR. Sometimes ownership is entirely
| theoretical - nobody who wrote it is still employed and nobody
| among the notional owners understands it and none of them want to
| approach it within 100 metres in case it blows up and becomes
| their problem. I simply _had_ to approach such code - no choice -
| but I kept having to send diffs to people who didn 't want to
| bother to look at them. It was a case of pushing wet spaghetti
| and took forever to do very simple changes.
|
| Insufficient tests that run infrequently. What else is there to
| say?
|
| Complicated code with no "why" or "what this is for" type
| comments. The kind of thing you trawl around in for weeks and
| cannot make head nor tail of what is going on overall.
|
| Code with so much dependency injection and general SOLID that you
| have to bounce all over the place to understand a very simple
| action.
|
| Code where writing tests is an enormous ballache. In one Go
| codebase the reason was because somone decided that the standard
| Go practise of an array of test data being run through a kind of
| "test engine" was the only way anyone should be allowed to write
| tests. Hence you had to do lots of weird things to make your test
| cases into data. Generally we use a kind of "religious" approach
| to try to get consistency out of a group of people but then take
| it much too far.
|
| codebases without automated reformatting - so everyone wastes
| time arguing about line spacing or camel-case names or whatever
| in their PRs.
| gwbas1c wrote:
| > Code with so much dependency injection and general SOLID that
| you have to bounce all over the place to understand a very
| simple action.
|
| I find that happens when people get religious about patterns
| and methodology; without understanding the "why", the language,
| and how a computer works.
|
| Case in point: I once worked on a C# project that used a port
| of Spring for dependency injection: Ultimately, it was near
| impossible to know when something was constructed, and what was
| calling what. There were classes that couldn't call themselves
| through "this" because of certain weird dependency injection
| features used.
|
| Later, I decided to use dependency injection as a design
| pattern: Instead of a complicated DI framework, there was just
| a few files of code. It was very easy for newcomers to
| understand. It was also easy to swap in mock objects, and easy
| to swap dependencies based on the target platform. It was also
| easy to see when a dependency was constructed; because it
| wasn't hidden behind a giant framework.
| nijave wrote:
| I was debugging some issues with Thanos and had pretty good
| success tweaking the codebase to add additional telemetry.
|
| The code was fairly well organized and more importantly worked
| out of the box with a Makefile and IDE integration (GoLand). All
| it took was `git clone` and opening GoLand to get started.
|
| For C (maybe it's C++), fluentbit seemed pretty straight forward
| (I don't have much experience in C though)
| sibit wrote:
| Pretty much any internal tool/TUI/CLI/library I've created. If I
| had to guess I'd say at most 25% of the company projects I've
| worked on have launched AND have consistent usage. Working hard
| on something just for it to wither crushes my soul but internal
| projects are different. They're all skunk works projects. No
| tickets. No project/board. No PM pushing back on how many points
| (read: hours) something should be. I'm solving real problems that
| directly impact the quality of life for myself and my coworkers.
| The best part is getting real, genuine, feedback. If something
| sucks they'll tell you and they won't sugarcoat it.
| jftuga wrote:
| I love this take. What language(s) do you typically use to
| write CLI programs? I'm also interested in learning about what
| types of internal TUI tools you have created.
| wruza wrote:
| The ones that were straightforward and close to the business. It
| starts at the obvious, works in an obvious way and has comment
| blocks at hard parts.
|
| For this reason I despise most modern [web] projects, which have
| a weak start, immediately drop into "services" and "components",
| do one action per source file per 30-50 lines of code, which are
| mostly imports and boilerplate, and have hundreds of these files.
| You can never tell what it does because it does almost nothing
| except formalities.
|
| I also noted a tendency to use wrong paradigms for a language.
| E.g. it may have no closures (imagine that in 202x) so they use
| events as continuations for asynchronicity, which results in a
| mess. Or it isn't immutable/functional, but they pretend it is,
| which results in fragility.
|
| The best projects are both close to their business and written in
| a paradigm of the language used.
|
| _Was there someone enforcing good practices top down?_
|
| Natural time pressure is the best bs cleaner, imo. You write
| effing code, maybe have few hours a week to refactor strange
| parts. With no time pressure a project naturally becomes massaged
| by all members into the "likeable" form of their age.
| 8474_s wrote:
| I can't name a good one, but i have strong dislike for big
| project codebases where even finding one file out of thousands
| where the relevant code resides is a challenge: its never
| something isolated but acts like some "component". The best i can
| think of is one-person projects where organization is streamlined
| as its actually has to be used by the author, not like a cog in a
| giant project.
| 999900000999 wrote:
| Realistically any code base where the engineers had at least a
| basic understanding of programming. You do not know suffering
| until you've seen someone hard code basic variables, we're
| talking about strings all over the place, and then they just copy
| the function again to replace the strings .
|
| I've legitimately left jobs over bad code. We're talking about
| code that did nothing in reality. The best code bases have been
| ones where I've been able to lead the direction. I get to know
| exactly how things work. I'm privileged to have a job where I
| essentially created the initial framework right now .
|
| Plus I'm fully remote, life is pretty good.
| karmakaze wrote:
| Probably the microservices-based one for async video messaging
| (i.e. Slack for video) for workers in the field. Each service was
| small enough that we could do a blue-green deploy to prod in
| about 2 minutes running only the service's tests and a tiny
| (intentionally limited) set of about 8 system journey tests _(can
| onboard a new user, user can create a contact /group, user can
| send a common content-type message, user can receive messages,
| user can react/respond to a message)_. Every commit to
| main/master automatically either deployed to prod or broke the
| CI/CD pipeline and needed to be fixed ASAP. Each service was also
| well-known by team members that it literally could be rewritten
| in a week or two if desired to change a key part of its design.
| karmakaze wrote:
| Thinking about what I wrote, I suppose my criteria for a good
| codebase is one that has the lowest friction to change: a fast
| edit/run/test/debug loop, a meaningful sense of security
| (without dogma), and fast automated deployment/revert (via
| blue/green). Given those a bad codebase can become a good one
| again (by uncoordinated action of individuals) without
| _everyone having to buy into_ a large investment.
| llmblockchain wrote:
| Any code I have ever touched.
| ttfkam wrote:
| Postgres. I don't code in C if I can avoid it, since it often
| feels like an awful lot of extra typing while still having to
| worry about memory safety. But the Postgres codebase is
| extraordinarily well organized and respects the humans that work
| with it with its intelligent handling of memory and judicious use
| of macros.
|
| I consider the core Postgres codebase to be the gold standard in
| development even though it's in a language I do not prefer to
| write in if given the choice.
|
| Shout out to the pgrx folks. You're awesome!
| https://github.com/pgcentralfoundation/pgrx
| rurp wrote:
| That is nice to hear, albeit unsurprising. Their public
| documentation is some of the best that I have worked with.
| Postgres is such an impressive project overall.
| shutupnerd0000 wrote:
| This codebase is 99.6% Rust according to GitHub.
| radimm wrote:
| That's PGRX - not PostgreSQL
| ttfkam wrote:
| As I said, I don't like coding in C.
| rrr_oh_man wrote:
| https://github.com/postgres/postgres
|
| Postgres (core?) seems to be 85% C.
| mikewarot wrote:
| The run time library for Turbo Pascal/Delphi for Windows was
| completely documented, sane, and very easy to work with. The
| working examples really helped.
|
| The free Pascal RTL seems opaque in comparison. Their reliance on
| and archaic help file build system keeps contributors away. Thus
| it's poorly documented at best.
| tiffanyh wrote:
| I've seen a few people say 'google3'.
|
| Q: is it actually the code that you loved, or simply the tooling
| that exists?
|
| (and if it's tooling, why can't that type of tooling be
| replicated for other codebases outside of google?)
| okdood64 wrote:
| > (and if it's tooling, why can't that type of tooling be
| replicated for other codebases outside of google?)
|
| The elegance of the tooling from what I hear is that there's
| tons of different tools maintained by different teams that work
| seamlessly (and fast) together to produce google3 and all of
| its supporting pieces.
|
| But to answer your question, sure it can. But good luck
| building your own. Google has been doing this since the 2000s.
|
| And if you're a big company already, you've already bought into
| your existing patterns & design choices; things like that are
| VERY hard to change.
| aleksiy123 wrote:
| Its both. The tooling has a very direct impact on the quality
| of the code.
|
| I think the reason its not easy replicable is:
|
| 1. It takes a ton of initial investment and ongoing maintenance
| but its worth it when your code base is gigantic.
|
| 2. There is a consistent set of top down enforced rules. With
| the consistency it becomes much, much easier to build tight
| integrations between tools.
|
| (almost?) everything is buildable by a single build system
| (blaze). When anyone can consistently build/test/run anything
| in your codebase it becomes a lot easier to build a whole host
| of potential tools like code search.
|
| Probably someone can dive deeper than I can. But one thing I
| learned the most important property for a code base to be
| maintainable/scalable is consistency.
| aleksiy123 wrote:
| One more thought, is also how much other systems can utilize
| the same tooling/workflows by just storing things in source
| code. Things that would probably traditionally stored as
| application state in a database are often stored instead in
| google3 as config.
|
| Things like on team rosters, on call rosters, code review
| permissions, service permissions, feature flags.
|
| All of it stored in google3 and can all utilize the same
| consistent set of tooling like presubmit test, integration
| tests, deployment automation, permissions, code search.
|
| Its sort of like Infrastructure as Code but more.
| ryukafalz wrote:
| > When anyone can consistently build/test/run anything in
| your codebase it becomes a lot easier to build a whole host
| of potential tools like code search.
|
| As someone who hasn't worked for Google, how does Google's
| implementation of this differ from e.g. Guix/nixpkgs? Being
| able to easily build/test any available package is a big
| reason I like using tools like those.
| aleksiy123 wrote:
| I'm unfortunately not that familiar with the internals.
|
| I think the public bazel (public version of blaze) docs is
| probably the best place to look.
|
| https://bazel.build/basics/hermeticity
| fragmede wrote:
| That type of tooling _can_ be replicated. That 's why every
| Xoogler tries so hard to get Bazel adopted on everything they
| touch. Sometimes that's appropriate, sometimes it's not, but
| that's why.
|
| Bazel isn't the whole of everything though, the other piece
| being exported is kubernetes, which isn't Google's borg, but
| that's its roots. There's Apache Airflow if you need a workflow
| engine like Sisyphus, no shortage of databases to choose from,
| though now we're drifting into the operations side of things.
|
| But basically, Google has invested untold millions of dollars
| in the form of SWE and other engineering hours into making
| Google3 operate. If anyone else invested that kind of dough,
| with enough smarts, they'd also be able to make it a good
| experience. The problem is few people have that kind of budget,
| and even fewer invest in that thing, preferring to use free
| tools instead.
|
| What tool do you use to edit code, and how much did your
| employer spend on that for you?
| tiffanyh wrote:
| So the development experience Github is marching towards
| doesn't compare?
|
| E.g., Codespaces, CoPilot, Actions, etc?
|
| (genuinely curious, not trolling)
| fragmede wrote:
| It does! Working like Google is like that but having had
| decades to build that dream, using the resources that only
| a megacorp could bring to bear in the problem, and in a
| monorepo. The fact that it's a monorepo is not to be
| dismissed. git doesn't work for monorepos.
|
| Codespaces is like CitC; I haven't been a Google since the
| rise of AI so I can't comment on how the internal
| equivalent to Copilot is; Actions is very primitive
| compared to what Google has, but yeah, you can see where
| it's going. it doesn't compare right now but it could,
| eventually.
| pradn wrote:
| Google's monorepo, and it's not even close - primarily for the
| tooling:
|
| * Creating a mutable snapshot of the entire codebase takes a
| second or two.
|
| * Builds are perfectly reproducible, and happen on build
| clusters. Entire C++ servers with hundreds of thousands of lines
| of code can be built from scratch in a minute or two tops.
|
| * The build config language is really simple and concise.
|
| * Code search across the entire codebase is instant.
|
| * File history loads in an instant.
|
| * Line-by-line blame loads in a few seconds.
|
| * Nearly all files in supported languages have instant symbol
| lookup.
|
| * There's a consistent style enforced by a shared culture, auto-
| linters, and presubmits.
|
| * Shortcuts for deep-linking to a file/version/line make sharing
| code easy-peasy.
|
| * A ton of presubmit checks ensure uniform code/test quality.
|
| * Code reviews are required, and so is pairing tests with code
| changes.
| pjungwir wrote:
| > Entire C++ servers with hundreds of lines of code can be
| built from scratch in a minute or two tops.
|
| Hundreds, huh? Is this a typo? It makes me wonder if the whole
| comment is facetious. Or do C++ programmers just have very low
| expectations for build time?
| jbyers wrote:
| I suspect they meant "hundreds of thousands"
| pradn wrote:
| Yes, oops - fixed!
| stefan_ wrote:
| That's the beauty of C++, an absurdly slow build is just an
| include away.
| thfuran wrote:
| Boost always helps prop up falling compile times.
| spease wrote:
| Has anyone done LLM training with template
| metaprogramming? That seems like another excellent way to
| keep Google's build servers warm.
| dskloet wrote:
| Code search not just across the entire code base but across all
| of time.
| ok_dad wrote:
| I'm surprised they didn't turn that into a product, it sounds
| great.
| twunde wrote:
| Parts have been. Sourcegraph is basically the code search
| post built by ex-Googlers originally. Bazel is the open
| source build tool. Sadly, most of these things require major
| work to set up yourself and manage, but there's an alternate
| present where Google built a true competitor to GitHub and
| integrated their tooling directly into it.
| snotrockets wrote:
| Building tools for others is a competency that is under
| rewarded at Google. They would never.
| tallowen wrote:
| I always find these comments about interesting, having worked
| at Facebook and Google, I never quite felt this way about
| Google's Monorepo. Facebook had many of the features you listed
| and quite performantly if not more so. Compared with working at
| Facebook where there are no owners owners files and no
| readability requirements, I found abstraction boundries to be
| much cleaner at FB. At google, I found there was a ton of cruft
| in Google's monorepos that were too challenging / too much work
| for any one person to address.
| pradn wrote:
| OWNERS files rarely get in the way - you can always send a
| code change to an OWNER. They are also good for finding
| points of contact quickly, for files where the history is in
| the far past and changes haven't been made recently.
|
| Readability really does help new engineers get up to speed on
| the style guide, and learn of common libraries they might not
| have known before. It can be annoying - hell, I'll have to
| get on the Go queue soon - but that's ok.
| sawyna wrote:
| This isn't true at all for OWNERS files. If you try
| developing a small feature on google search, it will
| require plumbing data through at least four to five layers
| and there is a different set of OWNERS for each layer.
| You'll spend at least 3 days waiting for code reviews to go
| through for something as simple as adding a new field.
| okdood64 wrote:
| 3 days for a new change on the biggest service on the
| planet? Not bad.
| tallowen wrote:
| I agree that it could be worse! Facebook has significant
| (if not more) time spent and I found adding features to
| news feed a heck of a lot easier than adding features
| that interacted with google search. Generally a lot of
| this had to do with the number of people needed to be
| involved to ensure that the change was safe which always
| felt higher at Google.
| tallowen wrote:
| I think I have heard similar things from other googlers and
| I think there might be two factors on why I think this:
|
| - I worked on Google Assistant which was responsible for
| integrating many services. This meant I had to work with
| other peoples code way more regularly that many at google.
|
| - I moved from FB to google - I'm not really sure how many
| people have had this experience. I think many of my
| colleagues at google found it surprising how many of the
| things they thought were unique to google actually also
| existed at FB.
|
| At the end of the day, any of these processes have
| pros/cons but I think the cruft of having APIs that are a
| couple steps harder to evolve due to finding
| Readability/Owners for everything you touch just makes
| things slightly less cohesive and a trickier place to have
| a "good" codebase.
|
| When I worked at FB, I would frequently rebase my code on
| Monday and find that, for example, the React framework
| authors or another smaller infra team had improved the API
| and had changed *every* callsite in the codebase to be
| improved. This type of iteration was possible in certain
| situations but was just much less common at google than at
| fb.
| snotrockets wrote:
| > I think many of my colleagues at google found it
| surprising how many of the things they thought were
| unique to google actually also existed at FB.
|
| Google workers are groomed to believe Google is the best,
| and hence they are too. A corollary of that, then, is
| that nobody else has it that good, when in fact, others
| sometimes have it better.
| AnotherGoodName wrote:
| Meta made the language fit their use case. Go into bootcamp
| change the search bar text to 'this is a search bar!' press
| F5 and see the change (just don't ship that change ;D). It's
| incredibly smooth and easy.
|
| Googles a mess. There's always a migration to the latest
| microservices stack that have been taking years and will take
| many more years to come.
|
| Like meta just changed the damn language they work in to fit
| their needs and moved on. Google rewrites everything to fit
| the language. The former method is better in a large
| codebase. Meta is way easier to get shit done to the point
| that google was left in the dust last time they competed with
| meta.
| robertsdionne wrote:
| * https://abseil.io/resources/swe-book/html/ch16.html
|
| * https://abseil.io/resources/swe-book/html/ch17.html
| WWWMMMWWW wrote:
| Google's code, tooling and accompanying practices are
| developing a reputation for being largely useless outside
| Google ... and many are starting to suspect it's alleged value
| even inside Google is mostly cult dogma.
| wiseowise wrote:
| > Google's code, tooling and accompanying practices are
| developing a reputation for being largely useless outside
| Google ...
|
| Not that I don't believe you, but where do you see this?
| matthewfcarlson wrote:
| I haven't worked at google, but this is something I have
| heard from a few people. Reputation is largely word of
| mouth, so it checks out for me. I suspect the skills/tools
| at most large companies are increasingly less transferrable
| as they continue to grow in scale and scope.
| worik wrote:
| > Google's code, tooling and accompanying practices are
| developing a reputation for being largely useless outside
| Google .
|
| It is almost a tautology
|
| Why would they be useful for domains they are not designed
| for?
| bbor wrote:
| I'm just one incompetent dev, but I'll throw this in the convo
| just to have my perspective represented: every individual part
| of the google code experience was awesome because everyone
| cared a ton about quality and efficiency, but the overall
| ecosystem created as a result of all these little pet projects
| was to a large extent unmanaged, making it difficult to operate
| effectively (or, in my case, basically at all). When you join,
| one of the go-to jokes in their little intro class these days
| is "TFW you're told that the old tool is deprecated, but the
| new tool is still in beta"; everyone laughs along, but
| hopefully a few are thinking "uhhh wtf".
|
| To end on as nice of a note as possible for the poor Googs: of
| all the things you bring up, the one I'd highlight the biggest
| difference on is Code Search. It's just _incredible_ having
| that level of deep semantic access to the huge repo, and people
| were way more comfortable saying "oh let's take a look at that
| code" ad-hoc there than I think is typical. That was pretty
| awesome.
| aleksiy123 wrote:
| Imho the reason for the deprecated and beta thing is because
| there is a constant forward momentum.
|
| Best practices, recommendations and tooling is constantly
| evolving and requires investment in uptake.
|
| I sometimes feel like everything is legacy the moment it's
| submitted and in a constant state of migration.
|
| This requires time and resources that can slow the pace of
| development for new features.
|
| The flips side is this actually makes the overall codebase
| less fractured. This consistency or common set of assumptions
| is what allows people to build tools and features that work
| horizontally across many teams/projects.
|
| This constant forward momentum to fight inconsistency is what
| allows google3 to scale and keep macro level development
| velocity to scale relative to complexity.
| bbor wrote:
| That's all well said, thanks for sharing your perspective!
| Gives me some things to reflect on. I of course agree
| re:forward momentum, but I hope they're able to regain some
| _grace_ in that momentum with better organization going
| forward. I guess I was gesturing to people "passing the
| buck" on hard questions of team alignment and mutually
| exclusive decisions. Obviously I can't cite specifics bc of
| secrecy and bad memory in equal amounts, so it's very
| possible that I had a distorted view.
|
| I will say, one of the things that hit me the hardest when
| the layoffs finally hit was all the people who have given
| their professional lives to making some seriously
| incredible dev tools, only to be made to feel disposable
| and overpaid so the suits could look good to the
| shareholders for a quarter or two. Perhaps they have a
| master vision, but I'm afraid one of our best hopes for an
| ethical-ish megacorp--or at least vaguely pro social--is
| being run for short term gain :(
|
| However that turns out for society, hopefully it ends up
| releasing all those tools for us to enjoy! Mark my words,
| colab.google.com will be shockingly popular 5y from now, if
| they survive till then
| teaearlgraycold wrote:
| My experience with google3 was a bit different. I was shocked
| at how big things had gotten without collapsing, which is down
| to thousands of Googlers working to build world-class internal
| tooling. But you could see where the priorities were. Code
| Search was excellent - I'd rate it 10/10 if they asked.
|
| The build system always felt more like a necessary evil than
| anything else. In some parts of google3 you needed three
| _separate_ declarations of all module dependencies. You could
| have Angular 's runtime dependency injection graph, the
| Javascript ESM graph, and the Blaze graph which all need to be
| in sync. Now, the beautiful part was that this still worked.
| And The final Blaze level means you can have a Typescript
| codebase that depends on a Java module written in a completely
| unrelated part of google3, which itself depends on vendored C++
| code somewhere else. Updating the vendored C++ code would cause
| all downstream code to rebuild and retest. But this is a multi
| billion dollar solution to problems that 99.99% of companies do
| not have. They are throwing thousands of smart people at a
| problem that almost everyone else has "solved" by default
| simply by being a smaller company.
|
| The one tooling I think every company could make use of but
| doesn't seem to have were all of the little hacks in the build
| system (maybe not technically part of Blaze?). You could
| require a developer who updates the file at
| /path/to/department/a/src/foo.java to simultaneously include a
| patch to /path/to/department/b/src/bar.java. Many files would
| have implicit dependency on each other outside of the build
| graph and a human is needed to review if extra changes are
| needed. And that's just one of a hundred little tricks project
| maintainers can employ.
|
| The quality of the code was uniformly at least "workable" (co-
| workers updating parts of the Android system would probably not
| agree with that - many critical system components were written
| by one person poorly who soon after quit).
| SR2Z wrote:
| > But this is a multi billion dollar solution to problems
| that 99.99% of companies do not have.
|
| I know it's trendy for people to advocate for simple
| architectures, but the honest-to-god truth is that it's
| insane that builds work ANY OTHER WAY. One of the highest
| priorities companies should have is to reduce siloing, and I
| can barely think of a better way to guarantee silos than by
| having 300 slightly different build systems.
|
| There is a reason why Google can take a new grad SWE who
| barely knows how to code and turn them into a revenue
| machine. I've worked at several other places but none of them
| have had internal infrastructure as nice as the monorepo; it
| was the least amount of stress I've ever felt deploying huge
| changes.
|
| Another amazing thing that I don't see mentioned enough was
| how robust the automatic deployments with
| Boq/Annealing/Stubby were. The internal observability library
| would automatically capture RPC traces from both the client
| and server, and the canary controller would do a simple
| p-test on whether or not the new service had a higher error
| rate than the old one. If it did? The rollback CL would be
| automatically submitted and you'd get a ping.
|
| This might sound meh until I point out that EVEN CONFIG
| CHANGES were versioned and canaried.
| ramses0 wrote:
| It's kindof a dumb answer, but `abcde` (A Better CD Encoder)
| continues to be my go-to reference for "everything shell".
|
| https://git.einval.com/cgi-bin/gitweb.cgi?p=abcde.git;a=summ...
|
| https://git.einval.com/cgi-bin/gitweb.cgi?p=abcde.git;a=blob...
|
| ...it's just so damned... like... it _is_ 5000 lines of bash, but
| given the constraints and the problem domain, it's an incredible
| (and well-organized) feat.
|
| If I ever question "How would I do something like XXX in shell?",
| I can usually find some clues, inspirations, or answers in the
| `abcde` codebase.
| e12e wrote:
| > https://git.einval.com/cgi-
| bin/gitweb.cgi?p=abcde.git;a=blob...
|
| Interesting. I was a bit surprised by the use of echo in the
| usage() function (and not cat and a heredoc) -- but that
| certainly looks like a very structured shell script.
|
| Thanks for sharing.
| paxys wrote:
| One that was written from scratch, by me. All others are
| terrible.
| ChrisMarshallNY wrote:
| Heh. Can relate.
|
| Even my own codebases/config. management systems are difficult,
| if I haven't dealt with them in a while.
|
| However, I am pretty good at understanding old stuff I did,
| mainly because I leave a legacy for myself.
|
| https://littlegreenviper.com/leaving-a-legacy/
| wvenable wrote:
| ...unless I haven't looked at it in a few years.
| tiffanyh wrote:
| The problem I have is, I'll revisit code I wrote 1-2 months
| prior and swear someone else wrote it.
| pavel_lishin wrote:
| Same.
|
| I _know_ I wrote perfect, maintainable code that can easily
| live in my brain.
|
| But when I go to work on it again a few months since I wrote
| it, it seems like some _asshole_ stepped in and overwrote my
| beautiful code with a bunch of jumbly mish-mash and had the
| gall to overwrite the git commit history with it, using the
| exact same commit messages, and my own name!
| lagrange77 wrote:
| Right. I even quietly mumble 'asshole' while reading it.
| takeda wrote:
| This is the weird thing. I heard it a lot and I don't know if
| people just over-exaggerate or maybe I'm just forgetting
| things quicker, but I don't have that. I come to the code
| after month or two and it is the same as if I came a week or
| two. I also don't feel ashamed of it.
|
| I also got compliments of my code in the past.
|
| I do try to make my code readable and avoid being clever.
| Basically my rule of thumb is that if I need to write a
| comment about block of code what it does, then that code is
| too complex.
| tazjin wrote:
| Same. I think it's a meme that people keep repeating, and
| for some fraction of them it's true.
| doctor_eval wrote:
| A month or two is nothing. Go back to it after 5 years.
| grotorea wrote:
| Past me is someone else.
| https://en.wikipedia.org/wiki/Ship_of_Theseus
| intelVISA wrote:
| We were all thinking it, thank you for having The Courage.
| ryancnelson wrote:
| Illumos, and specifically the SmartOS distribution.
|
| Being ringside for the fork of Sun's OpenSolaris, and watching
| absolute geniuses steward a thing like that into a useful product
| and a build process was foundational for my understanding of how
| and why process is important, as well as how to get things fixed
| in the real world, while not muddying the idea of the pure design
| principles. A balance can exist!
| jowdones wrote:
| >> Ask HN: What is the best code base you ever worked on?
|
| LOL. Obviously. Mine! :)
| lanstin wrote:
| AOL's server code was excellent. It was written by some people
| that had done similar things for BITNET and had a deep
| understanding of Unix programming and how to use event loops for
| scalability.
|
| It was in C, but the process was expected to run for months
| without crashing or running out of memory; if it had an abnormal
| exit, you'd (the owner as set in some code thing) get an email
| with the core backtrace. If it had a memory leak, ops would be on
| you quickly. Ops was considered a primary driver of requirements
| to dev, and absolutely everything could be reloaded into a
| running server without restarting it. There was a TCP control
| port where a TCL interpreter inside the service was exposed and
| generally you wrote the (very simple, CS 101 style) TCL commands
| to manage the server. It was a "No Threads Kernel", scaled to the
| dozens or hundreds of physical machines communicating over a very
| well managed network, and most 1 process per core, and 1 core for
| the OS. The 200 or so unix developers (as we were called) had a
| common understanding of how the framework worked and if you were
| just writing app code it was basically impossible to write slow
| services. We had technical writers that would interview the
| developers and write books that could be handed to outside
| developers and lead to a successful integration with no developer
| time spent.
|
| The NTK was primarily for sending msgs over the network - we had
| a principle to never write to disk (which were pretty slow in the
| 1990s), so everything was just a server to get network messages
| and then send out other messages in reponse and then assemble the
| replies/timeouts and send back a msg to the caller. All done over
| persistent connections established by the infrastructure, the
| applications just registered callbacks for msg type 'X' which
| would present one with the caller information, the msg as a
| buffer, and potentially a "user word" which would be a way to
| keep server state around between calls.
|
| The layering, from the main select loop thru different layers of
| TCP connection, SSL handling (if used, not 100% was SSL in the
| 90s), thru persistent link handling, application msg handling,
| timers, memory allocation, etc. was done with such art that I
| felt it to be a thing of beauty.
| mrpippy wrote:
| Fascinating, what kind of hardware/OS was used in production?
| lanstin wrote:
| Started with Tandem and Stratus pre Unix then migrated to
| HPUX then Solaris then Linux on stock hardware. Also Digital
| for big machines and early usage of 64 bit. I ported to Mac
| OSX but only for local dev.
|
| Funny thing about HPUX to Solaris migration is we found a ton
| of null pointer dereferences which didn't cause SEGV on HPUX
| but did on Solaris. And Linux migration was to a different
| endianness so had to fix any copying to/from ints to a
| buffer.
| tibbydudeza wrote:
| I worked at a bank where we used Tandem NonStop on Guardian
| OS - loved the programming environment using TAL language
| and the built-in SQL database.
| patrick451 wrote:
| Sadly, it seems like the moral of this account is that
| engineering quality is orthogonal to business success.
| jsrcout wrote:
| It's refreshing to read the replies here as I am currently
| working in the worst codebase I've ever seen in my life. It's
| just unimaginably bad.
| tonymet wrote:
| Facebook but for different reasons than most. We called this
| swapping out the airplane parts mid-flight
|
| A lot of effort goes into language design and tooling to enable
| continuous migration of code. Rather than re-writing entire
| repos, existing code is continuously upgraded through semi- &
| fully-automated code-mods. Every day thousands of patches are
| landed to upgrade APIs with new type safety, security
| restrictions, deprecations and other code maintenance.
|
| Most other company repos I worked on had major re-writes outside
| of the mainline until one day there was a sudden and often
| troublesome switch-over to the new code.
|
| Code is constantly changing, and unless you have a continuous
| process for landing and testing those changes, you are going to
| suffer when you try to apply the changes needed to address the
| accumulated tech-debt.
| lanstin wrote:
| I really wish more people believed in continuous improvement
| and the malleability of software. Giant heroic refactors or
| redos are so annoying, but so are ten year old compiler or
| kernels.
| ceejayoz wrote:
| > We called this swapping out the airplane parts mid-flight...
|
| I work fairly heavily with Facebook's Graph API, and I think
| they may have employed too many Boeing engineers for the part
| swaps lately.
| fragmede wrote:
| > We called this swapping out the airplane parts mid-flight...
|
| At ksplice we swapped the code out from under the kernel
| without rebooting, so I know what you mean.
| mrazomor wrote:
| In my case, it was the integration testing framework built for a
| large Python service.
|
| This was ~10y ago, so my memory might not serve me well. A bit of
| context:
|
| - proprietary service, written in Python, maaany KLOC,
|
| - hundreds of engineers worked on it,
|
| - before this framework, writing the integration tests was
| difficult -- you had a base framework, but the tests had no
| structure, everyone rolled out their own complicated way of
| wiring things -- very convoluted and flaky.
|
| The new integration tests framework was build by a recently
| joined senior engineer. TBF, it's wrong to say that it's was a
| framework, if you think in the xUnit sense. This guy built a set
| of business components that you could connect & combine in a
| sound way to build your integration test. Doesn't sound like
| much, but it significantly simplified writing integration tests
| (it still had rough edges, but it was 10x improvement). It's rare
| to see the chaos being tamed in such elegant way.
|
| What this guy did:
|
| - built on top of the existing integration tests framework
| (didn't rollout something from zero),
|
| - defined a _clear semantic_ for the test components,
|
| - built the initial set of the test components,
|
| - held a _strong ownership_ over the code -- through the code
| review he ensured that the new components follow semantics, and
| that each test component is covered by its own test (yep, tests
| for the test doubles, you don 't see that very often).
|
| Did it work well longterm? Unfortunately, no. He stayed
| relatively short (<2y). His framework deteriorated under the new
| ownership.
|
| Travis, if you are reading this and you recognized yourself,
| thank you for your work!
| MathMonkeyMan wrote:
| I wonder if this is a tragedy of "the org wasn't ready for his
| solution" or more of a "if I were to go back now, I'd notice it
| wasn't so good."
| bornfreddy wrote:
| Why only those two options? It could be just "the company had
| other priorities and developers, while they appreciated
| Travis's work, didn't find it worthwhile to carry the torch
| themselves". Having a great testing framework is just one of
| the things that need devs' attention.
| MathMonkeyMan wrote:
| Fair point, it's a false dichotomy; though your example is
| close to "the org wasn't ready for it," which is itself
| vague.
|
| Maybe the issue is that it easily comes apart without a
| dedicated censor, so to speak, and nobody wanted to have
| that role.
| cwbriscoe wrote:
| One that I built and maintained 100% on my own. Nothing like
| doing things the way you like to without someone else messing
| everything up.
| antishatter wrote:
| Had to work through a weirdness in kubernetes many years ago when
| using it on prem for a project (not at google). It stands out as
| the best code base I have had to go through to figure stuff out.
| cvwright wrote:
| libjpeg 6 is pretty good. It's oldschool C so it's pretty
| spartan, but it's easy enough to read it and follow what it's
| doing if you have a basic understanding of the concepts.
| (Discrete cosine transform, Huffman coding, etc)
|
| And it's only had a couple of vulnerabilities, in nearly 30 years
| of being on the Internet. That's not quite like DJB code, but
| it's darn close.
|
| My understanding is that it was written by a very small number of
| people who were experts in the field, as part of the standards
| process way back in the early 90s.
| brynet wrote:
| OpenBSD. Good people, consistent style, correctness.
| JensRantil wrote:
| The first few years at a previous startup I worked on. Java.
| CI/CD in place when merging PRs. But mostly, the thing that made
| the code base great was the fact that the code wrapped external
| libraries in small interfaces that were _owned by the company_,
| and dependency was used to not depend on implementation
| interfaces. In short, adhering to the basics on the SOLID
| principles.
| kristopolous wrote:
| NetBSD
| OnlyMortal wrote:
| Used: the Boost C++ libraries. In particular ASIO. Variant has
| also been very useful.
|
| The Mac GUSI BSD socket library is worth a mention too. I built a
| multithreaded Quark XPress XTension with it that used PowerPlant
| for UI. Quark even wanted to interview me.
|
| Added to: a pre-Boost cross platform library for Mac (Classic)
| and Windows NT. It did much of what Boost did way back when.
| elwell wrote:
| ClojureScript + Re-frame + GraphQL subscriptions over WebSockets:
| https://github.com/Vetd-Inc/vetd-app
| fooker wrote:
| The LLVM infrastructure.
|
| Other compilers don't even come close.
| sunir wrote:
| I worked on an incredible C++ codebase back in the dot.com. We
| built a mobile SVG renderer and packed into 128kb on a Palm 3 and
| similar code sizes on Blackberries, Symbian (the worst OS ever),
| Fujitsu phones, and whatever you threw at us.
|
| I built a few systems. The DOM, a build system that could cross
| compile to a completely new target phone while the phone
| executives were at lunch with our sales team (that's great
| demo!), and an optimizing build step in Perl that rewrote the C++
| to minimize bytes, cycles and joules of energy.
|
| The rendering engine was built by a genius. It was the most
| efficient and accurate vector graphics renderer I have ever seen.
|
| But of course no one could understand how this system worked.
|
| I mean, Perl that transpiles C++ into more optimized C++ just is
| abusive. I was not popular with the senior devs.
|
| The main coders including me were coop students who worked
| furiously at all hours for the fun of it.
|
| But of course the product had no revenue. The wiser devs probably
| knew this and didn't care.
|
| Nevertheless, we eventually were let go during the dot.com bust
| and the remaining main engineers had to throw the code out and
| try to rewrite it in Java. That didn't work because in that era
| Java was too slow to fit on a phone. Or maybe the market was
| dead.
|
| I learnt a solid lesson.
|
| What's amazing technically is not an amazing code base.
|
| What is an amazing code base isn't an amazing product.
|
| What is an amazing product isn't an amazing business.
|
| Literally every product I have ever seen that is profitable is
| not a good code base. But if it pays me to code, I love it.
| jeffrallen wrote:
| The Go standard library. The level of detail, the politeness of
| the reviews, the fact that the features of the language allowed
| me to put more attention into correct code than "at least it's
| working" code, the pragmatism and personalism of the core team.
| malkosta wrote:
| The best codebases I worked on were from startups that failed due
| to lack of product market fit. Oh man, their code was so
| beautiful and optimized...
| yobert wrote:
| I tried to add a new SQL function to CockroachDB. I had never
| seen the source before, but since it was such nice Go code, it
| only took me a few hours to have a working build, with tests.
| jchrisa wrote:
| CouchDB's HTTP API used to fit in just one file (less than 1k
| lines of Erlang), and it was a great jumping-off point for
| learning databases and web programming:
| https://github.com/apache/couchdb/blob/0.8.1/src/couchdb/cou...
|
| It was written by one engineer, and then later refactored by a
| team to look like this (and many other files):
| https://github.com/apache/couchdb/blob/main/src/chttpd/src/c...
|
| It's an interesting exercise to see how something grows and
| changes as it transitions from inspiration to real-world usage.
| CSMastermind wrote:
| I've seen enough code bases that I feel like I have a good
| spectrum from truly exceptional to horrific.
|
| The best code bases had these things in common:
|
| 1. _Consistency_ : they had clear patterns that were followed. It
| doesn't matter what the pattern was; some were strictly object-
| oriented, some were purely functional, some used RESTful API, and
| others leveraged gRPC or GraphQL. The important thing was that it
| was all consistent across the codebase so even if you were
| looking at a part you'd never seen before you could still orient
| yourself and reason about it quickly.
|
| 2. _Ownership_ : there was always a single individual person who
| was ultimately responsible for each section of code. That doesn't
| mean they wrote all the code themselves; they were the final
| arbiter on the rare occasion a truly contentious conflict arose.
| They also had the unilateral power to make changes within their
| section of the code if they felt it was needed. This was always a
| rarely exercised power but they could, if they had to, push a
| change through. There could be many such people spread out across
| the codebase but for each discrete part the number was always
| one.
|
| 3. _Clear Boundaries_ : it was clear what each part of the
| codebase's purpose was, and the boundaries were rigidly enforced.
| Things were never tightly coupled across these boundaries.
| Business logic was always isolated from things like
| serialization/deserialization, each system was forced to maintain
| its own models of the world that it cared about.
|
| The worst code bases had these things in common:
|
| 1. _Lack of Eng Representation_ : Product controlling development
| schedules and insisting everything is high priority and needs
| done right now! Project managers who always wonder, "yes but what
| if 9 women could make a baby in a month. Why don't we try adding
| more resources? Can we just try?" Business types who see software
| as a cost center not a profit driver. This can also happen if
| your engineer leadership didn't come up through software
| engineering but rather QA or IT or started out in academia or is
| just plain and MBA type with no eng background.
|
| 2. _" Time saving" Tech_: We don't need a database schema, we can
| just go NoSQL and have JSON blobs, it will save so much time! We
| can share models between the front end and back end, it will save
| so much time! Don't write SQL, use this ORM, it will save so much
| time! Don't think about DevOps, just use this all in one hosting
| solution, it will save so much time! What about this low code/no
| code solution? It will save so much time!
|
| 3. _Misaligned Incentives_ : This could be because they were
| contractors with no stake in the company or this could be because
| it was a large company and there was no realistic way they'd ever
| be fired. Either way there were no consequences for writing bad
| code.
___________________________________________________________________
(page generated 2024-06-28 23:00 UTC)