[HN Gopher] Ask HN: What is the best code base you ever worked on?
___________________________________________________________________
Ask HN: What is the best code base you ever worked on?
And what made it so good? Was there someone enforcing good
practices top down? Just being in a group of great engineers? Or
something else?
Author : pcatach
Score : 496 points
Date : 2024-06-28 08:40 UTC (1 days ago)
| cqqxo4zV46cp wrote:
| A blank slate, so I can make the first mistake :)
| escapecharacter wrote:
| I agree, the best code base is always a zero-line code base.
| alex_suzuki wrote:
| After that first line, it's all downhill from there... :-)
| azangru wrote:
| So true!
|
| I spend so much time obsessing over how what I am about to
| write ties in with what has already been written; or fuming
| over the stupidity of earlier decisions (usually made by
| myself). A blank slate is incredibly refreshing.
| mulholio wrote:
| Although I'm only on job three and have not had that much
| involvement with open source, I think my current employer (Attio)
| has one of the best codebases I've seen.
|
| Qualitatively, I experience this in a few ways: * Codebase
| quality improves over time, even as codebase and team size
| rapidly increase * Everything is easy to find. Sub-packages are
| well-organised. Files are easy to search for * Scaling is now
| essentially solved and engineers can put 90% of their time into
| feature-focused work instead of load concerns
|
| I think there are a few reasons for this:
|
| * We have standard patterns for our common use cases * Our hiring
| bar is high and everyone is expected to improve code quality over
| time * Critical engineering decisions have been consistently
| well-made. For example, we are very happy to have chosen our
| current DB architecture, avoided GraphQL and used Rust for some
| performance-critical areas * A TypeScript monorepo means code
| quality spreads across web/mobile/backend * Doing good migrations
| has become a core competency. Old systems get migrated out and
| replaced by better, newer ones * GCP makes infra easy * All the
| standard best practices: code review, appropriate unit testing,
| feature flagging, ...
|
| Of course, there are still some holes. We have one or two dark
| forest features that will eventually need refactoring/rebuilding;
| testing needs a little more work. But overall, I'm confident
| these things will get fixed and the trajectory is very good.
| kleton wrote:
| google3, all the devex tooling was taken care of by other teams.
| Tons of useful library functions available to import, accumulated
| over decades.
| chrisvenum wrote:
| My favourite projects are small, with very focused goals and
| features.
|
| I have a Laravel project that I have maintained for a customer
| for seven years. The app is straightforward and allows users to
| create portals that list files and metadata, such as expiration
| dates and tags.
|
| Every other year, they ask me to add a new batch of features or
| update the UI to reflect the business's branding. As the app is
| so small, I have the opportunity to review every part of the app
| and refactor or completely rewrite parts I am not happy with.
|
| It is a joy to work on and I always welcome new requests.
| gepardi wrote:
| This sounds like wonderful! Maybe one day I can have a few of
| these "babies" that I can lovingly raise, providing joy to both
| the dev and the customer.
| 2Pacalypse- wrote:
| Any codebase that I had complete control over.
|
| No, but more seriously, I've found that familiarity with the
| codebase is more important than having it be perfectly
| engineered. Once you're really familiar with the codebase, you
| know where dragons be, and you can make changes more easily. And
| God (PM) forbid, if you ever find yourself with some extra free
| time you might even reduce the size of dragons over time.
|
| This brings me to my final point. Any codebase that I really
| enjoyed working with was the one that was constantly evolving. I
| don't mean rewriting everything from scratch every few months,
| but as long as I have permission (and time) to refactor the
| things that have been bothering me for months as patterns emerge,
| I'm a happy bee.
| sam_lowry_ wrote:
| > Any codebase that I had complete control over.
|
| No excuses. Code ownership is important. Sometimes it works for
| a team, sometimes only for individuals.
|
| But not having to submit to core teams, architects and self-
| proclaimed experts of all kinds is a blessing.
|
| I now work for an organization that discourages code ownership,
| and it struggles on many fronts: 1. core
| teams are dysfunctional 2. people find niches and
| stick to them 3. top talent is leaving, although
| pay is good and business creates real value for citizens
| 4. there is virtually no horizontal communication
| 5. mediocre ones rise to the level of their incompetence and
| infest the lives of others 6. and so on and so
| forth...
|
| And I think the root cause of all this is lack of individual
| (code) ownership.
| Aeolun wrote:
| I've had exactly the same issues but because I couldn't
| change anything without getting approval from 16.5 code
| owners on every PR submitted. It's a real pain if you start
| modifying your coding for 'least code owners hit' instead of
| 'best architecture'.
| rsanek wrote:
| I like how my workplace does it -- there are rigorous
| codeowners and usually you only need approval from 1-2. if
| you do need approval from 5+, you can request a select
| 'super' codeowner review which will approve it for all.
| ownagefool wrote:
| I think core teams being helpful or harmful really comes down
| to the individuals.
|
| The problem is at this level, most orgs don't have anyone to
| really judge or contest competency, so they hire the salesmen
| rather than the doers and when they don't, they tend to cheap
| out and just get inexperienced people.
|
| Logically it makes a bunch of sense, though.
|
| Why rebuild yet another platform? Why is your central
| platform bad? Usually it's not self-service, sometimes it's
| because it's built in cumbersome ways, other times its
| because it actually enforces good standards on you rather
| than just giving app your apps admin.
|
| It's difficult for the person who hires the core team to
| differentiate between those complaints, unless they
| themselves both have the technical competency and the empathy
| to really understand the problem. They usually don't.
|
| Point being, done well, it's great, but most folks can't do
| it well.
| threatofrain wrote:
| Also nice is when an entire community has agreed to architect a
| codebase more or less the same. You're basically psychic when
| that happens.
| blitzar wrote:
| > Any codebase that I had complete control over.
|
| Anyone other than myself would instantly observe it as the the
| worst codebase they have ever seen.
| sam_lowry_ wrote:
| For that, there is a saying: "code as if your kids will
| maintain it".
|
| But I think it does not convey the right meaning. When I code
| something I will have to maintain for a long time, I try to
| make it as simple as possible for my future, older, less
| motivated and weary self.
|
| The worst codebases are written by people who landed the gig
| a few months before and do not expect to stay around longer
| than a year or two.
| null_glyph wrote:
| This is very much true. Initially when I joined the industry I
| used to work for a product from its inception. So I was aware
| of what section of the code affects what part of the product.
| Although I hadn't worked on all of those, I kept an eye for the
| all the changes that were coming in. I knew where I should look
| immediately a bug is reported even if it is not something
| related to my line of work.
|
| Recently I switched teams and now I find myself taking up bugs
| that are only related to my line of work. Not being familiar
| with the codebase decreases productivity and wants you to rely
| on other people in the team for most of the time.
| sgbeal wrote:
| > Once you're really familiar with the codebase, you know where
| dragons be, and you can make changes more easily.
|
| This is an interesting, and often overlooked, point. A month or
| two ago someone asked us, the Fossil SCM maintainers, if we'd
| be open to them refactoring the tree to something which better
| matches modern sensibilities (i'm paraphrasing here). Despite
| its many proverbial dragons, the long-time maintainers are
| comfortable with it and know where those dragons are and how to
| defeat them (or, in some cases, sneak around them), so, despite
| our collective geek tendencies to prefer "perfect code," we're
| happier with the dragons we know than those we don't. (That's
| not to say that fossil's code is awful, but its structure
| varies significantly from what one might write today if one was
| to start such a project from the ground up.)
| nevi-me wrote:
| I also echo the code ownership part.
|
| I have a part-time gig where I maintain accounting software for
| a former client of mine. It takes up a few months' weekends a
| year.
|
| I wrote about 60-70% of it when I was working for the owner of
| the software. It's something where as long as the client's
| happy, and they get new integrations and updates on time, they
| could keep using it for a decade longer.
|
| I had almost complete ownership of the architecting of the
| software. It's broken down into a few microservices (think
| database, core business logic, reporting, auth, logging etc).
| The best thing I did at the time was pushing to use gRPC even
| though management felt it was too new tech.
|
| The UI is in Angular, pain-free periodic upgrades. I've even
| rewritten some perf-sensitive code in Rust, and everyone's
| happy with snappier calculations.
|
| The code hygiene is relatively good.
|
| The only downside's that if someone else were to take over the
| code, they'd struggle (it's one of those things where I'm
| wearing many hats). I've been fortunate to be a professional
| accountant who moved into software engineering, so everything
| makes sense to me.
| KronisLV wrote:
| > Any codebase that I had complete control over.
|
| This is probably one of the pillars of good codebases, or at
| least decoupling the bits that you don't control as well as you
| can (this includes external services). I remember needing to
| write a wrapper around another JWT library, but because it was
| quite important, I aimed for >95% test coverage, some of the
| tests acted as documentation for how to use the code, there was
| also a nice README, there was CI configuration for pushing the
| build artifacts to Maven and suddenly even managing dependency
| updates became easy because of the test suite. Years later,
| it's been integrated in a bunch of services and "just works".
|
| Come to think of it, things always get less pleasant once you
| add a bunch of complex dependencies and libraries/frameworks.
| Need to make a few RESTful Web APIs in Java? Something like
| Dropwizard will probably give you fewer headaches than Spring
| (or Spring Boot) and all of its inherent complexity, in the
| case of the former you might even need to do configuration in
| XML and that has honestly never been pleasant. If you need to
| integrate with a bunch of other stuff and want something
| opinionated, going for Spring Boot will make sense then, but
| for simple use cases it's overkill. Same for ASP.NET, Ruby on
| Rails, Laravel and many others, while they might be easier to
| use, updates (especially across major versions) and breaking
| changes will give you a headache and just add a bunch of churn
| to the project.
|
| Similarly, if you need a message queue, externalizing that into
| RabbitMQ might make a lot of sense, same with storing files in
| an S3 compatible store, using something like Redis or Valkey
| for key-value storage, as well as not trying to shove too much
| logic into your RDBMS. Just pick whatever tool feels best for
| the given task at hand, instead of shaping things into what
| they're not (using the database for blob storage, for example),
| unless you have a whole bunch of constraints to contend with.
| Otherwise, sometimes you just get the worst of both worlds,
| like needing to use Oracle for a project, not having easily
| launchable local environments (because Oracle XE doesn't
| support some features), having to share DB instances with
| others during development and also running into weird obscure
| issues like DATABASE LINK queries taking 100 times longer in
| some cases, even when executing the same SELECT query on the
| remote DB works without issues.
|
| To not go into a rant, I'd sum it up like this: be in control,
| isolate the things that you cannot control, pick the correct
| technologies, do the simplest thing that you can get away with
| without overengineering and think about the person who'll have
| to maintain, debug and grow the system in the following months
| and years (which might also be yourself, sans knowledge about
| what the code did, if you don't make it explain itself or don't
| comment the non-trivial stuff).
| idrios wrote:
| > And God (PM) forbid, if you ever find yourself with some
| extra free time you might even reduce the size of dragons over
| time.
|
| Honest question, what is the company like where you can do
| that? Everywhere I've worked (only been working in industry for
| 6 years) has had such rigid agile development that even when I
| do find myself with free time, there's no flexibility to work
| on things that haven't been assigned to you and the best I can
| do is work on profiling/debugging tools.
| arp242 wrote:
| Usually at smaller companies with just a handful of
| developers. Can be a "start-up", or just a smaller business
| running fine with small teams. Or sometimes in a larger
| company where there's a small team doing its own thing.
|
| There's up-and-downsides to everything. The pay is usually
| less, and "fewer processes" often also means fewer processes
| surrounding HR and such. This can mean you're subject to the
| whims and mood swings of one person. I was once fired over a
| pretty far-fetched misunderstanding, but 1) they didn't
| mention it because "I should know myself", so I didn't really
| have a chance to clear it up, and 2) when I got fired they
| agreed it was a complete misunderstanding and that they made
| a huge mistake, but fired me anyway because that was the road
| already taken now. Great ... thanks ... It's hard to imagine
| that happening at a larger company with an actual HR
| department and stuff.
|
| Also: a Wikipedia-esque "be bold" attitude helps. Obviously
| refactoring things just for the craic isn't a good idea, but
| if there's something that can really make an impact then I'd
| just do it and send a PR, regardless of what I've been
| assigned. Obviously "refactor everything" on your own
| initiative isn't a good idea, but usually there's tons of
| smaller stuff that really helps: "introduce new
| function/package that helps reduce boiler plate", "split up
| this difficult to understand function", things like that.
|
| Most PMs will realize they don't have a 100% in-depth
| understanding of the code, and will trust you when you say it
| helps you do your work better. Usually it's fine as long as
| you apply proper judgement and don't spend too much time on
| it. That said, there are some places where this really isn't
| appreciated regardless. IMHO that's pretty dysfunctional and
| when I start looking for somewhere else to work.
| james_marks wrote:
| Counter-point: whitespace PR's, refactors I didn't ask for,
| aren't attached to a larger initiative, etc actively annoy
| me as an owner.
|
| Every PR takes time to manage, compounds complexity, and
| carries risk. We only get so many out a week, and I want
| them to count.
|
| Maybe it sometimes goes in hand with disfunction, but I'm
| guessing there's more going on there.
| arp242 wrote:
| > apply proper judgement
| lanstin wrote:
| Your successors just hope you can only remember 3 things at
| once so you end up with elegant designs rather than one of
| those monsters who can remember "to add a new wrinkle to this
| feature, just edit these 25 files."
| ysris wrote:
| Considering the biases, the one I wrote for the company I
| created.
|
| When we have the opportunity to be in this context, keeping in
| mind what bothered us in the codebases with which we were able to
| work in the past, we can force ourselves not to reproduce the
| same errors. Like the unmaintained unit and integration tests,
| the lack of refactoring, other developers that use fancy
| technologies instead of simpler concepts more for the opportunity
| to play with technologies than real need..
|
| And also, I guess, because we are more aware that the code is a
| reflection of the company that we want to have, that the simpler
| the better is a key point when we need to debug.
| anonzzzies wrote:
| Generally code that hasn't been tested commercially. Unencumbered
| by pesky client driven features, just code for dreamt up features
| that are fun to code but perhaps will never be used.
| jmartin2683 wrote:
| Hopefully the last one you were responsible for.
| jmartin2683 wrote:
| Hopefully the most recent one you were responsible for.
| bittermandel wrote:
| Any code base that doesn't use the advanced features of it's
| language(s) are always better in my experience. Heavy usage of
| e.g. meta-programming in Python or perhaps uber's fx (dependency
| injection) in Go makes projects infinitely harder to get into.
| jamestimmins wrote:
| Such a great point. I audibly groan when I come across Python
| meta-programming.
|
| While not an advanced feature, I have a similar response when I
| see lots of decorators in Python. They quickly become a
| debugging nightmare.
| sebstefan wrote:
| I can see a few cases where that depends...
|
| Really simple languages: Ruling out meta-programming is really
| going to limit you in Lua for example. Just being able to do
| `mySocket:close()` instead of `Socket.close(mySocket)` involves
| meta-programming.
|
| Older languages: For C++ the "simple" features are going to
| include raw pointers and macros. Maybe it's not so bad to allow
| smart pointers and templates to avoid those
| wruza wrote:
| Both of these examples are examples of an under-programmed
| core though. Lua is notorious for lacking batteries, so
| everyone has to reinvent their own. There's literally no
| serious Lua program without some sort of classes, but they
| still resist adding them into lauxlib.
| BiteCode_dev wrote:
| Depends who is providing it.
|
| Django and pydantic meta programming usually make the code
| easier to deal with.
|
| In shop written meta programming usually sucks.
| tommypalm wrote:
| I worked at GOV.UK for a few years on what was effectively
| specialised CMSs all written in Rails. Mostly basic CRUD stuff.
| A contractor came along and built the most insane CMS I've ever
| seen. I found out later it was flavor of the Java Repository
| Pattern using dependency injection. It became so difficult to
| work with that the last thing I worked on there was to delete
| it and rebuild it using standard Rails patterns.
|
| The KISS philosophy exists for a reason and that includes over-
| using advanced language features just to show off.
| nijave wrote:
| Besides just KISS, a lot of messes I've seen have been
| implementing patterns outside the framework or implementing
| complex patterns that didn't add value.
|
| Besides KISS (or maybe as an extensive), try to keep
| framework-based codebases as close to the official documented
| setup as possible. You automatically get to s of free, high-
| quality documentation available on the Internet.
| andrei_says_ wrote:
| Props for gov.uk! I've looked at its documentation and design
| system and see both as peak user experience and clarity.
| RangerScience wrote:
| I've had a consistent experience in Rails where I think for a
| day or two I've got a legitimate use case for one the whacky
| things you can do... and then as I work the problem more, it
| turns out: nope, the simple stuff is still the Right Way.
|
| Someday, Ruby shenanigans, someday...
|
| PS - Being able to pull the shenanigans _is_ super useful
| during the dev process, usually to skip some yak shaving
| during exploration, so it 's nice to have anyway.
| Mikhail_Edoshin wrote:
| Having just removed a metaclass from my Python code I totally
| agree.
| rvdginste wrote:
| I think one should not use advanced language features just
| because, but I also think one should not avoid using advanced
| language features where it is useful.
|
| Why would the code base be worse when advanced language
| features are used?
| lanstin wrote:
| Because unless you hire steadily more intelligent developers
| you will be headed towards a mass of code that is hard and
| scary to change.
| peelle wrote:
| I disagree with this, but I understand where it's coming from.
| I think you have a form of whiplash from things like: - Novices
| overusing the new shiny. - Java/C++/etc Jr programmers
| overusing design patterns. - Perl programmers solving
| everything with regexes. - Small startups with GraphQL, or any
| other large enterprise tool. - Metaprogramming, Maco's,
| Dependency injection, Recursion, etc when a simpler solution is
| a better fit.
|
| IMHO, a "best codebase" will be just a bit more advanced than I
| am, with good resources for me to grok it. I want to be able to
| learn from it. I also don't want to be so far out of my depth
| that I can't make a reasonable contribution.
| arp242 wrote:
| A pinch of salt can really liven up a dish. Not every dish
| needs it, but when used appropriately it's almost magic in
| how much difference it can make.
|
| A lot of salt always make everything disgusting.
| tracerbulletx wrote:
| Huh? Salt is one of the most important foundational
| elements of basically all cooking.
| pphysch wrote:
| It would be cool if newer (imperative) languages had a clear
| protocol to delimit the simple core of the language from the
| advanced, library-building features. Like Rust has `unsafe`
| blocks, imagine a `advanced` or `metaprogramming` block that is
| required in order to use advanced metaprogramming features.
| Then you could tell a junior to implement $MUNDANE_INTEGRATION
| without use of `advanced`, and that constraint could be
| statically verified in PRs, etc.
|
| It seems like it would vastly simplify language evolution. The
| core language can have rigid limits on changes and follow the
| Zen of Python or whatever, while the `advanced` language
| extensions can be a bit looser. Features could graduate from
| advanced to core without breaking anything. You get a single
| powerful+expressive language with a "porcelain interface"
| without necessarily giving juniors ammo to shoot themselves in
| the foot.
| teleforce wrote:
| I think D language fits your very descriptions.
|
| You have GC by default that's Python like and intuitive
| programming constructs. Since for the most part D compilation
| is much faster than C++ and Rust compilation as long as you
| stick to the core non-CTFE. Heck you can use D as a compiled
| scripting language with REPL using rdmd [1].
|
| Then if you want to go gung-ho, you can use the other advance
| features like D's excellent template meta programming, CTFE,
| bit banging in-line assembly, etc. D has so many advanced
| features that can be explored later on, even modern C++ is
| playing catch up with D in every releases. Nowadays D
| compiler also support C natively [2] and GDC compiler is
| included inside the venerable GCC eco-system.
|
| There are university that teach software development and
| engineering class with D due to its managed, on-demand and
| gradual complexity, not like "in your face" complexity of the
| big kahuna programming language like C++ and Rust [3]. Like
| other compiled languages you can build real-time system even
| firmware device drivers in D unlike Python.
|
| [1] https://dlang.org/areas-of-d-usage.html#academia
|
| [2] Adding ANSI C11 C compiler to D so it can import and
| compile C files directly:
|
| https://news.ycombinator.com/item?id=27102584
|
| [3] Teaching Software Engineering in DLang [pdf]:
|
| https://news.ycombinator.com/item?id=37620298
| burutthrow1234 wrote:
| I worked with someone who insisted on using fx for DI in go.
| It's so antithetical to the entire Go philosophy, I don't even
| think it's an "advanced feature". It's just bringing Java cruft
| to a language where it isn't necessary and making everything
| worse.
| ethegwo wrote:
| A Python version about core features of the Hindley-Milner type
| inferencer: https://github.com/ethe/typer
| donatj wrote:
| The latest Go micro-service I have built.
|
| About once a year roughly, for the last couple years, the
| opportunity has arisen to greenfield a Go micro-service with
| pretty loose deadlines.
|
| Each time I have come into it with more knowledge about what went
| well and what I wasn't particularly happy with the last time.
| Each one has been better than the last.
|
| I've been building software professionally for twenty years, and
| these micro-services have been one of the few projects in that
| time that have had clear unified vision and time to build with
| constant adjustments in the name of code quality.
| lanstin wrote:
| Sounds fun. I have a similar policy but only every other year.
| It means I know what the hell I am talking about when in
| meetings with people who have slow or inefficient services. It
| also means my job of mostly telling other people how to solve
| problems is still a lot of fun for me.
| intelVISA wrote:
| Go micro-service with pretty loose deadlines
|
| Sounds like A Money Printer, congrats.
| Traubenfuchs wrote:
| Every code base I have ever worked on was a legacy nightmare.
| Every "greenfield" project I joined turned into a legacy
| nightmare within weeks. I have never encountered enjoyable code.
| I had the displeasure of wading through Spring, Hibernate and
| Apache HTTP client code before and they were all an
| incomprehensible mess.
|
| My conclusion: You know the claim "any medication that really has
| an effect must also have side effects". I would like to adapt
| that for code: Any code that does a lot of useful and complex
| things must be an arcane, barely maintainable mess that can only
| be understood by deep study.
| alex_suzuki wrote:
| IMHO library code is especially challenging, as cruft has a
| tendency to accumulate, historical behavior needs to be
| preserved, and APIs are set in stone once they're built.
| lukan wrote:
| "Any code that does a lot of useful and complex things must be
| an arcane, barely maintainable mess that can only be understood
| by deep study."
|
| "Every code base I have ever worked on was a legacy nightmare.
| Every "greenfield" project I joined turned into a legacy
| nightmare within weeks"
|
| I am sorry to say this, but it really sounds like you were
| either really, really unlucky, or part of the reason why it
| became a mess.
|
| Complicated things are complicated. Nothing can ever change
| that and it requires study to understand it.
|
| But it still does matter a lot, how one organizes the whole
| thing. How it is structured, documented(!), refactored. Are
| there competent people in charge who understand it all and kick
| peoples asses if they make even a temporary mess or forget to
| document, or do random people make changes wherever they see
| fit, because a deadline is ticking?
|
| Modularisation is usually the key. Small modules do one thing
| and are as seperated as much as possible with as little side
| effects as possible.
|
| And if one has to ship things, it is not always possible to
| keep it pure and if the code is not intended to live forever
| then this is often fine. But if the codebase is supposed to
| stay, then there needs to be the time to clean up the hacks. Or
| don't and then you end up scared touching anything.
|
| That being said, the technologies you mentioned, I would not
| like to touch either..
| t43562 wrote:
| I think that's a bit cruel. Usually mess happens because lots
| of people with different ideas about the future needs and
| best structure meet up in the code - and it's hard to develop
| a consistent culture.
|
| Also codebases get too large for any one person to refactor
| them into shape in the time they have each day. So you end up
| needing people who are responsible just for keeping things in
| shape.
| lukan wrote:
| Well, that is why I said with those words it _sounds_ like
| this to me, not that he or she _was_ in fact responsible.
| (at least this is what I meant)
|
| "Also codebases get too large for any one person to
| refactor them into shape in the time they have each day."
|
| Which is why modules, or subprojects were invented. Or
| however you want to call it, if one person is only
| responsible for a small part and not everyone for
| everything. And yes, there is also the non trivial problem
| of time management.
| t43562 wrote:
| yup, but the style of the modules starts to diverge as
| different people maintain them.
|
| IMO one does want people whose fulltime job is to looking
| at a codebase orthogonally to those who are just
| implementing a feature. People who make sure it builds
| fast, is secure tries to be consistent to some degree
| etc.
|
| Many companies call teams/indivuduals like this a "cost
| centre" and disparage it because they are dolts.
| lukan wrote:
| If those code checkers are experienced programmers, who
| do not annoy people with arbitary guidelines and
| standards, then yes, this might also work. But I do not
| think they are always necessary as a seperate full time
| role, if everything works normal otherwise. And with
| working normal, I mean there is enough time to refactor
| and document and clean up and is actually done.
| RangerScience wrote:
| > Modularisation is usually the key. Small modules do one
| thing and are as seperated as much as possible with as little
| side effects as possible.
|
| 100% agree. The nuance I find is that a lot times, people
| want to draw the lines of the module along something that's
| more immediately intuitive, but results in longer-term
| complexity.
| lukan wrote:
| Yes, the hard part is to define what goes inside one
| module. There are rules of thumb, but lots of exeptions.
| Taking any advice too literally can also make things
| awkward.
| arp242 wrote:
| > Modularisation is usually the key. Small modules do one
| thing and are as seperated as much as possible with as little
| side effects as possible.
|
| I think the key aspect is a bit different: understandability.
| That is, if you open two pages of random code, then can you
| roughly understand and follow what it roughly does?
|
| Everything in a single package/module/directory can be quite
| understandable. It doesn't necessarily _need_ to be modular,
| but it often does help. "Understandability" is more broad
| though, and also includes things such as not having weird
| hard to understand functions nested 8 levels deep, and things
| like that.
|
| It also means not over-engineering things. "Our code is hard
| to understand, therefore, we need more architecture" is a
| fallacy I've seen a few times. The code wasn't hard to
| understand due to lack of modularity or architecture, but
| because it was just badly written unclear code.
| lukan wrote:
| "It also means not over-engineering things. "Our code is
| hard to understand, therefore, we need more architecture"
| is a fallacy I've seen a few times. The code wasn't hard to
| understand due to lack of modularity or architecture, but
| because it was just badly written unclear code."
|
| For sure, if I have 10 tightly separated micro modules,
| just for the sake of it, then this is not helping clarity.
|
| But if everything is an entangled mess of spaghetti code,
| then modularisation helps, so I do not have to look into 10
| places to get what a certain function is doing.
| mrweasel wrote:
| Something in your answer triggered a flash back to when I
| worked for a phone company, 20 years ago. One team, under the
| leadership of our questionable chief architect, had produced
| our new, all encompassing backend system. This was to be the
| corner stone of all future development and integration, dog
| slow and complicated as it where. I worked as a .Net developer
| and had the misfortune to be among the first to integrate with
| this monster. Try as I might, I could not get .Net to
| interoperate with these services. Finally I figured out that
| the SOAP services was using some old deprecated versions. Going
| back to the architect, I ask "did you build this on Apache
| Axis, and please say Axis2", but no, it was just Axis, a
| deprecated version, that would generate webservices not
| supported by newer .Net version. That wasn't an problem,
| because: "None of our project have upgraded to those .Net
| versions yet"... DUDE, we've launched a brand new system based
| on that .Net version a year ago, and what was you f-ing plan
| for the future, to redo every single service using Axis2?
|
| This guy had based a brand new system on a framework/library
| that was no longer maintained, even before the system was
| launched.
| noelwelsh wrote:
| My open source projects. One in particular I've been working on
| for about 10 years. The code is consistent and always getting
| better, even though there is a lot more work that could be done.
| structural wrote:
| Worked on a codebase for a large safety-critical system where
| everything was 100% documented, and the development guide for the
| project was followed so closely that you couldn't tell, across
| millions of lines of code, that the whole thing wasn't written by
| one person. Absolutely impressive levels of attention to detail
| everywhere, down to not even being able to find typographical
| errors in comments or documentation (a typo in a comment was
| treated just as seriously as any other bug).
| n_ary wrote:
| Let me guess, it was very well funded and there were no fake
| deadlines and cross-team dependencies, am I correct or am I
| very correct?
| contingencies wrote:
| Cancel the rocket launch people, we have a semantic
| incongruity! Correct is a binary state.
| kridsdale3 wrote:
| Unless you're working with Quantum Correctness.
| nurettin wrote:
| If you allow for composable statements, the whole can be
| half correct.
| Hasu wrote:
| "When people thought the Earth was flat, they were wrong.
| When people thought the Earth was spherical, they were
| wrong. But if you think that thinking the Earth is
| spherical is just as wrong as thinking the Earth is flat,
| then your view is wronger than both of them put together."
| f1shy wrote:
| I bet you are very correct
| unboxedtype wrote:
| I am almost sure this is because the system would have to pass
| a certification procedure somewhere, and for that they would
| need this level of clarity. Am I right?
| jamesmunns wrote:
| Having worked in a couple of safety critical companies, there
| are things you definitely HAVE to do, but some companies do
| it better than others.
|
| Some companies have process and development practices down
| pat: things go smoothly, and meeting the qualification
| process objectives is easy, because the work has been done
| right the whole way.
|
| Other companies have less established or less consistent
| process. They generally meet the process objectives and
| deliver a working product, but the development process is
| often more of a struggle, and there is often a lot of
| "cleaning up" of missed pieces at the end of the release
| process.
|
| This is just to say: companies and products in the safety
| critical space don't necessarily have some intrinsic quality,
| just a higher minimum bar.
| f1shy wrote:
| In my experience there is exactly 0 correlation between
| certification needed and code Quality. Right now working in a
| multi billion company doing SW that must pass many
| certifications. The code is absolut trash. The tooling is
| terrible. The people confuse make and cmake. All said. And
| the SW gets certified, because it is all a matter of how much
| it costs to certify. It is a kind of high level corruption,
| that is not seen as corruption.
| unboxedtype wrote:
| Sorry, I was talking about safety critical system
| certifications, like for avionics. The systems you describe
| would never pass that for sure.
| oasisaimlessly wrote:
| > The systems you describe would never pass that for
| sure.
|
| I wouldn't assume that.
| rr808 wrote:
| I'd love to know if working like this is enjoyable or a chore.
| simoncion wrote:
| After way too many years of working on several sprawling Ruby
| and Rails projects, I would enjoy the everloving fuck out of
| working on this.
| jamesmunns wrote:
| Having worked in both kinds personally, it's _different_.
|
| You spend more time on less topics: the variety of work is
| much lower, which some people love, and some people are
| frustrated by. There is often much less "drastic innovation",
| or when there is, it takes much longer than you might be used
| to in other industries. Favor is given to proven, predictable
| technologies and choices, even if that leaves some
| opportunities on the table.
|
| That being said, it's something I often miss. The ability to
| have such solid confidence on the things you've built, and
| the ability to drop into nearly any piece of the system and
| have everything be consistent and predictable is a quality in
| itself. It makes debugging (often VERY rare) issues much more
| tractable, both at a higher systems level, as well as digging
| deeper into the code itself.
| noisy_boy wrote:
| Sounds like a well maintained tractor from 1950s that is
| still chugging along.
| Aeolun wrote:
| > Favor is given to proven, predictable technologies and
| choices
|
| I feel like this goes two ways too. Sometimes people favor
| technologies they've used before, regardless of how many
| problems they know it causes.
|
| If it's predictable that certain tech will cause you issues
| years down the line, do not choose it again.
| jamesmunns wrote:
| Yep. Sometimes it's "known issues are avoidable issues",
| and other times it's "hey I wish we didn't spend xx% of
| our time avoiding (incl. spending time auditing syntax)
| or doing mitigations for the same limitations/issues over
| and over and over again".
|
| The wheels do turn, just slowly.
| JacobThreeThree wrote:
| If lives are at stake, it would probably elicit different
| emotions altogether.
| pavlov wrote:
| For me, the most eye-opening codebase of my career was Cocotron,
| around 2007:
|
| https://github.com/cjwl/cocotron
|
| I was looking for a way to port my native Mac Cocoa apps to
| Windows. I had been already disappointed by the aimless sprawl of
| GNUstep.
|
| This one-person project implemented all the essential APIs for
| both Foundation and AppKit. Reading the code was a revelation:
| can it really be this simple and at the same time this
| effortlessly modular for cross-platform support?
|
| I contributed a few missing classes, and successfully used
| Cocotron for some complex custom GUI apps that needed the dual-
| platform support.
|
| Cocotron showed me that one person with a vision can build
| something that will rival or even outgun large teams at the big
| tech companies. But focus is essential. Architecture astronauts
| usually never get down from their high orbits to ship something.
| hannofcart wrote:
| +1 to this.
|
| I remember my boss asking me to build a MacOS UI we had on
| Windows with this. I was very sceptical that it would go
| anywhere.
|
| But no, it worked. I was just open mouth shocked to see the GUI
| worked pretty flawlessly in Windows.
| cjwlloyd wrote:
| :)
| cjwlloyd wrote:
| Thank you!
| tlarkworthy wrote:
| Google3 codebase. It's just so vast and it works. It's a miracle.
| I feel lucky to have seen it. Everytime you change it, it reruns
| the dependencies. Everyone has different view concurrently.
| Commits are efficient immutable snapshots. It's just incredible
| multiplayer. So massively beyond what can be done with GitHub.
| Really I feel it's peak codebase. I've not seen what the other
| big techs do but it blew my mind
| asymmetric wrote:
| What is Google3?
| kqr wrote:
| I would assume Google's monorepo.
| Daub wrote:
| This link might help...
| https://dl.acm.org/doi/10.1145/2854146
|
| The linked pdf has lots of details.
| erikpukinskis wrote:
| HTML version in case anyone prefers that to PDF:
| https://cacm.acm.org/research/why-google-stores-billions-
| of-...
| bushbaba wrote:
| The replacement of google2
| arcade79 wrote:
| It was wonderful to work with. :-) It's one of the things I
| truly miss from Google.
| VirusNewbie wrote:
| Google3 codebase very consistently has _clean_ code, but some
| of the architecture there is very much not great.
|
| Some is great, some not so much.
|
| Some of Verizon's code was _much_ more elegant (though much
| smaller scope) from an API perspective, and really leaned into
| advanced type systems in a way Google has not.
| dilippkumar wrote:
| +1
|
| I can not understate how much I agree with parent comment.
|
| The opposite of move fast, build a shitty prototype and iterate
| is a deliberate problem solving approach undertaken by the
| highest caliber of engineers. The actual challenges to be
| addressed are effectively addressed right at the design stage.
|
| The result is a thing of immense beauty and elegance.
|
| I will forever be grateful for the opportunity I had to see
| this magnificent piece of engineering in action.
| robertsdionne wrote:
| Cannot overstate.
| musiciangames wrote:
| (Aside). To expand slightly, what robertsdionne is
| highlighting is the changing usage of this expression. In
| its original sense, e.g an issue is so important that it is
| impossible to overstate its importance. It is now
| increasingly used the other way around.
|
| Old me would have said it's used wrongly, but this happens
| all the time with language. Especially things being used in
| the opposite of their original sense, e.g. inflammable for
| flammable.
| smcameron wrote:
| In my mind, "cannot overstate" always meant "impossible
| to overstate", but I think some people interpret/intend
| "cannot understate" to mean something like "must not
| understate". I don't know if that's really what they're
| thinking, but it is how I make sense of it. I have come
| to just avoid such constructions.
|
| Edit: reminds me of an ancient SNL skit with Ed Asner in
| which he's a retiring nuclear engineer and as he heads
| out the door he says to his incompetent co-workers "Just
| remember, you can't put too much water in a nuclear
| reactor".
| deanishe wrote:
| > opposite of their original sense, e.g. inflammable for
| flammable
|
| Inflammable was _never_ the opposite of flammable. Those
| word have always been synonyms. The opposite was always
| non-flammable.
| semitones wrote:
| I, like many others here, echo this sentiment. While I
| disliked working in ads, the experience of working with that
| repo+tooling is unmatched by anything else in my career.
| bufferoverflow wrote:
| If the tooling is so far beyond anything publicly
| available, why don't you guys make something like that and
| make millions?
| thimabi wrote:
| Perhaps they make more than that just by using the
| tooling internally and creating/maintaining other stuff
| with it -- that's a major competitive advantage
| vundercind wrote:
| I strongly suspect it's not that useful for a lot of
| businesses.
|
| So many have their code split between a two dozen clouds,
| BA tools (... does Google put that in the monorepo too?
| Or is that, which is a _lot_ of the code at most
| businesses, not " _really_ " code at Google?), vendor
| platforms, low-code tools you all hate and are trying to
| deprecate but god damned if you aren't still spending dev
| hours on new features in it, et c...
|
| I bet achieving anywhere near the full benefits would
| first require retooling one's entire business, including
| processes, and bringing a whole shitload of farmed-out
| SaaS stuff in-house at enormous expense, most places.
| znkr wrote:
| It's not just tooling; processes, infrastructure and
| dedicated teams for centralized infrastructure are what
| makes Google's monorepo what it is. FWIW, most of the
| tools are publicly available or have good publicly
| available counterparts. What's likely missing elsewhere
| is funding for infrastructure teams.
| wmanley wrote:
| I'd love to see something like that applied to a project
| like Debian. The tools already exist. The cost of
| switching is too great, and you'd need everyone to learn
| your new system before you'd see the benefit.
|
| I wrote a bit about this here:
| https://blog.williammanley.net/2020/05/25/unlock-
| software-fr...
|
| I understand that nix have made some progress in this
| direction, but I don't know any more than that.
| lesuorac wrote:
| millions sounds like a lot but it's like just 3
| employees.
|
| To quote broccoli man, "i've forgotten how to count that
| low" [1].
|
| [1]: https://www.youtube.com/watch?v=3t6L-FlfeaI
| kridsdale3 wrote:
| I dunno man, I work at google and this comment feels like
| circlejerk. It's not held to THAT high a standard. We
| absolutely commit janky MVPs and iterate.
| elevatedastalt wrote:
| He is referring to the codebase itself, not individual
| projects or CLs.
| simoncion wrote:
| As a Google outsider, that's not at all the impression I
| got from the paragraphs.
|
| This:
|
| > It's not held to THAT high a standard. We absolutely
| commit janky MVPs and iterate.
|
| seems to very directly address this:
|
| > The opposite of move fast, build a shitty prototype and
| iterate is a deliberate problem solving approach
| undertaken by the highest caliber of engineers. The
| actual challenges to be addressed are effectively
| addressed right at the design stage.
|
| If your claim is that "Well, if you look at the codebase
| AS A WHOLE, there's absolutely no iteration and shitty
| prototypes, it's all designed right from the start."...
| well, I can't see any way that codebase came into
| existence without being built up by individual projects
| and changesets.
|
| So, yanno, when folks on the ground report that
| individual projects and changesets/PRs/whatever ARE using
| the "Commit something barely serviceable, test it out,
| and iterate." process, statements like "We always get the
| design right before we write even a single line of code!"
| definitely come off as a circlejerk.
|
| Google's a very large software house, it has a very
| exclusive hiring process, and it (reportedly) has
| internal tooling that's very well-adapted for the
| problems a company of its size faces. But Google is still
| hiring from the same pool of programmers as everyone
| else, and the odds that zero of those that it hires will
| work best with the "Get something out there to field-
| test, and use the test results to make it more suited for
| field use." method of development are absolutely zero.
| Given Google's size, the odds that zero of _those_ will
| never be able to negotiate to work in the way they work
| best are ALSO zero.
|
| (And -frankly- I expect that this method of development
| gets used a lot in the company. You can burn an assload
| of time on simulators (and what is the "What if?" game,
| but a in-brain simulator?), but in the realm of software,
| it's not-infrequently the case that the simulator with
| the best ROI is real-world deployment.)
| remram wrote:
| He is referring to the codebase not the code?
|
| What does that mean?
| kichimi wrote:
| I think skill is also an issue here, in both directions. I
| have worked with a company that followed the opposite of move
| fast and it just turned into a 'who is the most correct' and
| 'what is the most elegant code' competition. We didn't push
| out a single new feature in the three years i'd worked there
| at that point. There was so much focus on code that we gave
| almost no time to business requirements.
|
| The wider implication of this was that the number of tickets
| we got dropped dramatically, because users knew they'd never
| be resolved anyway.
|
| Balance is key.
| inoffensivename wrote:
| I honestly can't tell if this comment is sarcastic...
|
| google3 is a quagmire and it's getting worse by the day
| ignoramous wrote:
| Well, first impressions: It is part of our
| human condition to long for hard lines and clear concepts.
| When we have them we have to either face the fact that some
| realities elude them, or else bind ourselves to the
| inadequacy of the concepts.
|
| https://fs.blog/impressions-are-schematically-determined /
| https://archive.is/F8uPE
| thwd wrote:
| Agreed. Once long (10+ years) ago, google3 may have been peak
| codebase. Nowadays, not so much.
|
| Edit: I guess it also depends at what level of abstraction
| you work. High: can be easy breezy. Low: oh boy.
| phreeza wrote:
| I agree but would add that within the codebase there are also
| vastly different experiences. The search ads serving stack for
| example is phenomenal, it is >10 years old, very complex and
| gets contributions from hundreds of engineers every week, yet
| it is quite easy to understand what is going on anywhere and it
| has great performance. But I also had to work with other code
| which had been through 3 rewrites, none of which had ever been
| totally completed, so there were overlapping paradigms
| everywhere and working with it was just a nightmare. Tests
| never caught anything leading to several post mortems.
| lokar wrote:
| It was certainly great up to like 2015, perhaps later?
|
| You could spend all day finding fun code to read.
|
| I always like the main signal (eg sigsev) handler.
| cpojer wrote:
| https://github.com/nkzw-tech/athena-crisis
| joeshachaf wrote:
| I worked with a very experienced engineer who created his own WAF
| (software-based) using Java. From a purely architectural
| perspective, it might not be the best (I might have used an ISAPI
| filter back then), but the code itself was very efficient, well-
| written and documented. I used it several times as a teaching
| example.
| Agingcoder wrote:
| I haven't contributed code to it, but I've read lots of code in
| it ( for work reasons ) - I really like the golang compiler and
| library codebase.
|
| About codebases I've written code for, the best one strived for
| simplicity, and was driven by very strong engineers who actively
| considered code hygiene ( in the broadest possible sense ) a
| first class citizen.
| christophilus wrote:
| Go, SQLite, Redis are the three that came to mind for me.
| Simple, readable, fairly flat hierarchy.
| prinny_ wrote:
| The one in my previous job, which was an admin board for a market
| intelligence application. Ultimately, the reason it was good was
| because the engineers had zero ego on top of having excellent
| skills. The team that set the codebase were basically 4 seniors
| and 3 principals (the client actually did pay for top talent in
| this case) so not only everything was based on industry
| standards, written elegantly and organized perfectly, but every
| time some new requirement came up, these senior / principal
| engineers would discuss it in the most civilized matter I have
| ever seen.
|
| E.g, "we need to come up with a way to implement X". Person A
| gives their idea, person B gives another idea and so on until
| everybody shared their thoughts. Then someone would say "I think
| what person C said makes the most sense" and everybody would
| agree and that was it. 30 minutes to hear everybody out, 3
| minutes to discuss who will do it and when and the meeting was
| over.
|
| I think the biggest testament to this code base was that when
| junior members joined the team, they were able to follow the
| existing code for adding new features. It was that easy to
| navigate and understand the big picture of.
| brailsafe wrote:
| This is the approach I tried to take as an IC frontend on a
| building management system dashboard about a year ago. It was
| basically me and one other frontend, one full-stack, and a QA
| or 2, plus a manager. My manager I guess had written a lot of
| the code we'd be interacting with, and I guess was somewhat
| protective and reluctant to delegate authority over decisions
| around it. It was a big refactoring project, and I just
| encouraged my colleagues to take the things they wanted, like
| rebuilding the state management system. We'd discuss the
| requirements and why it was necessary, and I'd look for reasons
| to agree rather than disagree, then we'd be off. Something
| burnout has taught me is that the marginal differences between
| one implementation detail or another are not worth getting hung
| up on unless they pose a real problem, especially when someone
| else gets to decide that doing it fast is priority (sprints).
| robertlagrant wrote:
| > Something burnout has taught me is that the marginal
| differences between one implementation detail or another are
| not worth getting hung up on unless they pose a real problem,
| especially when someone else gets to decide that doing it
| fast is priority (sprints).
|
| I think this is true. And a self-imposed problem (should a
| problem arise) is much less frustrating to fix than one that
| came from a decision imposed by someone else, even if the
| latter avoided loads of other problems. Sometimes it's better
| to let people make mistakes (as you believe them to be) and
| correct them later.
| konschubert wrote:
| And often, the cost of indecision is much bigger than the
| margin in utility between the optimal and the second-to-
| optimal solution.
| psychoslave wrote:
| Great example as it shows that code base really is a side
| consideration and a proxy to team quality.
| cainxinth wrote:
| Another reminder that soft skills multiply the effectiveness of
| hard skills.
| alickz wrote:
| Exactly.
|
| What is development if not communication? Between the
| clients, the coworkers, the users, and the computer
|
| I'd take a mediocre dev with no ego and ok social skills on
| my project any day over an egotistical genius who can't work
| with other people or effectively communicate their ideas
| zamalek wrote:
| Otherwise known as "you are not your code" and "you are not
| your ideas." Essentially, criticism against your code is not a
| personal attack (and other corollaries). Its definitely
| something I aspire to, but its surprisingly hard to get right.
| BiteCode_dev wrote:
| Last year I worked for a client that gave me a lot of time, money
| and autonomy to lead dev on a critical software rewrite.
|
| We got a small team of competent people, with domain experts to
| peer code with the devs.
|
| It was wonderful. We could test, document and clean up. Having
| people who knew the trade and users at hand removed second
| guessing.
|
| The result was so good we found bugs even in competitors'
| implementations.
|
| We also got x5 in perfs compared to the system it was replacing
| and more features.
| leetrout wrote:
| Good for you. Magical moments in careers are hard to find in my
| experience but they are so satisfying when you get there.
|
| Glad whomever was over this didnt just drop the "dont rewrite"
| joel spolsky article and fight making it happen.
| BiteCode_dev wrote:
| I actually was the one telling them not to rewrite, lol.
|
| But the original code was a mess of matlab spaghetti, they
| couldn't find a way to hire for that. Not to mention turning
| it into a web service was already a big hack of java parsing
| a raw dump of matlab datastructures that nobody dared to
| touch.
|
| I had to read the matlab code, and it tooks hours to decypher
| a few lines. Plus the language doesn't have great debugging
| and error handling capabilities and the tooling is quite
| terrible.
|
| So rewriting to python won, and for once, I must say it was a
| good call.
| junon wrote:
| Joel has since said that that he doesn't really agree with
| that advice anymore, at least not in the same way. Super
| annoying that it gets parroted over and over again as though
| it's the word of the lord.
| kevin_nisbet wrote:
| I agree it shouldn't be parroted as though it's the word of
| the lord, like any advice it will be more or less
| applicable on specific situations. I've been on both sides
| of the let's redesign a thing plenty of times.
|
| The balance is somewhere in the middle, it's valuable
| advice for what can go wrong when you don't think through
| the implications of the decision making process, understand
| how and why the system works the way it does, and what
| risks exist for what can go wrong with a big redesign. But
| like anything, if the risks are understood, then those
| risks can be accepted, mitigated, or rejected if
| appropriate, or provide guidance on why the redesign
| investment isn't worth the associated risks.
| thecupisblue wrote:
| Similar thing.
|
| Had time and autonomy from a client, so took sweet time
| examining the domain, the existing systems et al. Spent a few
| months writing the basis and the framework around what will be
| done, based on years and years of experience I had with bad
| frameworks and codebases, combined with working on the same
| domain for their parent company years ago.
|
| And it worked. We delivered features insanely fast, hundreds of
| forms were migrated, feature generators would create 90% of the
| boilerplate code and the code was small, readable and neatly
| clustered. Maintaining it was a piece of cake, leading to us
| not having enough work after a while so I negotiated our time
| to half a week for the same money.
|
| After a while, client deemed us too expensive to pay for only
| 2.5 days of work - after all, how does it make sense - if we
| are paying them that much, they should work 5 days!
|
| So they cut us out. Two things happened:
|
| 1. Devs that got moved to other projects in the company told me
| they didn't know development could be so smooth and tried to
| replicate it in future projects, even tho they say it failed a
| lot of lessons they picked up from the framework were highly
| relevant in their future careers.
|
| 2. The company found a cheaper developer, he said "this is
| unusable and has to be rewritten" and rewrote it into "clean
| code", taking longer than the original project took. At least
| he works 5 days a week now.
| Aeolun wrote:
| We were going to do this. Get several months to set up a
| clean base for our new system with only the four most
| competent people working on it.
|
| Then I went on vacation for a week.
|
| Come back to a system that needs to be delivered to prod next
| month with 20 randos submitting PR's...
|
| WTF happened. Did you learn _nothing_ from previous failures?
| bboygravity wrote:
| What was the lesson they where supposed to have learned?
| Aeolun wrote:
| 9 women cannot have a baby in a month.
|
| There's a limited amount of people that can work on a
| system at the start of a project if you want it to be a
| coherent whole, so that you can have everyone iterate on
| it in the same style later. If you start out with 20
| (junior) engineers, you get a kind of frankenstein where
| all issues are waved away because 'we need to deliver in
| a week'
| simfoo wrote:
| My past three employers code bases: mono-repos, Bazel, lots ot
| C++ and Python, thousands of libraries and tools, code generation
| and modeling tools that are fully integrated into the build, easy
| cross compilation, large integration tests just one bazel test
| invocation away, hermetic and uniform dependencies...
| fire_lake wrote:
| Bazel Python does not cross compile IIRC. How was this
| achieved?
| simfoo wrote:
| Python is only used for build-time tooling (modeling, code
| generators) and developer tooling. All the on-target code is
| C and C++, and only that is cross compiled (Linux, QNX,
| various RTOS and x86+aarch64)
| Claudus wrote:
| Laravel, once you get the hang of it everything just works, and
| using a debug bar to optimize database calls is very satisfying.
| neilv wrote:
| One of them stands out, due to being super-productive, over
| years, and then decades.
|
| A large system that was originally written by only two super-
| productive engineers (I mean real engineers, both with PhDs in an
| area of Engineering). And a comparably capable and essential IT
| person.
|
| The reasons for the super-productivity include one of the
| developers choosing great technology and using it really well, to
| build a foundation with "force multiplier" effects, and the other
| developer able to build out bulk with that, while understanding
| the application domain.
|
| Another reason was understanding and being pretty fully in
| control of the code base, so that, as needs grew and changed,
| over years, someone could figure out how to do whatever was
| needed.
|
| One of the costs was that most things had to be built from
| scratch. Over time that also proved to be an advantage, because
| whenever they needed (put loosely) a "framework" to something it
| couldn't do, they effectively owned the framework, and could make
| dramatic changes.
|
| When I said "costs", I mean things like, many times they needed
| to make a component from scratch that would be an off-the-shelf
| component in some other ecosystem. So if someone looked closely
| at how time was sometimes spent, without really understanding it
| or knowing how that panned out, it would look like a cost that
| they could optimize away. But if they looked at the bigger
| picture, they'd see a few people consistently, again and again,
| accomplishing what you'd think would take a lot more people to
| do.
|
| It helped that the first programmer also became the director for
| that area of business, and made sure that smart engineering kept
| happening.
|
| Someone might look for a reason this couldn't work, and think of
| bus factor. What I think helped there was the fact that the work
| involved one of those niche languages that attract way more super
| programmers than there are jobs. "Gosh, if only we had access to
| a secret hiring pool of super programmers who were capable of
| figuring out how to take up where the other person left off, and
| we had a way to get them to talk with us...")
|
| It was easy to imagine a competitor with 100 developers, not able
| to keep up, and at many points getting stuck with a problem that
| none of them were able to solve.
| klibertp wrote:
| I assume you avoided identifying (or even hinting at) this
| "great technology" on purpose, but could you persuaded to
| divulge what it was?
| neilv wrote:
| It started as a bespoke structured data and Web architecture,
| which did somewhat complex things that would've been a huge
| headache to build and maintain in the dominant languages of
| the time, but were viable for one person to figure out and
| implement in Scheme and PostgreSQL.
|
| That bespoke architecture and implementation language lent
| itself to a lot of rapid domain-specific functionality, as
| well as doable architecture changes over time.
| klibertp wrote:
| Thanks! :) I hoped it'd be a Lisp or Smalltalk - great to
| hear I almost guessed right, though I never imagined an
| RDBMS to be in the mix :)
| neilv wrote:
| There's actually a bespoke object metamodel mapping atop
| the RDBMS, and it permits customers who want to use SQL
| direct to the database to do so.
|
| And, IMHO, PostgreSQL is the default choice for most
| storage backend purposes, unless one has figured out a
| good reason something else is better. :)
|
| There were also multiple kinds of large blob storage
| needs, and changing architecture needs over time (e.g.,
| multi-tenant, changing authn implications of things like
| browser plugins, scalability, move to cloud), so systems
| programming skills and controlling your framework comes
| in handy, for not getting stuck or blocking on vendors,
| but just solving the problem.
| bruce343434 wrote:
| What is this niche language attracting super programmers?
| neilv wrote:
| Scheme, Common Lisp, Haskell, [edit] Smalltalk, probably more
| obscure ones...
|
| Also, Erlang, Rust, and Clojure, though those have been
| rumored to be employable, so no longer get as much of the
| filter as you get when it was just people caring strongly
| enough to want to use a particular language _despite_ the
| unemployability.
|
| Whether that caring happened because of skill (to identify
| something good), or skill happened because of caring (to
| invest the time in exploration and thinking and practice),
| these communities seem to get more than their share of great
| programmers.
|
| And so you know where to look for them, and you have a reason
| that they might want to talk to you.
| bear8642 wrote:
| >probably more obscure ones.
|
| Apl/BQN worth adding for the recent CUDA/vector language
| explosion, and has it's fair share of extraordinary
| programmers!
|
| Forth an interesting one too look, although never properly
| used.
|
| miniKanren/Prolog another vein to branch down
| (unification/logic programming)
| nonameiguess wrote:
| No name because the project code name changes with each new
| contract, but the first-level ground processing system for the
| NRO that turns raw satellite downlinks into some kind of human-
| intelligible files.
|
| I think a lot was just the restrictions built into the way
| development happens required a high level of discipline, care,
| and planning. Also, requirements were pretty tightly coupled to
| sensor platform capabilities, which are known well in advance and
| don't change unexpectedly, so waterfall development actually
| works and you don't have to deal with the chaos of not really
| knowing what will and won't work and customers constantly
| changing their minds.
|
| Code base was overwhelmingly C++, some Fortran, a lot of it was
| very old. It was all developed on airgapped networks, and the
| difficulty of bringing in external dependencies meant there
| largely were not any. All of the library functionality we
| required was mostly developed in-house, so we had extremely well-
| documented and stable functions available to do damn near
| anything you could want, with a good chance that whoever first
| wrote it was still there. All development had always been tied to
| a ticketing system of some sort that included all of the original
| discussion, design documents, and that kind of thing might add
| process overhead upfront, but it means that forever new
| developers can simply read the history and learn exactly why
| everything works the way it works.
|
| The system itself was very Unixy. In production, it was meant to
| be run as a server with many instances striped across high-
| performance compute nodes, but it did not have to be run that
| way. Every individual product flow could also be built as its own
| transient executable, so that working on a single component could
| easily be done locally. You didn't have to rebuild the world or
| spin up your own server. Performance requirements were enough
| that we had our own customized networking stack and filesystem in
| production, but nothing depended on this for function, so you
| could still develop and test on plain Linux with ext4.
|
| The culture was terrific. We were part of one of the big five
| defense contractors, but an acquisition and this program was
| largely still staffed by employees of the original company that
| had been acquired. We were de facto exempted from most of the
| bullshit any other program had to deal with. I don't know if that
| was part of the original terms of being acquired or just a
| consequence of having so many long-time developers that couldn't
| afford to be lost if you subjected them to nonsense. This was the
| kind of project that people intentionally stayed on and retired
| from because the experience was so much better than any other
| project you could get assigned to.
|
| Ironically, it had none of the characteristics that high-
| performing companies often tout. You work in private. The rest of
| the company, including your own management chain, doesn't even
| know what you're working on. You'll never get any recognition or
| publicity. The pay is mediocre. We weren't attracting the best
| and brightest from all of the world. You had to be American, have
| a top-secret clearance, and be geographically close enough to the
| facility to get there every day, so this was a pretty constrained
| hiring pool. I still worked with some of the smartest people and
| best engineers I've ever known. The upside of this kind of
| environment is you have no mercenaries or publicity hounds.
| Everyone who sticks around is a person who really loves and cares
| about what they're working on, and a lot of people did stick
| around. The sanity and organization of the code was heavily
| facilitated by having a whole lot of people working on it who'd
| been working on it for 30+ years.
| rthnbgrredf wrote:
| The best codebase is the one you fully understand. I prefer
| codebases that are small enough to understand within a week. This
| is why I like Microservices. Large codebases can be overwhelming
| and even senior developers working a decade in the company of
| might not fully understand them. Instead, I prefer maintaining a
| few Microservices that our team fully comprehends, where the
| entire codebase fits into a clear mental model. We then interact
| with other codebases, that have active mental models in other
| teams, via APIs.
| 01HNNWZ0MV43FF wrote:
| I'm skeptical of this because I think that a well-architected
| large codebase should be as good as multiple microservices
| stapled together at the seams. But then I don't do much web
| dev.
| _kb wrote:
| I don't think the 'code' side of code base can be considered in
| isolation.
|
| What makes a _project_ objectively good (from subjective
| experience) is a combination of code, design, documentation, and
| often the humans involved.
| fer wrote:
| One that had a sort of improvised facade/adapter pattern (it
| didn't really follow either) in a clearly cut multilayered and
| pipelined structure, with actor model bits where it made sense.
|
| The code wasn't simple, at all. It took active training of new
| arrivals for them to understand it. But it was very well thought
| out, with very few warts given the complexity, and extremely easy
| to extend (that was the main requirement, given constant changes
| in APIs and clients).
|
| We had an API, with multiple concurrent versions, that
| transformed requests into an intermediate model, on which our
| business logic operated, later targetted external APIs (dozens of
| them, some REST, some SOAP, some under NDAs, some also with
| multiple versions), whose responses turned again into the
| intermediate model, with more business logic on our end, and a
| final response through our API. Each transaction got its context
| serialized so we could effectively have what was an, again
| improvised, "async/await"-like syntax in what was (trigger
| warning) C++03 code.
|
| The person who engineered it didn't have formal CS background.
| Maledictus wrote:
| Ruby on Rails.
|
| It is the only framework I have read top to bottom.
|
| Also the FreeBSD kernel, if you want to see a C code base that's
| quite beautiful (for C).
| sgt wrote:
| In this category I would nominate Django as well. It's very
| well designed (opinionated, but usually for good reasons).
|
| In terms of large C code bases I enjoy reading the PostgreSQL
| source code.
| rc_mob wrote:
| whenever I'm stuck on how to structure some code, I ask
| myself how would Laravel do it? and look up their code and
| structure mine similarly
| chuckadams wrote:
| My story is similar, but using Laravel as the canonical
| example of how not to structure things. Even without
| Facades, the internals are an impenetrable maze of magic
| methods, with things like Macroable adding methods from
| dog-knows-where. Even adding phpdoc types doesn't work when
| dealing with things like scopes.
| t43562 wrote:
| I just cannot face the thought of calling any of them "best".
|
| Every one had good and bad features though. One or two were OS-
| sized and I think a codebase that compiles and links to 85GB of
| output for 20+ devices without being a total disaster inside is
| harder to do than a neat small python module or whatever.
|
| GOOD FEATURES:
|
| Maintenance of the build and test: I worked on tools that helped
| builds go faster so I saw a lot of codebases where people were
| not maintaining the build partly because nobody had that a s a
| responsibility. There was bad management of dependencies leading
| to build failures, poor performance, incorrect build output.
| Android would be a counter example to that - I don't know if
| people like developing in it but it was always hard to accelerate
| it as the maintainers fixed performance problems regularly
| leaving our tools with little to improve.
|
| Using appropriate languages. Writing everything in C++ was a fad
| at one time. All projects work better, port better, have faster
| build times, are easier to test etc if they use memory safe
| "build once" languages to a maximum (e.g. java) and unsafe ones
| (e.g. C/C++ which have to be rebuilt and tested for each
| device/os) to a minimum. IMO Android beat Symbian amongst other
| reasons because it wasn't all C/C++ and that meant a lot of code
| didn't have to be rebuilt for different devices. This made builds
| faster and fast builds lead to better quality because of a short
| dev-test cycle.
|
| Use of faster compilers over "better" compilers. Ultimate code
| performance and quality depends on a fast development cycle more
| (IMO) than on having the most optimizing compiler. GCC versus the
| older ARM compilers for example. Now the ARM compiler is based on
| LLVM and I know that happened indirectly from a suggestion I made
| to someone who then made it to ARM who then did it.
|
| The setup and build of one codebase I worked on was as easy as
| one could expect, the build errored out if you tried to use the
| wrong tools so you never ended up debugging weird failures
| because of an incorrect tool in your PATH somewhere. I made this
| feature happen :-D. With big codebases the tools could be
| included in the version control system so you knew you had the
| right compiler, right object dumper etc. This is another strength
| of Android and yet I was in a project for Symbian to do the
| opposite because of some utter bonehead who never touched a build
| in his life who was trying to make a name for himself with his
| slimy bosses as a "doer" and "reformer."
|
| Codebases (especially big ones) benefit a lot from some kind of
| codesearch/index where you could find out where some
| function/class/variable was defined and what version of the
| source base it was introduced in.
|
| BAD FEATURES:
|
| Exclusively Owned code - we need to know who understands code
| best and who is best to review it but I don't think anyone should
| have totally exclusive control. It was a nightmare for me at one
| job - trying to get another team to make some needed change (like
| fixing their stupid makefiles to work properly in parallel). We
| (build team) should have been able to do it ourselves - maybe
| including them in the PR. Sometimes ownership is entirely
| theoretical - nobody who wrote it is still employed and nobody
| among the notional owners understands it and none of them want to
| approach it within 100 metres in case it blows up and becomes
| their problem. I simply _had_ to approach such code - no choice -
| but I kept having to send diffs to people who didn 't want to
| bother to look at them. It was a case of pushing wet spaghetti
| and took forever to do very simple changes.
|
| Insufficient tests that run infrequently. What else is there to
| say?
|
| Complicated code with no "why" or "what this is for" type
| comments. The kind of thing you trawl around in for weeks and
| cannot make head nor tail of what is going on overall.
|
| Code with so much dependency injection and general SOLID that you
| have to bounce all over the place to understand a very simple
| action.
|
| Code where writing tests is an enormous ballache. In one Go
| codebase the reason was because somone decided that the standard
| Go practise of an array of test data being run through a kind of
| "test engine" was the only way anyone should be allowed to write
| tests. Hence you had to do lots of weird things to make your test
| cases into data. Generally we use a kind of "religious" approach
| to try to get consistency out of a group of people but then take
| it much too far.
|
| codebases without automated reformatting - so everyone wastes
| time arguing about line spacing or camel-case names or whatever
| in their PRs.
| gwbas1c wrote:
| > Code with so much dependency injection and general SOLID that
| you have to bounce all over the place to understand a very
| simple action.
|
| I find that happens when people get religious about patterns
| and methodology; without understanding the "why", the language,
| and how a computer works.
|
| Case in point: I once worked on a C# project that used a port
| of Spring for dependency injection: Ultimately, it was near
| impossible to know when something was constructed, and what was
| calling what. There were classes that couldn't call themselves
| through "this" because of certain weird dependency injection
| features used.
|
| Later, I decided to use dependency injection as a design
| pattern: Instead of a complicated DI framework, there was just
| a few files of code. It was very easy for newcomers to
| understand. It was also easy to swap in mock objects, and easy
| to swap dependencies based on the target platform. It was also
| easy to see when a dependency was constructed; because it
| wasn't hidden behind a giant framework.
| nijave wrote:
| I was debugging some issues with Thanos and had pretty good
| success tweaking the codebase to add additional telemetry.
|
| The code was fairly well organized and more importantly worked
| out of the box with a Makefile and IDE integration (GoLand). All
| it took was `git clone` and opening GoLand to get started.
|
| For C (maybe it's C++), fluentbit seemed pretty straight forward
| (I don't have much experience in C though)
| sibit wrote:
| Pretty much any internal tool/TUI/CLI/library I've created. If I
| had to guess I'd say at most 25% of the company projects I've
| worked on have launched AND have consistent usage. Working hard
| on something just for it to wither crushes my soul but internal
| projects are different. They're all skunk works projects. No
| tickets. No project/board. No PM pushing back on how many points
| (read: hours) something should be. I'm solving real problems that
| directly impact the quality of life for myself and my coworkers.
| The best part is getting real, genuine, feedback. If something
| sucks they'll tell you and they won't sugarcoat it.
| jftuga wrote:
| I love this take. What language(s) do you typically use to
| write CLI programs? I'm also interested in learning about what
| types of internal TUI tools you have created.
| wruza wrote:
| The ones that were straightforward and close to the business. It
| starts at the obvious, works in an obvious way and has comment
| blocks at hard parts.
|
| For this reason I despise most modern [web] projects, which have
| a weak start, immediately drop into "services" and "components",
| do one action per source file per 30-50 lines of code, which are
| mostly imports and boilerplate, and have hundreds of these files.
| You can never tell what it does because it does almost nothing
| except formalities.
|
| I also noted a tendency to use wrong paradigms for a language.
| E.g. it may have no closures (imagine that in 202x) so they use
| events as continuations for asynchronicity, which results in a
| mess. Or it isn't immutable/functional, but they pretend it is,
| which results in fragility.
|
| The best projects are both close to their business and written in
| a paradigm of the language used.
|
| _Was there someone enforcing good practices top down?_
|
| Natural time pressure is the best bs cleaner, imo. You write
| effing code, maybe have few hours a week to refactor strange
| parts. With no time pressure a project naturally becomes massaged
| by all members into the "likeable" form of their age.
| 8474_s wrote:
| I can't name a good one, but i have strong dislike for big
| project codebases where even finding one file out of thousands
| where the relevant code resides is a challenge: its never
| something isolated but acts like some "component". The best i can
| think of is one-person projects where organization is streamlined
| as its actually has to be used by the author, not like a cog in a
| giant project.
| 999900000999 wrote:
| Realistically any code base where the engineers had at least a
| basic understanding of programming. You do not know suffering
| until you've seen someone hard code basic variables, we're
| talking about strings all over the place, and then they just copy
| the function again to replace the strings .
|
| I've legitimately left jobs over bad code. We're talking about
| code that did nothing in reality. The best code bases have been
| ones where I've been able to lead the direction. I get to know
| exactly how things work. I'm privileged to have a job where I
| essentially created the initial framework right now .
|
| Plus I'm fully remote, life is pretty good.
| naikrovek wrote:
| > any code base where the engineers had at least a basic
| understanding of programming
|
| I felt this in my soul as soon as I read it. The number of
| people writing code who should not even be speaking to others
| because of how clearly they don't understand _anything_
| technical is unfathomably high.
|
| Our industry hires seat fillers and tells them to write
| software. _And then they do_ , and it's every bit as bad as it
| sounds.
| karmakaze wrote:
| Probably the microservices-based one for async video messaging
| (i.e. Slack for video) for workers in the field. Each service was
| small enough that we could do a blue-green deploy to prod in
| about 2 minutes running only the service's tests and a tiny
| (intentionally limited) set of about 8 system journey tests _(can
| onboard a new user, user can create a contact /group, user can
| send a common content-type message, user can receive messages,
| user can react/respond to a message)_. Every commit to
| main/master automatically either deployed to prod or broke the
| CI/CD pipeline and needed to be fixed ASAP. Each service was also
| well-known by team members that it literally could be rewritten
| in a week or two if desired to change a key part of its design.
| karmakaze wrote:
| Thinking about what I wrote, I suppose my criteria for a good
| codebase is one that has the lowest friction to change: a fast
| edit/run/test/debug loop, a meaningful sense of security
| (without dogma), and fast automated deployment/revert (via
| blue/green). Given those a bad codebase can become a good one
| again (by uncoordinated action of individuals) without
| _everyone having to buy into_ a large investment.
| llmblockchain wrote:
| Any code I have ever touched.
| ttfkam wrote:
| Postgres. I don't code in C if I can avoid it, since it often
| feels like an awful lot of extra typing while still having to
| worry about memory safety. But the Postgres codebase is
| extraordinarily well organized and respects the humans that work
| with it with its intelligent handling of memory and judicious use
| of macros.
|
| I consider the core Postgres codebase to be the gold standard in
| development even though it's in a language I do not prefer to
| write in if given the choice.
|
| Shout out to the pgrx folks. You're awesome!
| https://github.com/pgcentralfoundation/pgrx
| rurp wrote:
| That is nice to hear, albeit unsurprising. Their public
| documentation is some of the best that I have worked with.
| Postgres is such an impressive project overall.
| shutupnerd0000 wrote:
| This codebase is 99.6% Rust according to GitHub.
| radimm wrote:
| That's PGRX - not PostgreSQL
| ttfkam wrote:
| As I said, I don't like coding in C.
| rrr_oh_man wrote:
| https://github.com/postgres/postgres
|
| Postgres (core?) seems to be 85% C.
| HaZeust wrote:
| He's referencing pgrx:
|
| https://github.com/pgcentralfoundation/pgrx
|
| To be fair pgrx _was_ mentioned in OP under a positive
| light, and OP _did_ say he likes avoiding coding in C, so
| it was something to bring up.
| jokoon wrote:
| I laughed at this comment
|
| sorry
| mikewarot wrote:
| The run time library for Turbo Pascal/Delphi for Windows was
| completely documented, sane, and very easy to work with. The
| working examples really helped.
|
| The free Pascal RTL seems opaque in comparison. Their reliance on
| and archaic help file build system keeps contributors away. Thus
| it's poorly documented at best.
| nurple wrote:
| Was always an Anders fan because of his work in the
| turbo/delphi community.
| tiffanyh wrote:
| I've seen a few people say 'google3'.
|
| Q: is it actually the code that you loved, or simply the tooling
| that exists?
|
| (and if it's tooling, why can't that type of tooling be
| replicated for other codebases outside of google?)
| okdood64 wrote:
| > (and if it's tooling, why can't that type of tooling be
| replicated for other codebases outside of google?)
|
| The elegance of the tooling from what I hear is that there's
| tons of different tools maintained by different teams that work
| seamlessly (and fast) together to produce google3 and all of
| its supporting pieces.
|
| But to answer your question, sure it can. But good luck
| building your own. Google has been doing this since the 2000s.
|
| And if you're a big company already, you've already bought into
| your existing patterns & design choices; things like that are
| VERY hard to change.
| aleksiy123 wrote:
| Its both. The tooling has a very direct impact on the quality
| of the code.
|
| I think the reason its not easy replicable is:
|
| 1. It takes a ton of initial investment and ongoing maintenance
| but its worth it when your code base is gigantic.
|
| 2. There is a consistent set of top down enforced rules. With
| the consistency it becomes much, much easier to build tight
| integrations between tools.
|
| (almost?) everything is buildable by a single build system
| (blaze). When anyone can consistently build/test/run anything
| in your codebase it becomes a lot easier to build a whole host
| of potential tools like code search.
|
| Probably someone can dive deeper than I can. But one thing I
| learned the most important property for a code base to be
| maintainable/scalable is consistency.
| aleksiy123 wrote:
| One more thought, is also how much other systems can utilize
| the same tooling/workflows by just storing things in source
| code. Things that would probably traditionally stored as
| application state in a database are often stored instead in
| google3 as config.
|
| Things like on team rosters, on call rosters, code review
| permissions, service permissions, feature flags.
|
| All of it stored in google3 and can all utilize the same
| consistent set of tooling like presubmit test, integration
| tests, deployment automation, permissions, code search.
|
| Its sort of like Infrastructure as Code but more.
| kridsdale3 wrote:
| I also love the built-in history diffing that this gives
| you on those files.
| nurple wrote:
| Org as code
| ryukafalz wrote:
| > When anyone can consistently build/test/run anything in
| your codebase it becomes a lot easier to build a whole host
| of potential tools like code search.
|
| As someone who hasn't worked for Google, how does Google's
| implementation of this differ from e.g. Guix/nixpkgs? Being
| able to easily build/test any available package is a big
| reason I like using tools like those.
| aleksiy123 wrote:
| I'm unfortunately not that familiar with the internals.
|
| I think the public bazel (public version of blaze) docs is
| probably the best place to look.
|
| https://bazel.build/basics/hermeticity
| fragmede wrote:
| That type of tooling _can_ be replicated. That 's why every
| Xoogler tries so hard to get Bazel adopted on everything they
| touch. Sometimes that's appropriate, sometimes it's not, but
| that's why.
|
| Bazel isn't the whole of everything though, the other piece
| being exported is kubernetes, which isn't Google's borg, but
| that's its roots. There's Apache Airflow if you need a workflow
| engine like Sisyphus, no shortage of databases to choose from,
| though now we're drifting into the operations side of things.
|
| But basically, Google has invested untold millions of dollars
| in the form of SWE and other engineering hours into making
| Google3 operate. If anyone else invested that kind of dough,
| with enough smarts, they'd also be able to make it a good
| experience. The problem is few people have that kind of budget,
| and even fewer invest in that thing, preferring to use free
| tools instead.
|
| What tool do you use to edit code, and how much did your
| employer spend on that for you?
| tiffanyh wrote:
| So the development experience Github is marching towards
| doesn't compare?
|
| E.g., Codespaces, CoPilot, Actions, etc?
|
| (genuinely curious, not trolling)
| fragmede wrote:
| It does! Working like Google is like that but having had
| decades to build that dream, using the resources that only
| a megacorp could bring to bear in the problem, and in a
| monorepo. The fact that it's a monorepo is not to be
| dismissed. git doesn't work for monorepos.
|
| Codespaces is like CitC; I haven't been a Google since the
| rise of AI so I can't comment on how the internal
| equivalent to Copilot is; Actions is very primitive
| compared to what Google has, but yeah, you can see where
| it's going. it doesn't compare right now but it could,
| eventually.
| kridsdale3 wrote:
| The Gemini-powered copilot is fine, but has so far, for
| me, only done automated like "heres a simple method based
| on the comment you wrote", or "I patched in a change to
| the other callsites when you renamed this thing". I have
| also not used it in 2 months so it's probably better
| already.
|
| I still mainly do my LLM assisted coding in chat-style
| interfaces that are more like pair programming by mail.
|
| I think the Gemini code assist is coming out as a
| consumer product, I forgot the name, because Google is
| awful at branding.
| pcatach wrote:
| > git doesn't work for monorepos.
|
| Why not? Is it related to partial checkouts?
| simoncion wrote:
| > That's why every Xoogler tries so hard to get Bazel adopted
| on everything they touch.
|
| After burning an entire weekend like five years back trying
| to get a released version of tensorflow to build from source
| [0], I'm catastrophically disinclined to use Bazel for
| anything.
|
| I found its diagnostics to be utterly unhelpful, its
| documentation to say nearly nothing I needed to know, and the
| various Internet resources for the build system to be
| somewhere between "as confused and lost as I was" and "total
| fanboy who is so expert in the system that they are incapable
| of speaking like anything other than an architecture
| astronaut".
|
| I'm sure it's legitimately fuckin amazing when you learn it
| at a company that's big enough to have one or more entire
| teams dedicated to internal developer tooling (and training
| for and documentation of the same), but (at least in my
| experience) for those of us on the outside, it's just bad,
| bad, bad.
|
| [0] The fucking thing wouldn't even build in the officially-
| supplied "build tensorflow" Docker image. I was utterly
| unable to find out why. I get that this indicates that the
| tensorflow folks fucked up somehow, but the fact that I was
| utterly unable to figure out how to understand WHAT they
| fucked up is pretty damning.
| dieortin wrote:
| I drove the adoption of Bazel for my C++ project, which
| certainly does not have an entire team (nor even an entire
| person) dedicated to maintaining the build system.
|
| We're having incredible build performance with it, and I
| find its language much more readable than CMake or Make.
| We're also using Bazel for things like packaging.
|
| I would encourage you to give it another go.
| pradn wrote:
| Google's monorepo, and it's not even close - primarily for the
| tooling:
|
| * Creating a mutable snapshot of the entire codebase takes a
| second or two.
|
| * Builds are perfectly reproducible, and happen on build
| clusters. Entire C++ servers with hundreds of thousands of lines
| of code can be built from scratch in a minute or two tops.
|
| * The build config language is really simple and concise.
|
| * Code search across the entire codebase is instant.
|
| * File history loads in an instant.
|
| * Line-by-line blame loads in a few seconds.
|
| * Nearly all files in supported languages have instant symbol
| lookup.
|
| * There's a consistent style enforced by a shared culture, auto-
| linters, and presubmits.
|
| * Shortcuts for deep-linking to a file/version/line make sharing
| code easy-peasy.
|
| * A ton of presubmit checks ensure uniform code/test quality.
|
| * Code reviews are required, and so is pairing tests with code
| changes.
| pjungwir wrote:
| > Entire C++ servers with hundreds of lines of code can be
| built from scratch in a minute or two tops.
|
| Hundreds, huh? Is this a typo? It makes me wonder if the whole
| comment is facetious. Or do C++ programmers just have very low
| expectations for build time?
| jbyers wrote:
| I suspect they meant "hundreds of thousands"
| pradn wrote:
| Yes, oops - fixed!
| stefan_ wrote:
| That's the beauty of C++, an absurdly slow build is just an
| include away.
| thfuran wrote:
| Boost always helps prop up falling compile times.
| spease wrote:
| Has anyone done LLM training with template
| metaprogramming? That seems like another excellent way to
| keep Google's build servers warm.
| mianos wrote:
| Specially when your juggling is getting out of practice.
| robodan wrote:
| The public version of Google's build tool is Bazel (it's
| Blaze internally). It has some really impressive caching
| while maintaining correctness. The first build is slow, but
| subsequent builds are very fast. When you have a team
| working on similar code, everyone gets the benefit.
|
| As with all things Google, it's a pain to get up to speed
| on, but then very fast.
| citizen_friend wrote:
| Just wait until you try a "modern" language
| dskloet wrote:
| Code search not just across the entire code base but across all
| of time.
| ok_dad wrote:
| I'm surprised they didn't turn that into a product, it sounds
| great.
| twunde wrote:
| Parts have been. Sourcegraph is basically the code search
| post built by ex-Googlers originally. Bazel is the open
| source build tool. Sadly, most of these things require major
| work to set up yourself and manage, but there's an alternate
| present where Google built a true competitor to GitHub and
| integrated their tooling directly into it.
| snotrockets wrote:
| Building tools for others is a competency that is under
| rewarded at Google. They would never.
| kridsdale3 wrote:
| I've published my Google proprietary stuff (when we decided
| to open source it) on GitLab, but they wouldn't let me do
| it on GitHub.
| tallowen wrote:
| I always find these comments about interesting, having worked
| at Facebook and Google, I never quite felt this way about
| Google's Monorepo. Facebook had many of the features you listed
| and quite performantly if not more so. Compared with working at
| Facebook where there are no owners owners files and no
| readability requirements, I found abstraction boundries to be
| much cleaner at FB. At google, I found there was a ton of cruft
| in Google's monorepos that were too challenging / too much work
| for any one person to address.
| pradn wrote:
| OWNERS files rarely get in the way - you can always send a
| code change to an OWNER. They are also good for finding
| points of contact quickly, for files where the history is in
| the far past and changes haven't been made recently.
|
| Readability really does help new engineers get up to speed on
| the style guide, and learn of common libraries they might not
| have known before. It can be annoying - hell, I'll have to
| get on the Go queue soon - but that's ok.
| sawyna wrote:
| This isn't true at all for OWNERS files. If you try
| developing a small feature on google search, it will
| require plumbing data through at least four to five layers
| and there is a different set of OWNERS for each layer.
| You'll spend at least 3 days waiting for code reviews to go
| through for something as simple as adding a new field.
| okdood64 wrote:
| 3 days for a new change on the biggest service on the
| planet? Not bad.
| tallowen wrote:
| I agree that it could be worse! Facebook has significant
| (if not more) time spent and I found adding features to
| news feed a heck of a lot easier than adding features
| that interacted with google search. Generally a lot of
| this had to do with the number of people needed to be
| involved to ensure that the change was safe which always
| felt higher at Google.
| kridsdale3 wrote:
| I have the same background: I find the code quality at G
| to be quite a lot higher (and test pass-rate, and bug
| report-rate lower) than News Feed, which was a total
| shit-show of anything-goes. I still hold trauma from
| being oncall for Feed. 70 bugs added to my queue per day.
|
| The flip side is of course that I could complete 4 rounds
| of QuickExperiment and Deltoid to get Product Market Fit,
| in the time it takes to get to dogfooding for any feature
| in Google.
| codethief wrote:
| I'm only an outside observer in this conversation but
| could it be that the review process (or the lack thereof)
| and the ease with which you can add new features has had
| an impact on the quality of the software?
|
| The thing is, in my experience as a user Facebook (the
| product, not the former company) is absolutely riddled
| with bugs. I have largely stopped using it because I used
| to constantly run into severe UI/UX issues (text input no
| longer working, scrolling doing weird things, abysmal
| performance, ...), loading errors (comments & posts
| disappearing and reappearing), etc. Looking at the
| overall application (and e.g. the quality of the news
| feed output), it's also quite clear that many people with
| many different ideas have worked on it over time.
|
| In contrast, Google search still works reasonably well
| overall 25 years later.
| esprehn wrote:
| There are pretty different uptime and stability
| requirements for a social product and web search (or
| other Google products like Gmail). When news feed is
| broken life moves on, when those products break many
| people can't get any work done at all.
|
| One of Google's major cultural challenges is imposing the
| move slow and carefully culture on everything though.
| zeroonetwothree wrote:
| It's not considered ok for newsfeed to break. It would be
| a massive issue that would command the full attention of
| everyone.
| esprehn wrote:
| And yet folks who are on call for it say things like
| this: https://news.ycombinator.com/item?id=40826497
| tallowen wrote:
| I think I have heard similar things from other googlers and
| I think there might be two factors on why I think this:
|
| - I worked on Google Assistant which was responsible for
| integrating many services. This meant I had to work with
| other peoples code way more regularly that many at google.
|
| - I moved from FB to google - I'm not really sure how many
| people have had this experience. I think many of my
| colleagues at google found it surprising how many of the
| things they thought were unique to google actually also
| existed at FB.
|
| At the end of the day, any of these processes have
| pros/cons but I think the cruft of having APIs that are a
| couple steps harder to evolve due to finding
| Readability/Owners for everything you touch just makes
| things slightly less cohesive and a trickier place to have
| a "good" codebase.
|
| When I worked at FB, I would frequently rebase my code on
| Monday and find that, for example, the React framework
| authors or another smaller infra team had improved the API
| and had changed *every* callsite in the codebase to be
| improved. This type of iteration was possible in certain
| situations but was just much less common at google than at
| fb.
| snotrockets wrote:
| > I think many of my colleagues at google found it
| surprising how many of the things they thought were
| unique to google actually also existed at FB.
|
| Google workers are groomed to believe Google is the best,
| and hence they are too. A corollary of that, then, is
| that nobody else has it that good, when in fact, others
| sometimes have it better.
| kridsdale3 wrote:
| I also made the move from FB to G and echo everything
| said above. Googlers have a massive superiority complex.
| In reality, it's naivete.
|
| My 2 cents: OWNERS is fairly useful, if only as a form of
| automating code reviewer selection. Readabilty is a
| massive drag on org-wide productivity. I have had
| diffs/CLs take MONTHS to be approved by every Tom Dick
| and Harry whose claws were added to my code and made me
| re-design whole project approaches, and they were only
| there because they're supposed to check if my new-lines
| are in the right spot for that language. I thought about
| quitting.
| wavemode wrote:
| People really underestimate how much productivity drain
| there is in having a bad code review culture. One of the
| worst things about working at Amazon was that any
| feedback on a merge request, no matter how small,
| required you to request a re-review.
| ynx wrote:
| +1.
|
| Going from FB to $REDACTED to Oculus was a pretty wild
| ride, there were a lot of different cultures, though I
| think generally speaking the best qualities filtered
| through.
|
| (also, howdy former teammate)
| kridsdale3 wrote:
| (trying to recall unixname to human name mapping.....)
| elevatedastalt wrote:
| Huh? Facebook has a lot of that infra because ex-Googlers
| built it there. It takes an insane amount of delusion to
| notice something common between a father and a son and
| say that the dad inherited it.
| pradn wrote:
| No, we're quite aware the world outside has been catching
| up. There's even a famous doc by a senior director about
| it...
| AnotherGoodName wrote:
| Same and another vote for meta. Meta made the language fit
| their use case. Go into bootcamp change the search bar text
| to 'this is a search bar!' press F5 and see the change (just
| don't ship that change ;D). It's incredibly smooth and easy.
|
| Googles a mess. There's always a migration to the latest
| microservices stack that have been taking years and will take
| many more years to come.
|
| Like meta just changed the damn language they work in to fit
| their needs and moved on. Google rewrites everything to fit
| the language. The former method is better in a large
| codebase. Meta is way easier to get shit done to the point
| that google was left in the dust last time they competed with
| meta.
| quicklime wrote:
| I think what you're saying is true for www, but not fbcode,
| and the later starts to look a lot like google3. I agree
| though, Meta's www codebase has the best developer
| experience in the industry.
| codethief wrote:
| For non-Googlers like me, here's some background about
| "readability":
|
| https://www.moderndescartes.com/essays/readability/
| mattnewton wrote:
| Huh, also having worked at both I had exactly the opposite
| experience. Google's tools looked ugly but just worked. At
| meta there were actually multiple repos you might have to
| touch and tools worked unreliably across them. Owners files
| made sure there was less abandoned code and parent owners
| woild be found by gwsqueue bots to sign off on big changes
| across large parts of the repo by just reading these files.
| tantalor wrote:
| > no owners owners files and no readability requirements
|
| Move fast and break things, right?
| zeroonetwothree wrote:
| I don't think that owners files are the best way to ensure
| things don't break.
| robertsdionne wrote:
| * https://abseil.io/resources/swe-book/html/ch16.html
|
| * https://abseil.io/resources/swe-book/html/ch17.html
| WWWMMMWWW wrote:
| Google's code, tooling and accompanying practices are
| developing a reputation for being largely useless outside
| Google ... and many are starting to suspect it's alleged value
| even inside Google is mostly cult dogma.
| wiseowise wrote:
| > Google's code, tooling and accompanying practices are
| developing a reputation for being largely useless outside
| Google ...
|
| Not that I don't believe you, but where do you see this?
| matthewfcarlson wrote:
| I haven't worked at google, but this is something I have
| heard from a few people. Reputation is largely word of
| mouth, so it checks out for me. I suspect the skills/tools
| at most large companies are increasingly less transferrable
| as they continue to grow in scale and scope.
| robertakarobin wrote:
| I can vouch for it. It's the main reason I quit: none of
| the "hard" skills necessary to code at Google were
| transferrable anywhere outside of Google. It would have
| been easy enough to skate and use "soft" skills to move up
| the management ladder and cash big checks, but I wasn't
| interested in that.
|
| The reason it's not transferrable is that Google has its
| own version of EVERYTHING: version control, an IDE, build
| tools, JavaScript libraries, templating libraries, etc,
| etc. The only thing I can think of that we used that wasn't
| invented at Google was SCSS, and that was a very recent
| addition. Google didn't even use its own open-source
| libraries like Angular. None of the technologies were
| remotely usable outside Google.
|
| It might sound cool to use only in-house stuff, and I
| understand the arguments about licensing. But it meant that
| everything was poorly-documented, had bugs and missing
| features that lingered for years, and it was impossible to
| find a SME because whoever initially built a technology had
| moved on to other things and left a mess behind them.
|
| Some people may be able to deal with the excruciating
| slowness and scattered-ness, and may be OK with working on
| a teeny slice of the pie in the expectation that years
| later they'll get to own a bigger slice. But that ain't me
| so I noped out as soon as my shares vested.
| Banditoz wrote:
| Their own IDE? Is it web based? So they don't use, say,
| IntelliJ/Eclipse for Java projects?
| robertakarobin wrote:
| It's basically a fork of VSCode.
| makeramen wrote:
| That's a recent development, used to be something else
| altogether.
| audiodude wrote:
| 12 year current Googler here. You are absolutely correct
| about "Google has its own version of EVERYTHING". Midway
| through my current career, I started to get existential
| dread about the fact that I wasn't "up to date" on any
| current development practices or frameworks.
|
| Partly, this was assuaged through participating in open
| source projects in my free time. That's how I learned
| Docker, Github workflow, React, Vue, Bootstrap, Tailwind,
| etc.
|
| But at the same time, I think it is a mistake to consider
| working with tools/languages/frameworks to be the only
| "hard" skills. Galaxy brain is realizing that anyone can
| learn a language/framework/workflow in a month or so. The
| real work is applying sound principles to the design and
| production of meaningful artifacts within those systems.
| kaba0 wrote:
| Though credit where it's due, some of their tools really
| have been years ahead of anything outside of google, e.g.
| the closure compiler that made javascript development
| scalable.
| openmajestic wrote:
| I have seen this discussed in hiring decisions. I don't
| know that it played a large factor in a decision, but lack
| of experience in the standard tools/practices/terms of
| software development because of a career at Google was
| definitely a discussion point.
| makeramen wrote:
| I had a bunch of very tenured teammates that didn't really
| know how to use git, so there were only a few of us
| comfortable enough integrating and interacting with an open
| source dependency repo.
| worik wrote:
| > Google's code, tooling and accompanying practices are
| developing a reputation for being largely useless outside
| Google .
|
| It is almost a tautology
|
| Why would they be useful for domains they are not designed
| for?
| tonynator wrote:
| Most companies don't use proprietary tools for everything.
| jart wrote:
| Google used to have a near monopoly on the most expensive,
| educated, devoted, and conscientiously willful people and
| imposed very few demands on their time. The lengths to which
| they were willing to go, to make everything they did with the
| tools pleasant and elegant, was orders of magnitude beyond
| anything I'd seen.
|
| Some of us thought that the magic of these people would be
| imbued in the dev tools that they created, so if enterprises
| adopted the tools, then they'd reap the benefits of that same
| magic too. But this simply wasn't true. The tools didn't
| actually matter; it was the way they used them.
|
| For example, when other companies started adopting tools like
| Bazel (open source Blaze) they wanted features like being
| able to launch ./configure scripts inside Bazel, which
| totally violates the whole point of Bazel, and never would
| have been allowed or even considered inside Google. The Bazel
| team was more than happy to oblige, and the users ended up
| with the worst of all worlds.
| WWWMMMWWW wrote:
| Google's systems were designed to index mountains of low
| value data at hitherto unseen scale, and they're good at
| that. But, to-the-second system-wide precision with full
| audit trails ... not so much.
|
| You keep seeing startups with ex-Googlers that think they
| can "disrupt" Fintech with Google's "secret sauce" ... this
| tends to go badly.
|
| I've had to clean up one of these messes where, in all
| seriousness, even a pre-2000 LAMP stack (never mind Java)
| implemented by people who understood the finance domain
| would have worked better.
| dieortin wrote:
| If Google open sourced their BUILD files for public
| libraries, we wouldn't be able to use workarounds...
| Migrating something complex like ffmpeg to Bazel is not
| something trivial.
|
| Bazel is an awesome tool though, I'm very glad it was open
| sourced and receives constant attention from Google.
| bbor wrote:
| I'm just one incompetent dev, but I'll throw this in the convo
| just to have my perspective represented: every individual part
| of the google code experience was awesome because everyone
| cared a ton about quality and efficiency, but the overall
| ecosystem created as a result of all these little pet projects
| was to a large extent unmanaged, making it difficult to operate
| effectively (or, in my case, basically at all). When you join,
| one of the go-to jokes in their little intro class these days
| is "TFW you're told that the old tool is deprecated, but the
| new tool is still in beta"; everyone laughs along, but
| hopefully a few are thinking "uhhh wtf".
|
| To end on as nice of a note as possible for the poor Googs: of
| all the things you bring up, the one I'd highlight the biggest
| difference on is Code Search. It's just _incredible_ having
| that level of deep semantic access to the huge repo, and people
| were way more comfortable saying "oh let's take a look at that
| code" ad-hoc there than I think is typical. That was pretty
| awesome.
| aleksiy123 wrote:
| Imho the reason for the deprecated and beta thing is because
| there is a constant forward momentum.
|
| Best practices, recommendations and tooling is constantly
| evolving and requires investment in uptake.
|
| I sometimes feel like everything is legacy the moment it's
| submitted and in a constant state of migration.
|
| This requires time and resources that can slow the pace of
| development for new features.
|
| The flips side is this actually makes the overall codebase
| less fractured. This consistency or common set of assumptions
| is what allows people to build tools and features that work
| horizontally across many teams/projects.
|
| This constant forward momentum to fight inconsistency is what
| allows google3 to scale and keep macro level development
| velocity to scale relative to complexity.
| bbor wrote:
| That's all well said, thanks for sharing your perspective!
| Gives me some things to reflect on. I of course agree
| re:forward momentum, but I hope they're able to regain some
| _grace_ in that momentum with better organization going
| forward. I guess I was gesturing to people "passing the
| buck" on hard questions of team alignment and mutually
| exclusive decisions. Obviously I can't cite specifics bc of
| secrecy and bad memory in equal amounts, so it's very
| possible that I had a distorted view.
|
| I will say, one of the things that hit me the hardest when
| the layoffs finally hit was all the people who have given
| their professional lives to making some seriously
| incredible dev tools, only to be made to feel disposable
| and overpaid so the suits could look good to the
| shareholders for a quarter or two. Perhaps they have a
| master vision, but I'm afraid one of our best hopes for an
| ethical-ish megacorp--or at least vaguely pro social--is
| being run for short term gain :(
|
| However that turns out for society, hopefully it ends up
| releasing all those tools for us to enjoy! Mark my words,
| colab.google.com will be shockingly popular 5y from now, if
| they survive till then
| kridsdale3 wrote:
| I guarantee you that the master vision is exactly what
| you wrote.
|
| Google is not a software company, it is an advertising
| system providing cash-flow to a hedge fund that comprises
| a large part of every pension and retirement fund in
| America. It's far too important as simply a financial
| entity to risk anything on .... _product development_.
| teaearlgraycold wrote:
| My experience with google3 was a bit different. I was shocked
| at how big things had gotten without collapsing, which is down
| to thousands of Googlers working to build world-class internal
| tooling. But you could see where the priorities were. Code
| Search was excellent - I'd rate it 10/10 if they asked.
|
| The build system always felt more like a necessary evil than
| anything else. In some parts of google3 you needed three
| _separate_ declarations of all module dependencies. You could
| have Angular 's runtime dependency injection graph, the
| Javascript ESM graph, and the Blaze graph which all need to be
| in sync. Now, the beautiful part was that this still worked.
| And The final Blaze level means you can have a Typescript
| codebase that depends on a Java module written in a completely
| unrelated part of google3, which itself depends on vendored C++
| code somewhere else. Updating the vendored C++ code would cause
| all downstream code to rebuild and retest. But this is a multi
| billion dollar solution to problems that 99.99% of companies do
| not have. They are throwing thousands of smart people at a
| problem that almost everyone else has "solved" by default
| simply by being a smaller company.
|
| The one tooling I think every company could make use of but
| doesn't seem to have were all of the little hacks in the build
| system (maybe not technically part of Blaze?). You could
| require a developer who updates the file at
| /path/to/department/a/src/foo.java to simultaneously include a
| patch to /path/to/department/b/src/bar.java. Many files would
| have implicit dependency on each other outside of the build
| graph and a human is needed to review if extra changes are
| needed. And that's just one of a hundred little tricks project
| maintainers can employ.
|
| The quality of the code was uniformly at least "workable" (co-
| workers updating parts of the Android system would probably not
| agree with that - many critical system components were written
| by one person poorly who soon after quit).
| SR2Z wrote:
| > But this is a multi billion dollar solution to problems
| that 99.99% of companies do not have.
|
| I know it's trendy for people to advocate for simple
| architectures, but the honest-to-god truth is that it's
| insane that builds work ANY OTHER WAY. One of the highest
| priorities companies should have is to reduce siloing, and I
| can barely think of a better way to guarantee silos than by
| having 300 slightly different build systems.
|
| There is a reason why Google can take a new grad SWE who
| barely knows how to code and turn them into a revenue
| machine. I've worked at several other places but none of them
| have had internal infrastructure as nice as the monorepo; it
| was the least amount of stress I've ever felt deploying huge
| changes.
|
| Another amazing thing that I don't see mentioned enough was
| how robust the automatic deployments with
| Boq/Annealing/Stubby were. The internal observability library
| would automatically capture RPC traces from both the client
| and server, and the canary controller would do a simple
| p-test on whether or not the new service had a higher error
| rate than the old one. If it did? The rollback CL would be
| automatically submitted and you'd get a ping.
|
| This might sound meh until I point out that EVEN CONFIG
| CHANGES were versioned and canaried.
| kridsdale3 wrote:
| I've worked at the majority of FAANG.
|
| Facebook's build system works the same as Googles, because
| most of FB's infra was made by ex-Googlers around 10-15
| years ago. The worst thing I can say about Blaze is
| basically already pointed out above, sometimes you need to
| write little notes to the presubmit system to ensure cross-
| boundary updates. Whatever, it's all text files in the end.
|
| The wildest was at Apple. It's just as you said, 300 build
| systems. Not only that, but 300 source code repositories!
| Two teams in the same hall that hang out all the time could
| be using git and svn, for no good reason besides what
| someone wrote "init" in 20 years ago. There was no cross
| team communication, by design, because Steve Jobs was
| paranoid. Their sync mechanism was to build the entire
| stack once a night, and force everyone to full-reinstall
| their OS and toolchain to "rebase". Insane.
| teaearlgraycold wrote:
| I definitely agree most companies should use a monorepo.
| Most companies don't need Blaze, though.
|
| And the whole rollout system was excellent. I wish that
| tech was standard but I have a vague idea of how much work
| that would be to implement and few companies will be able
| to afford to get that right.
|
| Edit: I forgot to mention - I absolutely hated Cider and
| all of the included plugins. Sure the code at Google was
| fine but the code editing experience destroyed all of the
| fun of coding. Is that function signature correct? You'll
| find out in 45 seconds when the Intellisense completes! And
| when I was there Copilot was a thing outside of Google but
| we were not allowed to use any AI (even Google's own AI) to
| write code. The whole situation was so bad I wrote a few
| paragraphs about it in my offboarding survey.
| codethief wrote:
| > You could require a developer who updates the file at
| /path/to/department/a/src/foo.java to simultaneously include
| a patch to /path/to/department/b/src/bar.java.
|
| Could you elaborate on how this worked exactly?
| teaearlgraycold wrote:
| There's a configuration directive you put in a plain text
| file in the monorepo which lets you configure:
|
| * File A's path
|
| * File B's path
|
| * The message shown on the commit's review page if B isn't
| updated when A is updated
|
| * How a developer can override the alert (Which would be a
| custom named directive added to a commit message, like
| "SKIP_FILE_SYNC_ALERT=true")
|
| You then need to either commit a diff to file B when file A
| is changed or override it in order to get the commit added
| to HEAD. This is just one of many different "plugins" for
| the CI system that can be configured with code.
| codethief wrote:
| Thanks!
| scubbo wrote:
| Interesting to note that almost all of these are to do with
| tooling _around_ the codebase, not the contents _of_ the
| codebase!
| kridsdale3 wrote:
| Just like the man is the product of his genetic code, the
| codebase is invariably the product of the constraints on its
| edits enforced by tooling.
| nathan_douglas wrote:
| so we beat on, commits against the tooling, borne back
| ceaselessly into the technical debt
| remram wrote:
| Is that true, though? Is the code itself good? Because it
| is sorely absent from GP's list...
|
| If you are trying to say that people can't make bad code
| with good tools, I don't agree.
| dheera wrote:
| Why do so many people like monorepos?
|
| I tend to much prefer splitting out reusable packages into
| their own repos with their own packaging and unit tests and
| tagging to whatever version of that package. It makes it MUCH
| easier for someone to work on something with minimal overhead
| and be able to understand every line in the repo they are
| actually editing.
|
| It also allows reusable components to have their own
| maintainers, and allows for better delegation of a large team
| of engineers.
| Tyr42 wrote:
| I can change a dependency and my code at the same time and
| not need to wait for the change to get picked up and deployed
| separately. (If they are in the same binary. Still need cross
| binary changes to be made in order and be rollback safe and
| all that.)
| radicality wrote:
| Have you ever worked at FB / Google / whatever other company
| has huge mono repo with great tooling?
|
| I went from many years at FB, to a place like you describe -
| hundreds of small repos, all versioned. It's a nightmare to
| change anything. Endless git cloning and pulling and rebase.
| Endless issues since every repo ends up being configured
| slightly differently, and very hard to keep the repo metadata
| (think stuff like commit rules, merge rules, etc) up to date.
| It's seriously much harder to be productive than with a well-
| oiled monorepo.
|
| With a monorepo, you wanna update some library code to
| slightly change its API? Great, put up a code change for it,
| and you'll quickly see whether it's compatible or not with
| the rest of the whole codebase, and you can then fix whatever
| build issues arise, and then be confident it works everywhere
| wherever it's imported. It might sound fragile, but it really
| isn't if the tooling is there.
| dheera wrote:
| I have worked at a company that has huge monorepos and bad
| tooling.
|
| Tooling isn't the problem though, the problems are:
|
| - multiple monorepos copying code from each other, despite
| that code should be a library or installable python package
| or even deb package of its own
|
| - you will never understand the entire monorepo, so you
| will never understand what things you might break. with
| polyrepos different parts can be locked down to different
| versions of other parts. imagine if every machine learning
| model had a copy of the pytorch source in it instead of
| just specifying torch==2.1.0 in requirements.txt?
|
| - "dockerize the pile of mess and ship" which doesn't work
| well if your user wants to use it inside another container
|
| - any time you want to commit code, 50000 people have
| committed code in-between and you're already behind on 10
| refactors. by the time you refactor so that your change
| works, 4000 more commits have happened
|
| - the monorepo takes 1 hour to compile, with nothing to
| compile and unit test only a part of it
|
| - ownership of different parts of the codebase is difficult
| to track; code reviews are a mess
| joshuamorton wrote:
| All of these are solved with tooling.
|
| - dont have multiple monorepos
|
| - use blaze or similar and run all downstream tests, the
| binary for your ml model includes pytorch whether you
| build from source or requirements.txt.
|
| - other people committing doesn't matter if you are
| isolating presubmits and similar.
|
| - using a blaze-like you never compile the whole monorepo
|
| - code owners etc. makes this straightforward.
|
| Like, as someone whose career has been mostly at Google,
| these are not problems I encounter at all, or only in
| cases where you'd have similar scope of problems no
| matter the repo structure.
| hughesjj wrote:
| > other people committing doesn't matter if you are
| isolating presubmits and similar.
|
| Could you elaborate on this one?
| joshuamorton wrote:
| If you only need to run the tests you affect, and only
| need to sync and update files touched in your CL,
| external changes are generally not impactful, sync &
| submit is a quick process even if people are submitting
| things elsewhere.
|
| It's only a problem if someone submits a file you're
| touching, in which case you just have standard merge
| conflict issues.
| audiodude wrote:
| I think all of your problems are root caused by the
| phrase "multiple monorepos". This sounds more like
| "polyrepos" or something, multiple siloed repos that
| themselves might contained disparate projects, languages
| and tooling.
|
| Google3 is a true monorepo. 99% of code in the company is
| one repo, with minor exceptions for certain open source
| projects and locked down code.
|
| Edit: for example, you can change YouTube code, Google
| Assistant code, a Maps API, some config for how URLs are
| routed at top level load balancers, etc all in one CL if
| you really wanted to/needed to.
| Thaxll wrote:
| How do you handle tagging, the whole repo gets a new tag /
| version every time something change?
| audiodude wrote:
| Yes, all changes (CLs) are atomic operations. The CL
| number reflects a specific moment in time for the entire
| repo.
| sghill wrote:
| Up to what scale?
|
| This works well for a couple dozen repos per team in my
| experience. It's also my preferred way to work.
|
| It doesn't scale so well to hundreds of repos per team
| without significant tooling. At some point anything cross-
| cutting (build tool updates, library updates, etc) becomes
| hard to track. Repos are left behind as folks change teams
| and teams are reorg'd.
|
| I've never worked in a monorepo, but I can see the appeal for
| large, atomic changes especially.
| zerd wrote:
| If you have a reusable component in a separate repository and
| need a change, you have to submit that, merge, release, then
| bump the version in the downstream project to use it. Then if
| someone else uses the project updates but hits an issue that
| you introduced they have to go fix it, perhaps a month later,
| with no context of what changed. Or they just don't upgrade
| the version, and reimplement what they need. With a monorepo
| it would be one change, and your change breaking someone
| else's would get flagged and fixed with the code change. I've
| seen the amount of shared code get less and less and more
| stale with polyrepo.
| rkagerer wrote:
| Question I've always wondered: Does Google's monorepo provide
| all its engineers access to ALL its code?
|
| If yes, given the sheer number of developers, why haven't we
| seen a leak of Google code in the past (disgruntled employee,
| accidental button, stolen laptop, etc)?
|
| Also how do they handle "Skunkworks" stlye top-secret projects
| that need to fly under the radar until product launch?
| rkagerer wrote:
| Edit - I guess there hasn't been zero leaks:
| https://searchengineland.com/google-search-document-leak-
| ran...
| pradn wrote:
| The very very important stuff is hidden, and the only two
| examples anyone ever gives are core search ranking algorithms
| and the self-driving car.
|
| Even the battle-tested hyper-optimized, debugged-
| over-15-years implementation of Paxos is accessible. Though
| I'm sure folks could point out other valuable
| files/directories.
| kccqzy wrote:
| Former employee here. I remember a third example: the anti-
| DoS code is hidden. I remember this because I needed to do
| some very complicated custom anti-DoS configuration and as
| was my standard practice, I looked into how the
| configuration was being applied. I was denied access.
|
| Fourth example: portions of the code responsible for
| extracting signals from employees' computers to detect
| suspicious activity and intrusion. I suspect it's because
| if an employee wants to do something nefarious they
| couldn't just read the code to figure out how to evade
| detection. I only knew about this example because that
| hidden code made RPC calls to a service I owned; I changed
| certain aspect of my service and it broke them. Of course
| they fixed it on their own; I only got a post-submit
| breakage notification.
| robodan wrote:
| Partial check outs are standard because the entire code base
| is enormous. People only check out the parts they might be
| changing and the rest magically appears during the build as
| needed.
|
| There are sections of the code that are High Intellectual
| Property. Stuff that deals with spam fighting, for example. I
| once worked on tooling to help make that code less likely to
| be accidentally exposed.
|
| Disclaimer: I used to work there, but that was a while back.
| They probably changed everything a few times since. The need
| to protect certain code will never go way, however.
| cmrdporcupine wrote:
| Google3 monorepo source isn't, by policy, supposed to leave
| the corp network workstations, and can't even be on your
| corporate provided laptop (except for some edge cases in
| mobile apps dev). Even during full COVID lockdown WFH we had
| to remote into our machines. (I worked on embedded stuff and
| had to compile on my office workstation, scp the binaries
| home, and flash my device, and repeat. Super slow cycle.)
|
| So, anyways, source code being basically on-premise only and
| on machines that they can fully audit and control... Would
| you be stupid enough to "cp -r srccheckout /media/MYUSBSTICK"
| on such a box?
|
| Also believe it or not they used to have a very open internal
| culture at Google because the bulk of employees genuinely
| liked the company and its stated mission and there was a bit
| of a social contract that seemed to be upheld. Stuff didn't
| generally leak out of the wide open all hands, even. Past
| tense.
| tfsh wrote:
| Within the monorepo there is the notion of "silos" where
| access to directories can be restricted to groups of
| people/bots. Though I believe that's exceedingly rare, I've
| never come across one.
| kccqzy wrote:
| It's extremely easy to detect a disgruntled employee making a
| copy of source code. There's no accidental button to leak.
| There's no source code on laptops as policy doesn't allow it,
| with limited exceptions only.
|
| But there was a giant leak a long time ago. It was called
| Operation Aurora done by China. Legend has it that to this
| date the Chinese search engine Baidu still uses stolen code
| from Google.
| bhawks wrote:
| Google laptops and workstations (anything that can actually
| access srcfs to get this data) are extremely monitored and
| controlled.
|
| Very critical stuff (ranking, spam/abuse, etc) can be further
| protected via silos which lock down sections of the code base
| (but still allow limited interactions with the build).
|
| Google spent significant engineering $$$ into its development
| tools and policies (generally building custom with no intent
| to ever monetize vs buying). I don't see a company today, in
| this climate, that would emulate that decision.
| emmelaich wrote:
| I suspect this is part of the interview process and why it
| takes so long and so many people.
|
| Character and trustworthiness is extremely important.
| kccqzy wrote:
| The Google interview process is overly focused on algorithm
| skills and absolutely does not select for character and
| trustworthiness. In fact the leaks from Google to the news
| started circa 2017 and in response the leadership basically
| neutered internal forums like TGIF and memegen. Remember
| the Damore incident? While Damore was wrong, it wouldn't be
| as big of a deal if the incident wasn't leaked to the
| press. It's clear that Google would be a much better
| company if its interview process actually accounted for
| character and trustworthiness.
|
| The old article, _Three Years of Misery Inside Google, the
| Happiest Company in Tech_ is still the best description of
| what went wrong inside Google:
| https://www.wired.com/story/inside-google-three-years-
| misery...
| Ocerge wrote:
| I recently left Google and knew it was going to be a step down
| from Google's build ecosystem, but I wasn't prepared for how
| far a step down it would be. It's the only thing I miss about
| the place, it' so awesome.
| pmb wrote:
| People who have never had it have no concept of how much they
| are missing. It's so frustrating.
| Thaxll wrote:
| Most of those arguments are not about code quality though.
| ramses0 wrote:
| It's kindof a dumb answer, but `abcde` (A Better CD Encoder)
| continues to be my go-to reference for "everything shell".
|
| https://git.einval.com/cgi-bin/gitweb.cgi?p=abcde.git;a=summ...
|
| https://git.einval.com/cgi-bin/gitweb.cgi?p=abcde.git;a=blob...
|
| ...it's just so damned... like... it _is_ 5000 lines of bash, but
| given the constraints and the problem domain, it's an incredible
| (and well-organized) feat.
|
| If I ever question "How would I do something like XXX in shell?",
| I can usually find some clues, inspirations, or answers in the
| `abcde` codebase.
| e12e wrote:
| > https://git.einval.com/cgi-
| bin/gitweb.cgi?p=abcde.git;a=blob...
|
| Interesting. I was a bit surprised by the use of echo in the
| usage() function (and not cat and a heredoc) -- but that
| certainly looks like a very structured shell script.
|
| Thanks for sharing.
| paxys wrote:
| One that was written from scratch, by me. All others are
| terrible.
| ChrisMarshallNY wrote:
| Heh. Can relate.
|
| Even my own codebases/config. management systems are difficult,
| if I haven't dealt with them in a while.
|
| However, I am pretty good at understanding old stuff I did,
| mainly because I leave a legacy for myself.
|
| https://littlegreenviper.com/leaving-a-legacy/
| wvenable wrote:
| ...unless I haven't looked at it in a few years.
| tiffanyh wrote:
| The problem I have is, I'll revisit code I wrote 1-2 months
| prior and swear someone else wrote it.
| pavel_lishin wrote:
| Same.
|
| I _know_ I wrote perfect, maintainable code that can easily
| live in my brain.
|
| But when I go to work on it again a few months since I wrote
| it, it seems like some _asshole_ stepped in and overwrote my
| beautiful code with a bunch of jumbly mish-mash and had the
| gall to overwrite the git commit history with it, using the
| exact same commit messages, and my own name!
| lagrange77 wrote:
| Right. I even quietly mumble 'asshole' while reading it.
| takeda wrote:
| This is the weird thing. I heard it a lot and I don't know if
| people just over-exaggerate or maybe I'm just forgetting
| things quicker, but I don't have that. I come to the code
| after month or two and it is the same as if I came a week or
| two. I also don't feel ashamed of it.
|
| I also got compliments of my code in the past.
|
| I do try to make my code readable and avoid being clever.
| Basically my rule of thumb is that if I need to write a
| comment about block of code what it does, then that code is
| too complex.
| tazjin wrote:
| Same. I think it's a meme that people keep repeating, and
| for some fraction of them it's true.
| doctor_eval wrote:
| A month or two is nothing. Go back to it after 5 years.
| grotorea wrote:
| Past me is someone else.
| https://en.wikipedia.org/wiki/Ship_of_Theseus
| intelVISA wrote:
| We were all thinking it, thank you for having The Courage.
| stephenhandley wrote:
| This entire website in a single comment
| ohyes wrote:
| Mine is always terrible and the others are terrible as well.
| mikeqq2024 wrote:
| One day I got an unexpected email from someone asking
| permission to use a piece of code on github I forgot I wrote
| almost 10 years ago without a license clause. I was very
| surprised and confused because I forgot I wrote that code, and
| I was not involving coding for quite some years and my github
| profile was blank without any contact information. I asked. It
| happened the guy simply made a search and found a post on a
| forum with my nickname, and email address - newly created
| recently. At that moment, it clicked and I felt some unusual
| arrangment. I quickly added an open source license, and the
| guy, CTO of a decent company, offered to make a donate. That
| made me a day.
| ryancnelson wrote:
| Illumos, and specifically the SmartOS distribution.
|
| Being ringside for the fork of Sun's OpenSolaris, and watching
| absolute geniuses steward a thing like that into a useful product
| and a build process was foundational for my understanding of how
| and why process is important, as well as how to get things fixed
| in the real world, while not muddying the idea of the pure design
| principles. A balance can exist!
| jowdones wrote:
| >> Ask HN: What is the best code base you ever worked on?
|
| LOL. Obviously. Mine! :)
| lanstin wrote:
| AOL's server code was excellent. It was written by some people
| that had done similar things for BITNET and had a deep
| understanding of Unix programming and how to use event loops for
| scalability.
|
| It was in C, but the process was expected to run for months
| without crashing or running out of memory; if it had an abnormal
| exit, you'd (the owner as set in some code thing) get an email
| with the core backtrace. If it had a memory leak, ops would be on
| you quickly. Ops was considered a primary driver of requirements
| to dev, and absolutely everything could be reloaded into a
| running server without restarting it. There was a TCP control
| port where a TCL interpreter inside the service was exposed and
| generally you wrote the (very simple, CS 101 style) TCL commands
| to manage the server. It was a "No Threads Kernel", scaled to the
| dozens or hundreds of physical machines communicating over a very
| well managed network, and most 1 process per core, and 1 core for
| the OS. The 200 or so unix developers (as we were called) had a
| common understanding of how the framework worked and if you were
| just writing app code it was basically impossible to write slow
| services. We had technical writers that would interview the
| developers and write books that could be handed to outside
| developers and lead to a successful integration with no developer
| time spent.
|
| The NTK was primarily for sending msgs over the network - we had
| a principle to never write to disk (which were pretty slow in the
| 1990s), so everything was just a server to get network messages
| and then send out other messages in reponse and then assemble the
| replies/timeouts and send back a msg to the caller. All done over
| persistent connections established by the infrastructure, the
| applications just registered callbacks for msg type 'X' which
| would present one with the caller information, the msg as a
| buffer, and potentially a "user word" which would be a way to
| keep server state around between calls.
|
| The layering, from the main select loop thru different layers of
| TCP connection, SSL handling (if used, not 100% was SSL in the
| 90s), thru persistent link handling, application msg handling,
| timers, memory allocation, etc. was done with such art that I
| felt it to be a thing of beauty.
| mrpippy wrote:
| Fascinating, what kind of hardware/OS was used in production?
| lanstin wrote:
| Started with Tandem and Stratus pre Unix then migrated to
| HPUX then Solaris then Linux on stock hardware. Also Digital
| for big machines and early usage of 64 bit. I ported to Mac
| OSX but only for local dev.
|
| Funny thing about HPUX to Solaris migration is we found a ton
| of null pointer dereferences which didn't cause SEGV on HPUX
| but did on Solaris. And Linux migration was to a different
| endianness so had to fix any copying to/from ints to a
| buffer.
| tibbydudeza wrote:
| I worked at a bank where we used Tandem NonStop on Guardian
| OS - loved the programming environment using TAL language
| and the built-in SQL database.
| patrick451 wrote:
| Sadly, it seems like the moral of this account is that
| engineering quality is orthogonal to business success.
| winrid wrote:
| I wouldn't say it was a failure. They had a good run in a
| very quickly evolving time. That they got a slice of business
| for years during that period isn't bad!
| CPLX wrote:
| When did they fail? They acquired Time Warner, creating a
| company that after continued consolidation is now ranked
| #13 on the Fortune 500 list.
| foooorsyth wrote:
| AOL was an incredibly successful business. Nothing lasts
| forever.
| __loam wrote:
| The creative destruction of the business cycle grinds ever
| onward.
| huhtenberg wrote:
| Orthogonal implies independence, not inverse correlation
| (which is what I'm guessing you meant).
| justin66 wrote:
| I bet they meant what they wrote, since it makes more sense
| than what you're suggesting they meant.
| dmd wrote:
| Er, what? AOL was one of the most successful companies...
| ever. Things moved on, but this is like saying the roman
| empire was never successful.
| deanishe wrote:
| AOL was extremely successful in its day.
|
| They had 40 million customers paying $20/month 30 years ago.
| mtoddsmith wrote:
| AOL's primary business was dialup modems. Those died off
| after internet became ubiquitous.
|
| They pre-dated Yahoo and Google as an index of the internet.
| vmsp wrote:
| Is this it?
|
| https://aolserver.github.io/
| edgyquant wrote:
| Can't be the same thing GP is talking about as the repo says
| it's multithreaded and the poster said what they worked on
| wasn't
| mtoddsmith wrote:
| Might have been updated later on.
| mtoddsmith wrote:
| Wild seeing "19 years ago" as the last commit time.
|
| https://github.com/aolserver/aolserver
| jsrcout wrote:
| It's refreshing to read the replies here as I am currently
| working in the worst codebase I've ever seen in my life. It's
| just unimaginably bad.
| mythrwy wrote:
| You must be the guy who replaced me at my last job.
| tonymet wrote:
| Facebook but for different reasons than most. We called this
| swapping out the airplane parts mid-flight
|
| A lot of effort goes into language design and tooling to enable
| continuous migration of code. Rather than re-writing entire
| repos, existing code is continuously upgraded through semi- &
| fully-automated code-mods. Every day thousands of patches are
| landed to upgrade APIs with new type safety, security
| restrictions, deprecations and other code maintenance.
|
| Most other company repos I worked on had major re-writes outside
| of the mainline until one day there was a sudden and often
| troublesome switch-over to the new code.
|
| Code is constantly changing, and unless you have a continuous
| process for landing and testing those changes, you are going to
| suffer when you try to apply the changes needed to address the
| accumulated tech-debt.
| lanstin wrote:
| I really wish more people believed in continuous improvement
| and the malleability of software. Giant heroic refactors or
| redos are so annoying, but so are ten year old compiler or
| kernels.
| ceejayoz wrote:
| > We called this swapping out the airplane parts mid-flight...
|
| I work fairly heavily with Facebook's Graph API, and I think
| they may have employed too many Boeing engineers for the part
| swaps lately.
| tonymet wrote:
| Most of the compatibility issues over the past few years at
| least have been due to added privacy restrictions and reduced
| permissions scoping.
| ceejayoz wrote:
| The same few years have seen the developer bug tool closed
| down in favor of the mostly abandoned "community forums" no
| one uses.
|
| We had big multi-year bugs in "assigned" state before its
| closure, too.
| tonymet wrote:
| That's true too. The developer api is no longer a major
| growth generator as it once was. What remains is the bare
| minimum to keep partners happy, typically ads vendors
| fragmede wrote:
| > We called this swapping out the airplane parts mid-flight...
|
| At ksplice we swapped the code out from under the kernel
| without rebooting, so I know what you mean.
| 01HNNWZ0MV43FF wrote:
| For curious readers, the "building a plane as it flies"
| metaphor dates back to at least 2000, to these commercials for
| EDS: https://www.youtube.com/watch?v=L2zqTYgcpfg
|
| I thought it was IBM but I was quite mistaken, so good thing I
| looked it up lol
| mrazomor wrote:
| In my case, it was the integration testing framework built for a
| large Python service.
|
| This was ~10y ago, so my memory might not serve me well. A bit of
| context:
|
| - proprietary service, written in Python, maaany KLOC,
|
| - hundreds of engineers worked on it,
|
| - before this framework, writing the integration tests was
| difficult -- you had a base framework, but the tests had no
| structure, everyone rolled out their own complicated way of
| wiring things -- very convoluted and flaky.
|
| The new integration tests framework was build by a recently
| joined senior engineer. TBF, it's wrong to say that it's was a
| framework, if you think in the xUnit sense. This guy built a set
| of business components that you could connect & combine in a
| sound way to build your integration test. Doesn't sound like
| much, but it significantly simplified writing integration tests
| (it still had rough edges, but it was 10x improvement). It's rare
| to see the chaos being tamed in such elegant way.
|
| What this guy did:
|
| - built on top of the existing integration tests framework
| (didn't rollout something from zero),
|
| - defined a _clear semantic_ for the test components,
|
| - built the initial set of the test components,
|
| - held a _strong ownership_ over the code -- through the code
| review he ensured that the new components follow semantics, and
| that each test component is covered by its own test (yep, tests
| for the test doubles, you don 't see that very often).
|
| Did it work well longterm? Unfortunately, no. He stayed
| relatively short (<2y). His framework deteriorated under the new
| ownership.
|
| Travis, if you are reading this and you recognized yourself,
| thank you for your work!
| MathMonkeyMan wrote:
| I wonder if this is a tragedy of "the org wasn't ready for his
| solution" or more of a "if I were to go back now, I'd notice it
| wasn't so good."
| bornfreddy wrote:
| Why only those two options? It could be just "the company had
| other priorities and developers, while they appreciated
| Travis's work, didn't find it worthwhile to carry the torch
| themselves". Having a great testing framework is just one of
| the things that need devs' attention.
| MathMonkeyMan wrote:
| Fair point, it's a false dichotomy; though your example is
| close to "the org wasn't ready for it," which is itself
| vague.
|
| Maybe the issue is that it easily comes apart without a
| dedicated censor, so to speak, and nobody wanted to have
| that role.
| f1shy wrote:
| Without knowing anything, and before reading this comment, I
| had the feeling of nr. 2. Strong feeling. I think the reason
| is ,,many klocs of Python". I have developed alergy to big
| Python programs. I like python for small things, probably
| wrapping some C code.
| bboygravity wrote:
| Why would KLOC in Python be worse than in any other
| language?
| munch117 wrote:
| Because you're doing more work per line.
|
| A 20 KLOC Python program is not more complicated than the
| 100 KLOC C program performing the same function. Quite
| the contrary. But if you're comparing 100 KLOC of C to
| 100 KLOC of Python, the Python program may seem unwieldy.
| yosefk wrote:
| So 500 KLOC of C doing the same thing would be better?..
| munch117 wrote:
| Not at all.
|
| But when you come across 500 KLOC of C, then you say "wow
| this is big" and you're forewarned that this is going to
| be unwieldy. You may underestimate the 100 KLOC of
| Python. That's all I'm saying.
| mrazomor wrote:
| It's about the different ways the language allows you to
| shoot yourself in the foot.
|
| I worked on large Python, C++, Java & Go services. I have
| 10y+ of experience with the first 3. C++ allows you to
| write incomprehensible code (even to the experienced C++
| devs) and justify its existence (because of the
| performance gains). But you need to be a top expert to
| write a compileable code of that type. I'm comfortable
| with diving in any C++ codebase except for the libraries
| like std, boost, abseil, folly, etc. Most of the code
| there is absurdly difficult to comprehend.
|
| On the other hand Python leads in the ways a junior dev
| can introduce hell in the code. Especially if the team
| doesn't rely on the strict type check. I have seen
| horrors.
|
| I was bewildered when I realized that working with
| JavaScript with type checks (Closure compiler) was
| insanely more productive and smooth than working with
| Python (before the type checks).
|
| That's why Java won the enterprise world. It takes an
| effort to make a mess in Java (but people still manage).
| Go is in a similar place.
| PartiallyTyped wrote:
| > Did it work well longterm? Unfortunately, no. He stayed
| relatively short (<2y). His framework deteriorated under the
| new ownership.
|
| I think the issue is that integ tests are not really a place
| that sees "development". You write the integ tests, and move
| on, you are not actively introducing more integ tests.
|
| I think that's a shame that this is the common view of the
| people that need to fund this. Bad integ tests mean bad dev
| exp, which then results in increased attrition and
| dissatisfaction.
| cwbriscoe wrote:
| One that I built and maintained 100% on my own. Nothing like
| doing things the way you like to without someone else messing
| everything up.
| antishatter wrote:
| Had to work through a weirdness in kubernetes many years ago when
| using it on prem for a project (not at google). It stands out as
| the best code base I have had to go through to figure stuff out.
| cvwright wrote:
| libjpeg 6 is pretty good. It's oldschool C so it's pretty
| spartan, but it's easy enough to read it and follow what it's
| doing if you have a basic understanding of the concepts.
| (Discrete cosine transform, Huffman coding, etc)
|
| And it's only had a couple of vulnerabilities, in nearly 30 years
| of being on the Internet. That's not quite like DJB code, but
| it's darn close.
|
| My understanding is that it was written by a very small number of
| people who were experts in the field, as part of the standards
| process way back in the early 90s.
| brynet wrote:
| OpenBSD. Good people, consistent style, correctness.
| JensRantil wrote:
| The first few years at a previous startup I worked on. Java.
| CI/CD in place when merging PRs. But mostly, the thing that made
| the code base great was the fact that the code wrapped external
| libraries in small interfaces that were _owned by the company_,
| and dependency was used to not depend on implementation
| interfaces. In short, adhering to the basics on the SOLID
| principles.
| meowtimemania wrote:
| Kinda sounds like abstraction hell. What did you gain by
| wrapping all external libraries in interfaces?
| oasisaimlessly wrote:
| In my experience, any third-party dependency will inevitably
| eventually be in-sourced. Third-party dependencies are
| technical debt - they let you make fast progress initially,
| but have recurring maintenance costs.
|
| Wrapping the API lets you see, all in one place, the surface
| area of an external API that is in use, and minimizes the
| changes required here when reimplementing it.
|
| ---
|
| Also, not that Clean Code is any great authority (but it is
| popular!):
|
| > [Wrapping] third-party APIs is a best practice. When you
| wrap a third-party API, you minimize your dependencies upon
| it: You can choose to move to a different library in the
| future without much penalty. Wrapping also makes it easier to
| mock out third-party calls when you are testing your own
| code.
|
| > One final advantage of wrapping is that you aren't tied to
| a particular vendor's API design choices. You can define an
| API that you feel comfortable with.
| kristopolous wrote:
| NetBSD
| OnlyMortal wrote:
| Used: the Boost C++ libraries. In particular ASIO. Variant has
| also been very useful.
|
| The Mac GUSI BSD socket library is worth a mention too. I built a
| multithreaded Quark XPress XTension with it that used PowerPlant
| for UI. Quark even wanted to interview me.
|
| Added to: a pre-Boost cross platform library for Mac (Classic)
| and Windows NT. It did much of what Boost did way back when.
| elwell wrote:
| ClojureScript + Re-frame + GraphQL subscriptions over WebSockets:
| https://github.com/Vetd-Inc/vetd-app
| fooker wrote:
| The LLVM infrastructure.
|
| Other compilers don't even come close.
| sunir wrote:
| I worked on an incredible C++ codebase back in the dot.com. We
| built a mobile SVG renderer and packed into 128kb on a Palm 3 and
| similar code sizes on Blackberries, Symbian (the worst OS ever),
| Fujitsu phones, and whatever you threw at us.
|
| I built a few systems. The DOM, a build system that could cross
| compile to a completely new target phone while the phone
| executives were at lunch with our sales team (that's great
| demo!), and an optimizing build step in Perl that rewrote the C++
| to minimize bytes, cycles and joules of energy.
|
| The rendering engine was built by a genius. It was the most
| efficient and accurate vector graphics renderer I have ever seen.
|
| But of course no one could understand how this system worked.
|
| I mean, Perl that transpiles C++ into more optimized C++ just is
| abusive. I was not popular with the senior devs.
|
| The main coders including me were coop students who worked
| furiously at all hours for the fun of it.
|
| But of course the product had no revenue. The wiser devs probably
| knew this and didn't care.
|
| Nevertheless, we eventually were let go during the dot.com bust
| and the remaining main engineers had to throw the code out and
| try to rewrite it in Java. That didn't work because in that era
| Java was too slow to fit on a phone. Or maybe the market was
| dead.
|
| I learnt a solid lesson.
|
| What's amazing technically is not an amazing code base.
|
| What is an amazing code base isn't an amazing product.
|
| What is an amazing product isn't an amazing business.
|
| Literally every product I have ever seen that is profitable is
| not a good code base. But if it pays me to code, I love it.
| jeffrallen wrote:
| The Go standard library. The level of detail, the politeness of
| the reviews, the fact that the features of the language allowed
| me to put more attention into correct code than "at least it's
| working" code, the pragmatism and personalism of the core team.
| malkosta wrote:
| The best codebases I worked on were from startups that failed due
| to lack of product market fit. Oh man, their code was so
| beautiful and optimized...
| BobbyJo wrote:
| Is this true or a bit of humor? Of the three startups I worked
| for, the one that had the best exit also had the best code. Ime
| shitty code hamstrings startups into slow execution during the
| growth phase, and kills them despite pmf, because competition
| winds up moving much faster.
| yobert wrote:
| I tried to add a new SQL function to CockroachDB. I had never
| seen the source before, but since it was such nice Go code, it
| only took me a few hours to have a working build, with tests.
| jchrisa wrote:
| CouchDB's HTTP API used to fit in just one file (less than 1k
| lines of Erlang), and it was a great jumping-off point for
| learning databases and web programming:
| https://github.com/apache/couchdb/blob/0.8.1/src/couchdb/cou...
|
| It was written by one engineer, and then later refactored by a
| team to look like this (and many other files):
| https://github.com/apache/couchdb/blob/main/src/chttpd/src/c...
|
| It's an interesting exercise to see how something grows and
| changes as it transitions from inspiration to real-world usage.
| phito wrote:
| So little blank lines, I need to breathe
| CSMastermind wrote:
| I've seen enough code bases that I feel like I have a good
| spectrum from truly exceptional to horrific.
|
| The best code bases had these things in common:
|
| 1. _Consistency_ : they had clear patterns that were followed. It
| doesn't matter what the pattern was; some were strictly object-
| oriented, some were purely functional, some used RESTful API, and
| others leveraged gRPC or GraphQL. The important thing was that it
| was all consistent across the codebase so even if you were
| looking at a part you'd never seen before you could still orient
| yourself and reason about it quickly.
|
| 2. _Ownership_ : there was always a single individual person who
| was ultimately responsible for each section of code. That doesn't
| mean they wrote all the code themselves; they were the final
| arbiter on the rare occasion a truly contentious conflict arose.
| They also had the unilateral power to make changes within their
| section of the code if they felt it was needed. This was always a
| rarely exercised power but they could, if they had to, push a
| change through. There could be many such people spread out across
| the codebase but for each discrete part the number was always
| one.
|
| 3. _Clear Boundaries_ : it was clear what each part of the
| codebase's purpose was, and the boundaries were rigidly enforced.
| Things were never tightly coupled across these boundaries.
| Business logic was always isolated from things like
| serialization/deserialization, each system was forced to maintain
| its own models of the world that it cared about.
|
| The worst code bases had these things in common:
|
| 1. _Lack of Eng Representation_ : Product controlling development
| schedules and insisting everything is high priority and needs
| done right now! Project managers who always wonder, "yes but what
| if 9 women could make a baby in a month. Why don't we try adding
| more resources? Can we just try?" Business types who see software
| as a cost center not a profit driver. This can also happen if
| your engineer leadership didn't come up through software
| engineering but rather QA or IT or started out in academia or is
| just plain and MBA type with no eng background.
|
| 2. _" Time saving" Tech_: We don't need a database schema, we can
| just go NoSQL and have JSON blobs, it will save so much time! We
| can share models between the front end and back end, it will save
| so much time! Don't write SQL, use this ORM, it will save so much
| time! Don't think about DevOps, just use this all in one hosting
| solution, it will save so much time! What about this low code/no
| code solution? It will save so much time!
|
| 3. _Misaligned Incentives_ : This could be because they were
| contractors with no stake in the company or this could be because
| it was a large company and there was no realistic way they'd ever
| be fired. Either way there were no consequences for writing bad
| code.
| jokoon wrote:
| * medical imaging software to drive a X ray machine
|
| * software was relatively new, and well made. age is probably the
| main reason why it was good "at the time".
|
| * there were good practices in place, yes, led by an opinionated
| team lead, which felt like it was the main cause for such a
| project to work well under medical standards.
|
| I feel like I will never find another job like this one again.
| cbanek wrote:
| The Xbox codebase was pretty good. We were doing enforced code
| reviews back in 2011-ish, which was a big thing at the time.
| There was also a good layer cake going on and a few good
| architects.
|
| The worst codebase by far was Outlook for Mac. It had code going
| back to Entourage, and was never properly cleaned up.
| ruff wrote:
| Ha, if you think that was bad, you should have been there when
| it was just Entourage ;).
| christophilus wrote:
| Office in general is a mess. I think it's a good example of the
| fact that the sales team / marketing power of a business is the
| real driver of success, much as that pains me as a developer.
| cbanek wrote:
| I feel that Office's trouble is always backward
| compatibility. Which makes sense.
| raggi wrote:
| Code wise, probably Fuchsia. I worked up and down the Fuchsia
| stack regularly, from bootloaders to GUI programs and all over
| in-between. For a sizeable code base with so many layers and
| concepts, and regularly moving between these without much memory
| for what's there (as it also moved around quite quickly in my
| time), it was the lowest average cost of entry to new areas of
| most other millions of lines of code repositories I've dealt with
| in my career. Still plenty of bugs came up, lots of polish and
| more design work to do, it's not at all to over-sell it, but
| almost everything was approachable with low impediment.
|
| There were also lots of Ruby libraries back in the hayday of the
| language that were very clean for what they are. I've rarely seen
| programs structured so well since - some of this is essentially a
| forced practice by the language - if you don't do it, you'll
| suffer as there's no types, lsps and whatnot to otherwise save
| you, but still, it's worth calling out, as I think back
| occasionally with fond memories of being unimpeded anywhere I
| needed to go.
|
| Tooling wise it's got to be noted as others have that the Google
| monorepo was well integrated, but it wasn't what I'd describe as
| "best" at all. I had a fairly broad time at Google having worked
| in most of the sub orgs over the years and in doing so saw a lot
| of the variance even with the chosen integrations. I saw teams
| who regularly built from no where near head cherry picks out of
| the repo, I ended up owning stuff that always built off of the
| main branches (most Googlers don't even know exist) setup before
| my time to avoid running into otherwise hardline constraints
| around single version policy that couldn't be adhered to for
| specific product reasons (hi cloud). Exactly how much and which
| parts of buganizer teams used varied hugely, which service
| management tools they used varied, which database systems, etc.
| The single patch system while also fairly well integrated
| combined with code review practices that often stymied medium
| scale refactors (large refactors within a single product area),
| though making large scale changes that touch every single top
| level directory, while a bit rough on the edges of the tooling,
| the tooling existed and worked really well - pushes out hundreds
| or even thousands of patches and automates pinging owners and
| informing you of failures, etc. In many ways a lot of smaller
| systems with far less tooling, and far far less integration were
| more productive - I've never seen such a bad way of _organizing
| work to be done_, or such a poor volume management of bugs as I
| have at Google in various teams/areas, despite all the tooling.
| Any delivery that involved client programs (desktop programs,
| mobile programs, etc) was always pretty remarkably awful, and
| only very very small teams were on the hook for making any of
| this better. They'd always plaster their new thing with "don't
| you dare make another or use the deprecated thing" meanwhile they
| never had enough time to make the new thing ready - a common
| trope in the company, but really highlights there are edges
| outside of the "pure saas" side of the business which were pretty
| bad. The best part of the monorepo and monotooling culture for
| the saas side is the security properties of it all, which were
| outstanding, and are among the reasons why I still trust Google
| as the holder of some main accounts. I hope they never start to
| lose those properties given the scale and impact they could have.
| didip wrote:
| Many of the large Go codebases. Including Kubernetes.
|
| It is super easy to find what I am looking for.
| rrdharan wrote:
| I work at Google on database systems and would still pick the
| VMware codebase (specifically the virtual machine monitor
| component).
|
| It was by far the most impressive piece of software engineering
| I've ever had privilege of perusing.
| bitwize wrote:
| I haven't contributed to it yet, but one of my personal projects
| involves hacking NetBSD. Absolutely, legendarily good code base,
| so thoroughly documented you can actually begin kernel hacking
| just by _reading the man pages_. The code itself is really
| consistent and easy to read; it 's pretty clear the NetBSD team
| enforces a consistent style as well as high standards for
| thorough documentation. NetBSD is how C code _should_ be done.
|
| The other code base I really liked was QRes, ITA Software's
| airline reservation system written in Common Lisp. I contracted
| at Google a few years ago doing maintenance on this code base
| before it was shuttered for good. Aside from being just high-
| quality Lisp code, it had its own testing language that allowed
| new tests to be written in just a few lines or so -- tests that
| sent XML queries with generated data to the running system and
| checked for specific results in specific XPath paths. Because the
| system was Lisp, implementing and extending this testing language
| was relatively easy. Using it even more so. Truly the only system
| I was happy to write tests for, rather than seeing it as a
| necessary chore.
| fnordpiglet wrote:
| SecDB at Goldman. There were runtime issues aplenty but the SDLC
| was top notch and I've never heard of or seen better - and it's
| 30 years old. Instant deployments and rollbacks globally, you
| show up to work and commit to prod in the first half day. Within
| a week you can go on rota. It's why Goldman was so nimble in the
| 90's, and 00's. It's still a remarkable system but
| underinvestment has taken its toll after margins evaporated in
| 2010.
| arriu wrote:
| Blender is incredible, harfbuzz equally so, pretty much all
| modern browsers do the impossible so they need to be included as
| well.
| bogwog wrote:
| Open 3D Engine's codebase is pretty great: https://o3de.org/
| zombiwoof wrote:
| Java SDK 1.0
| codezero wrote:
| I'm not a software engineer, but the best codebase I worked on
| was at Quora. I attribute this to them having several years of
| time to build a base that was solid. That said, I found plenty of
| bugs, but it was just so nice to work with. Almost everything was
| Python. DOM, CSS, JS embedding, etc... keeping in mind this was
| way before Angular/React, but the system worked very similar,
| doing DOM diffing via a C++ service on the backend and pushing
| updates out over a long polling JS endpoint.
|
| On top of that the data which was read heavy went through several
| layers of cache that were very easy to understand (process,
| system, global - called L1, L2, L3 like a system as a CPU). The
| front-end had a "Konami code" that let you see the rendering
| hierarchy from the backend while viewing it from the frontend
| that let you also understand the caching state of each level of
| hierarchy that was rendered as well as all the CSS/JS handlers
| affecting the element, not unlike current "inspect" view of most
| browsers.
|
| What made this all so good? A very strong foundation that was
| opinionated, but which was consistent. This was ~ 2012-2014 -
| they had Flask as an underlying framework, and much was built out
| on top of Python decorators.
|
| Besides the day-to-day code - they had a bespoke packaging
| system, auto-scaler, and a number of other services that "just
| worked" - it was very different from any other startup I've
| worked for.
| ryukoposting wrote:
| This is going to sound insane, but I worked on a firmware
| codebase at an industrial automation company that was remarkable.
|
| Everything was working against it. No RTOS. Subversion. GCC
| 5-point-something (I think?).
|
| It was an incredible mass of preprocessor crimes. I'm talking
| about #including the same file 10 times and redefining some
| macros each time to make the file do something different.
|
| It used a stackless coroutine library called Protothread, which
| itself is a preprocessor felony.
|
| And yet? It was brilliant. Compilation was lightning quick. F5,
| lean back in your chair, and boom, you're running your code. I
| understand that this kind of thing is normal for web/backend/etc
| folks, but I yearn for the days of sub-15 second firmware compile
| times.
|
| It was easy to flip a couple flags and compile it to a
| Win32-based simulator. Preprocessor felonies are felonies, but
| when you stick to a small handful of agreed-upon felonies, you
| can actually reap the benefits of some very sophisticated
| metaprogramming, while staying close enough to the hardware that
| it's easy to look at a datasheet and understand exactly what your
| code is doing.
| jhayward wrote:
| > Preprocessor felonies are felonies, but when you stick to a
| small handful of agreed-upon felonies, you can actually reap
| the benefits
|
| There's a saying, "Only commit one crime at a time". Knowing
| when to cheat is a fairly high-risk but pro move.
| agentultra wrote:
| I never got to submitting my own patches but the Linux kernel is
| nice.
|
| Where I work now I get to work in Haskell. And that's been a
| pleasure. Recently rewrote over 1k lines. Deployed to production
| with one reviewer. No issues.
|
| Just good test coverage and a great type system with no-nonsense
| code. Refactoring in Haskell is a heady drug.
|
| If this was C++ or even Python there would have been much more
| intense reviews and probably some fail-forward patches to fix
| things we may have missed.
|
| But I dunno. Maybe not. Maybe I'm just that good. /s
| sgtnoodle wrote:
| A former coworker texted me out of the blue yesterday, saying he
| missed working on a "<my name> codebase". He specifically
| appreciated that "doing something simple things were kept
| simple." Made my day!
| Joel_Mckay wrote:
| It was a 63kB assembly project that compiled down to 384 bytes.
| However, that project handled phase-locked control of a power
| dimmer, logged up to 2 days of power-factor stats information
| into an eeprom, integrated clock timers, and several user inputs.
|
| It was the smallest, and most bizarre interleaved task-structure
| I'd seen fit into a $0.17 micro-controller. I kind of admired
| that the code itself formed the delays for other areas of the
| multiple tasks, and essentially achieved near perfect resource
| utilization for actual work.
|
| Never saw anything that efficient again for several decades. =3
| theusus wrote:
| Worked on a domain driven design code. It was simple and
| intelligible. Now, I want to copy that methodology everywhere.
| Aeolun wrote:
| I'm firmly convinced that any codebase written by any 4 people in
| the same room is better than any written by any higher number or
| distributed.
| tinco wrote:
| You'll be surprised, I think the best codebase I ever
| contributed to is the Rust compiler, which is being built by a
| huge distributed team.
|
| Despite it being complex and me being in there for the first
| time, I could quickly track down a bug in the parser and fix
| it. As a cherry on top the communication around the pull
| request was top notch too.
| Aeolun wrote:
| I'm not trying to say it can't be good despite that factor,
| but if the rust compiler were written by the top 4
| contributors sitting in the same room, it'd be even better
| (and possibly not finished yet ;) )
| whoknowsidont wrote:
| You could at least contribute to the thread instead of
| grandstanding about an opinion you have.
|
| Name the codebases.
| f1shy wrote:
| I've seen plenty of examples of exactly the opposite.
| bboygravity wrote:
| So NVIDIA must write terrible code?
| Aeolun wrote:
| That would not surprise me. There's a reason they need all
| that hardware.
| sensanaty wrote:
| It took my company a team of 8 fully-remote devs to fix the
| clusterfuck of a codebase that the "4 guys in a room" (the CTO
| and early devs) built up with spit and tree bark,.
| JSDevOps wrote:
| The entire world disagrees, Windows, The Linux Kernel, Anything
| at Netflix, Google, Facebook etc. Try getting them all in a
| room.
| unkulunkulu wrote:
| The one I'm working at right now!
|
| What made it great: the company is 7 years old, I work basically
| since inception, half year late.
|
| The codebase had some serious problems due to migrating build
| systems, rapid team growth and lack of proper communication while
| developing important infrastructure tools, lack of foresight at
| the start of a crucial requirement for the whole system that
| became clear 3 years in.
|
| Aaand... Due to unique sequence of events half a year ago we had
| to start everything from scratch! We could not use a line of code
| from the old repo !
|
| This brought miracles out! We started designing the right things
| right. People started talking to each other on the global scale,
| without the code limiting them!
|
| The old code not only had problems on the engineering level, but
| it held people and team responsibilities along suboptimal
| interface boundaries! This is the biggest magic! After the
| rewrite started, all the people in the team landed exactly where
| they wanted and their areas of responsibility became clear and
| highly efficient. Some people started shining like I could not
| imagine. Because their organizational and communication skills
| were limited by a huge suboptimal codebase.
|
| Also, it is very empowering to design a system when your whole
| team has real experience 5 years into the future. Also, everyone
| turned out to be caring about code simplicity and quality, I
| finally feel at home.
|
| I could not wish for a better developer experience. The sexiest
| C++ coding time in my life :)
| pcatach wrote:
| Interesting! Seems like you were uniquely well-positioned to
| tackle the infamous system rewrite in a way that worked out
| great.
|
| Fixing the interface boundaries is a strong case for it despite
| all of the downsides (mostly opportunity cost, I'm thinking).
| ephaeton wrote:
| NetBSD.
|
| The docco, the culture, the clarity and simplicity of design from
| first principles. The coherency of code across kernel, user-
| space, accompagnying material. The vibe.
|
| You may know NetBSD (if, at all) as the BSD that wants to be
| "portable" and may dismiss it against FreeBSD's focus-claim of
| "performance" and OpenBSD's focus-claim of "security".
| Interestingly, trying to remain wildly portable requires a
| cleanliness inside that is very soothing.
| ajay_san wrote:
| Question, I want to get into more lower level programs like
| compositors or docker daemon or kernels (I do realize they all
| are different things) but I find tech behind them so
| intimidating and don't know where to get started. Do you have
| any advice where I can learn them? I'm about to grad next year
| spai2 wrote:
| https://stevens.netmeister.org/631/
| v3ss0n wrote:
| Litertar.dev and Tornado web . Litestar code base, architecture
| and core design is what modern python code should be. Very clean,
| fully type annotated, DTO, and many cool stuff happening there.
| m00x wrote:
| redis's C code is incredible
| purple-leafy wrote:
| My own, lol. Which just means I need to work on more codebases.
|
| I'm making a tool to convert data schemas to SQL via a UI for
| lay-users. Just like https://react-querybuilder.js.org/ which is
| basically a UI based SQL generator. For work.
|
| Except my version extends the idea much further, blending Excel
| like functionality with functions that can act on Fields and
| Rules.
|
| What makes it good?
|
| For one, it's a from scratch project with almost no third party
| libraries.
|
| For two, I fortunately chose a recursive data structure to
| represent the data schema, and that has really worked out well.
| Early on I tried 4 other approaches to represent the data, but
| went back to recursive feeling it was the best choice.
|
| Furthermore I'm using React, but it's heavily leveraging
| reducers. Specifically Groups have a reduced, Rules have a
| reducer, and Fields have a reducer. The reducers are linked in a
| chain top to bottom, where changes on a Field change a Rule, and
| changes on a Rule change a Group. It's been extremely clean to
| work with.
|
| Because the base data schema is recursive (Groups contain groups
| contain groups), most of the functions that manipulate the schema
| are recursive. There is a real elegance to the code, each
| recursive function has a very obvious base case, and a very
| obvious recursive path.
|
| And for the final outcome, walking the query data structure and
| spitting out the equivalent SQL is also recursive, and feels
| elegant, coming in at under 40 lines.
|
| Literally as I've been writing this codebase, everything somehow
| perfectly fell into place. I was marvelling near the end that it
| felt like I chose all the best possible logic paths to build this
| somehow.
|
| I'm hoping to get the okay from work to open source it (fully
| open source). The only cruft of the project is the types I'm
| using, the interface of the code could be improved with generics
| w10-1 wrote:
| Java, because: Mark Reinhold
|
| He learned from the best.
| lolive wrote:
| Not wanting to start a flame war, but the appearance of jquery
| has been an eye-opener for me. The official API of the Web was
| instantly outdated thanks to the incredible cleverness of a bunch
| of developpers. Wow ... that was, for me, exactly what open
| source was all about.
| somenameforme wrote:
| Unreal Engine, by far.
|
| It's not only just extremely well written in general (with your
| only chance of 'really' learning the engine to go through it,
| read comments, and the like) but it also defies the wisdom
| 'everybody knows.' That wisdom being that premature optimization
| is the root of all evil - like we're supposed to just benchmark
| things, see where the bottleneck is and then try to work on
| optimizing spots like that.
|
| Unreal doesn't do this. There are countless thousands of micro-
| optimizations everywhere. For one striking example there's even a
| fairly substantial system in place that cache the conversion
| between a quaternion and a euler rotation. This would never, in a
| zillion years, be even close to a bottleneck. But with thousands
| of these little micro-optimizations everywhere you get a final
| system that just runs dramatically better than any comparable
| engine.
|
| In more general terms, they've also taken advantage of the
| 'idiomatic flexibility' that C++ offers to create a sort of
| Unreal C++ language that is also just lovely to use and feels
| much closer to something like C# in terms of luxuries like the
| lack of manual memory management, garbage collection, reflection
| (!!), and so on. The downsides are that compile times are
| horrible (even though a cached compile might only take 10 or 15
| seconds, it feels like _forever_ when trying to work out one
| specific issue) and C++ intellisense, especially in a
| preprocessor heavy environment, is pretty mehhhh.
| 01HNNWZ0MV43FF wrote:
| > there's even a fairly substantial system in place that cache
| the conversion between a quaternion and a euler rotation
|
| I hope that does show up on a benchmark though, cause that
| sounds complex
| ecuaflo wrote:
| The one I made from scratch after iterating over 3 years. You can
| see the technologies here [0]
|
| [0] https://gitsell.dev/u/bitofbreeze/r/bitofbreeze/git-sell
| hightechnical wrote:
| For me best code base I've read so far is Redis code. Feels like
| a poem to me.
| liampulles wrote:
| The best codebases have been my own, because I can monopolize the
| style and design to my own exacting vision of how things should
| be.
|
| The best work codebases are ones where I may disagree with a lot
| of the style, but the lack of enforcement at the level of classes
| and functions gives us creative freedom, builds mutual respect,
| and enables ego-less, pragmatic, tradeoff-driven discussions
| about things which are actually important. If these discussions
| end with a majority but not universal agreement, then its fine,
| the minority is not offended, and we happily continue with our
| work.
|
| That might also be because we're all pragmatists and somewhat
| cynical of shiny things, or the latest fad.
| jhon-crypt wrote:
| The best codebase I've ever worked on was when I started software
| development officially, I had zero knowledge of the stack for
| real, the codebase was run by just 5 mid level engineers, but
| based on how well structured and understandable it was, i was
| able to pick up the stack without reading a tutorial or a
| documentation, i would later read up on it when i started doing
| complicating stuff tho, but it's important for engineers to write
| good code to the extent that another person reading it does not
| have to look up something just to understand why this does that,
| and how stuff works in the code
___________________________________________________________________
(page generated 2024-06-29 23:02 UTC)