[HN Gopher] Repeating Yourself Thrice Doesn't Turn You into a 3x...
___________________________________________________________________
Repeating Yourself Thrice Doesn't Turn You into a 3x Developer
Author : vira28
Score : 102 points
Date : 2023-07-30 16:24 UTC (6 hours ago)
(HTM) web link (yrashk.medium.com)
(TXT) w3m dump (yrashk.medium.com)
| andybak wrote:
| He must be using a fairly crappy tech stack for the categories
| feature to be as complex as he makes out. For the sites I've
| worked on (Django/light js for progressive enhancement) it's a
| hell of a lot simpler. Either he's exaggerating or we've truly
| gone backwards from the halcyon days of Django/Rails
| vvaibhav_desai wrote:
| it turns you into a recursion XD
| anonuser123456 wrote:
| Loop unwinding is a compilers job.
| holmesworcester wrote:
| The folks at Braid (braid.org) and Ink & Switch
| (inkandswitch.org) think part of the answer (at least for team
| collaboration applications) is to use CRDTs to mirror frontend
| state between devices collaborating on a dataset, making the
| backend mostly just one more device, maybe using encryption to
| keep users' data private from the backend. For something like a
| kanban board or a collaborative document editing app I think this
| could work really well, though I'm not sure how it generalizes.
|
| People from those communities say it's a relief building this
| way, though they're building simple proofs of concept still and
| it's not clear to me how well the approach holds up in fully
| fleshed out products. But it does seem to make a lot of sense in
| situations where a lot of the work involves keeping a bunch of
| devices in sync with each other.
| Zetice wrote:
| Or you could autogenerate large swaths of this just from your
| schema.
|
| Is this entire article basically forgetting that as an option?
| nerdchum wrote:
| Yeah lol I was thinking the exact same thing.
|
| Autogeneration is a thing.
| t1mmen wrote:
| That's my thinking, too. I've recently been using ts-rest.com
| for a relatively small project at work (<20 API endpoints,
| NextJS frontend, Postgres). Its been such a joy writing the
| "source of truth" as API "contracts", and having everything
| else just work. With zero added effort, I get fetch/react-query
| clients 100% typesafe. Request & response validation on the API
| layer (which can easily be moved from eg NextJS API routes to
| Express or another framework). OpenAPI spec. Typescript and Zod
| types. All of that for free, without repeating myself. I like
| it a lot.
| mkl95 wrote:
| > Ultimately, what is just a tiny line of text at the top of blog
| posts for the users becomes a daunting task, representing tens of
| hours of engineering work to implement.
|
| Something I have noticed about Fowler-esque / Uncle Bob-esque
| codebases is that usually only the guys who wrote it understand
| how it works. Which is either a blessing or a curse depending on
| whether you wrote the thing yourself or somebody else did it. And
| it also seems to defy the point of "making it easy to swap
| implementations by writing a ton of interfaces".
| sch00lb0y wrote:
| For frontend people shipping application is easy because of tools
| like firebase
| lorenzotenti wrote:
| Are we repeating history though? I've worked for a company that
| used Oracle plsql for everything (shall we return html snippets
| from the database as a reactive frontend, why not!, the whole
| business logic is in huge stored procedures anyway) and it was
| clearly an utter mess. Now, new tools may make this better, but
| every time I see too much business logic getting close to SQL I
| get suspicious. Supabase is another example of doing everything
| with postgres. Sounds cool, but is it maintainable?
| city41 wrote:
| Supabase now has edge functions: https://supabase.com/edge-
| functions
| brtkdotse wrote:
| Tangentially, it's curious there hasn't emerged A Proper Way of
| version controlling and deploying stored procedures outside of
| "stick a bunch of sql scripts in a folder in the project root"
| vanviegen wrote:
| Is there anything wrong with that approach? It seems pretty
| optimal to me, since you'll probably want to commit the
| stored procedures together with regular code.
| brtkdotse wrote:
| Not really! It's just weird that every places I've worked
| at basically invents it from first principles rather
| knowing about it.
| eddd-ddde wrote:
| Exactly, the reason the other alternatives feel better, is
| not because of how they work, but because of the tooling.
| failuser wrote:
| I don't work with 3-tier applications so I was surprised by the
| solutions, I was expecting a single origin for the schema at
| least to eliminate the need to triplicate some code. Is that a
| deprecated approach?
| zlwaterfield wrote:
| With the advent of AI, a substantial portion of the laborious
| tasks involved in the 3-tier model will likely be automated,
| making it less likely for most to move away from this approach.
| In my opinion, the 3-tier pattern was established for valid
| reasons, and any attempts to simplify it by removing tiers might
| inadvertently constrain developers, leading them to eventually
| revert back to the original model.
|
| Regarding solo projects, I agree that simpler stacks like BaaS or
| other innovations can be sufficient. However, fast-scaling
| companies often require the unparalleled flexibility and
| customizations offered by an in-house 3-tier model. This tailored
| approach ensures they can effectively meet the evolving demands
| of their growing operations.
| lawn wrote:
| Using a magical construct to autogenerated the three instances
| also doesn't turn you into a 3x developer.
|
| Because they're never exactly the same, and you end up with heaps
| of special cases and handling and it would've been easier to
| write it three times from the beginning.
|
| And even if they start out as exactly the same, in any non-
| trivial codebase that won't hold true for long.
| moffkalast wrote:
| Yeah depending on the codebase size, it's often better to opt
| for some copied code and keep the ravioli encapsulation than
| trying to abstract everything into interfaces and layers of
| inheritance that just end up as a massive bowl of spaghetti as
| soon as requirements change ever so slightly.
| mjw1007 wrote:
| I think Master Kaimu agrees with you:
| http://www.thecodelesscode.com/case/97
| adra wrote:
| Business logic / rules are vertically integrated. You need your
| frontend, middleware, and databases to all align on how to store,
| transform and present information to meet business goals.
| Vertically developed software are the least efficient because you
| miss out on the core similarities of each vertical, so we use
| horizontally oriented frameworks that can reuse a lot of the
| boiler plate. Do you.nerd to add a cache layer later? With
| horizontally developed code, you can do that application wide
| with some annotations, properties, and library imports. If you
| wanted to do the same on a purely vertically developed code,
| you'd be changing N features with a bunch of duplication in each
| insertion point.
|
| One one winner with splitting tech on horizontal boundaries is
| that changing a feature is a largely high cohesion change. All
| the code bung updated in that commit are related to one another,
| and despite the fact that there are "many" places that the code
| needs updates, at least they all relate to one another.
|
| There was some effort in the java community to meet the problem
| half way with something called point cuts. This allowed some
| level of contracts which you could "insert behaviour into all
| instances of X" which had some success, but I haven't see it in
| the wild for a while, so I'm not entirely sure it survived.
| noduerme wrote:
| I just don't buy it about adding a "category" field to a blog.
| Add the db field to production and make it defualt to null. Did
| you write your query to SELECT * instead of the fields you
| wanted? Tisk. Okay, fix that. Add the property to your back and
| front end. Don't paint the html if it's null. Optionally make a
| 'categories' table and do a join. 30 minutes of work, max.
|
| If you're writing code where the front or back end data objects
| will break if you add a new db field, you're doing something
| wrong.
| fatnoah wrote:
| Ah, yes, the old "look how easy building your simple CRUD app in
| [new tech] is" article. These always seem to work great (and do
| work great for some use cases) until things evolve beyond, and
| then one spends their day fighting the technology instead of
| actually building the product. Meanwhile, the n-tier dev you
| laughed at is still plugging away and getting some extra help
| because because the loose coupling between tiers made it easier
| to divide-and-conquer.
| forgetfulness wrote:
| ORMs when you have to do the most basic selects and joins, with
| naive pagination: look at how easy it is, it's magic!
|
| Also ORMs when you have to do anything more complex, specially
| if they involve aggregations: welcome to my awkward
| undocumented APIs, you now embark on a journey through hard-to-
| search-through class definitions and source dives that you'll
| share with every programmer that will touch your code in the
| future.
| semicolon_storm wrote:
| Every ORM I've ever used has some raw SQL escape hatch you
| can use when you hit that edgecase. For the 90% of DB access
| that really is simple, ORMs are a pleasure. For that other
| 10%, if your alternative is raw SQL, just use the escape
| hatch and you're not worse off than if you had skipped the
| ORM.
| mrits wrote:
| You can also map or redirect the objects to database views
| javcasas wrote:
| Ooh, database views! What would be next?? Permissions?
| Triggers?? Stored procedures??
|
| Go shove your views up yours, you maniac!
|
| /s, except for way many more ORM lovers than you think.
| pessimizer wrote:
| Plenty of people who use ORMs have absolutely no ability to
| even begin to use the "raw SQL escape hatch." People who do
| know SQL know exactly how their ORM performs, and know
| exactly when and how not to use it.
| whstl wrote:
| Yep. But also, I've met plenty of people that know SQL
| but have an aversion to using inline SQL escape hatches
| out of purity.
|
| One fun case I witnessed involved a junior developer
| adding the desired/resulting SQL as a comment to every
| complicated Rails AREL queries, so that people could know
| what the query was doing.
|
| Then, after seeing that, one of the tech leads determined
| that EVERY query should have SQL on top of it, for
| consistency, even things like User.all had the `SELECT *
| FROM users` on top.
|
| In hindsight it's funny but it was a terrible team and a
| terrible software.
| beesnotincluded wrote:
| I don't know how to react to this. It seems like the author
| trivializes the task to prove a point. It is never just a
| 'category'. Wrapped up in that is a whole bunch of functionality
| and expectations that always differ between projects. For example
| users want to search by, edit, manage and delete categories. Who
| should have permission to change them and edit them? How should
| they be shown in the UI, are they clickable, do they have perma-
| links? What category should old posts be given. How do you want
| to represent "no-category" state. Do we need to support multiple
| categories? What other side-effects happen when a category
| changes.
|
| Unless all product managers get in a room and define the
| canonical implementation of all web app features i think we are
| destined to do a lot more plumbing for a long time to come.
| lovasoa wrote:
| That is a good point. And that's why developing a new feature
| in, say, facebook, will always take a lot of efforts.
|
| But when you are a team of 3 with a startup to launch, for
| instance, you don't really _care_ about permissions to edit
| categories and the no-category state. You just want that line
| of text at the top of the post that says which category it
| belongs to.
|
| And you want to do it in a way that will allow you to _later_
| easily come back to it and start thinking about the "no-
| category" state and multiple categories for a single post.
| fragmede wrote:
| Problem is, even if you could get every single product manager
| in a room to hash it all out, three years down the line, when
| half of them have changed companies, and there is a whole new
| batch of them; when the business needs have evolved so that
| there are now two types of wholly orthogonal "categories" tags
| for every post that have their own separate management systems,
| and the product managers can't even agree on their
| functionality and expectations, what then?
|
| Job security for one, but it's hard to say in the abstract
| which coding style will be better.
| smrtinsert wrote:
| Spring boot is pretty close to compressing the effort involved
| but the ui remains an issue
| jackblemming wrote:
| Yes it's tedious to write plumbing code, but it's also dead
| simple. Just write the damn code. Don't try to create some weird
| beast that "automagically" does the n different things. Just.
| Write. The. Code.
|
| Yes it does suck. You know what sucks worse? Zero separation of
| concerns and the tar pit you get from it.
| bob1029 wrote:
| I find similar arguments around SQL.
|
| So much time & frustration expended simply to avoid typing out
| the magic database commands... And the _constant_ ego trips
| attempting to outperform 30+ year old query planner codebases
| on 7-way+ joins by using baby 's first ORM.
|
| > the tar pit
|
| If you find yourself stuck in one of these, I strongly
| recommend giving this a shot:
| https://curtclifton.net/papers/MoseleyMarks06a.pdf
|
| "but it won't scale"
|
| We are in the era of _hyperscale_ SQL engines. Database engines
| that are spread out across multiple servers, racks and
| buildings. Engines so vast & complex the compute & storage
| responsibilities have to be separated into different stacks.
| But, they (the good ones) still work just like the old school
| approach from an application perspective. The workload
| necessary to actually saturate one of these databases would be
| incredible. I some days wonder if Twitter could be rewritten on
| top of one without much suffering.
|
| And, if you aren't trying to go big and bold or spend a bunch
| of money, there's always SQLite. It also supports basically all
| the same damn things. It can run entirely in memory. It has FTS
| indexing. Your CTE-enabled queries will work just fine on it.
| If you find SQLite doesn't scale with you, swapping to a
| different engine really isn't that big of a deal either. You
| will have some dialect conflicts but it's generally very
| workable, especially if you use some thin layer like Dapper
| between your code and the actual connection instances.
| DarkNova6 wrote:
| To be fair, a good IDE can give you low-effort tools to one-
| click typical use-cases.
|
| Other than that I completely agree. Devs get hang-up on trivial
| syntax topics waaaay too often, when the actual time-killer
| lies in reasoning and performing test-cycles.
| amelius wrote:
| > Yes it's tedious to write plumbing code, but it's also dead
| simple.
|
| Don't we have ChatGPT/Copilot to do it for us now?
| dgb23 wrote:
| I've become a fan of code generation (data driven).
|
| The benefits: you write code faster, automatically uniform and
| the result is "dumb" and less abstract AKA easy to debug and
| modify. Tedium/boilerplate is gone, you focus in the overall
| model.
|
| The costs: you think more up front, you have to see the result
| first (hand written). It's easy to see common patterns too
| early.
|
| With some patience, caution and experience some of the costs
| can be mitigated.
| semicolon_storm wrote:
| I work at a company that does a lot of code generation, and
| it gets uglier the longer you do it. It's much harder to
| write the code that generates the code you want than to just
| write the damn code in the first place. The abstractions &
| assumptions made for your code generator will eventually
| begin to break down, and when that finally happens everything
| goes from a simple refactoring to way overly complicated
| update to the generator.
| epolanski wrote:
| We too do lots of code generation, but I have the opposite
| experience.
|
| The articles example would imply in our use case:
|
| 1) add one key to the schema (which is database
| independent), which will generate encoders, decoders, apis
| (to work with the data structure, not in network-sense)
| automatically
|
| 2) add the key in the views you want to add it (when
| updating/reading or more complex network apis)
|
| 3) specify how the key is retrieved/saved in the use cases
| (controller-like)
|
| 4) use the key in the frontend.
|
| It took me longer to write this post from mobile than it
| would've taken me do the first 3 steps.
| lovasoa wrote:
| I loved the idea of code generation when I first encountered
| it, but I've since come to hate it.
|
| A large code base that was auto-generated and then subtly
| modified in some places is hard to refactor, and if you need
| to change the signature of a function that is used thousands
| of time across the generated code, you are in for a long
| ride.
| dgb23 wrote:
| Yes it's easy to go overboard with it. I don't have a clear
| recipe for which parts it makes sense, except that it
| emerges from hand written code.
| PartiallyTyped wrote:
| I asked some developers to implement something with guidelines
| over how to do it.
|
| Ultimately they tried to do more than asked which then caused
| problems because maintenance is now harder, and some types were
| removed while others were "enriched", and much like uranium,
| became more dangerous to wield.
| lovasoa wrote:
| It's not just writing the code. Writing the code is easy. It's
| maintaining it. And then debugging it. There is a limit to how
| many lines of code a single person can maintain.
| PNWChris wrote:
| I'm of two minds on this, I both agree and disagree.
|
| Once a code base is a certain size, explicit but bigger can
| be a boon. Magic dynamic dispatch systems and other tools
| that simplify plumbing make onboarding and routine, drive-by
| maintenance way harder IME.
|
| I find that once you understand systems that have a dash of
| "magic", though, it is easier to add features and stuff.
| Single points of maintenance and all that.
|
| It's a continuum, with each side having different benefits.
| mawadev wrote:
| That is true, add more developers :)
| delusional wrote:
| No there is not. A line of code takes no resources, has no
| overhead, requires no upkeep. I think you may be referring to
| the drag complexity imposes in future development. That I
| agree with, but LOC is a poor proxy for complexity, and code
| that is static costs nothing.
| fragmede wrote:
| Every line of code has an overhead; has a chance of bugs,
| and demands upkeep just for existing. Having class A, class
| B, and class C, that do _almost_ the same, but slightly
| different thing means that when the business rules change,
| that you have to be sure that similar, but _slightly_
| different changes to class B and class C, which _aren 't_
| neatly going to be self-contained in B.cpp and C.cpp (or
| .py, .rs, .rb; you get the point) have to be made, and then
| you can't ever be sure that A.cpp doesn't also have some
| long-forgotten but similar and crucial bit of functionality
| that this one customer relies on (because that was written
| before TDD became popular).
|
| ---
|
| LoC itself is a bad proxy for complexity, but I think
| taking the log of the number of LoC tells you enough to
| build some expectations. A codebase where log LoC is ~6 (so
| in the neighborhood of ~1M LoC) is different enough from
| one where log LOC is ~3 (so ~1,000 LoC) that you have an
| idea of what you're getting into if someone asks you to
| make a change to either one of those.
| delusional wrote:
| The key to understanding our (apparent) disagreement is:
|
| > that when the business rules change
|
| Yes, when things change complexity has a cost. The
| inverse is also true however, if nothing changes, it has
| no cost. If class A, B, and C do almost the same thing,
| then nobody cares because the computer will gladly
| execute almost the same thing in different locations in
| memory. The modern computer built today is essentially
| perfect. It will execute the same thing every time, it
| will not suddenly require changes because there was some
| degradation in an adder, and no cogs need changing. All
| the maintenance is stuff we make up because we want it to
| do something it never did before.
| fragmede wrote:
| A friend's friend's company absolutely relies on this
| bespoke computer program running on an un-networked
| desktop computer running Windows XP from the 2000s. There
| _will_ be a degradation in its hard drive, its power
| supply, its fan; _something_. All the lines of code that
| comprise that program (which are lost to the sands of
| time) are a liability because that code has been lost.
| All we can do now is virtualize the application and move
| it to newer hardware that isn 't on the verge of failing.
| Rewriting the app is out of everyone's budget so that's
| all we can do, and hope for the best.
|
| The lower the log LoC of their Visual Basic app, the
| easier it should be to replace and rewrite atop a modern
| tech stack.
|
| If it ain't broke... you point out. It's old and creaky,
| and everyone's just _afraid_ of the thing. There 's no
| real backup (working on that!), there's no accessibility
| to it from the Internet - looking up info on that
| computer via a smartphone or tablet would be a boon to
| the company. It's absolutely load bearing, but it's like
| a bridge that's too small for the city that's grown
| around it.
|
| The world moves forwards around software that's sat in
| place, so the software wants to move as well. We're not
| "making up" maintenance stuff just for the hell of it.
| Unless you work on the same chair and desk you used when
| you were 5. I don't fit in mine, and they were lost to a
| move anyway.
| Supermancho wrote:
| > Yes, when things change complexity has a cost.
|
| Things always change. Software does not perform in a
| vacuum. It's subject to the inexorable progression of
| hardware decay and business knowledge loss, at the very
| least.
| klodolph wrote:
| Saying "a line of code requires no resources" can only be
| true under a particular set of assumptions and particular
| system for accounting. It's not a useful or interesting
| argument by itself, because it doesn't explain the
| assumptions and accounting system that it implies.
| lawn wrote:
| Extending and debugging complex code (eg autogenerating
| tools, macros etc) is much more difficult than simple code,
| even if the before can be written in fewer lines than
| replicating (nearly) identical but simple code.
| praptak wrote:
| Debugging is easier when you have a backend server which logs
| the API calls.
|
| I did debug apps where UI and DB access lived in a single
| code space (VB/Delphi style). This was pretty hard to debug
| and logic was so tightly coupled with the UI code that it was
| nearly impossible to write tests for it.
| FpUser wrote:
| Because those Delphi apps were written by less capable
| people. I've done tons of Delphi's applications in the past
| and still do some now (both Delphi and Lazarus). In every
| case the UI and backend business logic was clearly
| separated.
| hyperman1 wrote:
| In my experience, the limit does not depend on the volume as
| such, but more the complexity. This complexity can be
| intrinsic frombthe business domain, or accudental from
| technical choices. If frontend, backend and storage have
| parallell structure based on predictable patterns, the triple
| line cost is easily ignorable by skimming.
|
| Development heavily slows down under unpredictability.
| Maintainance is slower partially because knowledge loss
| hightens unpredictability. One-off half-documented pseudo-
| frameworks create much higher knowledge loss in maintenance,
| and are a much worse time eater than simple code, even if
| tripled.
| lovasoa wrote:
| Hey, I'm Ophir, the co-author of the post, and main
| contributor to the SQLPage one-off half-documented pseudo-
| framework :)
|
| I'm not sure if you had a look at what SQLPage really does.
| It is not a framework in the same sense as Django, Rails,
| or Laravel. It doesn't have a large set of functions you
| need to interact with.
|
| It lets you write the database queries you would have
| written anyway to get data out of your database, and just
| renders that as a nice frontend. All the components you can
| use for rendering are heavily documented with many examples
| on https://sql.ophir.dev/documentation.sql
| hyperman1 wrote:
| OK, here is a severe misunderstanding brewing. I
| definitely did not mean SQLPage when I said one-off half-
| documented pseudo-framework. In fact, I did not mean any
| real, standalone, named product with this. I do however
| see very much how you could think so from my description,
| so my apologies.
|
| What I meant: consider any random big software
| development. It might be mind-numbingly boring, very
| technically repetitive, you might have devs who never did
| any maintenance, or devs being expensive got the command
| to start building something anything while the business
| has yet to start delivering something resembling
| requirements.
|
| In this kind of case, programmers tend to start building
| abstractions based on their imagined needs, with an We-
| will-add-the-business-stuff-later attitude. The results
| are generally some kind of architecture astronaut horror.
| Abstraction will be very high, weird features and
| handling of useless corner cases will abound. In-code
| documentation, logging, debugging features will be
| absent. Higher level documentation was either not written
| or lost long ago. That's your average one-off half-
| documented pseudo-framework.
|
| I've seen plenty of these (and committed a few crimes of
| my own). From the top of my head, some of the worst:
|
| * A full-blown 3000 lines templating library, for
| rendering exactly 1 report that was basically a for loop
| dumping an sql query to a html file.
|
| * A C10K database connection manager built on top of
| apache commons pooling (which while a good library was
| not fit for this purpose at all), hyperoptimized for TCP
| port open/close speed, for an application making at most
| a few connections per minute.
|
| * A cache manager for files, deciding when to remove a
| file based on either AI or linear regression, with a web
| UI for configuring this decision and all the zillion
| config parameters and strategies, but the time to
| generate the cached data was shorter than the time to
| read it from disk and the files easily lived for months.
|
| * A java message building code that did everything
| humanly possible to only allocate a big buffer once at
| the beginning because 'GC is too slow', but the coder
| forgot how joining strings together created temporaries
| that were of course cleaned up by the GC.
|
| Needless to say, the people maintaining these beast
| cursed the devs who implemented them, and tended to rip
| them out on sight if possible, or pay the very heavy
| maintenance cost.
| whstl wrote:
| _" In this kind of case, programmers tend to start
| building abstractions based on their imagined needs, with
| an We-will-add-the-business-stuff-later attitude"_
|
| I wish I could publish examples from my current codebase,
| because that's exactly what happened. Difficult and
| verbose abstractions, with sometimes 50 classes being
| involved in displaying a simple table (one class per
| column display, one class per filterable column), and
| that's just the "R" part of CRUD.
|
| And there are 6 or 7 different teams working on it, and
| each one uses different methodologies to do their work.
| In some cases it's abstractions on top of GraphQL.
|
| Everyone involved had the best intentions possible, but
| the end result doesn't reflect it.
| vendiddy wrote:
| Relevant quote from the book "A Philosophy of Software
| Design"
|
| Complexity is what a developer experiences at a particular
| point in time when trying to achieve a particular goal. It
| doesn't necessarily relate to the overall size or
| functionality of the system. People often use the word
| "complex" to describe large systems with sophisticated
| features, but if such a system is easy to work on, then,
| for the purposes of this book, it is not complex. Of
| course, almost all large and sophisticated software systems
| are in fact hard to work on, so they also meet my
| definition of complexity, but this need not necessarily be
| the case. It is also possible for a small and
| unsophisticated system to be quite complex.
| simonw wrote:
| "There is a limit to how many lines of code a single person
| can maintain."
|
| I used to believe that, but I don't think it holds true any
| more.
|
| The trick is to write code with automated tests and
| comprehensive documentation.
|
| If you do this, you can leave projects in a state where you
| can pick them up in the future as if you weren't the original
| author.
| lovasoa wrote:
| It's true that if all the code works well, is tested and
| all the features are supposed to stay the way they were
| when the code was written, then, any developer can maintain
| any amount of code, there is just nothing to do.
|
| The problem arises when there is a change in what we want
| the code to do. Changing a feature that is implemented over
| three codebases in three different languages is definitely
| much more work than updating something that was written in
| SQLPage, for instance.
| soulofmischief wrote:
| When writing tests, my goal is to verify a given routine works
| as intended.
|
| I don't want to write tests for the same functionality over and
| over. Repeated functionality should be extracted, tested in
| isolation and then used in composition with other tested code.
|
| This is how you write correct code without stress or worry.
| People that take "just write the code" as dogma have produced
| some of the most untestable, bug-ridden code I've ever
| encountered.
| echelon wrote:
| You're talking about two entirely different things.
|
| OP is saying don't write a "magic thingcombobulator factory"
| that "simplifies X endpoints with Y and Z similar behavior".
| This might be an earnest attempt to try to speed development,
| but it all collapses under its own weight at scale. The
| maintainers after you will be left holding the bag and have
| immense difficulty refactoring, adding a new set of
| requirements, migrating to a new data model, or moving to an
| entirely new service.
|
| Clever abstraction kills.
|
| I've dealt with undoing insane balls of twine left by
| unthoughtful devs, mostly in magic method dispatch, included
| behavioral overrides, and monkey patching (some of these
| behaviors are a hallmark in Ruby land).
|
| One person once exposed the entire database as a "safe" SQL-
| like query parameter DSL. No more endpoints to write - just
| use the thing.
|
| There are so many problems with this. For example, when
| millions of transactions per day on mobile clients or via
| third party integrators bake these assumptions in, you can't
| easily migrate them away. You have to keep serving the same
| data assumptions, even while you're gutting and changing
| everything under the hood. You have to understand the
| callers, the data flows, the read and write paths. For
| complex spider webs of business critical logic, it can take
| several people entire quarters to even years to unwind the
| mess.
|
| Simple endpoint logic is best. Your data model should be well
| thought out, and the CRUD code serves as a well-defined,
| super literate, super maintainable means to manipulate it.
|
| Simplicity of design is important from the simplest Django
| endpoints all the way up to the most battle-hardened
| active/active 500k transaction per second endpoints.
| CuriouslyC wrote:
| The end goal is to minimize software TCO. In addition to
| being semantically less clear, repeated plumbing code tends
| to diverge over time, which makes it difficult to refactor
| and more bug prone if people assume behavior is homogenous.
|
| The best way to handle cases that will be almost the same
| but may diverge over time is to create a functional mini
| DSL that describes the domain behavior, and create a
| template implementation that can be used if desired. Then
| everything is using a common language, and a non-template
| implementation indicates the presence of non-standard
| logic.
| klodolph wrote:
| Agreed. I've also gone into a codebase and seen the most
| boring code ever. It looked like examples from an intro to
| web programming class. The backend did simple parameteried
| SQL queries. It was a pleasure to work with.
|
| My conclusion is that the real "star" developers will, most
| of the time, write code that's so simple, it looks like
| anyone could have written it. They ship a project on time,
| with good performance and availability, and then they move
| on. Anyone can come in and maintain it because the code is
| so obvious.
| lovasoa wrote:
| I think your parent comment was making a good point. The
| "just write the code" and "don't try to be smart" mentality
| is good only up to a certain point.
|
| Too much "just write the code" ends up creating huge
| unmaintainable monstrosities.
|
| When you have a lot of time in front of you and a large
| team, it's okay to just put two junior developers at work
| for two weeks, and get a big CRUD REST api in the end.
|
| But when you are trying to iterate quickly with a small
| team, exposing your database is not as stupid an idea as it
| sounds. And that's why things like Firebase, Hasura,
| Apollo, Postgraphile, etc. are so popular.
|
| The post is not trying to convince people to build custom
| DSLs just for querying their database (sorry you had to
| work with that). It is saying that there are things that
| exist today, that dramatically reduce the complexity of
| full stack applications. And that whether or not we like
| it, this is probably the direction the industry is taking.
| dkarl wrote:
| Dogma leads to shitty code no matter which way a person
| leans. One of the worst pieces of code I ever worked on was a
| query generator. Somebody noticed that there were recurring
| patterns in some BI-ish queries that were used to generate a
| dashboard for customers that wanted to see their usage, and
| they decided to factor out the redundant parts and eliminate
| the boilerplate.
|
| What did they end up with? The few hundred lines of code
| expressing the BI queries shrank in half, but behind the
| simplicity was close to a thousand lines of dense,
| inscrutable magic. It was a net increase in LOC, but the
| value of the magic was supposed to compound as they added
| more queries. What happened was, the original programmer
| moved on, and every attempt to add more queries failed, until
| I joined and it was my turn to be sacrificed to the monster.
| (I did manage to figure it out. The key was realizing that
| the whole thing was stupid, from conception to execution --
| the other engineers had put the original programmer on a
| pedestal, and they were trying to make the code make sense,
| which it didn't.)
|
| After making the query generator work for a few queries, I
| had established the credibility to say that we shouldn't use
| it anymore, and we should just write out all the boilerplate
| instead. Suddenly adding and modifying queries became
| something that anybody could do.
|
| It isn't just custom code that ends up this way. I'm
| currently working on a project that uses SQLAlchemy, and as
| the glutton for punishment I am, I'm the person who cleans up
| all our SQLAlchemy difficulties. I virtually always have the
| documentation open in a tab, and I have the source code
| checked out to the version we use. If we just wrote raw SQL
| and wrote our own row mappers, we'd have twice as much
| database code, but we'd understand it, and anybody could
| write and debug it. Instead, half the team treats it as
| witchcraft, and I feel like I've invested more time learning
| SQLAlchemy in the last year than I ever spent learning SQL.
|
| This is not to say I'm against abstraction, just that it can
| be done so poorly that it's counterproductive. You always
| have to compare -- are we better off with this, or without
| it? Saying that something reduces boilerplate or reduces
| repetition isn't the end of the conversation, even if it's
| true. You have to ask what the cost is.
| lovasoa wrote:
| This is very true. And this is what the blog post was
| advocating too ! It was not about using some smart custom
| ORM, but about writing dead simple raw SQL queries in
| SQLPage instead of hundreds of lines of python and
| typescript.
| dgb23 wrote:
| I like to go in the other direction:
|
| Write the raw SQL and then generate the boilerplate from
| that.
|
| This has very few surprises because it's a bottom up
| approach. And even better: you can do the exact same thing
| by hand.
|
| There's tools/libs that help with that like hugsql
| (Clojure) or sqlc (Go and other languages).
|
| Doing it top down (ORM etc.) is what can cause so many
| problems outside of the happy path and trivial cases. These
| tools basically need to reinvent SQL and map it into a
| procedural language.
|
| Just use SQL!
| actionfromafar wrote:
| Query builders are a good golden path for me. Just SQL
| but extra hand holding in type coercion and syntax.
| gabereiser wrote:
| 3x gripe for the fully expected "How are we trying to solve this"
| pitch. Nice. Putting logic in your database is stored procedures
| all over again. Switch to a different storage engine? F#%ked.
| epups wrote:
| There's a value to compartmentalization, and this solution does
| not capture it. If you have to make one small change and it
| cascades through individual modules of your code, it may be true
| that more work is required, but you going through the work of
| implementing it "three times" comes with some advantages. For
| example, if there's a business need to change the database
| system, you have already taken care of most of the work to do
| that. Meanwhile, the proposed solutions sound like they would
| require a huge commitment to move all of your codebase to an
| obscure framework, with the presumed upside that you can sort of
| rely on them to properly abstract the other work for you.
| benzible wrote:
| I've found the Phoenix LiveView approach in the PETAL stack
| elegantly solves many of these problems. By rendering templates
| directly on the backend server, you can build the entire
| application - frontend to database - all within the same Elixir
| codebase.
|
| There's no need for a separate API layer or painstaking
| synchronization with a standalone frontend. Features that took
| days of work across all three tiers now take just hours in a
| single unified backend context.
| h4l wrote:
| I've not used it myself, but https://htmx.org/ combined with a
| traditional web framework like Django or Rails seems like it
| should greatly reduce the need to triplicate logic. At least for
| apps where the UI needs to be good enough rather than as good as
| possible.
| aaronbrethorst wrote:
| ...or you could use server side rendering and remove about half
| of the tasks on the bulleted list of "what you'll probably have
| to do"
| mock-possum wrote:
| Is 3-tier architecture just MVC pattern? (Or I guess vice versa?)
| khaledh wrote:
| Not necessarily. Three tier architecture means separating the
| client, the server, and the database into 3 different tries.
| MVC can be all in the server (e.g. for server rendered views)
| or separated between the server (model and controller) and the
| client (view).
| slondr wrote:
| No - 3-tier is an infrastructural pattern, not a software
| design pattern. It means you have a front-end, back-end, and
| database.
| JHorse wrote:
| [dead]
| cjfd wrote:
| It is an interesting question. If there really is nothing besides
| the same sets of fields repeated three times one could have some
| metadata that is used to generate what is necessary in all three
| layers. But... very often something special must happen in one of
| the layers. In the GUI it may be that the layout is not uniform,
| e.g., some fields appear below each other and some next to each
| other. Perhaps one field should not appear when some other field
| has particular value. In the between front end and back end there
| may be something special when one of the fields happens to be
| readonly and comes from some different source. In the database
| there may be something special because the legacy part of the
| code base also needs to read some fields and it has some special
| needs. And so on, and so on. It then becomes difficult to have
| anything besides three layers that mostly repeat fields.
| lovasoa wrote:
| Hey Ophir here, I'm the co-author of the post.
|
| What you say is on-point, and we should have mentioned it in
| the post.
|
| The way I see it is: at the beginning, everything is repeated
| three times on the three layers. Then, as time advances,
| complexity grows, and you start having much more specific
| requirements that will need one of the layers to differ
| slightly.
|
| The common approach is to just duplicate everything three times
| at the beginning to be ready for the moment when something
| needs to diverge.
|
| What SQLPage [1] is saying, is: when you start, just think
| about the database. Make it the single source of truth, and
| iterate quickly to find out what form the data you work on will
| need to have. You won't get it right the first time, so it's
| crucial you don't find yourself having to do the work three
| times for every change. And then, when you need some frontend-
| specific feature, make just a react component for it and
| integrate it in the application. Then, as the app grows, you
| will progressively write a full frontend for it, and an
| external backend, but you will never have to re-do the work you
| have done in the beginning. This has allowed me to make some
| applications that I wouldn't even have thought I would have the
| courage to start before.
|
| [1] https://sql.ophir.dev
| Pxtl wrote:
| For a trivial case like this, something like odata and entity
| framework will get you 90% of the way there. The ORM provides
| both the webserver and data tier copies of the data. The problem
| I run into where I have to drop down into manual SQL is
| migrations. Every tool that promises free migrations fails me.
| kosasbest wrote:
| Writing plumbing and boilerplate only has to be done once.
| Likewise, a solid API to a database only needs to be learned
| once. Put in the grunt work early and you'll be flying.
___________________________________________________________________
(page generated 2023-07-30 23:01 UTC)