[HN Gopher] Garbage collect your technical debt (2021)
___________________________________________________________________
Garbage collect your technical debt (2021)
Author : gfairbanks
Score : 128 points
Date : 2024-06-08 11:54 UTC (1 days ago)
(HTM) web link (ieeexplore.ieee.org)
(TXT) w3m dump (ieeexplore.ieee.org)
| Rochus wrote:
| It even rhymes ;-)
| oriel wrote:
| Schoolhouse Rock for Coders?
| gfairbanks wrote:
| A quirk of IEEE's publishing system is that it drops the
| abstract, instead using the first paragraph. Here is the abstract
| [1]:
|
| The iterative process that a team follows is a bit like a garbage
| collection algorithm, and we can compare software development
| processes like we can any algorithm. A process can help
| developers do two things: clean up tech debt after it exists, or
| avoid creating it. When an iterative process does neither, tech
| debt buildup will lead to bankruptcy, so it is only suitable for
| projects with a short lifespan. A process that does both has the
| best chance at minimizing tech debt over a long lifespan. In
| particular, focusing on the system's design will keep tech debt
| low.
|
| [1] https://www.georgefairbanks.com/ieee-
| software-v38-n5-sep-202...
| clcaev wrote:
| It's not always easy to find the right words to describe why a
| feature pause to refactor is needed. This quote was a succinct
| statement that avoids analogy:
|
| "Postponing a small cleanup can transform it into a big cleanup
| because, over time, code builds up around the problem, and it too
| must be refactored."
| WalterSear wrote:
| "I'm sorry, I didn't understand much of that. Are you saying
| you can commit to finishing the feature on a shorter timeframe
| than you originally asked for? Would it help if we forgo
| writing tests?"
| nxicvyvy wrote:
| Yes, but every other feature will take ten times as long as
| over the next three years we will lose 90% of our best
| developers.
| RadiozRadioz wrote:
| "This is the last feature we will tell you to rush, we
| promise. We really need just this one, and then we're
| good."
| FridgeSeal wrote:
| "Well done on delivering everyone! We knew you could do
| it, now, our next priority is this new thing......"
| kaffekaka wrote:
| Please stop, it hurts.
| Seattle3503 wrote:
| Says the PM, right before they get promoted.
| WalterSear wrote:
| "I'm confused. When I greenfielded this app, several
| thousand commits ago, I took half the time you've already
| spent on this feature. You said you were a senior
| engineer!!"
|
| True story. Twice.
| physicles wrote:
| I'm lucky that I don't have a boss like this, but I've
| asked myself this question recently. I've been with the
| same company for almost eight years working on the same
| code base, which I and another dev greenfielded. I know
| it very, very well.
|
| I'm often dismayed at how long things seem to take these
| days compared to when we first started out. Am I getting
| slower? Lazier? So far, I've identified the following
| factors:
|
| - We have more customers, and those customers are much
| more demanding (used to be b2c, now we're b2b). The cost
| of making a mistake is much higher.
|
| - It's just a lot of code. Parts of it are fairly
| complex, as much as I try to keep it simple. When a core
| component is changed, multiple services might need
| refactoring.
|
| - We have many more features, and they sometimes interact
| in surprising ways. I've been around longer than our
| product people, so we often have to spend time iterating
| when they come up with a design that doesn't fit with
| what's already there.
|
| - It's important to refactor your database from time to
| time as you learn more about the domain and find simpler
| ways to do things. But refactoring a database is
| terrifying. I spend a lot of time triple-checking my
| work.
|
| Working on such a large code base for years is super
| satisfying though. I've learned so much about system
| design, just from noticing how easy or hard stuff is to
| maintain.
| benji-york wrote:
| I wish I had more than one upvote to give you for this
| post.
|
| > I've asked myself this question recently.
|
| I suspect both are true. There are real complexities that
| have grown around you _and_ working in the same way on
| the same stuff for so long has caused you to habituate to
| a few inefficiencies. I suggest shaking up your world
| view a little and seeing what falls out. There are
| probably a few big gains you could make.
|
| Good luck!
| triplesec wrote:
| what was your response?
| WalterSear wrote:
| The first time, I lacked the self-confidence to speak up
| in the face of dominating people. So I internalized their
| behavior as "proof" of my incompetence - in the face of
| the patently obvious facts to the contrary, in the face
| of my own experience and judgement. Naturally, this made
| things much worse - I consider myself "part of the
| problem" in that case, though not the largest part by any
| means. Anyway, I was fired a few months later.
|
| The second time, it was a breaking point - the CEO, who
| said those things was incorrigible, and the situation was
| unworkable. I called my boss in the morning (the fucker
| CEO had been yelling at me at 11 at night) and gave
| notice. He quit too - exhausted of losing engineers and
| being party to the abuse. Within the month, they lost
| most of their engineering team, and the few who stayed
| had received promotions and substantial pay increases to
| incentivize sticking around. They also "saw the light"
| and halted feature work for several months while (I
| assume) the worked on fixing their tech debt problem.
|
| I've done really substantial work on myself since then,
| and I feel like I'm in a much better place to
| appropriately execute the soft skills required by my
| position. So I'd like to think that if (or rather, when
| :) ) the first situation occurs again, I will be self-
| assured enough to push back in an effective non-
| confrontational way, or at least speak my mind instead of
| being silenced by the unreasonable shame of an
| inappropriate dressing down. I would find ways of halting
| the narrative every time bullshit was spoken, and address
| the "inaccuracy" instead of behaving in a way that that
| manager took as confirming his suspicions that I was the
| problem.
|
| And, in the second case, I'd have quit way, way earlier,
| when I saw all the previous red flags.
|
| Mostly though, I'm not going to work for hotheaded,
| first-time founder-engineers so recently graduated from
| college, so bereft of the experience required to lead an
| engineering team. :)
| TheDudeMan wrote:
| Triggered
| yarekt wrote:
| Ouch, right in the feels.
|
| I've come to realise that there's no valid business case for
| dealing with tech debt early, nor adding tests to an existing
| project (bar some special circumstances / legacy change,
| critical outage, etc)
|
| It's like a lot of things have to be aligned for "good"
| development practices to reap benefits, most shops are much
| less organised, and a bit of chaos and early/quick iterative
| shipping will always yield better results.
|
| Having said that, my core belief is still that if you take
| care and do things properly you'll go fast in places where
| all other companies get bogged down
| WalterSear wrote:
| I'd argue that there's definitely a business case for
| frequent and early refactoring - it just needs to be
| refactoring worth doing in the first place. IMHE a most
| code gets touched rarely, and some parts get touched all
| the time, so refactoring needs to be strategic if it's
| going to have any utility.
|
| This also argues for ongoing refactoring - addressing pain
| points as they arise, while they are fresh in people's
| minds, rather than suffering through them until Stockholm
| Syndrome sets in, people can no longer see the forest for
| the trees, and much of the velocity refactoring would
| provide is lost because the subsequent work is already
| done.
|
| But the biggest issue I've seen with not handling tech debt
| on an ongoing basis is that there's never a good time to
| start. So if it's not built into the development cadence,
| then resistance builds for doing it at all. Product starts
| pointing fingers at "slow" engineering, who point back at
| the "breakneck" demand for feature work, and negotiations
| start for unrealistic (for both sides) halts in feature
| work - neither sufficient to resolve the problems nor short
| enough to avoid hurting the business.
|
| Then product breathes a sigh of relief - the tech debt is
| "resolved" and will never need to be addressed again,
| engineering returns to wading through a codebase that is
| only marginally less swampy than it was before the cursory
| refactoring sprint, and the downward spiral (and finger
| pointing) resumes.
|
| At least that's how it always seems to happen around me :)
|
| As far as tests, IMHO refactoring without them (with BDD
| style tests being greatly preferable) is fraught with
| peril. But unless the team is bought in on BDD tests and
| using them to guide development, I agree they are a time
| sink to write them early. However, writing them later (and
| around code that might have been touched by multiple hands)
| rather than maintaining them is flavor of pain - like the
| refactoring, it's harder to be sure they won't miss things
| and you'll break something.
| 0xDEAFBEAD wrote:
| This suggests a "just in time" approach to cleaning up tech
| debt. Clean it up just before you write code which would
| otherwise depend on the debt.
| physicles wrote:
| I recently needed to make a change in how something common is
| done across several services. I made the change in one place
| (and have some passing unit tests), but then I realized that
| I'd have to duplicate more code than I'd like. So I'll take a
| couple days to pull all related functionality out into a
| shared library.
|
| This refactoring could've been done ages ago -- there's
| already some messy and duplicate code there -- but now's the
| perfect time to do it because 1) I'm already touching and
| destabilizing that area of the product, and 2) the clearer,
| de-duplicated code will be easier to spot check.
| 0xDEAFBEAD wrote:
| Makes sense. I guess for any given refactoring, the
| advantage of doing it sooner is that you get residual
| recurring benefits sooner. And the advantage of doing it
| later is you have more information about the requirements
| of your application and the "correct" way to do the
| refactor (or even if that code is going to be a long-
| lasting part of the app, cc Lindy effect). So whenever you
| touch any given piece of code which could use a refactor,
| you could try to figure out which advantage is more
| important right now.
|
| Of course, if the total effort of doing the refactor plus
| building the feature on top of the refactor is less than
| building the feature on top of the debt, you should always
| do the refactor first. That's pretty much a no-brainer.
|
| Another idea I've thought about is informally keeping track
| of possible refactors, maybe give them an informal
| prioritization, and taking a glance at your list before
| each task. Maybe there's some value in letting a refactor
| stew in your brain a little bit before actually putting it
| in to action. And continually maintaining a rough
| refactoring plan could help prevent your codebase from
| entering a state where it's actually unrecoverable.
| 0xDEAFBEAD wrote:
| Also, as you alluded to, if you're changing related code
| which will require manual testing, then that decreases
| testing costs, if they're shared across the refactor and
| the new code.
| physicles wrote:
| Yeah I've got about five ideas for refactors that will
| make my life noticeably better. I've been pondering some
| of them for more than a year, slowly improving the
| design, but I'm still waiting for the right time when the
| ROI is higher.
| yarekt wrote:
| Interesting, I think this is part of what the article is
| alluding to: what happens with tech debt is more to do
| with the team's process rather than with the tech debt
| itself. Some teams are stuck in a process that makes them
| unable to tackle the problems even if they want to due to
| that process.
|
| Of course you could just say "change the process" but
| it's also obvious that there are many other factors and
| pressures that are non trivial
| hnthr_w_y wrote:
| I'm looking for a job where this is feasible. And I'm not
| even being sarcastic, I'd take a pay cut for such a workflow.
| 0xDEAFBEAD wrote:
| Seems like maybe shops have a tendency to get stuck in one
| extreme or another, either shoving crap out the door or
| else excessive concern with prettifying minutia.
| yarekt wrote:
| I've literally never seen the latter: I think companies
| that do that just disappear. The former though is a great
| driver: Let your debt pile up sky high because it doesn't
| matter as long as it lets you make more money in short
| term vs dealing with it.
|
| you may say it's short term thinking, but if you're
| optimising for investment, that's the right thinking.
| Make money now, you can always move it elsewhere that
| makes better use of it.
|
| It's not what I personally agree with, but just my
| observations so far
| 0xDEAFBEAD wrote:
| With regard to "excessive concern with prettifying
| minutia", I'm thinking of e.g. code reviews where people
| spend significant attention going back and forth on minor
| stylistic issues.
|
| Another example: I remember a coworker reviewing code I
| wrote many years ago. I came up with a solution that was
| reasonably simple and workable, but wasn't the "right"
| way to do it. My coworker complained, I pointed out
| various reasons the "right" way wasn't practical, and
| they said something like "yeah but it has to be right"
| (without suggesting any concrete plan). Very frustrating
| -- I don't think it was particularly important code.
|
| Working on my own, I already have a sense of how much
| effort I want to spend on code quality. When you add code
| review on top of that, it can feel a little excessive,
| depending on the importance of the application.
|
| I actually have a lot more memories of code review
| frustration than code review gratitude. At my next job I
| would like to experiment more with code "previews" or
| design reviews -- that seems more efficient than
| rewriting code which already works.
| Tade0 wrote:
| I recommend the pharmaceutical industry.
|
| When the study for which you're producing software is only
| going to conclude in the next 18 months or so, you get to
| allocate some time for fixes.
|
| The downside is that it attracts people who want to take
| advantage of this to slack off or just aren't that good at
| software engineering in general.
|
| I'm currently in the process of clearing tech debt that
| ultimately lead to hard to resolve bugs and, ahem, staff
| rotation.
|
| I can spend days on this so long as I report progress on a
| daily basis.
| theshrike79 wrote:
| "Always leave the code you're editing a little better than
| you found it" - Robert C. Martin (Uncle Bob)
|
| There's no point in refactoring the whole thing. Maybe add a
| longer comment explaining the logic you had to decipher when
| you encountered the code. Rename a few variables from foo,
| bar and baz tom something more descriptive etc.
| MaxBarraclough wrote:
| > There's no point in refactoring the whole thing
|
| Deep architectural/design flaws in a codebase can't always
| be addressed using a series of small independent changes.
| theshrike79 wrote:
| Yes, but that's also not a thing you do on the side with
| extra hours you have left over here and there.
|
| It's basically refactoring/rewriting the whole part of
| the system in most cases, which is a full project in
| itself.
| naasking wrote:
| > Deep architectural/design flaws in a codebase can't
| always be addressed using a series of small independent
| changes.
|
| Not sure that's true. At the extreme end, you introduce a
| replacement with a better architecture and run it side by
| side with the old one, incrementally switching over
| dependents. Of course, that may take _more overall work_
| , but maybe the incrementality is sometimes worth it.
| gfairbanks wrote:
| Agreed. The architecture mismatch paper [1] identifies
| common assumptions that software can make, such as "I own
| the main thread of control and other modules will do my
| bidding", that tend to be baked-in from the start.
|
| [1] Garlan, Allen, and Ockerbloom. Architectural
| Mismatch: Why Reuse Is So Hard. IEEE Software 1994. https
| ://ics.uci.edu/~andre/ics223w2006/garlanallenockerbloom..
| .
| WalterSear wrote:
| Please no!
|
| This isn't refactoring - it's just hygienic coding.
|
| "Refactoring as you go" sets up an adversarial relationship
| between dealing with tech debt and the feature work that that
| the time and effort was actually assigned to. IMHE, human
| nature and engineering politics then means that tech debt
| doesn't get addressed at all until it's already too late.
|
| Approaching with this intention also takes refactoring
| entirely off the roadmap - from the eyes of product
| management, refactoring becomes invisible and "free", and no
| longer requires budgeting for. "Why do you need time to
| handle tech debt? I thought we were refactoring as we go?"
|
| By all means, it's good practice to leave the code better
| than you find it, but don't just leave the tech debt that
| isn't trivial, is too large to address immediately, to
| fester.
|
| Instead, take notice of inefficiencies and cruft while
| working, and write tickets for it, so it can be seen and
| brought up for discussion, prioritized against other tasks.
| Perhaps even leave a comment on the ticket whenever you run
| up against the same issue, so there's a paper trail of the
| how serious the issue.
|
| This way, at least management has visibility into the state
| of the codebase and can't act surprised when tech debt brings
| productivity to a grinding halt.
|
| A lesser issue, but still important IMHE, is that actually
| trying to "refactor as you go" makes PRs inscrutable. What's
| refactoring? What's feature work? A little bit of noise here
| and there is unavoidable, but any refactoring substantial
| enough to increase development velocity is likely to make
| meaningful code review very difficult.
| atomicnumber3 wrote:
| Agreed about mixing a refactor with the incremental feature
| add makes PRs confusing.
|
| "and write tickets for it, so it can be seen and brought up
| for discussion, prioritized against other tasks"
|
| I have been doing this for almost a decade now and I have
| at no point ever seen even a top-ticket tech debt item get
| prioritized. Ever. It's always too much effort for little
| payoff (in the eyes of the business).
|
| I explicitly tell juniors that when I cut this kind of
| ticket, I do it because it is right, not because I think it
| will ever get worked unless I myself (or someone similarly
| insane) directly disobey marching orders to work on it.
|
| And even when someone does disobey orders and refactor,
| it's often a pet peeve that isn't one of the more major
| ones. So that's fine but nobody ever one-man-armies against
| the real problems.
|
| And to illustrate what I mean by "the real problems", let
| me explain. I like to talk about tech debt as high interest
| or low interest.
|
| Low interest tech debt is the shitty python script I wrote
| the other day that took 1-2 hours of manual monkey work
| we'd have to do in Presto sql and that only 1-2 people even
| knew how to do, and makes it so literally anyone at the
| company can do it perfectly right in a minute or two. It's
| a shitty script, but it knows it's shitty, and tries to
| have no pretenses about it and make it as simple as
| possible to debug and make incremental changes to it. So
| yes, if I spent more than an hour or two on it, I would not
| have done it like this at all. And yea it'll probably be a
| little annoying to change the next time someone has to do
| it. But it saves a lot of time, didn't cost much, and it's
| not going to change how we design the system at all.
|
| That last bit hints at what I mean by "high internet tech
| debt". High interest tech debt is how it takes 8 hours of
| babysitting to deploy a schema change. It causes blip
| outages in every region as you deploy it. It's so painful
| it perverts our schema design to minimize how often we have
| to migrate the database. And it's all because, shortly
| after the Big Bang, right in between photons existing and
| the first hydrogen nucleus existing, someone had a cool
| idea for a simple trick about how they could eat their cake
| and have it too with how our databases work. And we have
| paid for it ever since. And they built on top of the
| primitives this offers so deeply that we'd have to change a
| ton of shit to make it work. The best time to fix this was
| 5 years ago, the second best time is now, and I guarantee
| you we will never ever do it. We'll replace the whole thing
| with something else because it's shiny and will be worse
| for years before we actually just pay down this tech debt
| codr7 wrote:
| I agree, but then you need buffers in the process, which
| mostly isn't happening in corporate agile land from my
| experience.
| nicbou wrote:
| The way it was taught to us was that the earlier you catch and
| fix a mistake, the cheaper it is.
| mcronce wrote:
| This follows the "debt" analogy pretty well - it's effectively
| the interest you have to pay on your technical debt
| apantel wrote:
| In my experience you need to do 'the big refactor' when a
| codebase has to handle something big that was not part of the
| original design such that you find yourself having to break the
| simplicity, elegance, completeness, coherence, etc of the
| existing system by tacking on some lopsided or alternate route
| or structure. What you really want in place of doing the 'tack-
| on' is a new simple, elegant, coherent, etc system that can
| handle both the old requirements and the new. In other words,
| you want to do the big refactor when you have new requirements
| that really should have been known at the time the system was
| designed such that you would have done things differently to
| accommodate them along with all of the requirements that
| _were_known at the time. This is easier to do the more
| monolithic and strongly-typed the application is.
|
| Naturally then, you do want to know as many requirements up
| front as possible, which is the basic point of the article.
| Even though it's not always possible, it's still the best path
| to try.
|
| All of this 'screw design lets just roll up our sleeves and
| start coding' is a great way to end up with spaghetti code and
| technical debt.
|
| The key is to do as much requirements gathering as you can up
| front because your initial design will address only the
| requirements you know about, and the initial design constraints
| future updates.
| notjoemama wrote:
| > pushed a routine re-acceptance of those terms
|
| Oh, of course, of course. Why, this is so mundane, why would
| anyone even bother to read it?
|
| > For content processed or stored on Adobe servers, Adobe may use
| technologies and other processes, including escalation for manual
| (human) review, to screen for certain types of illegal content
| (such as child sexual abuse material), or other abusive content
| or behavior (for example, patterns of activity that indicate spam
| or phishing).
|
| They are going to use AI to scan through all content...for your
| protection. Does Adobe have a problem with people using their
| cloud to produce child abuse material? I think we all recognize
| Bitcoin is used for illegal activity. But I never considered
| Adobe being associated with CP. I guess that's what they're going
| with...
|
| I feel like this is a trope at this point. Just like Microsoft
| Recall, no one believes or trusts this won't eventually turn into
| them using your IP for themselves via AI, and then eventually by
| the government. That's the bottom line.
|
| To really put a point on it, they are saying they're swipping a
| spider off us while they slip their hands down our pants.
| rrdharan wrote:
| commented on the wrong post?
| caseyohara wrote:
| > Building software iteratively leads inevitably to tech debt
| because we choose to deliver systems before we have looked at all
| the requirements. Not knowing what's next distorts our designs,
| and that distortion is the tech debt.
|
| This article frames technical debt as something that happens
| passively because you can't know future requirements. That's
| sometimes true, of course, but in my experience the majority of
| technical debt is accrued deliberately in a much more active
| process.
|
| When developing a new feature that doesn't neatly fit into the
| existing system, you must choose between two compromises:
|
| 1. Build it the "fast way", shoehorning the feature into the
| system just enough to work, compromising quality for velocity and
| accruing technical debt; or
|
| 2. Build it the "right way", adapting the system to accommodate
| the new feature, compromising velocity for quality to avoid
| technical debt.
|
| This is usually a deliberate decision, so choosing to accrue
| technical debt is an active process. The only way it could be
| passive like the article describes is if the developers don't
| know or otherwise don't consider the "right way" and go straight
| for the "fast way". I hope to never work on a team that operates
| like that.
| dec0dedab0de wrote:
| The problem is that there is never an objective right way.
| There are infinite wrong ways, and usually a handful of ways
| that are just fine with different pros and cons that are not
| always clear up front.
|
| A lot of the time when people say technical debt they mean a
| developer not taking the time to understand some code they
| inherited (or they wrote and forgot about), and wanting to
| throw the baby out with the bathwater.
|
| If think you ask ten developers the best way to refactor a
| complex program you'll get 100 answers.
|
| But I do agree that deliberate technical debt is more common on
| a decent team. I definitely have left many comments like "I
| know it would be better if I did XYZ, or ABC, but the boss
| wants it now, and I'm tired, so it's going to be a monstrosity"
| yarekt wrote:
| the 10 -> 100 bit is absolutely right. a cross functional
| team has to learn how to work together well, and spend a non
| trivial amount of time discussing the problem and tempering
| solutions until something is achieved where each of them are
| some state of "happy". This is I think the crux of many tech
| debt issues in companies
| fmbb wrote:
| There is often a third alternative: do not shoehorn it into
| something, nor rebuild what you have to fit this new thing,
| instead build the new thing on the side.
|
| I sometimes have worked with engineers who believe they know
| what "the right way" is, or spend a lot of time trying to
| figure it out. And I have certainly worked on legacy systems
| persons like that have built. It's not all fun and games.
|
| The less we entangle things the easier it is to remove cruft
| when it is no longer needed.
| yarekt wrote:
| You're completely right about entanglement: Id go further to
| say that simplicity in its purest form is almost always more
| useful than clever upfront design, but it's funny because
| clever upfront design process can be tweaked to bias
| simplicity.
|
| What I'm trying to say is that there are two ways of
| designing systems: Make them flexible to meet unknown future
| objectives by incurring tech debt, or build them so simple
| that they are easy to change when those unknowns come in.
|
| You may say that there's no "right way" to build systems, but
| some ways are certainly better, perhaps it's possible to
| distil a way from that, but i agree that it can be paralysis
| inducing. can't go wrong with building Simple working
| systems, imo.
| sublinear wrote:
| > The only way it could be passive like the article describes
| is if the developers don't know or otherwise don't consider the
| "right way" and go straight for the "fast way". I hope to never
| work on a team that operates like that.
|
| I've been on teams like that and it's absentee management.
| There's two reasons: the management is technically inept or the
| execs and other stakeholders are taking up too much of their
| time. Sometimes it's both.
|
| This creates a situation where either the most technically
| competent team member takes over responsibilities or the team
| just loses coherence.
|
| Beyond that there's also often a severe lack of involvement
| from the broader organization where everything is siloed.
|
| The quality of the dev team reflects the quality of the
| business. Implementation details are not so different from
| "operations" in other fields. Makes no sense we have such a
| terrible state of things in software at some places other than
| a lack of talent and residual people who should have retired or
| pivoted careers a long time ago.
| sophiabits wrote:
| The other possibility (which is common in startups) is that
| often the "right way" is different depending on the scale of
| the system you need to design for. In cases like this you end
| up with technical debt a year down the line, but at the time
| the feature was shipped the engineering decisions made were
| extremely reasonable.
|
| I've seen a few colleagues jump to writing off all technical
| debt as being inherently bad, but in cases like this it's a
| sign of success and something that's largely impossible to
| avoid (the EV of building for 10-100x current scale is
| generally negative, factoring in the risk of the business going
| bust). There's a kind of entropy at play here.
|
| Big fan of tidying things up incrementally as you go [1],
| because it enables teams to at least mitigate this natural
| degradation over time
|
| [1] https://sophiabits.com/blog/be-a-tidy-kiwi
| marcus_holmes wrote:
| How would you recommend implementing a feature that might get
| rolled back or fundamentally changed depending on user reaction
| to it?
|
| Doing it the "right" way incur a lot of up-front expense that
| will be wasted if the feature is rolled back.
| gumby wrote:
| Incremental collection is preferred in this case, as stop-and-
| copy collection leads to Second System Syndrome.
| digger495 wrote:
| This is one of the dumbest and most obvious journal articles I've
| ever read.
| bitwize wrote:
| RAII your technical debt: make a note of every time you slap
| something together the expedient way, and schedule a future time
| to refactor after the legitimacy of expediency goes "out of
| scope", e.g., after a major product demo.
|
| One could even add a CI/CD task to remind the user of when
| refactorings are due (and refuse to successfully build until they
| are addressed). Since debt is created by borrowing, one could
| call this the "borrow checker".
| fiddlerwoaroof wrote:
| A couple times I've put a "if (now() > xxxxxx) fail()` in tests
| for this reason.
| rglynn wrote:
| How did you get that past peer review?
| fiddlerwoaroof wrote:
| Because people agreed that the failing test represented
| technical debt that needed to be cleaned up later.
| yarekt wrote:
| Good in theory, but try that on a project with real customers
| and you'll get a very simple "remove it and we're done here"
| /shrug. Team's processes and decision chain must have agreed
| to enforce this already, and that above is just a symptom
| ralferoo wrote:
| I think this is the absolute worst way of dealing with the
| issue, and I truly hope everyone who sees this in a code-
| review has the sense to reject it.
|
| By all means, have such a check as a _compile-time_ error,
| but not as something that gets shipped to customers. The code
| you have now works, even if as a developer you 'd like to
| refactor it. However, what if you ship this software to a
| paying customer, and then your company goes bust? At unknown
| point in the future, their perfectly good application
| suddenly stops working, without warning or obvious cause, and
| your company is no longer in business to simply change the
| time to a later one and recompile. Even if your company
| hasn't gone bust, why should the customer have to spend ages
| trying to figure out if they've done something wrong or it's
| the software that's at fault, before contacting your company,
| possibly being told to pay more money for an update even if
| they were otherwise happy with the old version they'd
| purchased.
|
| Basically, this change is a liability for everybody working
| on the project and your customers. It'll soon get forgotten
| about, until you have people complaining that something just
| suddenly stopped working without warning. You might be able
| to "fix" it in seconds, but it can still require hours to
| triage the issue to discover what caused it before it ends up
| as your problem again, and probably significantly longer
| after your few seconds to fix it before the working version
| is back in the hands of the customer. For all that time,
| they've been inconvenienced just to save you the hassle of
| sticking a reminder in your own personal calendar or raising
| an issue in your bug tracking system or wherever else.
| bitwize wrote:
| > By all means, have such a check as a compile-time error,
| but not as something that gets shipped to customers.
|
| Tests don't get shipped to customers.
| ralferoo wrote:
| Ooops, I guess I missed the part where he said "in
| tests". That makes it a bit better, although when it
| triggers, it'll still need someone to triage why the test
| failed and either nudge the date up / delete the test /
| do the refactor. While that's going on, potentially
| you'll miss other failed tests because your overall build
| is red, or maybe you'll force someone to backout their
| change for no reason and stop other checking in that day.
| I actually worked at a place where if the build ever went
| red, all commits since the last successful build had to
| be backed out and nobody was allowed to check in again
| until the build was green again. This sounds drastic, but
| having lived with it for a couple of years, it grew on me
| - it stopped the repeated "Oh, that's just a one-line
| fix" that then broke something else and so on... Better
| to immediately revert the change and re-commit when
| you're sure everything will work. We also disabled and/or
| removed tests that will intermittently fail due to a
| fault in the test, and created an issue to write a better
| test if it was still relevant. The GP's test would have
| just deleted when it was discovered, rather than
| achieving its aim of getting someone else to do the
| refactor.
|
| The best approach is still to do the refactor straight
| away if it's genuinely needed, or else create a task in
| your issue tracker to do it a later time if it's going to
| require resource that you don't have right now. Then PM
| can prioritise it accordingly, not have to react to an
| arbitrary timebomb placed in the code, which will
| invariably be discovered at a time that's not good for
| anyone.
| m463 wrote:
| realtime systems just prealloc everything at the beginnging.
| noisy_boy wrote:
| I do opportunistic collection - when I am working on a feature,
| and spot opportunities for a refactoring/cleanup in code that is
| more or less directly related to the code I'm touching, I will
| keep making small incremental changes and keep testing them until
| my feature is implemented _and_ the cleanup is done as well. I
| also ensure to not make a breaking change while doing this e.g.
| no change to the user facing api signature. If I see issues in
| unrelated code, that just gets a TODO and waits its turn when we
| have to do changes there for some feature request.
|
| The reason is that I can atleast somewhat justify the changes
| under the umbrella of my feature while utilizing the allocated
| time budget. If I don't do this, we will never get a dedicated
| release to do tech cleanup - the backlog of feature requests is
| just too big and too little appetite on the decision makers' side
| for purely tech debt releases.
| TeeMassive wrote:
| This is the only way I've found that we can do the needed
| refactor. As consultants, we never ask the permission of the
| client to refactor the code. Ever. That's not for them to
| decide. The reason they hired us in the first place is because
| we're the experts and they are not. We evaluate the needed
| refactor that puts the code in a decent state and put the time
| in the quote we provide the client. And even then we never
| create tasks or log time; when we find something that needs to
| be done, then our time is billed to the task relating to that
| issue.
| teeray wrote:
| I have done this in the past and it has worked well. On some
| teams though, I have been met with:
|
| "Why did you do this refactor with the feature? Can you pull
| the feature into another PR, then we'll leave the refactor in
| the original PR to be merged at another time? (read: never)"
| ralferoo wrote:
| I'd definitely agree with the putting the refactor into a
| separate PR and then the new functionality in another.
|
| Aside from the obvious that people are more likely to be
| willing to review your change when it's small, it makes it
| much easier reason about both when they are separate.
|
| There will be a lot of noise generated from the refactor that
| will drown out the new feature, but self-contained in its own
| PR it's a lot easier to understand the new feature and spot
| mistakes.
|
| A simple refactor can likewise be likely skimmed over
| quickly, looking for the patterns of how it was done and
| assuming that most of the changes were similar and mostly
| just looking out for the differences.
|
| As for accepting the feature PR but not the refactor PR, that
| suggests that the feature doesn't actually rely on the
| refactor. In that case, it's even clearer that they should be
| separated.
|
| Personally, I'd always even create a new ticket for the
| refactor so that there's some justification for the work, and
| maybe you can say that this ticket blocks the one for the
| actual work. Maybe that's just me though, because I like to
| make sure that every non-trivial commit is tied to an issue
| in the bug/issue tracking software. This means that every bit
| of code is a simple "git blame" away from a justification of
| _why_ it was changed.
| sophiabits wrote:
| Big fan of this. Here in New Zealand we have a slogan "be a
| tidy kiwi" that encourages people to pick up their litter and
| be good stewards of for our natural environment
|
| Imo the same mentality is good to have in software, and I've
| always appreciated being in a team that makes codebase
| improvements alongside feature additions. It makes things a lot
| more pleasant
| yarekt wrote:
| Consider a case where every non trivial refactor ends up in
| several rabbit holes, requiring lengthy meetings with lots of
| people to discuss, and many disagreements about direction. And
| in light of that the trivial refactors start to feel like
| you're taking a bucketful out of a tsunami.
|
| What ends up happening is it's much more impactful for the team
| to just focus on shipping what they can, partly because they
| don't have the tools/procedures/experience (anymore) for
| dealing with tech debt in this context
| noisy_boy wrote:
| And that is exactly how the cruft I see has built over many
| years (well before I joined the team). It is not that devs
| before me were not smart; they just focused on doing the bare
| minimum without any attention to keeping the code easier to
| manage and reason with. An append only garden.
| marcus_holmes wrote:
| So, from my 30+years building software products, I have a couple
| of problems with this:
|
| - we don't choose iterative development as an alternative to
| waterfall because "waterfall is bad mmkay". When developing a new
| product, we don't know all the requirements. Part of the process
| of iterative development is _discovering_ the requirements. Every
| waterfall project is secretly an iterative project because there
| 's always a Phase 2 where the requirements get updated (even
| <especially> when every stakeholder pinky-swore that the
| requirements were final before Phase 1 started).
|
| - The optimum amount of tech debt is not zero. Tech debt is a
| product of changing the product plan/vision/design in reaction to
| learning from customers. If you're not accruing tech debt then
| you're not learning from your users.
|
| - Changing the design to accommodate the new feature is great in
| theory, but in a lot of cases the new feature is an experiment
| itself. We're optimising for customer learning: get the new
| feature in quickly with the minimum of effort, see if the users
| actually use it, and if not then roll back. If the users do like
| it, then refactor to tidy up any tech debt. Doing all the
| refactoring first is a complete waste of time if the feature ends
| up getting rolled back.
|
| - Tech debt slows down development but only as a result of
| delivering early features quicker. It's the sharpening the axe
| analogy - sooner or later you have to stop chopping the tree to
| sharpen the axe. Changing the way you chop the tree so you don't
| have to sharpen the axe is not optimising for getting the most
| timber quickest. The trick here is communicating with
| stakeholders about what tech debt is and how it accumulates.
| They're usually OK with feature pauses if they understand the
| situation correctly. In other words, like a lot of problems in
| this industry, this is a communication problem not actually a
| tech problem.
| naasking wrote:
| > Every waterfall project is secretly an iterative project
| because there's always a Phase 2 where the requirements get
| updated
|
| This is a false equivalence. Waterfall is iterative in
| _requirements_ but _not in code_. Changing the requirements
| document is much, much simpler than changing the code to
| conform to new requirements.
|
| > The optimum amount of tech debt is not zero. Tech debt is a
| product of changing the product plan/vision/design in reaction
| to learning from customers. If you're not accruing tech debt
| then you're not learning from your users.
|
| Not sure I agree with the implication "learning from users
| entails accruing technical debt". Technical debt is a rough
| measure of the _inherent resistance_ of your program to adapt
| to customer requirements. Some architectures are flexible
| enough that you could learn how to configure the architecture
| differently based on what you 've learned from users, but I
| don't see how that necessarily entails the addition of
| technical debt, ie. as a rule.
| woodpanel wrote:
| Ctrl+F "@TODO"...
|
| or have your build pipeline keep track of Todos automatically.
| Also some IDE integrations keep track and provide quicklinks to
| lists of TODOs.
| yarekt wrote:
| Curious. Got any recommendations for CI tooling that analyses
| todos? What sort of things that you've seen are possible
| MeteorMarc wrote:
| The garbage collector already has a 100% coverage test suite.
| anymouse123456 wrote:
| Wow, what a weird feeling. The authors of this article seem like
| super smart and experienced guys, but the article itself is
| introduced with incorrect statements in the very first two
| sentences and slides downhill from there.
|
| "There is a kind of design distortion that happens when a team
| chooses to build iteratively instead of looking at all of the
| requirements at once. Ward Cunningham coined the term technical
| debt to describe those design distortions."
|
| 1) There isn't an option to build software by "looking at all of
| the requirements at once." The requirements of any (sufficiently
| complex) project will emerge over the course of development,
| regardless of whether it's built in an iterative style, or with a
| much larger investment in planning and design up front. We often
| work iteratively because we acknowledge this particular aspect of
| reality and it helps everyone when we work closer to reality,
| rather than fighting it.
|
| 2) Technical Debt does not describe design distortions that arise
| due to "iterative" development, it describes design problems that
| arise in every project.
|
| Quantizing the choices for dealing with tech debt into 4 buckets
| doesn't feel right either. Changing the design of a running
| system happens along a continuum and the need and benefit vary
| widely over the course of any given software lifecycle and
| surrounding (business or other) environment.
|
| Maybe I'm just grumpy this morning, and I think reasonable folks
| could disagree, but the metaphor at the center of the thesis
| (tech debt [?] soon-to-be-deallocated-memory) doesn't hold up for
| me at all.
|
| Tech debt is sometimes left in place, intentionally or not, for
| many years. Sometimes we chip away at it, sometimes we stop the
| world and push on it. Sometimes it's bad enough to throw the
| whole system away and start over. It's like debt for businesses.
| It can be valuable to take on some debt if it lets you stay alive
| long enough to pay it back.
|
| Finally, I'm struggling with this article because tech debt is
| already a metaphor, that includes information about how, and when
| to take it on and pay it down.
|
| It's not helpful to layer another, unrelated metaphor on top of
| it.
| gfairbanks wrote:
| Ward Cunningham's original idea of tech debt (see [1], a beauty
| of concision at just 300 words) is that iterative development
| distorts your code because you start writing code before you
| know the requirements, but even so, it's better than waterfall.
| "The traditional waterfall development cycle has endeavored to
| avoid programming catastrophy by working out a program in
| detail before programming begins. We ... [instead use] the
| alternative, incremental growth ..."
|
| Today, the term "tech debt" includes sloppy code, shortcuts,
| novice code -- really any kind of bad code. The original
| conception of tech debt is more limited and tied to waterfall
| vs iterative process choices. [2]
|
| [1] Ward Cunningham, The WyCash Portfolio Management System,
| OOPSLA 1992. https://c2.com/doc/oopsla92.html
|
| [2] George Fairbanks, Ur-Technical Debt, IEEE Software 2020.
| https://ieeexplore.ieee.org/document/9121630
| zarathustreal wrote:
| I would argue that there is the option to look at all the
| requirements at once. The disconnect with your mental model is
| in the definition of the piece of software being designed.
| You're considering all iterations to be the same piece of
| software, despite ostensibly having different sets of features
| and functionality (e.g the ones added over time).
|
| Consider the alternative perspective that each new feature
| actually makes it a new piece of software. You were able to
| fully design and specify the original piece up front, thus
| avoiding technical debt on that piece of software.
| wavemode wrote:
| I don't think that perspective really holds up to practical
| scrutiny. The most common sources of tech debt in the real
| world are when new features are tacked onto existing software
| without integrating with them fully/smoothly.
|
| As a relatable example, think about how the Control Panel in
| Windows still uses a 2000s-era UI, even as the Settings menu,
| and most of the rest of the OS, uses a new UI. The likely
| reason for this is that it was faster to tack on a new UI and
| leave the Control Panel as-is. It would've taken more time to
| refactor the Control Panel from the ground up to accomodate a
| new UI.
|
| The end result is that there are now two separate settings
| interfaces, and there is probably some ongoing engineering
| effort required in maintaining both and keeping them coherent
| with one another. That's a classic example of tech debt -
| save time now, but it may cost you later.
|
| But, by your definition, the old UI and the new UI are
| separate pieces of software, therefore there is no debt. How
| does that track?
___________________________________________________________________
(page generated 2024-06-09 23:02 UTC)