[HN Gopher] Tech debt gets worse before it gets better
___________________________________________________________________
Tech debt gets worse before it gets better
Author : piinbinary
Score : 124 points
Date : 2022-01-29 18:00 UTC (1 days ago)
(HTM) web link (jeremymikkola.com)
(TXT) w3m dump (jeremymikkola.com)
| tconfrey wrote:
| Nicely put. I've always liked the 'rotating the tires while
| driving 65 on the highway' metaphor for a big tech-debt/refactor
| project.
|
| I've worked in a couple of large and messy codebases where we
| undertook a big refactor, it took longer than expected and then
| the resources were pulled to other things before it was
| completed. Net result: an even larger, messier and harder to
| understand codebase!
| errantmind wrote:
| Tech debt is only a useful label for communicating with non-
| technical people.
|
| As a concept it lacks all nuance. I argue it is one of those
| vague, amorphous blobs that means anything to anyone, to the
| point that no one agrees on anything when discussing it, except
| that it exists. Everyone thinks they know what it is though,
| which makes it all the worse.
|
| I advocate we discuss real issues with code and architecture
| instead of using the term 'tech debt'. Articles like this are
| like going to a self-help speaker. You walk away feeling great
| but also totally unequipped to change anything because the advice
| was contrived and only works in non-existent general situations.
| [deleted]
| mmcnl wrote:
| It doesn't mean anything. Does a defined work package improve
| the product you are building in any way? Then plan the work. If
| not, then don't do it. I discourage using this term within my
| team, because it's not a transparent term.
| Retric wrote:
| It's a useful way to frame things in technical discussions.
| Everyone wants to refactor ugly code, but unless the team is
| paying interest because the code is bad it's rarely worth it
| just because.
|
| Aka if it's 5 years old and nobody had needed to edit it then
| yes it's ugly but the interest rate is probably low. Meanwhile
| stuff that just a little off but gets touched every week can be
| much more meaningful.
| errantmind wrote:
| I don't think it is at all useful in technical discussions as
| it anchors the discussion in backwards reasoning (deduction).
|
| Better to start with real observations about the workings of
| a particular piece of software or architecture, then reason
| about why it may have certain 'bad' properties. This more
| often leads to action and mutual understanding, but requires
| actually putting in some mental effort.
|
| In your example, calling some code 'ugly' is another example
| of what I'm saying not to do, it is starting at a conclusion
| and reasoning backwards.
|
| For an example of what I'm advocating, say you started a
| project and defined certain architectural principles after a
| series of discussions with stakeholders. Let's say one of
| those principles is efficiency/speed of execution because the
| stakeholders use the software synchronously. So, _that_ is
| how you define code quality in that project. You benchmark
| and eliminate hotspots until you reach acceptable
| performance. Independent of context, fast code may appear
| ugly to some people, but in this context it is not, perhaps
| it is even beautiful as it so embodies and elevates the
| architectural principle it was designed around.
|
| Developers now have a frame to surface issues and everyone
| can agree on what an issue is, because you defined what an
| issue looks like and why it is important, up front, as one of
| your architectural principles. This is basically a context-
| bound value system. There is no need to even use the words
| 'technical debt'.
|
| In a real project you have multiple competing architectural
| principles but these are prioritized so you know how to
| evaluate whether or not something is 'good' or 'bad'.
|
| As a project leader I believe it is your job to define these
| architectural principles, to enable developers to frame 'good
| and bad' as well as to prioritize their work.
|
| I used this approach when leading a project and it was very
| effective.
| Retric wrote:
| If your code isn't fast enough to meet spec, that's just a
| defect. QA should create a ticket even if you don't.
|
| There are however a meaningful class of issues that have
| zero direct impact outside the team, but are still issues
| for your team. The best example of that is how long it
| takes to setup your build environment, frankly your users
| have zero reason to care but it can still slow your teams
| progress over time. Really the cost and payoff of fixing it
| is simply time / a better work environment.
|
| Different teams will of course use technical debt to
| describe stuff outside of that context, but if it directly
| benefits users then it's really something else.
| mmcnl wrote:
| What is the impact on the end user? Will anyone notice or
| care if you don't address the technical debt?
| Retric wrote:
| In the platonic ideal of technical debut it should have
| zero direct impact on users. Indirect impacts are a
| slowdown in the release of new features, and or bug fixes
| etc.
| mmcnl wrote:
| Impact is impact. Slower roll-out of future releases is
| already way more specific than a vague arbitrary term
| "technical debt", right?
|
| I challenge you to come up with a situation where
| technical debt is a better way to describe activities
| than indirect impact as you are doing here.
| Retric wrote:
| How about someone saying "The X has _."
|
| For one it's more specific. Saying the existing code _has
| indirect impact_ doesn't even define if it's a good or
| bad impact. If you want to say, the existing code has
| _long description that just means technical debt_ then
| sure they mean the same thing but the term technical debt
| is more concise.
| chinchilla2020 wrote:
| > ugly code
|
| Ugly code doesn't cause issues. Refactoring a reliable,
| functional, easily maintained codebase because it isn't
| elegant (or 'beautiful') is a waste of time.
| Retric wrote:
| I think you're describing something else. The defining
| characteristic of ugly code is it's hard to reason about
| and thus not easy to maintain. It can still reliable,
| functional, and ugly.
| dilyevsky wrote:
| People (myself included) usually overestimate their
| ability to ship more readable code that has bug-for-bug
| parity with complex existing codebase.
| Spivak wrote:
| I think step zero for this kinda thing should be writing
| a test suite using the current code as the reference
| implementation. Then you spike it out with a first pass
| and if your clean version can't get 90%+ without lots of
| edge case handling or abstraction breaking then you keep
| what you have.
| dilyevsky wrote:
| That's exactly my point - chasing that 10% remainder will
| make the new thing similarly hacks-ridden as the original
| NAHWheatCracker wrote:
| Talking about "real issues" doesn't always work.
|
| A few years ago, a director at my organization said at an all-
| hands meeting that he viewed "tech debt is a debt we never have
| to pay and have no interest on". In one swoop, he told the
| entire organization that technology concerns are not valid.
|
| People have been complaining for years using specific issues.
| We could point at times that important services went down. We
| could point at full time operations people who have to babysit
| applications. We could point at failed initiatives. The bi-
| annual company survey always has "we're fighting the same
| fires" in the top issues.
|
| I suppose the issue with talking about tech debt to non-
| technical people is that they don't care. It doesn't affect
| them. At least not directly, like it does for developers.
| ma2rten wrote:
| It sounds like that company was just dysfunctional.
| NAHWheatCracker wrote:
| There are good things and bad things. There is nuance. I
| haven't and can't provide the whole picture. I just
| provided one anecdote that I felt is relevant to the
| discussion here.
|
| In re-reading my comments, I think it's easier to interpret
| what I wrote in an only negative way. I think that dealing
| with technical debt is would be a topic that the company
| does poorly.
|
| On the flip side, they build new products quickly and use
| new tech often. I consider the organization to be similar
| to other big-company feature factories. There's a degree of
| dysfunction in that, but also has a purpose.
| mmcnl wrote:
| If it doesn't affect the non-technical people, is it an issue
| then? I tend to be on the side of your director to be honest.
| NAHWheatCracker wrote:
| It does affect them indirectly. More importantly, it
| affects the company.
|
| Engineer's lose motivation and put less effort in. The
| company has to spend more budget on operations. Emergencies
| come up. Projects fail to deliver on-time.
|
| These things are the tech debt and the interest being paid.
|
| He had a point in some cases. There are products that
| aren't used or aren't under active development.
|
| He was lumping all of the tech debt together. He was saying
| to everyone in the organization that he viewed all tech
| debt this way.
| mmcnl wrote:
| Ok, so there is impact for them? Then elaborate and be
| specific. I'm pretty sure that any sane manager or
| director will listen when you tell you them you can save
| on operational costs. Just be specific and explain what
| you can do and what it will achieve. There's literally 0
| need to use obfuscating non-specific vague words such as
| "technical debt".
|
| Everytime I see someone mentioning technical debt I take
| as a signal that there is a vague idea of opportunities
| for improvement, but no one has taken the effort to
| quantify the costs and benefit in a specific way. If you
| see yourself using the words "technical debt" as an
| engineer, it means you need to do your homework and be
| more specific and ask yourself what you're actually
| saying.
| Spivak wrote:
| I think I agree wit the general sentiment though. I don't
| think "tech debt" in abstract is an actionable thing.
|
| Instead it should be pitched to the business as "we should
| change how we architect $service because the time spent on
| cards which interface with it are mostly spent wrangling
| rather than actually implementing the desired features. With
| $new_model here's the code for what those same cards would
| have been and we expect that features of this nature will
| take 4 hours instead of 40."
| tyingq wrote:
| I agree. It's used for things that really do feel like debt.
| Perhaps a dependency on a library that no longer has support.
|
| But then it's also used for subjective things like "too
| monolithic" or "spaghetti code" where the value of fixing might
| not be clear. Or even that the issue is really an issue.
| drewcoo wrote:
| Calling this tech debt stretches the metaphor too much. We're
| missing terms for this common kind of problem.
|
| "We decided not to FOO" is already called tech debt.
|
| Say "we MUST BAR but never planned for it" is an _Oops_.
|
| And "SQUISH lacks any sense of consistency or intentionality" is
| a _SNAFU_.
|
| And a "we don't all even have common language to talk about what
| we need to do" is a _Whoopsie_.
|
| Then maybe we can also introduce sparkline graphs to help us
| visualize the effort needed to overcome it all: the _Whoopsie-
| Doodle_.
| softwarebeware wrote:
| Agreed. Tech debt had a specific meaning of intentionally
| taking out a loan with a known larger payoff for investment in
| a known horizon.
|
| The garbage crap dumpster fire of code at your company is not
| tech debt; it's incompetence on display.
| louwrentius wrote:
| Thanks, I feel that "Tech Debt" is an abused term to make
| mistakes sound like they're not mistakes but something else to
| evade responsibility.
|
| If you fucked up, that's not technical debt. If business
| requirements change, you don't have technical debt
|
| Calling every problem or challenge technical debt is so
| unhelpful and frankly a bit dishonest too.
| _puk wrote:
| Process debt is another one I've invariably come across and
| covers the need for loose requirements (especially early on).
|
| There's a lot of technical debt that's going to accrue before
| the process debt is sorted if you're trying to scale quickly.
| [deleted]
| louwrentius wrote:
| If you build with the explicit goal to learn and explore
| the process and problem domain, and then have to rework
| code because of new insights and understanding, we are as
| close to the original definition of technical debt.
|
| But it is my impression that this is the rare exception. In
| most cases, people are just bullshitting to cover up their
| ineptitude.
| nerdponx wrote:
| I tend to use "tech debt" as a euphemism for _other_ people
| 's bad code and design decisions. Work politics.
| louwrentius wrote:
| Ha, I think that's the most legit usage of the term.
| nijave wrote:
| I think these others still lead to "tech debt" when you decide
| to ignore or divert the work.
|
| >"SQUISH lacks any sense of consistency or intentionality"
|
| And once you acknowledge the problem and decide not to address
| it, it's tech debt again.
| wccrawford wrote:
| I don't see that as disagreeing with the parent in the end.
| The key words are 'decide to', which the parent already used.
| Once you recognize a problem and decide not to deal with it,
| it's technical debt. You're avoiding paying for the problem
| for some reason. Perhaps you hope to never deal with it, or
| perhaps you think you can get better return on your
| (time/energy) investment working on something else first.
|
| The point stands: Unforeseen problems aren't 'tech debt' in
| themselves. They become tech debt once they are recognized
| and not dealt with.
| nmyk wrote:
| I like this. Here are a few more:
|
| "We maintain a lot of BAZ we think no one needs anymore but we
| don't know for sure" is called a _Yikes_.
|
| "You just have to know which alerts matter and which don't" is
| called an _Uh-oh_.
|
| And of course "securing the QUUX wasn't an explicit requirement
| for launch so we haven't done it yet" is already called a
| vulnerability.
| recursivedoubts wrote:
| i worked at a company a while back that decided it was time to
| refactor due to application complexity, which was driving
| developer productivity down significantly
|
| a team of elite coders and a senior architect was put in charge
| of the project, and they decided to implement IoC & OSGi
| throughout the code base and split the repos up into cleaner
| modules
|
| a year and a half later, they were done. glowing company wide
| emails were sent, it was a new era
|
| things got significantly worse: the new system was very difficult
| to understand, testing cross-module issues was impossible to
| coordinate, OSGi added almost no value at a heavy cost across the
| code base, etc. developer productivity, already low, fell even
| further
|
| the senior architect left shortly after the project was completed
|
| careful with that refactor, eugene
| almostdeadguy wrote:
| I'll argue that things like this have nothing to do w/ the
| merits of refactoring in general and are usually due to poor
| project management.
|
| A small team orchestrating a large scale refactor in isolation
| for over a year seems appealing to project managers because it
| allows that team to focus without interrupting the pace of
| feature development, but it makes it exceedingly likely that
| the output of that effort will fail for these reasons:
|
| 1. Evolving requirements. A refactor is a design that is
| focused on solving challenges _at that time_. Things happen
| over a year. A product grows, its supported feature set
| requires rethinking the nature of the domain, the pressures of
| scale force new considerations, teams get more rigorous about
| the kind of automated testing they practice, etc. A team that
| doesn't have a feedback loop into is likely to build something
| that may have worked for the system of a year and half ago, but
| no longer makes sense.
|
| 2. Lack of buy-in and knowledge sharing. In isolation, most
| software engineers tend to solve their own problems. Senior
| engineers often make assumptions about the knowledge of the
| rest of the team and about the coherency of their abstractions
| when they don't have to teach it to anyone or battle test it in
| any meaningful way. This is why collaborative processes like
| code review and pairing are often a huge win, but sharing these
| efforts needs to go farther and a team should be pushed to try
| incremental adoption and feedback until there's more confidence
| about a refactor. Large scale refactors should be chunked and
| interspersed w/ a workload that attempts to validate the
| success of the design (i.e. working alongside teams doing
| feature development to make use of the refactored design). And
| perhaps most importantly: the team working on this needs to
| have clarity on what it means to be successful. How do they
| know that their proposed design is meeting the needs of the
| team at large? Infrastructure, dev ex, or any team that is
| orchestrating something like this needs to think about their
| task like a product team, and "what does success look like"
| needs to be a question from the onset.
| recursivedoubts wrote:
| problem w/ that is that "is refactoring a good idea" becomes
| unfalsifiable: you can always blame something else
|
| this is sort of like w/ the agile folks: when an agile
| implementation fails, "oh, you didn't do agile right"
|
| obviously, better code is always desirable, but the
| temptation to rewrite is very strong in developers (I feel it
| all the time) and experience suggests it often leads first to
| a worse situation, and then a worser one
| almostdeadguy wrote:
| Like everything, you need to have clarity on what you're
| trying to achieve and when to pull the ripcord. I'm saying
| you need to frame the task of a large scale refactor so
| that the planned implementation is falsifiable.
|
| A rewrite is usually a quagmire if you don't do these
| things, but even the need for a rewrite is usually
| avoidable. If a team is making frequent observations about
| pain points and places value on fixing them quickly, they
| can often be avoided. But there are discontinuities:
| product concepts and features that dramatically change the
| constraints of a system late in the game, unanticipated
| growth requiring you to reconsider architectural decisions,
| etc. There's a long tail of events that you could be
| proactive about, but would likely create over-engineered
| solutions that place more limiting constraints on the
| design if you tried to be. They still occur though. It's
| very important to sufficiently interrogate the question of
| if a rewrite is necessary to solve these challenges, but
| you also need to interrogate the cost of band-aid solutions
| w/ limited applicability.
|
| I don't think there's a silver bullet here, but most of the
| failed attempts at doing this that I've seen have been a
| result of teams that didn't sufficiently consider what the
| outcomes they were trying to provide were and how to test
| their candidate solution more rapidly.
| hedora wrote:
| I've noticed organizations with lots of tech debt promote
| exactly the people that created the problem in the first place.
|
| Giving that group a big budget and autonomy to fix the issue is
| like paying tobacco company to cure your lung cancer.
| Jtsummers wrote:
| The people who create a lot of tech debt (and other problems)
| are often highly visible and _appear_ to Get Things Done. In
| fact, they may _actually_ get things done, but in a poor
| fashion (in this context).
|
| Repenning & Sterman, _Nobody Ever Gets Credit for Fixing
| Problems that Never Happened_ [PDF] https://web.mit.edu/nelso
| nr/www/Repenning=Sterman_CMR_su01_.... (pretty sure this is
| the one I want)
|
| Being proactive and doing the right thing (maybe, potential
| higher costs earlier than they can be supported for instance)
| is a low-visibility activity. Even if it's your _job_ to do
| the cleanup work, the one who wrote it _first_ is the one
| most likely to get the credit no matter the quality. Because
| they got something out there for customers, not you.
| ZephyrBlu wrote:
| A funny comic describing this exact issue:
| https://twitter.com/shreyas/status/1391594520207233024.
| dilyevsky wrote:
| That's right if someone shipped a buggy unreadable codebase
| the first time what makes anyone think they are actually
| capable of doing the opposite second time around?
| gonzo41 wrote:
| Companies exploit their workers, and sometimes workers exploit
| their companies by selling them a dream and leveling up to a
| new gig. I would be wary of hiring someone who came off a
| project like that who bailed from a position just as things
| were delivered.
| chinchilla2020 wrote:
| Leaving during the middle of the project would be worse.
|
| This is an attrition issue. The company lost an employee,
| fair game.
| tartoran wrote:
| I've seen some of that as well. After a very large
| refactoring is done it is initially claimed a success by the
| lead but to only slowly dawn on the rest of the team that the
| codebase is not better, just more complicated without any
| significant gains. Mess is still there, it's just a
| unfamiliar one. Questions to the lead of the refactoring
| project cannot be answered as they are long gone some other
| place.
| ma2rten wrote:
| It actually seems like a good sign to me that they completed
| their current project before jumping ship.
| dntrkv wrote:
| Of course that could be the case, but in my experience , the
| inverse is way more common. A team is tasked with
| rearchitecting some part of a system, and by the time they're
| done, everyone is burnt-out and one-after-another the
| engineers leave the company.
|
| The goal was never to "level up", it's just that these large
| deliverables take a lot out of you.
| ptr wrote:
| Would be interesting to hear more about this -- why did they
| think the new way would improve the situation? Why was the new
| solution harder to work with? Why was it hard to understand?
| tartoran wrote:
| Wrong refactoring? Added new unnecessary complexity? I've
| seen refactoring gone wrong, not specifically OSGi, as I have
| no idea what it really is.
| umvi wrote:
| The rewrite should be done by the people that will be
| maintaining it, not "elite coders" that aren't going to touch
| it ever again afterwards
| hyperman1 wrote:
| Almost every split between maintainers and creators will kill
| your productivity stone dead a week after delivery.
|
| Not only have you guaranteed total loss of knowledge, someone
| who had never maintained anything has no chance of writing
| code that survives contact with production.
|
| You'll get all sorts of shiny frameworks and nice
| architectural dreams, but no logging, no monitoring, no
| validation of the actual user requirements, no nothing. You
| spend the next 5 years rewriting from scratch if you're
| lucky.
| Codesleuth wrote:
| Seems like the article is going through quite some effort to
| describe what is essentially Expand/Contract or Parallel Change
| patterns.
| throwdbaaway wrote:
| I think it is rephrasing
| http://mikehadlow.blogspot.com/2014/12/the-lava-layer-anti-p...
| from a different angle.
| zerop wrote:
| From my experience, the big bang rewrites for addressing tech
| debts or simplifying a complex system, doesn't work for most.
| Break the large tech-debt/simplification goal into smaller
| milestones which roll up to a clean end state. Business won't let
| you address large tech debt easily, as they don't care about
| "your" tech debts. Tech debts must be address slowly but
| consistently in background. The chances of this working is higher
| IMO.
| xorvoid wrote:
| Ha. As a kid, I tried using this argument with my parents when
| they told me to clean my room. They came back an hour later and
| it was messier and I said "It has to get messier before it gets
| cleaner". It legitimately was not an excuse. But I failed to
| convince them.
| lobocinza wrote:
| Does it get better? The debt can keep growing longer than you can
| remain solvent.
| matt7340 wrote:
| I've noticed that often this is an area where WIP is not limited,
| and the efforts are least likely to be well managed and
| documented (unsurprisingly, it can take a lot of work to keep
| up).
|
| I've also noticed numerous long term migrations that don't tie
| back to a strong long term value proposition. And when the claim
| is that it does, it's often just overly optimistic gazing into
| the crystal ball. Just as dubious as multi-year product roadmaps,
| for most businesses a ton of unpredictable things will occur.
| [deleted]
| 29athrowaway wrote:
| Consider 4 situations:
|
| 1) My house has structural problems and is about to collapse.
|
| 2) Each time I turn on the lights in my house, I start a fire.
|
| 3) Each time I need water in the kitchen, I need to go get it
| from the second floor.
|
| 4) I don't like the house color.
|
| You can refer to these 4 situations saying something vague like
| "My house has a problem". The term "tech debt" is similar in
| terms of vagueness.
|
| Not everything described as "tech debt" is objective or
| actionable. And the term "tech debt" facilitates neglecting
| things that need to be done by mixing them with things that do
| not need to be done.
|
| "There is an antipattern that prevents developers from testing
| code" is a problem that needs fixing. "I don't like tabs" is not.
| wccrawford wrote:
| The first 3 of those act like debt, forcing you to spend more
| of something in the future if it's not changed. The last one is
| only a debt if you consider the emotional cost, which I think
| is debatable.
|
| Yes, some people use 'tech debt' improperly and aren't actually
| describing something that acts like a dead. Poorly structured
| code means the coders need to spend more time implementing new
| features than they would if it were just correct, for example,
| so that's actually 'technical debt'. Not taking care of the
| business's needs isn't technical debt (faster onboarding, etc),
| it's just a problem.
| yetanotherjosh wrote:
| Your criticism applies to the word "debt" in the traditional
| financial sense as well. If you want to be more specific about
| how much and how severe the debt is, that is not a fault of the
| the term "debt" or a reason to stop using it.
|
| 1) I owe $0.25 to my 5yro niece because I lost a bet about
| whether grandma's pancakes would be overcooked. She will
| probably forget about this as soon as Harry Potter comes on the
| TV.
|
| 2) I owe $50 to my housemate for groceries but actually I paid
| the electric bill and he didn't so maybe he owes me instead?
|
| 3) I have an $8000 balance on my credit card but can pay my
| minimum balance plus $500 extra every month with no sweat.
|
| 4) I have $500000 left on my mortgage and didn't make my last 5
| payments...
|
| These are all still "debt" and have varying degrees of
| actionability. A word that implies a dimension does not need to
| specify the magnitude or severity on that dimension.
| ksec wrote:
| _Things in general when they start to fix them, get worst before
| they get better. And if you believe that is true, the world is
| going to get A LOT better soon_.
|
| -Steve Jobs.
| michaelcampbell wrote:
| Never heard this one from Steve, but this is akin to thinking
| moving your speedometer dial down makes the car go slower.
| ksec wrote:
| It was from one of the D8 conference.
___________________________________________________________________
(page generated 2022-01-30 23:02 UTC)