[HN Gopher] Balancing "If it ain't broke, don't fix it" vs. "Rel...
___________________________________________________________________
Balancing "If it ain't broke, don't fix it" vs. "Release early and
often"
Author : yarapavan
Score : 102 points
Date : 2022-09-23 12:10 UTC (10 hours ago)
(HTM) web link (www.redhat.com)
(TXT) w3m dump (www.redhat.com)
| pdpi wrote:
| Lord Vetinari in the Discworld lives by the mantra that "If it
| ain't broke, don't fix it". Other characters noted that, under
| his governance, "things that didn't work ... got broken."
| silentsea90 wrote:
| How about use Feature flags and ship daily? Decouple your
| release-schedule/deployments from your product iteration/launch
| timeline.
| allendoerfer wrote:
| Never miss an opportunity to add complexity!
| silentsea90 wrote:
| Centralized coordination complexity v/s decentralized FF
| rollout complexity. The bigger the team contributing to a
| deploy, the better the latter gets because engineers rolling
| out their FF have all the context they need.
| int0x2e wrote:
| Feature flags or another way to release bleeding edge
| features to only some of your users can sometimes be the best
| way to iterate over new features quickly when your most user
| base is risk-averse
| generalizations wrote:
| Unofficial RedHat motto.
| alldayeveryday wrote:
| I think I know your meaning, though I would rephrase it as
| decoupling your deployments from your releases/launches.
| silentsea90 wrote:
| Edited.
| jbverschoor wrote:
| Those are two different things.. Nothing to balance about. But
| put it in a graph or drawing, and it means it's true and
| supported by 'science'
| exabrial wrote:
| The problem is a bunch of overpaid devs conflate anything that
| isn't absolute bleeding edge as a "bug", and we get into and
| endless cycle of UI refreshes and updates that exist solely to
| exist.
| arminiusreturns wrote:
| Release to prod so you can fix stuff in lower envs.
| sylware wrote:
| Grotesque generalities. The devil hides in the details: planned
| obsolescence and redhad (IBM) with its glibc symbol versionning
| frenzy and not a clean ELF ABI set of libs (as an example).
|
| Yep. Open source is not from enough, lean and stable in time open
| source is required.
| GnarfGnarf wrote:
| _" If it ain't broke don't fix it"_
|
| Would you fly on an airline that followed this principle?
|
| Wait till the engine fails before fixing it.
|
| In the air.
| eloisant wrote:
| They're not opposite.
|
| "If it ain't broke don't fix it" => don't try to "optimize"
| something that works well. Maybe it will be able to handle more
| load after your big refactor, but if it already handle the load
| nicely and there is no indication load will go up, don't bother.
|
| "Release early and often" => release after each change, don't
| bundle everything into a big release that happen every 6 months.
|
| The first one is about deciding what to do, the second one is how
| you release the work you did.
| drewcoo wrote:
| Exactly! False dichotomy.
| JohnFen wrote:
| > "Release early and often" => release after each change, don't
| bundle everything into a big release that happen every 6 months
|
| After years of dealing with software that does this, I've
| soured on it completely. It has resulted in greater difficulty
| using software, and the software being less solid.
| mdtusz wrote:
| I've seen this happen as well, but I think it's more of a
| problem of bundling too small a changeset in a release. A
| partial implementation of a feature shouldn't warrant a
| release (without feature flags to disable it), but bundling a
| year's worth of work in a single release is, in my opinion, a
| recipe for disaster (and this is sadly something that happens
| all too often in enterprise and non-software engineering
| businesses).
| michaelcampbell wrote:
| I've seen this happen as well; perhaps for different reasons.
|
| My early career was mostly fin-tech and when installing into
| banks or other financial institutions, they _very much_ want
| the every year or 6 mo release cycle. And not just because of
| "old stodgy", but rather they want a good bit of ramp-up time
| to ensure things are running right. In a Wall St trading firm
| I worked at, a release ran in parallel with real data for
| MONTHS before the old version was retired.
|
| So, I'm used to that model.
|
| From the customer's standpoint, they could "gear up" for the
| next release, get their ducks in a line, get the right people
| to assess things. I get that some of that is because the
| infrequent releases are "big", but also as a human customer
| it's fun to see a shiny new truck toy rather than a coat of
| anti-rust paint on its exhaust pipe, then a replacement for
| the broken window crank, then a new color on the odometer
| numbers, then ...
|
| As a developer I also miss the Big Project Ramp Up, then a
| period of a lot of work (which can be Waterfall, or Agile, or
| bespoke artisinal project management, which is pretty much
| 99% of the time, just CALLED "Agile"), then a Big Project
| Release (+ party, some down time, and prepare for the next
| one).
|
| SAAS release early/often is like death by 1000 boring dull
| cuts. They cut out the spikiness _so much_ that there is so
| little novelty that it's just not fun to work on it.
| bradwood wrote:
| These are not mutually exclusive. It seems like the whole premise
| here just lacking nuance.
| hulitu wrote:
| > Customers often ask me, why pour time into updates when what I
| have runs just fine? The answer is that a bug in your old version
| might assert itself at the worst possible time and force you to
| forego sleep for the next five days to recover
|
| Why not do proper testing ? I know it's expensive. And when you
| have an OS as an init system is even difficult. It is sad the the
| UNIX philosophy is dying being replaced with the Windows
| philosophy.
| codegeek wrote:
| You cannot test everything to perfection. Shit will break in
| production. Question is: how to minimize shit breaking in
| production and hopefully eliminating critical bugs that can
| cause revenue/data loss.
| torstenvl wrote:
| Simple: Release Early. Release Often until it Ain't Broke. Don't
| change things that Ain't Broke.
|
| I don't think there's much consternation over actual bugfixes.
| There's consternation over constantly-shifting UIs and APIs.
|
| The only time there's any tension between those two things is
| when people conflate bugs with poor design. The messy but correct
| thing to do when you've correctly implemented a poor design is to
| maintain backward compatibility while _also_ offering the
| _option_ to use the newer, more correct behavior. See, e.g.,
| strcat() - strncat() - strlcat()
| moffkalast wrote:
| Meanwhile Canonical: "Let's fix what ain't broke at least once
| every 2 years."
| gw99 wrote:
| At least it's not Gnome "Let's rewrite again what we didn't
| get around to finishing and fixing 2 years ago"
| klodolph wrote:
| (I think it would be strcat() - strlcat(), since strncat() is
| not really safer / newer.)
| david2ndaccount wrote:
| Just always use memcpy, you should always know the size of
| your buffers and if you don't you _will_ get a buffer
| overflow at some point.
| klodolph wrote:
| I've heard this advice before, and the problem with this
| advice is that people who just use memcpy in practice often
| write code with bugs in it. So the advice is bad advice.
| Good advice will result in code that has fewer bugs, or
| bugs with lower severity.
| kevin_thibedeau wrote:
| It can also lead to accidentally quadratic behavior which
| is why the modernized string ops that get so much hate
| these days rely on one iteration over the input.
| cpeterso wrote:
| btw, strlcpy() is now being removed from Linux kernel code.
| strscpy() is the latest "secure" string copy function:
|
| https://lwn.net/Articles/905777/
| torstenvl wrote:
| Definitely not safer (perhaps even more dangerous due to its
| unexpected semantics), but I do think it's slightly newer. It
| doesn't appear in K&R 1st edition, released in 1978, but does
| appear in K&R 2nd edition from 1988.
|
| OpenBSD claims it first appeared in Version 7 Unix, which was
| released in 1979.
|
| https://man.openbsd.org/strncat.3
| https://en.wikipedia.org/wiki/Version_7_Unix
| pabs3 wrote:
| Preferably also provide assistance to people using the old
| things, in the form of making the old API a wrapper for the new
| API where possible, adding warnings for the old API, adding
| migration tools etc.
| proc0 wrote:
| The line of when it's ready to be used is not clear and I think
| it has been pushed forward such that the user increasingly
| becomes the tester. This is unacceptable in hardware products,
| imagine if kitchen appliances were sold with issues, and you
| had to take it back to the store for fixes and updates.
| Unfortunately, users don't know any better, and I think that's
| why so much of the industry gets away with serving them a bad
| user experience that is bad because in essence you are testing
| it.
| kqr wrote:
| The software we write is far more complex than a kitchen
| appliance.
|
| For simple systems, yes, test all components, build in
| redundancies, etc. All those things that made reliable system
| design possible in the 1950s.
|
| For complex systems you're quickly wasting your time on
| diminishing returns. The failure modes of complex systems lie
| in obscure interactions between components conditional on
| rare input combinations. You _have to_ find that stuff out in
| production. You _have to_ make users into testers.
|
| However, there are better and worse ways to do this. Ideally,
| a failure should only be experienced by a user with a
| fallback solution, not a plain failure to accomplish the task
| entirely.
| gdsdfe wrote:
| "If it ain't broke, don't fix it" is the absolute worst mindset
| to have! I've seen systems that are literally garbage because of
| it.
| scombridae wrote:
| tl;dr continually test, continually deploy.
| henriquecm8 wrote:
| Release early, don't fix it
| tomalaci wrote:
| Creating bugs? No, I am creating learning opportunities for
| more junior developers.
| devteambravo wrote:
| Are you my manager?
| BurningFrog wrote:
| "Move Fast and Break Things" at maximum speed!
| badrabbit wrote:
| This is the way.
| onetokeoverthe wrote:
| jrochkind1 wrote:
| I think libraries (used as dependencies by other software) and
| "top-level" software/applications (used by users directly,
| whether that user is a developer or not) -- have different
| pressures and concerns.
|
| Libraries should be much more conservative. I don't think
| "release early and often" was said about libraries.
|
| And yes, it's tricky that it's not always a clear line between
| these two categories. A unix command line utility is sort of top-
| level software, but also likely to be use in a scripted/automated
| fashion.
|
| So it's not always cut and dry, but the more you are aware of
| software depending on your software, the more careful you should
| be with releases.
| wongarsu wrote:
| A great example of an end-user application that is like a
| library is Microsoft Excel. Sure, the UX can change, but the
| formula engine has to be extremely conservative, staying bug-
| compatible with basically every version, or all hell breaks
| lose.
| wpietri wrote:
| I think it depends a lot on where the library is in its
| lifecycle.
|
| A new library should iterate rapidly and be very clear about
| that. Big warnings, obvious version numbering, and good
| communications. You look for an audience of actual users that
| can keep up with that. That's how you figure out what the right
| library is, both in terms of the big abstractions and the small
| details.
|
| But then once you hit a 1.0 release, things change. You're
| shifting from an audience of fellow explorers to an audience
| who wants stability. You can still do exploratory work, but it
| has to be additive in the 1.x series, and eventually you need
| to start a 2.x series as you learn more about what needs to
| change. So the iteration still happens, it just happens away
| from the people who just want the basic thing you got right via
| the initial burst of iteration.
| fendy3002 wrote:
| Then let me introduce you to https://0ver.org/.
| wpietri wrote:
| I am suitably horrified.
| tetha wrote:
| I notice the same pattern on an infrastructural level.
|
| If I work on a test system, or a system a small number of
| people depend on... a year or two ago this would've hurt my
| pride, but why spend 4 hours planning a change, if you can just
| muddle through problems in 2 hours. The user impact will be
| none or low based on the assumption, so move quickly, break
| things, fix things, document the problems and fixes.
| Everything's good.
|
| Yet, I also have systems a dozen teams or more rely upon. For
| these systems, we have to move much slower and more deliberate.
| We cannot touch some of our database with larger changes,
| unless we have a way in, a way out, and a prediction how long
| all of these take, as well as announcement to the customers,
| and so on, and so on. We'd love to update those faster, but
| it's rough.
| JohnFen wrote:
| I agree. As a developer using libraries and component, I really
| hate "release early and often".
|
| As a user, I also hate "release early and often".
|
| In both roles, releasing early and often effectively means I'm
| always using beta software, with all the headaches using such
| software brings.
|
| However, as part of an internal development process -- that is,
| before customers see the release, "fail fast" is a totally
| legitimate and decent approach.
| jjslocum3 wrote:
| I would love to read a Malcom Gladwell book on this topic. It
| touches on a core truth, not only in engineering but in society
| as a whole. Three examples from different spheres:
|
| In software, I've seen repeatedly how new hires with big ideas
| (and youthful confidence that they know a better way of doing
| things) will come into a company and want to rewrite things
| "properly." In one management position, much of my role involved
| the politics of defending a well-working, well-maintained, big-
| revenue-driving application from a steady onslaught of such
| ambitious exuberance.
|
| At the time, I thought "this must be the same phenomenon that
| drives shrinkflation": new MBA arrives at pickle-making company,
| seeks to bolster their career by demonstrably saving the company
| millions, convinces execs to put one less pickle in each pickle
| jar, consumers won't notice; step 3: profit!! In 2022, when you
| buy a cereal box, it's half air on the inside. In 1980, it was
| only 20% air.
|
| I also shake my head when media pundits equate the success of a
| particular congressional session with the amount of new
| legislation they pass. I can imagine no simpler way to cruft-
| paralyze a democracy (while following the rules) than "releasing
| laws early and often."
| klodolph wrote:
| > In software, I've seen repeatedly how new hires with big
| ideas (and youthful confidence that they know a better way of
| doing things) will come into a company and want to rewrite
| things "properly."
|
| In my experience, these new hires are right a certain
| percentage of the time, and wrong a certain percentage of the
| time. I can take no default position here.
|
| I tell new hires that they're gonna see something that offends
| their sensibilities about the way code should be written. That
| puts them in the center of a conflict. On the one hand, they
| believe the code should be written differently. On the other
| hand, someone else wrote the code that way for a reason. Their
| first step is to dig in and figure out the reasons on both
| sides--why it was written one way, why they think it should be
| written another. The second step is to figure out what the
| consequences and risks are of making changes or not making
| changes.
|
| The way I think about it, I want to simultaneously keep the
| software working, and protect the "flame" that new engineers
| have to improve things--something which can be all too easily
| extinguished.
| ftlio wrote:
| Reminds me of
| https://en.wikipedia.org/wiki/Wikipedia:Chesterton's_fence
| strawhatguy wrote:
| That's a sensible way to temper newbies enthusiasm with
| experience.
|
| Newbies may be right in that a system would be better
| rewritten; but often wrong that they think they can do it
| 'properly'. At least, if they dive straight into it.
|
| So making them use the Charleston's fence approach is a good
| one.
| strawhatguy wrote:
| Gah! Chesterton's
| SQueeeeeL wrote:
| Shrink-flation is driven by short term quarter earnings
| thinking. Reducing the amount of your product (and therefore
| it's overall quality compared to competitors), is an act of
| cannibalization of a brand.
|
| They may increase profits for a short time, but as consumers
| realize what's happened, ultimately they see the brand as not
| having standards and lead to it becoming a commodity. A small
| group of long term invested owners would never do such a thing,
| because that would be 'killing the goose who laid the golden
| egg', but modern shareholding has no loyalty to quality.
| esaym wrote:
| And if it ain't broke, fix it 'til it is!
| temporallobe wrote:
| We have relevant case of this. We're on Angular 8 and the app is
| working just fine - customer is happy, it's pretty easy to
| maintain, well-tested and hardened, etc., but we are now facing
| having to update to 14.x "or else". It's gonna be a long and
| difficult migration.
| kmitz wrote:
| Well depending on the size of your codebase, npm dependencies
| and Angular APIs used, it might or might not be a big deal. You
| can have a rough estimate of the complexity by running 'ng
| update'. Yes major releases come at a rapid pace and it is hard
| to always be on the latest version. We update regularly once we
| are 2-3 major versions behind and usually it's not so awful.
| chuso wrote:
| Break it often and don't fix it.
| ChrisMarshallNY wrote:
| Man, I feel this.
|
| I worked for a corporation that was so change-averse, that we
| needed to buy our development systems on eBay.
|
| They actually had a point, and it was hard to argue against, but
| I found it _absolutely infuriating_ , as the solution was to plan
| for change, and establish a process to be constantly evaluating
| and refining for new developments.
|
| Instead, it was "Wait until we can't bear it any longer, then
| have a huge screaming match meeting." A new budget would be
| approved for new machines (and, thus, new operating systems and
| development tools), processes would be updated, and that would
| sit, until it was no longer new.
|
| As I said, it was hard to argue against, because it was a
| 100-year-old company that had been successfully delivering
| _really_ high-end stuff, since Day One. They did that by being so
| conservative that they hadn 't discovered fire, yet. Measure 300
| times, cut once, etc.
|
| When there was a problem, it escalated quickly (and was used as
| fuel to tighten things down _even more_ ), as the company was
| held to standards that are probably up there with NASA.
|
| Speaking of NASA, I can't help but notice that this rather plucky
| little outfit, called SpaceX, seems to be running circles around
| them. They seem to have figured out how to "fail fast," yet also
| deliver _insanely_ high Quality stuff.
|
| Might be worth ignoring their CEO's tweets, and look at what they
| are doing...
| Spooky23 wrote:
| Don't fix what isn't broken is tough, because broken is
| debatable.
|
| Is your stable Univac mainframe application broken?
| Operationally, probably not. But your business cannot adapt
| becuase it relies on a platform for which everyone who knows it
| is dead.
| mgbmtl wrote:
| I'd be curious though, in 20 years, what kind of technical debt
| SpaceX will accumulate.
|
| I mean, as vendors, part of the debt we inherit is that of our
| customers. Then our customers switch to something else. As
| vendors, how do we support both the past and the future? By
| making changes opt-in if possible, but still, that technical
| debt is there, and requires devs and other support staff.
|
| I don't work in anything rocket-science, but I stopped asking
| clients if they want to upgrade. We decide when to do it for
| them, and we take the blame if something goes sideways.
|
| I still prefer to have something break, than lose a client
| because they didn't realize they could upgrade instead of
| opting for another product (and they probably silently grew
| resentful at the lack of features of the old version).
| soperj wrote:
| > I'd be curious though, in 20 years, what kind of technical
| debt SpaceX will accumulate.
|
| They're already 20 years in. Wonder if there's anything left
| from the beginning (or if it all got blown up).
| Beached wrote:
| I mean falcon 9 is the tech debt. they are no longer
| working on it, only maintenance mode. their goal is to get
| a new stack of tech working, before falcon 9 becomes too
| out dated. falcon 9 had such. ahead start, that it is still
| better than everything else in it's class. but it won't be
| long now until it because just another rocket in the
| market. I give 5 -10 years and it isn't special anymore.
| SpaceX is betting that starship will be larger and cheaper
| and working by then.
| izacus wrote:
| I mean it's pretty clear - the early Teslas all had dying
| onboard computers due to flash wear which they even refused
| to fix because the warranty has passed.
|
| Those aren't companies that build anything lasting or
| environmentally friendly. It's just all throwaway toys.
| Beached wrote:
| nasa also doesn't have the luxury of picking their own
| direction, or tech stack, or doing whatever they want. nasa is
| burdened by Congress dictation what rocket to build, even if it
| isn't the rocket nasa wants. look at the river program though,
| nasa still does cutting edge stuff with cutting edge hardware,
| at really high quality. just don't look at sls or iss
| pclmulqdq wrote:
| SpaceX shows the success of using modern, commodity hardware
| rather than purchased-by-government contract obsolete
| components for space applications. The same is true for the
| cubesat revolution.
|
| I honestly think the idea of "if it ain't broke, don't fix it"
| is pretty silly as applied to large, engineered systems. If
| your system is more than 1000 lines of code, I guarantee you
| have a latent bug somewhere. Even if you don't have any bugs,
| the ecosystem around you changes so fast that you eventually
| will.
| Gibbon1 wrote:
| SpaceX shows the advantage of not allowing aerospace
| lobbyists and congress to design things. That's how you get
| the SLS with it's inane H2/LOX first stage and solid
| propellant boosters.
| jillesvangurp wrote:
| Reducing cycle times for changes minimizes integration testing
| effort. Effort and risk increases non linearly with the amount of
| change and time. So, release as often as you can to minimize the
| risk and effort while balancing the cost of doing so. Some
| release processes just involve a certain amount of heavy process
| that takes time and money. E.g. app store releases are a PITA so
| doing that multiple times per day is not worth it. Server updates
| on the other hand are fine. Create a PR, tests pass, merge it and
| it goes live right away. We automated that process. A decision to
| merge implies rolling out the change. The minimum cycle time is
| the time it takes to build and deploy (about 7 minutes for the PR
| and another 10 to deploy) plus whatever time we need to do a
| change. Some changes are as small as 1 character. We have a
| production branch that we merge to from our master branch. We
| test the and use the master branch intensively and merge to
| production multiple times per week. No point in sitting on
| changes that work fine. Get it out and create some value for your
| customers and company and keep feedback loops short as well.
|
| Short cycle times is also why I use a rolling release linux
| distribution (Manjaro) and browser (Firefox). Always fresh and up
| to date. And even though I'm on the Firefox Beta channel I never
| have to deal with it breaking or being unstable. It's stable
| because they have frequent nightly builds. By the time builds hit
| the beta channel they are already rock solid. I was on the
| nightly channel for a while and never experienced many issues
| there either. Great example of short cycle times. With the Beta
| channel I'm a few weeks separated from changes happening and me
| seeing the feature. With the release channel it's another few
| weeks.
|
| Not updating because it aint broken is very valid until the time
| comes when you finally have to upgrade and all hell breaks loose
| because you are two years behind on dealing with breaking changes
| and have to do a massive project to make it happen. It was
| getting increasingly more broken while you were doing nothing;
| you just did not know about it. It's still technical debt. And
| now you get to deal with the non linear effort to fix it and pay
| the price.
|
| So, on all projects where I'm in charge we update everything very
| frequently. If something doesn't work I want to know ASAP and
| mitigate now instead of not even knowing stuff is not going to
| work for another few years. If you stay on top of changes like
| that, the effort for this is very low. Mostly stuff just works.
| Occasionally some library has an issue. And then we fix it, work
| around it or wait for the next version (and document why we can't
| update). Easy stuff. Basic project hygiene. The first thing I do
| when working on a project I haven't touched in a while is update
| dependencies. If I'm working on it all dependencies have to be
| current. I get annoyed with being a few minor versions behind. I
| might wait a few dot releases with major releases. But generally,
| I want to get that over with ASAP. If it breaks, I'll at least
| know that I need to deal with that. Rolling back is always an
| option.
| peteradio wrote:
| How do people disentangle "ship the incorrect thing and let
| customers let us know what they really want" vs "ship the correct
| thing but built incorrectly and let customers know why its
| wrong". I hate the second with a burning passion because it seems
| like its bound to give the impression that you really don't know
| what you are doing. I've been put into situations too many times
| where I just could not get the point across that "yes I
| understand that we may not be giving the customers what they
| want, but what you are telling me to build is clearly not
| correct" its maddening. Sorry if its off topic but it seems at
| least adjacent to the articles balance. This seems to happen with
| PMs who do not have a deep enough background in the technical
| detail and yet reject the feedback given because it will reflect
| that poor understanding. "Agile" environments seem to enable this
| kind of behavior. I'm beginning to think I need to pivot to PM in
| my career so I can avoid the madness as a developer.
| MetaWhirledPeas wrote:
| My personal take is that "built incorrectly" should almost
| never be tolerated. The customer doesn't need to know how it
| was built, merely how long it will take. But if by "built
| incorrectly" you mean an unwise business decision (logging in
| with a phone number for instance) then yeesh, I wish I had an
| answer. Putting your foot down sometimes means leaving a job.
| sumtechguy wrote:
| That depends on your customer. Some are cool with 'something is
| better than nothing'. Some are not. Some also are very 'I put
| it in the contract just do it that way' others are 'I just want
| the right thing'. Some just do not know what they want at all
| and just want you to do it and put all the liability off onto
| you but do not really want to spell out what that means at all.
| Being a PM is that turned up to 11 but now you do not control
| the code at all.
| peteradio wrote:
| I guess what I'm talking about is this:
|
| 1) PO wants feature X for customer P.
|
| 2) Architect in meeting with PM and PO indicates that feature
| X is a simple addition to Y
|
| 3) Engineer is assigned X and says no this isn't a simple
| addition to Y because on deeper inspection it will fail for
| cases 1,2,3, we would need a different architecture.
|
| 4) PM says build it the naive way, let the customers find
| case 1,2,3 before we fix, we are an Agile team after all.
|
| 5) I quit.
| nightpool wrote:
| Without details, it is really, really, really hard to say
| whether cases 1, 2, 3 are real, important issues that need
| to be fixed or just needless complexity that 99.9% of users
| are never going to care about. The PM is trying to
| distinguish between those two cases because they've been
| burned before on developers double, tripling, and
| quadrupling their estimates as they find "one more thing"
| that is inelegant or could potentially be improved and
| delaying the launch schedule. You need to convincingly make
| the case _to the architect_ (and to the PM) that issues 1,
| 2 and 3 are critical to the users, fundamental to the
| design and going to be much harder to clean up later if we
| don 't spend the time to get them right now. Or you're
| wrong, and you should just suck it up and build the simpler
| thing today and leave the complexity of the new thing for
| the future, when you can decide whether the feature is even
| worth the cost of having whatever new architecture it would
| require.
| MetaWhirledPeas wrote:
| This is a good answer. Determine how much the
| deficiencies matter and make your case. If you're being
| asked to ship something with a major deficiency and being
| ignored by the team you need to inform someone higher up.
| Not always the most fun part of the job.
| sumtechguy wrote:
| Exactly. Also to add it depends on your business model of
| how you are selling software. Some customers are
| perfectly fine with MVP and iteration. Others will want
| everything spelled out beforehand in a contract 3 inches
| thick. That many times depends on how you are selling
| your work and what the company you are selling to
| expects. One place I worked we always shipped MVP. Then
| would get customers to pay for any new features. New
| features could be cases 2 and 3 do not work correctly for
| the customer, however case 1 works just fine for the
| first customer but they never use 2 and 3. It can come
| down to how your companies budget works which will drive
| the business model you have. Now if you are shipping
| 'boxed' software that mentality could end your product.
| As instead of a reputation of 'works nicely with the
| customers' you are 'this junk software is broken out of
| the box'. It is one of the things I ask during an
| interview. I want to know what sort of shop it is 'how do
| you sell your software'. Each are viable methods to make
| money but some people do not like working that way. I am
| flexible but I would like to know up front.
| peteradio wrote:
| You'd have to take my word for it I guess that these were
| real cases.
|
| The first time this happened was a physical product with
| intention to sell to a very large American company with
| which my company had a relationship going back decades. I
| did as I was told (did not get to step 5 until after
| release, there were steps in between 4 and 5 which caused
| me significant political blowback). Ultimately they did
| not appreciate the corner cutting and declined to
| purchase. My quitting moment came when I was asked to be
| involved in mislabeling product to indicate it was up to
| standard X when it did not come to our supplier that way.
| The pressure I was put under to do that put me in
| therapy. I was a recent graduate with a young child,
| tough times. A year after I quit, my former company was
| parted out.
|
| The last time this happened was software used for
| planning national infrastructure. Case 1,2,3 were common
| cases that would give very obviously incorrect results.
| The naive case was if a sliding parameter was always set
| to one particular value, opening that value up to change
| was a can of worms that required a whole new
| architecture. Ultimately I held my ground with the PM and
| said I would not build it incorrectly and implied it was
| a dealbreaker for me. I convinced the architect and PO of
| the correctness (PM was former frontend and had no clue
| what any of the backend stuff meant). Ultimately I ended
| up on the PM's (my manager) shitlist and the PO
| (apparently with the memory of a goldfish) couldn't
| understand why that feature slipped several sprints. I
| had better things to do so again I quit. I don't really
| expect that company to last too much longer at least in
| its current form. There were already "pivots" on the
| horizon which gave me deja vu (I expect company owner was
| gearing up for a sale).
|
| In both cases there were pretty bad shakeups (unbeknownst
| to me) in the year prior to my joining. In the last case
| I would have appreciated a friendly heads up from a buddy
| at the company (they no longer had a real QA dept, they
| were all fired for "poor communication").
|
| I guess I joined these companies because I have a soft
| heart for companies with deep history. I certainly
| learned a lot but my beard is a bit greyer than I'd like
| for my age. I think the rot was too deep to point out in
| each case without royally pissing of some of the longest
| tenured, better than betraying my ethics I guess.
|
| I've had good work in between the first and last where my
| raising the flag was taken quite seriously and procedures
| were updated to mitigate, but still I feel I have a black
| mark on my name.
| strgcmc wrote:
| Some truths to keep in mind:
|
| - Every company is navigating the marketplace, and making
| decisions with imperfect information
|
| - Not every decision will be perfect (or even, good)
|
| - Not every decision-maker will be perfect (or even,
| good)
|
| - Even a collection of individually smart/reasonable
| people, can end up collectively making pretty
| awful/illogical decisions
|
| - "Good" decisions don't guarantee market success;
| conversely, "bad" decisions can still result in good
| outcomes
|
| - Judging the quality of decisions and decision-makers
| based on outcomes, is an imperfect measure of the actual
| "quality" of those things/people
|
| In your first example, you were presumably an entry-level
| engineer, but you either mistakenly took on too much
| burden (emotional or practical) in terms of decision-
| making yourself, or you misunderstood what types of
| expectations you should have for the actual decision-
| makers.
|
| Decision-makers are allowed / expected to make such bets:
| "how many and which corners can we cut as a company, to
| get a product out to market, that clients will want to
| purchase, in a sensible time-frame?" This is not unusual,
| this happens all the time, at every single company, all
| around the world. The companies who do this more
| successfully, are the ones who find a sweet spot between
| cost-cutting, efficiency, time-to-market, and customer
| demands/satisfaction/delight. This is a very difficult
| thing to juggle, and really really smart business leaders
| consistently fail to find the right balance, or make the
| wrong calls. Hopefully the mistakes aren't fatal to a
| company, but unavoidably sometimes they will be. So yes,
| your company leaders made a bad call based on the
| outcome, but that on its own is not enough to indict the
| decision or the decision-makers as being fundamentally
| wrong.
|
| The fact that your company made a set of decisions that
| ultimately led to failure, doesn't necessarily prove that
| they were a bad company. And to be a devil's advocate for
| a second, even "mislabeling standard X" might be
| forgivable under certain circumstances, such as launching
| a product with an "X pending" label, even though you
| didn't finish certification process for X yet, or maybe
| you didn't even start (but hey not starting doesn't mean
| it can't say "pending").
|
| As a manager, I actually actively filter-in for what
| Amazon would call "Have Backbone" as a value, when
| interviewing engineers, and I ask them to provide
| examples of times where they fundamentally disagreed with
| the product team, disagreed with what they were asked to
| build, disagreed with a proposed architecture, etc. I
| want engineers on my team who will speak up, who are
| opinionated, who care enough about their work to take
| pride in it and put forth effort to improve beyond the
| status quo.
|
| That being said, your examples seem to indicate a
| rigidity of black/white thinking, all-or-nothing
| thinking, and an inability to collaborate towards finding
| a solution. These were probably the most extreme examples
| you had, so I'm not judging every interaction or your
| entire personality as being so rigid, but hopefully you
| have by now experienced other examples in your career,
| where collaborative problem-solving was possible, where
| you did more than point out fatal flaws but also helped
| formulate a path to mitigate or solve them. The companies
| where that was more encouraged or made possible, are the
| ones you probably want to work for.
| peteradio wrote:
| Thank you for your kind advice.
| flerchin wrote:
| Trust the Engineer. PMs care only for date.
| helge9210 wrote:
| > PM says build it the naive way
|
| Why PM has an option to choose here?
|
| > will fail for cases 1,2,3
|
| a) tip QA to test these cases
|
| b) add cases 1,2,3 to "Known issues"
|
| After all that is the reason they call it "beta": "Cause it
| beta then nothing"
| antupis wrote:
| I would do 4.5) escalate to PM boss everytime 1,2,3
| happens.
| skellera wrote:
| Seems like you're working with a PM who thinks agile is an
| excuse to release bad software. MVP should still solve a
| customer pain point.
|
| To play devils advocate though, maybe case 1,2,3 are low
| enough risk to release. Having metrics set to watch if
| these are actually big problems could be "good enough."
| strgcmc wrote:
| Well, this isn't prima facie unreasonable without more info
| (or knowledge of the implicit assumptions you are making
| but not publishing).
|
| - Are cases 1,2,3 named that way, because they are the top
| priority cases (i.e. the #1, #2, and #3 most important
| product features that customers care about)? Even if they
| are, what is the cost of a new architecture? Will it take
| you 3 years and 20 engineers, to rebuild Y or to make Y.v2,
| just so you can support X "properly"? By then the market
| may have moved on, the feature may be worthless, so it may
| make perfect sense to deliver a bad version of X that
| relies on Y.
|
| - Or, are cases 1,2,3 legitimately either rare, or low-
| impact, or do have viable manual workarounds? If so, then
| it's entirely reasonable to defer/punt on doing new
| architecture right now, because either you know these cases
| are unimportant, or at least you don't have positive proof
| that these cases are important enough to justify new
| architecture. With more data, or clear customer demand, you
| can make a better case for rebuilding Y "properly". The
| real problem comes later: what happens if you do get strong
| signals of customer demand, you can prove the current
| solution is not scalable or extensible, and yet the
| business still decides that Y is good enough to never
| touch... well that's a business that doesn't want to stay
| in business.
|
| Agile is about practicality/pragmatism, over adherence to
| dogma or preconceived notions. Just because Y is the wrong
| architecture to deliver X, does not mean it is the wrong
| decision to ship partial feature X. Don't be dogmatic about
| "correct architecture", if you care about for-profit
| software engineering as a profession.
|
| Of course, if your goal is different, if SWE is a craft or
| a hobby or an ivory tower pursuit for you, then feel free
| to make whatever decisions you want that don't fit your
| vision of "correctness".
| peteradio wrote:
| See this comment for more specifics:
| https://news.ycombinator.com/item?id=32953007
|
| I think there its a little unclear that by case, I mean
| testcases that would fail to pass to fulfill a single
| feature. E.g. "I need an addition feature for a
| calculator" but naive implementation will result in it
| working for 1+1 and fail for all others.
| m463 wrote:
| I think there is a continuum...
|
| server folks want nothing to change, ever.
|
| desktop folks want the latest and greatest.
|
| mobile folks get spurious changes forced down their neck and
| want... well who cares what they want, this is what they are
| getting.
| unity1001 wrote:
| Just keep things backwards compatible, then do whatever you want.
| PaulDavisThe1st wrote:
| TFA seems to try to pin DEC's collapse & failure on their
| committment to back compatibility. Yet Microsoft has essentially
| done the same thing with Windows, and has not suffered the way
| DEC did. Seems like a weak argument.
| teawrecks wrote:
| The problem with "if it ain't broke, don't fix it" is that it's
| always broken for someone.
| jmartrican wrote:
| More often than not its "if it ain't broke don't fix it because
| we are too busy releasing early and often".
| AaronM wrote:
| Error Budgets are your friend
___________________________________________________________________
(page generated 2022-09-23 23:01 UTC)