[HN Gopher] How to think about software quality (2022)
___________________________________________________________________
How to think about software quality (2022)
Author : adityaathalye
Score : 121 points
Date : 2024-02-24 10:25 UTC (12 hours ago)
(HTM) web link (www.evalapply.org)
(TXT) w3m dump (www.evalapply.org)
| pavel_lishin wrote:
| So much software suffers from a large class of users having no
| choice in its use. For example, our local school district uses
| Bloomz, which is pretty terrible - both on the web, and in its
| native app.
|
| I can report all the bugs I want, but since I'm never the person
| making a purchasing decision, how highly do I think those reports
| are prioritized?
|
| I wonder if the teacher/administrator side of things is better.
| dboreham wrote:
| I found the best approach is to search LinkedIn for someone at
| the vendor that's closely connected to me, then compose a
| cheery message to that person introducing myself as a fellow
| developer and parent of a user, ask the best way to inject a
| bug report etc. This actually worked. Ymmv of course.
| kqr wrote:
| Ohh, this is actually clever. Subversive and I'm not sure I
| like it, but it sounds very effective.
| zx8080 wrote:
| Sounds like a hack to make a person to work overtime.
| amelius wrote:
| ... while having fun while doing that work.
| the_fury wrote:
| If I get a message on linkedin that pertains to a work
| issue, you had better believe that I'm going to be
| dealing with it on work time.
| computerfriend wrote:
| One of the reasons I never put my employer on my LinkedIn
| profile.
| datavirtue wrote:
| Most of life's fulfilling opportunities come from just
| showing up. Showing up on searches, at work, at the pub.
| deprecative wrote:
| If I got a message like this I'd just block the sender. As
| a developer I have no say in what my owner tells me to work
| on. I type what they want and nothing more.
| D-Coder wrote:
| They're not asking you to fix anything. They are asking
| where to submit a bug report that someone (someone else)
| will look at.
| binarysneaker wrote:
| As someone this happened to, I was actually impressed they
| looked me up and reached out. Knowing they cared that much
| meant that I cared too and I was able to prioritise the
| request with the product team. When we completed the request,
| we celebrated, and even implemented uservoice on some sites
| to improve the feedback loop.
| hitchstory wrote:
| This software also tends to be waterfalled to hell and back.
| Theyve already planned out their deliverables for the next 12
| months and will be held accountable for meeting them. Your
| bugfix doesnt fit into that picture unless an exec treats it as
| a priority.
|
| ...in which case it'll be jammed into the devs' task queue with
| no expectation that the pre-agreed delivery dates will shift.
| datavirtue wrote:
| The issues you describe have nothing to do with "waterfall."
| hitchstory wrote:
| >Theyve already planned out their deliverables for the next
| 12 months and will be held accountable for meeting them.
| unclebucknasty wrote:
| Not sure I understand your point. Are you saying product
| roadmaps are incompatible with agile?
| hitchstory wrote:
| If they include detailed plans and timelines for feature
| deliveries then yes.
| adityaathalye wrote:
| Post author here. I agree; so often constructive feedback loops
| from end users to builders are woefully absent from the
| equation.
|
| I feel the greater the distance between the purchase decision
| maker and the end user, the weaker the feedback gets. Riffing
| off the six degrees of separation idea and exponential drop off
| of signals though lossy media, it's like by the time someone
| gets to a third degree of separation, any feedback signal
| barely registers. Only mass action has any chance of being
| heard, much less heeded.
| AnimalMuppet wrote:
| Ah. So that's (part of) why enterprise software is so
| universally bad. The users are further from the purchasers
| than they are in, say, developer tools.
| adityaathalye wrote:
| I believe so.
|
| I'll pick on Government websites and the systems that back
| them as an all-too-familiar, internationally reviled
| category of software, as an example. These tend to be
| incredibly painful to use. Some tendering process somewhere
| awards a contract to the lowest bidder who then has the
| unenviable job of wrangling the delightful intricacies of
| government machinery to put something out that maybe does
| what it was supposed to do for the common citizen.
|
| In such a world, I feel gov.uk stands out as a remarkable
| counter-example, worthy of emulation. e.g.
| https://www.gov.uk/guidance/government-design-principles
| jSully24 wrote:
| You mention not being the person making a purchasing decision,
| who is? Can you connect with them?
|
| I would encourage you to put together a story that illustrates
| the pain being caused by this software and get with the
| purchasers. Not only might it help your organization figure out
| they need to do something different it could also show off your
| skills to the orgainization.
|
| Could you collect the time you and other people are spending
| working on these problems? How many issues are reported to you?
| How many bugs you've filed and have not had fixed by the
| vendor? Rolling this up into a real dollar coast it's having on
| you and the school district users.
| cratermoon wrote:
| > who is? Can you connect with them?
|
| For something like this, usually at the senior executive
| level, and organizationally. There's not really one person
| making the decision, it's done through the organization's
| decision-making system. Nominally there's a responsible
| person, but practically speaking that individual isn't
| completely autonomous.
|
| Large organizations have been making and dealing with
| decisions like this for ages, and rarely does dollar cost of
| usability ever influence. Products SAP and MS Teams get and
| retain market share by making the cost of moving off that
| software outweigh the gains.
| pavel_lishin wrote:
| The person making the decision is very like the
| superintendent of the school system, likely informed by
| someone from IT and maybe someone representing the teachers
| somehow.
|
| They might listen to me if I bring in a coalition of other
| parents, but there's two reasons why I'm probably not going
| to do this:
|
| 1. They're likely locked into at least a year-long contract,
| if not more
|
| 2. Doing this is a huge amount of work for me. I'd have to
| find a significant number of parents who are _also_ as peeved
| as me, and then get them to take action - even if it's
| signing something - and then I'd have to bring the petition
| up to the school board, etc., etc., etc.
|
| It's much easier to just mute the notifications on the app,
| and go through them once a week. 99% of them are just wildly
| inactionable anyway - things like reminding me that school is
| out, or that a student interest group that's irrelevant to me
| is having a meeting two Thursdays from now.
|
| If I was annoyed enough to take action, I'd rather just offer
| to build them a replacement app - shorten that feedback loop,
| ya know?
| germinalphrase wrote:
| As a former teacher, my experience of education specific
| software was almost universally negative. We may even get an
| in-person meeting with the software vendor only for the
| reported issues to remain unpatched indefinitely.
| dahart wrote:
| As a developer, I'm not at all surprised, I suspect it's
| negative from almost all points of view except at the
| district or city or state level where they pat themselves on
| the back for bringing technology into classrooms via spending
| some money, what little money they have, on tech that they
| don't use and don't understand. I don't even have that much
| experience with it, but I've tried making educational
| software, in the form of multi-user whiteboards teachers
| could use for their classes. The people approving purchasing
| decisions don't use the software, and don't ask teachers or
| students about the experience or value of the software. The
| insane sales cycles are a never-ending nightmare of
| requirements and delays, far beyond the reasonable concerns
| about kids & privacy. After years of run-arounds, we gave up
| trying to use any official channels and just let teachers use
| the software free.
|
| As a parent, the experience was also very negative. My kids
| hated almost everything the school forced them to use,
| because the software was terrible compared to non-educational
| software. Teachers didn't get any training on the software,
| and didn't have the resources or time to use it effectively.
| Hardware was always outdated or supremely cheap and crappy.
| Possible the single worst software experience was the
| automated essay grading software. It was beyond broken,
| actively harmful, and it literally didn't work. I came home
| from work one day and my oldest kid was in tears and my wife
| at her wits end trying to finish a homework assignment to
| write a short concise 2 paragraph essay on a specific topic
| and achieve an arbitrary 90% score or whatever within 10
| tries, and they'd used up 8 of them and scores were going
| down each time. This software isn't even supposed to be used
| unsupervised for grading, but the teachers didn't seem to
| understand, and weren't following recommendations. I figured
| out quickly the software doesn't understand english and would
| score higher if you fed it more words, regardless of their
| quality or relevance - the polar opposite of the assignment's
| goal. So I gave it a wikipedia article on why people
| shouldn't use software grading robots, which scored 100%, and
| then scheduled a meeting with the teacher, who proceeded to
| complain about the software and told me she didn't have a
| choice because it was district rules and a state software
| contract.
| cratermoon wrote:
| There's a whole thing around the nature of who makes the
| purchasing decisions vs who are the real users. For key
| organizational systems like Bloomz for schools, the decision is
| made at the senior executive level, and input from the people
| who will use the software on a day-to-day basis is rarely, if
| ever, taken into consideration. Sometimes the decision is made
| on the golf course between business buddies.
|
| In these situations, the vendor doesn't have to care much about
| user experience or feedback, all they need to do is be able to
| check all the boxes on the whatever list of features management
| cares about. It comes up on HN off and on, here's an example:
| https://news.ycombinator.com/item?id=21224209. Thread unrolled
| here:
| https://threadreaderapp.com/thread/1182635589604171776.html
| rqtwteye wrote:
| This plagues most enterprise software. Decisions are made by
| people who will most likely never actually use the software. So
| the decisions are made by cost and ease of dealing with the
| vendor. Usability is way down the list of priorities.
| makerdiety wrote:
| What about free software? That is, how can human society achieve
| good quality free software if it is physically impossible for
| free software to develop? Because personal income for free
| software developers is important.
|
| I guess since it's personal income, then it's a personal problem
| for free software developers then? That's fine, but income level
| coincidentally correlates with high quality software.
|
| I guess free software just doesn't matter then.
| mantas wrote:
| Correlation between in one level and quality software? Heh,
| lol.
| mch82 wrote:
| Options include... Volunteer. Donate. Ask the elected officials
| that represent you to choose open source software for
| government use cases. Encourage grants to fund open source. If
| you're at a company, contribute back to open source.
| makerdiety wrote:
| Government officials for republics function to serve a
| greatest common denominator that doesn't even exist in
| reality. So there's no political representation at all for
| free software.
|
| Well, the greatest common denominator for political
| representation _does_ exist. But it by its nature excludes
| free software as even a talking point. Namely that capitalism
| is mutually exclusive from free software. Capitalism hates
| free software to the point that free software cannot even
| literally get a word, a voice, or a conceptualization that
| can help refine it.
|
| That leaves us with the only realistic options for supporting
| quality efforts in free software being volunteer work that
| isn't empowered. Everything goes downhill from there. Grants
| and company lip service are only superficial reactions to the
| free software problem.
| bdw5204 wrote:
| Capitalism is perfectly fine with FOSS software especially
| if it can use the idealism behind it to save on labor
| costs. Especially since the definitions of both "free
| software" and "open source" explicitly reject non-
| commercial licenses as heretical.
| hiAndrewQuinn wrote:
| Most software exists to save someone somewhere time. I think this
| is the One True Metric by which we measure Software Quality.
|
| Software which sells well historically has done so by saving huge
| numbers of people huge amounts of time, for whatever other
| complicated business they're already involved in. Excel, ERP
| systems, and GPT-4 all do this to great commercial success. You
| only need to step back and wonder how many more man hours it
| would take to do what they currently do by hand to realise this.
|
| Software can also save the time of other developers by being
| simple, well-architected, reliably documented, etc. This too is
| an aspect of software quality. Its recursive nature can often
| become a competitive advantage in its own right: High quality
| code, can lead to faster change implementations (fixes and
| features alike), which can filter downward to a higher quality
| product for the end user.
| yarekt wrote:
| I like the sentiment, but you can have quality, well
| architected software that's pleasure to work in, and it being
| completely useless/obsolete to customers. Value != quality.
|
| I think software quality (and all the symptoms discussed in the
| post) are to do with how quickly the software can be changed:
| Lower tech debt results in software that's quick to change to
| new requirements, or change in business domain. It's also
| important to realise that just like with performance, you get
| diminishing returns at very high levels
| detourdog wrote:
| I think software quality is more related fo how little it
| changes while remaining useful.
|
| I see the current software environment has different stake
| holders all defining quality from their own perspective.
| yarekt wrote:
| Ah I see, quality can also mean ingenuity, but in my case
| as a software developer ingenious solutions don't always
| present themselves. I'd settle for old boring plain
| solution that works well enough, but is of enough quality
| that it doesn't cause fires, and is quick to change with
| the business
| doubloon wrote:
| when you make a post about software quality but did not test it
| on enough browsers to realize your constant width font ascii art
| is not rendering correctly in firefox or chrome, so the user has
| to grab an invisible scrollbar to see the picture... zooming out
| of course doesn't work.
| adityaathalye wrote:
| One of the little ironies of life. I know about this (and some
| other annoying problems), and have been meaning to fix the
| typography on my site for a long while. But that would draw me
| into a rabbit hole of some other improvements I want to make to
| my site maker. And that will be, well, there's an xkcd for it.
| Two actually.
|
| Automation: https://xkcd.com/1319/
|
| Efficiency: https://xkcd.com/1445/
| adityaathalye wrote:
| P.S. Also, I did not tell them to make invisible scroll bars
| (nor have my CSS rules, as far as I can tell)!
| mdaniel wrote:
| I think some of the bad outcome may be due to macOS being
| sneaky with the scrollbar display
| <https://support.apple.com/guide/mac-help/change-
| appearance-s...>, but your style.css for sure isn't helping
| matters with its overflow: specifications. The tradeoff, as
| best I can tell, is whether macOS folks get hidden
| scrollbars (I don't know anything about the zoom complaint,
| since I'm on desktop) versus omitting the overflow: and
| having the asciiart clip outside of its bounding box due to
| some other CSS interaction that I didn't bother to track
| down
|
| https://www.evalapply.org/static/css/style.css#:~:text=over
| f... and https://www.evalapply.org/static/css/style.css#:~:
| text=overf...
| mojomark wrote:
| Good read (reminder) and love the reference list.
|
| Question for the OP. I suspect we're of similar tenure, and I'd
| love to det your perspective...
|
| I'm a mechanical systems engineer in an industry that is growing
| from industrial automation to true autonomy (robotics). I've
| always liked coding, but don't do it enough to be a serious
| contributer to "production quality" code. Nevertheless, since
| last year I've transitioned from a more mechanical/hardware
| design firm to a much more mathy/software engineering firm. I
| find myself in a role, bridging robotics hardware and software
| teams working on a ~10yr old codebase. We have some great
| engineers! I've tended to stay on the customer requirements and
| algorithm development/analysis side of things, but I struggle
| with the lack of benchmarking and testing our group does, and I'm
| continuously surprised at the simple bugs we find that cause
| application crashes. Moreover, when a developer shows me code,
| I'm stricken by the number of undocumented "magic numbers" that
| seem like forgotten patches popped in there at one time to make
| things work for a test or demo, and the number if/then statements
| I see that are holding things together.
|
| I'm also faced with a growing cry for "how can we best make use
| of these LLM things" (for which there are a thousand and one
| answers).
|
| I see/feel the suffering, and what looks to me (from my 1000ft
| perch) like thousands of cuts.
|
| Bottom line, I'm increasingly thinking that I really need to take
| a month and try tracing through the thousands of lines of C++
| (which I haven't really written since grad school) in order to
| truly understand the state/configuration of pur code baseline so
| that I can provide some more explicit leadership in terms of - do
| we refactor everything that looks messy or the we don't 100%
| understand? Do we take a T.O. to build up our automated unit and
| system level testing? etc.
|
| As a manager of a SW team, from your experience is it a good idea
| for me to dig deep into the code to gain an intimate
| understanding? That feels an awful lot like micromanagement to
| me, but I will do it if that's what 'wisdom of experience' in
| this domain proves is the most value-add.
| kushie wrote:
| i find myself in a similar sounding role, i don't have answers
| for you though. i'm wondering, do you use git for version
| control and repositories?
| groos wrote:
| If you can't understand it yourself, you will have to find
| someone whose expertise you trust to answer the questions for
| you. But you don't have to understand the software down to bits
| and bytes, just enough for you to make good decisions. Having
| pragmatic underlings bring up problems and propose solutions is
| what you want. Watch for idealists for whom everything is
| broken and needs to be rewritten. Incremental improvement is
| the way.
| yarekt wrote:
| /me raises hand. idealist here, and I do want everything
| rewritten. Please ensure you provide your idealists with all
| business goals and limitations rather than shield/protect
| them. You'll find that then their suggestions take those
| things into account
| e28eta wrote:
| I think you should probably leverage your team's experience
| with the code, and understand what they would prefer.
|
| I don't have management experience and I don't know your team,
| but I suspect if they agree there's a problem and propose
| solutions, you'll have better buy-in, and at least as good odds
| of successfully addressing the problem.
| teucris wrote:
| Minor nit regarding this thing that will destroy quality:
|
| "We'll fix it if customers complain."
|
| You can't fix everything, and perfect is the enemy of good. While
| punting testing to users is a no-no, recognizing that edge case
| issues can be deprioritized is crucial to getting things out the
| door. This means shipping stuff with known defects that you can
| respond to if necessary.
| marcosdumay wrote:
| > You can't fix everything
|
| Oh, you certainly can't.
|
| But you can choose how many things needing fixing you will
| make. Doing some shit without thinking and waiting for people
| to complain about symptoms of deep issues is a very effective
| way to create lots and lots of need for fixing things.
| mdaniel wrote:
| And my experience has been that a quality culture like that
| can "resource starve" annoyances for _literally years_ due to
| while True: ship_feature() if bugs:
| to_fix = highest_priority(bugs) fix =
| make_patch(to_fix) fix = introduce_new_bug(fix)
| # defer bugs[1:] until they meet the complaint criteria
|
| and that's how meetings end up with "but, how hard can that
| possibly be to fix?" statements
| NomDePlum wrote:
| If you accept some bugs as "features" that you will accept,
| not fix, you can have a bug free policy.
|
| Doing this means you make decisions early and don't carry a
| ton of historical baggage that gets endlessly discussed.
| hinkley wrote:
| I knew a guy who said on a previous project they had a
| system that raised the priority of old bugs on some
| interval. I guess the idea was to either get them fixed or
| agree that they never would.
|
| There are morale and duplicate bug issues with keeping a
| backlog that grows faster than you can close them, so I
| guess that makes a bit of sense. And better than GitHub
| auto-shooting-the-messenger by clocking bugs after a month
| or two of inactivity.
| golergka wrote:
| > You can't fix everything
|
| But you can fix a lot. Some of the products I've worked at had
| Sentry alerts all the time that nobody paid attention to,
| whereas others (of very similar complexity) had them a few
| times a week, and every time somebody would immediately get on
| top of it and release a hotfix, usually in a span of the same
| day.
| sriharis wrote:
| This is well written, adi. I particularly like the book
| references.
| jiggawatts wrote:
| One thing I like about this is that the author doesn't talk in
| terms of absolutes. He assumes (correctly) that advice is not
| universally applicable and that each process needs to be traced
| back to fundamental axioms or requirements.
|
| As a random example I'm right now trying to explain this to a
| large government enterprise customer.
|
| They write "cold blooded" software, the type where they have a
| development project with a fixed budget and a fixed scope and
| then the code goes into hibernation for five years until they can
| "find the budget" again.
|
| So of course, they picked the Angular framework, one of the
| fastest moving, most "hot blooded" frameworks in the world.
|
| Angular might be the right choice for a startup writing highly
| interactive software. It is definitely the wrong choice for an
| org that thinks of maintenance as something that occurs twice a
| decade.
|
| It's these things that most fundamentally kill quality. Picking a
| framework, language, or even operating system because it's "hip"
| or "cool" or the "buzzword of the year". Not because it's
| suitable for the project.
|
| The thousands of cuts come later, but they barely rate in
| comparison to having your legs chopped off at the knees right
| from the beginning.
___________________________________________________________________
(page generated 2024-02-24 23:00 UTC)