[HN Gopher] The Software Crisis
___________________________________________________________________
The Software Crisis
Author : todsacerdoti
Score : 87 points
Date : 2024-07-05 13:38 UTC (9 hours ago)
(HTM) web link (wryl.tech)
(TXT) w3m dump (wryl.tech)
| Animats wrote:
| The ending: _" Things can be better. I'll show you how."_
|
| It's just an intro for clickbait.
| darioush wrote:
| If only they had unveiled the single unifying abstraction of
| everything :(
| unstruktured wrote:
| Oh that's a solved problem since 1969. It's called "unix".
| Everything is a file which can be processed as a byte stream.
| Composition is a breeze- can't be any more general than that!
|
| (I kid, mostly :)).
| bitwize wrote:
| That's the worse-is-better grand unified abstraction.
|
| The Right Thing, as any Lisp programmer can tell you, is
| lambda.
| Animats wrote:
| - UNIX: everything is a file.
|
| - LISP: everything is a lambda.
|
| - Tandem: everything is a database.
|
| - QNX: everything is a message.
|
| - IBM System/38: everything is a capability.
| aleph_minus_one wrote:
| > If only they had unveiled the single unifying abstraction
| of everything :(
|
| Here you are: https://en.wikipedia.org/wiki/Category_theory
|
| ;-)
| dimitrios1 wrote:
| The prime criteria for being clickbait is if it is
| sensationalized, deceptive, or intentionally misleading. This
| strikes me as none of those things. It's the last sentence of a
| blog post.
| Animats wrote:
| It's something where you have to read the introductory
| material, and then find out the important part is elsewhere.
| In this case, in a blog post yet to come.
| throwup238 wrote:
| It's the first (and only) blog post on that site:
| https://wryl.tech/log/index.html
| redleggedfrog wrote:
| Ad Hominem? Or are you just stating that as a fact and not
| questioning the veracity of the arguement?
| Swizec wrote:
| The "inscrutable layers unapproachable to beginners" of today are
| the bare metal low level computing of tomorrow.
|
| Sure I learned with DOS and Turbo Pascal and it was wonderful,
| but if you ask my teachers who learned with machine code and
| microcontrollers, they worried that computers have become too
| abstract and kids these days have little chance to learn the true
| details.
| EternalFury wrote:
| If it's done right, by documenting a mature interface and its
| inputs/outputs, no one needs to know what atoms make up the
| slice of bread that gets buttered.
| doug_durham wrote:
| Isn't that exactly what the author is complaining about?
| There is too much complexity to understand all of the layers
| no matter how well documented.
| ChrisMarshallNY wrote:
| _> It 's catastrophic when they don't."_
|
| I have found this not to be the case.
|
| Very often, an inaccurate mental model is the ideal user state,
| and it's my job to afford and reinforce it.
|
| But that's just me, and my experience. I'm sure there's a ton of
| folks that have found better ways.
| omnicognate wrote:
| Personally I can't bear software that contorts itself to
| conform to an incorrect mental model. I find it alien and
| unusable. I don't want the inaccurate mental model and cannot
| force myself to accept it, but the contortions and don't-look-
| behind-the-curtain prevent me from forming an accurate one.
|
| That doesn't mean I don't want abstractions. It means I think
| what constitutes a good abstraction is determined as much by
| how true it is as by how intuitively appealing it is. An
| abstraction that corresponds to a naive user's expectations but
| that doesn't accurately reflect (the essential/relevant aspects
| of) what is actually happening is not an abstraction, it's a
| lie.
|
| Edit: And the tragedy of it is that users are, by and large,
| extremely good at figuring out the reality behind the lies that
| well meaning developers make their software tell. When our
| software breaks and misbehaves, its internal realities are
| surfaced and users have to navigate them, and more often than
| not they do, successfully. Internet forums are full of people
| reasoning, experimenting and fiddling their way to success with
| faulty lying software. Apple Community is perhaps the purest
| example of a huge population of users navigating the broken
| terrain behind the abstractions they weren't supposed to think
| about, with absolutely no help whatsoever from the company that
| built them. We should have more respect for users. If they were
| as dumb as developers assume they are almost none of the
| software we write would ever be successfully used.
| ChrisMarshallNY wrote:
| _> We should have more respect for users_
|
| Agreed. In my case, I write software for a _lot_ of really
| non-technical users, and have found great utility in
| reinforcing inaccurate, but useful, user narratives.
|
| So "respect" doesn't just mean assuming users are smart. It's
| also making house calls. Meet them where they live, and do a
| really, really good job of it, even if I think it's silly.
| ukuina wrote:
| Apple Community is a perfect example of users assuming the
| underlying software is always correct, for every possible
| use-case, and that the user is at fault for going against the
| grain or, in many cases, *using the software as advertised*.
| neilv wrote:
| > _It seems as if this state of comfort is due to a sense of
| defeat and acceptance, rather than of a true, genuine comfort._
|
| A whole lot of, say, Bay Area software developer
| salaries/compensation have genuine comfort.
|
| Defeat and acceptance doesn't come into this, for most
| organizations: they face little-to-no accountability for security
| problems or other system defects, so... comfort for the
| developers.
| kmoser wrote:
| Abstractions themselves aren't a problem; they are in fact a
| necessity if you want to move anywhere beyond simple bit-
| twiddling. (Heck, even the idea of software itself is an
| abstraction over hard-wired instructions.)
|
| The real problem with abstractions is when they are implemented
| poorly, or have side-effects, or just plain bugs. In other words,
| we will always be at the mercy of human-produced software.
| ramesh31 wrote:
| >The real problem with abstractions is when they are
| implemented poorly, or have side-effects, or just plain bugs.
|
| Which is always, and is why clean maintainable code uses
| _minimal_ abstractions to accomplish the task. However it seems
| the default these days has become "pile it on".
| threatofrain wrote:
| The problem isn't minimal abstraction but imperfect
| abstraction. Imperfect abstractions convince people to dig
| underneath the surface. Now you've lost the benefit of
| abstraction because they're thinking of two layers at once.
|
| So when people run explain on their SQL query, IMO you've
| broken the declarative abstraction.
| kmoser wrote:
| Asking the SQL engine to explain the query doesn't mean the
| abstraction is broken or imperfect. It simply means the
| developer wants to understand how their query is being
| understood and, hopefully, optimized. This is essential
| when debugging why queries don't behave optimally, e.g.
| failure to use an index.
|
| Even with the perfect abstraction, people will need to dig
| beneath the surface. This is because every layer of code is
| subject to side-effects due to its actual implementation.
| That's why we have, e.g., multiple sorting algorithms, and
| documentation for their individual behavior, speed, etc.:
| so developers can understand the limits and idiosyncrasies
| of each and pick the correct one for the job.
|
| We may make incremental steps towards more bug-free code,
| but because everything is limited by the details of its
| actual implementation (whether real or an abstraction),
| there is no magic bullet that will make a huge leap forward
| in the realm of software development as a whole.
| marcosdumay wrote:
| > It simply means the developer wants to understand how
| their query is being understood and, hopefully,
| optimized.
|
| Or, in other words, the developer wants to break the
| abstraction.
|
| If your definition of "perfect abstraction" doesn't
| include the developer never needing to look beneath the
| surface, I'd say it's a pretty bad one.
| derekp7 wrote:
| There is also the issue that lower abstractions don't always
| give enough info to higher level extractions, and vice-versa.
| Take OS filesystem code for example. It would be useful for
| the low level storage driver to have info that the higher
| level FS possesses so it can tell the hardware that certain
| blocks of storage can go back into the write leveling pool
| (i.e., when a file gets deleted). Or, the other way around,
| the filesystem driver can make better file allocation and
| access patterns if it knows that the minimum write size for a
| block on an SSD is 128KB, or that starting a write on one
| block, and ending on another, gives better performance if
| that first block is picked to be on a stripe boundary on a
| RAID drive.
| UweSchmidt wrote:
| If you look at the resumes of engineering or automotive company
| leadership, you'll see people going through stages of ever
| expanding responsibilities of part, component and product design,
| or management of production facilities of increasing size and
| importance. The CEO will still emphasize their technical
| knowledge, non-technical staff will at least try and fake it.
|
| In agile software development on the other hand, technical
| competence usually ends at the lowest tier. A scrum team has
| folks on it who make software, that's it. Then, lots of scrum
| masters, business analysts have probably never coded much; the
| first actual boss in the hierarchy has mostly secretarial and
| managerial work and will hardly look at code at all.
|
| Point is, it's not just that software development is done in
| ticket-sized portions which does not invite philosophical
| considerations about the numbers of abstraction layers that one
| builds and maintains. It's that software developers don't even
| have a seat at the table( _); they get babysat by scrum masters,
| make compromises during code review, are discouraged from
| thinking beyond the ticket, and are then of course not usually
| promoted into leadership roles to which they would bring their
| technical competence.
|
| It appears therefore that any movements to bring awareness to the
| "software crisis" will be relegated to hobbyists, as the article
| states at the end: to "Handmade, Permacomputing, and various
| retro-computing circles".
|
| (_) I partly blame Hollywood and their incessant humiliation of
| software/IT people, while creating endless leading roles for
| doctors and lawyers, effortlessly weaving their complicated
| terminologies into fascinating storylines, which is aparently not
| possible to do for us? Maybe the scriptwriting AIs can come up
| with something here soon.
| steveBK123 wrote:
| I could buy this argument. Had I been a junior during the agile
| era, I'm not sure I would have developed as fast or as far as I
| have.
|
| The most agile pilled company I worked for just treated juniors
| & seniors as interchangeable cogs, except seniors should be
| able to clear more points per sprint. Active discouragement
| from thinking outside the scope of your ticket, keep your head
| down and mouth shut.
| heurist wrote:
| When I was starting out a bit more than a decade ago, I was
| on an agile/scrum software team in a hardware company. The
| team was fine but I found the process painful. I ejected as
| soon as a new director realized he could do algebra with the
| story points and began plotting stories quarters out. I've
| never been on a scrum team since and am happier for it. As a
| manager now, I would push back on scrum in almost every
| situation (and have).
| marcosdumay wrote:
| The problem with "agile" is that everybody claims to do it.
| Companies with a healthy environment cram their procedures
| into agile-derived names, and keep the healthy environment,
| while unhealthy ones also cram their procedures into agile-
| derived names, and use their universality as an excuse to
| never improve.
|
| I think scrum is irremediable, but even then, some places
| only practice it pro-forma: divide the work into tickets at
| the beginning of the sprint, rewrite the tickets into what
| they did at the end, and go home happy.
| intelVISA wrote:
| > juniors & seniors as interchangeable cogs
|
| welcome to the car factory
| fuzzy2 wrote:
| The perceived misery you describe I feel is self-inflicted.
| Many devs "below" me have become entirely disconnected from
| customer needs, instead only focusing on "interesting" dev
| problems.
|
| Why do developers only work on ticket-sized portions of the
| actual requirements? To put it succinctly: because they are
| simply too dumb. They cannot wrap their heads around it. They
| cannot grasp it.
|
| Do I sound frustrated? I am. It is inscrutable.
|
| Sorry.
| wryl wrote:
| Problems surrounding computing education do compound these
| frustrations, and I sympathize.
|
| Having worked for larger organizations (whatever FAANG calls
| itself these days, I can't keep track), as well as academia
| and independent education, I've seen both halves of the
| "production line" for newcomers to computing.
|
| Something has to change in how we bring individuals into our
| field. I have some ideas based on my experiences, but you're
| not in the wrong for feeling frustrated about this. It is the
| state of things, and many companies are not equipped to
| handle it, because it's unexplored territory.
| stackskipton wrote:
| I think categorizing it as "Too dumb" is also doing
| disservice to many developers who are stuck in feature
| factories. After a while you realize business is happy with
| status quo so do your Jira tickets, take your paycheck and go
| home. My puny stock options are extremely unlikely to be
| impacted by my work output.
|
| My boss doesn't care about Tech Debt. Get this ticket done,
| get it done quick and move on. He figures he will be long
| gone before tech debt racks up to point he would get in
| trouble for it. Hell, I'm not sure his higher ups even
| realize the tech debt problem so fact if he is here for 4
| years, they wouldn't realize what was cause of the tech debt.
| fuzzy2 wrote:
| Though my employer certainly could be categorized as a
| feature factory (individual software development), we kind
| of sell the opposite: Sustainable development producing
| software which can be changed easily. There's only direct
| monetary compensation. Hierarchies are flat. Like, too
| flat.
|
| I understand many do not have the energy to fight the
| status quo and some may not have the... eloquence to do so.
| I have worked very hard for many years to end up where I
| am. If others don't, I expect them to at least accept where
| they remain. Because they don't do "don't care". They are
| effectively sabotaging projects.
|
| They certainly aren't unintelligent. They still act pretty
| dumb. Again, I must apologize for my polemics.
| TeMPOraL wrote:
| > _Why do developers only work on ticket-sized portions of
| the actual requirements? To put it succinctly: because they
| are simply too dumb. They cannot wrap their heads around it.
| They cannot grasp it._
|
| I think the reason is entirely different: ticket-sized
| portions of requirements are the only thing that one can hope
| to estimate in any useful fashion. Business side needs
| estimates, so they create pressure to split work into pieces
| they know how to handle.
|
| Put another way, it's not that developers are "too dumb" to
| wrap their head around actual requirements. _They 're not
| allowed to_. The business prefers devs to consistently crank
| out small improvements, keep "velocity", even though it leads
| to "organically" designed systems - i.e. complex, full of
| weird edge cases, and smelling of decay. The alternative
| would be to let the dev talk, design, experiment, take the
| project holistically - but that also means the work becomes
| impossible to estimate more precisely than "sometimes next
| year, maybe", which modern businesses can't stand.
| rafaelmn wrote:
| > I partly blame Hollywood and their incessant humiliation of
| software/IT people, while creating endless leading roles for
| doctors and lawyers, effortlessly weaving their complicated
| terminologies into fascinating storylines, which is aparently
| not possible to do for us?
|
| Doctors and lawyers deal with people and everyday problems that
| are easy to turn into a interesting story. I don't see many
| contract lawyers or radiologists as protagonists - it's ER docs
| and criminal law.
|
| Software development is about rigorously talking to a computer
| all day, either solving mundane solved tasks in a new
| application, or problems that you can't even grasp without a
| technical background. I'm a developer who started programming
| as a teen over 20 years ago for fun - and I'm bored out of my
| mind with most of the work - I don't even try to talk about it
| to non-developers because I know it's about as interesting as
| accounting, more people would get useful info out of their
| stories.
| asdfman123 wrote:
| In big tech lots of managers are highly technical all the way
| up.
|
| But there's two problems: they can't get into the weeds, and
| they also are subject to the perverse incentive of being
| rewarded for generating complexity.
|
| Some people fight it, sure, but those who fight it are less
| likely to be promoted. You don't get rewarded for having less
| staff under you or eliminating your own role.
| jahewson wrote:
| Counterpoint: Boeing.
| wryl wrote:
| Hi! Author here. I think it's important to address certain
| aspects of this post that people tend to misunderstand, so I'll
| just list them here to save myself the effort.
|
| * I do not argue against abstractions, but against the
| unrestricted application of them.
|
| * I do not advocate for a reversion to more constrained platforms
| as a solution.
|
| * I do not advocate for users becoming "more technical" in a
| "suck it up" fashion.
|
| The key to understanding the software crisis is the curves of
| "mastery of a platform" and "growth/release cycles". We have, in
| the past 40+ years, seen these curves diverge in all but a few
| sectors. We did not address the crisis when these curves were
| close in proximity, but the second best time is now.
|
| As for folks calling this clickbait, it is the first in my log,
| and reflects my thoughts on the situation we find ourselves in as
| developers. The sentiments are mirrored, in various forms, around
| multiple communities, some of them based in counterculture.
|
| I do want to deliver some part of the solution to these problems,
| so I do intend on following up on "I'll show you how". I am a
| single entity, so give me time and grace.
| morning-coffee wrote:
| It's hard to get past the hubris wrapped up in the statement
| "I'll show you how"... as if the tens of thousands of bright
| engineer's whose shoulders you stand on were incapable and
| you're the savior... maybe you are! (But just adding the word
| "try" in that sentence would reduce the perceived arrogance by
| orders of magnitude.)
| wryl wrote:
| Well, that certainly wasn't my intention, but in an
| environment of "I have a silver bullet and it'll cost you
| $X", I can understand the sentiment.
|
| At the same time, I do want to show that I have confidence in
| my ideas. Hubris and confidence must be applied in equal
| parts.
| jdbernard wrote:
| I didn't read it that way. I read it as an introduction to a
| counter-cultural paradigm shift away from the "move fast and
| break things" and "abstract over it" mindsets. I'm very
| interested in the path they are going to outline.
| beepbooptheory wrote:
| I just want to say this was a wonderful log. There are not many
| in this field who have an intuition or impulse towards
| materialist/historical analysis, but whether you are aware of
| it or not, that is certainly what I read here. Just to say, its
| not quite recognizing how we find ourselves thinking that is
| enlightening, but rather recognizing _why_ we find ourselves
| thinking like we do that can bring clarity and purpose.
|
| In a little more woo woo: abstractions are never pure, they
| will always carry a trace of the material conditions that
| produce them. To me, this _is_ the story of computing.
| wryl wrote:
| Thank you for the kind words! "I appreciate them" would be an
| understatement.
|
| And I agree entirely. Tracking the history of an abstraction
| will usually tell you its root, though it gets pretty muddy
| if that root is deep-seated!
| seansh wrote:
| I've been a fan of handmade and seems like a new blog in the
| same spirit is born. I look forward to reading more of your
| writings.
|
| I have recently started writing myself and paying more
| attention to what I read. I really liked your style and enjoyed
| reading it.
|
| So please keep writing!
| wryl wrote:
| Thank you very much! Very much in line with the handmade
| ethos, and I'm hoping to get involved with more handmade (and
| handmade-like) things in the future. :)
| hnthrow289570 wrote:
| I think this might be premature if you're trying to apply this
| to how the world makes money with software. There's too much of
| the economy that depends on adding another abstraction layer to
| make money, so you'd need to rewire that as well unless we just
| want to break things. You'd eventually have to tell a business
| "no" because they're +1 abstraction layer above the limit.
|
| I think most of us realize that working at corporate is still a
| necessary evil as long as we need to pay for stuff. Frankly
| that sector can burn because they've been taking advantage of
| developers. Most of the money goes to people above you,
| performance bonuses are not the norm, etc. We shouldn't be
| actively trying to give them improvements for free because of
| this behavior. Let them follow the hobby/passion projects and
| learn these practices and limits on their own.
| spirodonfl wrote:
| I could not agree with you more. I'm trying to formulate data
| and arguments and I think I actually have the data to prove
| these points.
|
| I'm working on a series of articles at
| enlightenedprogramming.com to prove it out.
|
| I'm also working "show you how" solutions because, absent data
| (and sometimes even with it), I still get people who believe
| that the industry has never been better, we're in some golden
| age and there is not a whole to improve on and it just boggles
| my mind.
| marcosdumay wrote:
| Well, I don't think many people would disagree about the
| crisis.
|
| Personally, I'm anxious to see your proposal on "how".
| GiorgioG wrote:
| Many abstractions do not hold over time. That is the basic
| problem I have with OOP.
| pmarreck wrote:
| The large majority of the work code does can probably be boiled
| down to a series of transformations done by a hierarchy of pure
| functions with a fixed number of inputs/dependencies, all of
| which are easily testable in isolation.
|
| It's unconstrained side effects and dependencies, resulting in an
| increase in complexity, that seem to cause the major issues and
| have to be managed.
|
| The real problem, of course, is the human capacity to comprehend
| (or not) the entirety of the system or subsystem by modeling it
| correctly in the brain.
| doug_durham wrote:
| I reject this thesis totally. It has never been easier to get
| things done with software. The APIs provided on many different
| platforms allow useful applications to be developed by a larger
| number of people than ever before.
|
| I started my career in the era is 8 bit microcomputers. Yes it
| was great to know the entire stack from processor up through the
| GUI. But I would never want to go back to those days. Development
| was too slow and too limited.
|
| We are in a golden era of software development.
| psychoslave wrote:
| > Very rarely do these models reflect reality.
|
| > It's a nice coincidence when they do.
|
| > It's catastrophic when they don't.
|
| Well, generally, it's not my experience. Most software out there
| is not critical. Many bloated crappy webapp might end up badly
| doing what user is expecting while sucking irrelevantly large
| amount of resources all day through with erratic bugs showing
| here and there, yes all true.
|
| But this is not as critical as the software that handle your
| pacemaker or your space rocket.
|
| Most software can afford to be crap because most projects are
| related to human whims for which a lake of quality is at worst a
| bit of frustration that will not cause a feeling of devastation
| or pure death penalty. All the more, I guess that most software
| developers out there are neither working on Sillicon-Valley-like
| money motivation, nor paying their bill with the kind of software
| project they love to build on passion. So this means most
| software that hits market are generated through exogenous
| miserable remunerations. Who will expect anything else than crap
| to be the output of such a process?
| wryl wrote:
| While I sympathize that the stakes don't appear to match up, I
| am talking about all software, and also taking into account the
| fact that these frustrations are numerous. It'd be one thing if
| it was taken in small pieces, small cracks in the stairs, but
| half of them are missing on average.
|
| I really don't like missing stairs.
|
| We're very removed from the usage of our software, and
| experience it in short-form (hopefully) actionable signals that
| we use to inform our development process. We don't get to
| appreciate the real pain in this "death by a thousand cuts"
| unless we can somehow switch bodies with a new user.
|
| I see programming as a trade, however, and we do have the power
| to govern the quality of our software. There are, however,
| incentives, financial or not, that can get us to look the other
| way.
| wvenable wrote:
| This article presupposes that this software crisis actually
| exists or is a significant problem. The crisis is all this these
| things: Projects running over-budget
| Projects running over-time Software was very inefficient
| Software was of low quality Software often did not meet
| requirements Projects were unmanageable and code
| difficult to maintain Software was never delivered
|
| Now take the word "software" out and how many human endeavours
| have one or all of these things? And then how much software is
| actually pretty great? We tend only see the failures and the
| flaws and success is just a baseline that we completely ignore
| even as it gets continuously better.
|
| When we press the power button on our computer and it gets to a
| desktop, our computer has already run through hundreds of
| abstractions. Just at that desktop it is already the most
| complicated machine we have or will interact with all day. This
| happens billions of times a day, all across the world, and mostly
| flawlessly. And that's just one tiny example.
| jdbernard wrote:
| The thing that is unique about software is the lack of physical
| constraints which serve as a natural forcing function or filter
| on quality. With a bridge, for example, at some level it must
| meet a minimum bar of structural integrity, quality of
| materials, etc. or it will fall over from its own weight. As a
| cook, there is a bare minimum I have to hit with the quality of
| my ingredients and skill of preparation in order for the food
| to be palatable and edible.
|
| No such limits exist on software beyond maybe performance and
| memory constraints. But both of those are in abundance, so we
| can and do patch over crap endlessly. Every one of us has had
| that moment where we think or say, "how does this even work?"
| But until the user hits the wrong edge case they have no idea
| how rickety the underlying code is.
|
| > This happens billions of times a day, all across the world,
| and mostly flawlessly.
|
| No, I'd argue it's much more common for there to be flaws.
| They're just not obvious. They're random crap like my phone
| continuing to vibrate after I've answered the call until I get
| another call or text. Or options disappearing from a web app
| because something didn't load 100% correctly. Or the joys of
| flaky Bluetooth pairing. The list is endless.
|
| "Have you tried turning it off and on again" is evidence of
| this. It's normal for software systems to get into such
| inscrutably subtle bad states that the only fix is to wipe
| everything and reload from scratch.
| wvenable wrote:
| > No, I'd argue it's much more common for there to be flaws.
| They're just not obvious. They're random crap like my phone
| continuing to vibrate after I've answered the call until I
| get another call or text.
|
| This is kind of what I'm talking about. The absolute massive
| complexity within your device that you and billions of people
| to seamlessly make calls from anywhere in the world to
| anywhere in the world with devices made by all different
| companies using infrastructure made by all different
| companies and it all works so incredibly well that we mostly
| take it entirely for granted.
|
| But yes, sometimes the phone doesn't stop vibrating.
| adamrezich wrote:
| One big difference between software and everything else is
| that software is built on top of other software, on top of
| other software, in an ever-increasing stack of abstractions
| and "good enough". This stack grows faster in software than
| in any other domain, because, as jdbernard said above,
| there's a lack of physical constraints preventing this from
| occurring. As this stack grows and grows, more points of
| failure are introduced, which can cause emergent,
| unexpected bugs and issues, which are hard to diagnose and
| pin down, because doing so involves traversing the stack to
| find the root cause(s).
|
| It's one thing to appreciate the natural world around us
| and how it all seems to work together flawlessly to provide
| the reality we all experience together--but that's because
| it's natural, not artificial, like the world of software we
| have created. When things work less than perfectly in
| software, there are human causes behind it, which, once
| identified, could be resolved in order to improve
| everyone's lives. But instead, most people share your "good
| enough--it's a miracle it all works!" mentality, which
| causes acceptance of any and all software issues, which
| leads to further issues down the line as we say "good
| enough" and build yet another layer atop it.
| wvenable wrote:
| What I've noticed is that most issues are at the leaves.
| Abstractions that have existed for a long time and that
| are used heavily tend towards being more solid over time.
| (And more documented, well known, etc)
|
| It's the stuff at the edges that appears to be less
| reliable but that's mostly because it's new. It doesn't
| really feel like that is getting worse though. We are
| constantly interacting with more and more software than
| ever before but it's not like everything is broken. The
| fact that you can reliably make a phone call is far more
| significant than the fact that the vibration doesn't
| stop. Both are build on ever-increasing stacks of
| abstractions.
|
| The difference in that example isn't some emergent
| complexity -- it's just that one is more important than
| the other. There are lots of analogies in the physical
| world where less-important things are crappier than more
| important things. We don't consider that some crisis of
| abstraction.
| jdbernard wrote:
| > But yes, sometimes the phone doesn't stop vibrating.
|
| I failed to communicate clearly. Yes, I agree, the scope of
| human achievement is amazing, software included. However,
| the issues with software go far deeper than just the
| trivial example I gave of the phone. It's pervasive and
| pernicious. I assume most software developers understand
| this as lived experience, but I'll elaborate more.
|
| Almost every single person I know who is not an IT
| professional of some sort regularly comes to me, my son,
| and others who are IT professionals for help with things
| that are broken and non-functional to the point that they
| cannot use it without assistance. And that's just the show-
| stoppers. There is tons of crappy software that they've
| just found workarounds for, and they've gotten so used to
| having to find workarounds that they've just stopped
| complaining. Not because it's working correctly, but they
| would rather be happy and accept it than get constantly
| worked up about something over which they have no control.
| This is not entirely unique to software, but there isn't
| really any other mass technology that is as bad as
| software.
|
| Software is simultaneously miraculous and horrible. Because
| of computing we have capabilities that would be seen as
| science fiction or magic not too long ago. But because of
| the things called out in this article, these magical tools
| and technologies are plagued by endless amounts of things
| that just don't work right, break randomly, and fail in
| unexpected ways.
|
| With physical systems and hard sciences we identify the
| physical constraints, model the system, and then engineer
| solutions around those models, refining our solutions as we
| refine our models. With software we make up the models,
| rarely fully document them, and slap things together as we
| go. Some companies do better than others, and some domains
| take quality and reliability seriously, but in my
| experience even that correlates to distance from physical
| constraints. In general the portions of our industry that
| are closer to real-world constraints (chip manufacturing,
| control interfaces, etc.) have historically been better
| about delivering quality software.
|
| If I come across as frustrated it's because I am. I love
| building software, and I used to love using software, but I
| am incredibly frustrated by the gap between what we've
| built and what is possible.
|
| I could keep going, but I'll end this comment with one
| other observation. I'm not an Apple fanboy by any means,
| but Apple used to be the exception to this general rules of
| "software just kind of generally sucks." Apple used to be
| "it just works." And that was because Steve Jobs was the
| forcing function on quality. It's possible.
|
| Again, this isn't unique to software. Excellence in any
| domain requires someone to be the standard bearer. At a
| restaurant it may be the head chef, refusing to let
| anything out of the kitchen that doesn't meet their high
| bar, for example. In every domain there exist various
| pressures that push quality down (time, cost, etc.) but
| software lacks the natural floor established by physical
| constraints, and is unique riddled with problems, IMHO.
| wvenable wrote:
| I think I agree with you but disagree with the article.
|
| > But because of the things called out in this article,
| these magical tools and technologies are plagued by
| endless amounts of things that just don't work right,
| break randomly, and fail in unexpected ways.
|
| There is _way_ more software out there than one can even
| imagine responsible for literally every aspect of human
| society. There is an insatiable need for more software
| and there simply isn 't enough software developers in the
| world to do it all. What you get is triage -- things half
| baked because of effort. There isn't some problem
| specific to software with regards to physical
| constraints. Cheap stuff breaks whether it's software or
| power tools or children's toys. It's that simple.
|
| > And that was because Steve Jobs was the forcing
| function on quality.
|
| Exactly. What does that have to do with abstractions or
| the lack of physical constraints?
| RaftPeople wrote:
| > _The thing that is unique about software is the lack of
| physical constraints which serve as a natural forcing
| function or filter on quality._
|
| Completely agree with this. The number of good+reasonable
| solutions is almost infinite, and the number of bad solutions
| is also almost infinite.
|
| What makes it even worse is that we really don't have a good
| method of communicating the design+structure of our models to
| others (tech and non-tech). As the system gets more complex
| the issue gets worse.
|
| We carry so much info in our heads about system models, and
| that info is painstakingly acquired through inefficient
| communication methods and trial and error and thoughtful
| analysis.
|
| It would be amazing if we could create tools that allow us to
| describe the essence of the model and make it directly
| available to our brains so we could all collectively reason
| about it and manipulate it collaboratively.
| asdfman123 wrote:
| The article doesn't do a good job of explaining what they
| determine the actual crisis is. Complexity in and of itself
| isn't a problem. The problems you listed, however, are.
|
| But I don't think that's the real motivation for articles like
| these: the real motivation is that things just _seem_ out of
| hand.
|
| But I think as an experienced programmer you have to balance
| that feeling of overwhelm with what needs to be done. It's
| taken me a while to arrive at this conclusion, but I believe
| it's important. Things will never be completely figured out and
| you have to be okay with it.
| wvenable wrote:
| I think being a software developer is always being in
| conflict with reality. Most of our job is logic and rules.
| Preciseness and completeness. Reality, however, is full of
| people, chaos, and change and we are uncomfortable by that. I
| think this is why programmers have such strong opinions on
| languages, tools, and techniques.
|
| Over the years I've started to tune out articles that say
| everything is doomed or articles that propose the silver
| bullet to fix everything. Things are not terrible -- they
| are, in fact, always getting better. It's never been a better
| or easier time to be a software developer. But, because of
| that, the problems we have to solve are harder. The demands
| are greater. I think this appears to many like we're stuck in
| place because we always feel like we're behind the 8 ball.
| The appetite for software is insatiable.
| marcosdumay wrote:
| > how many human endeavours have one or all of these things?
|
| You want a binary answer as in "it never happens with X"?
| Because for most kinds of project, we say it didn't go well
| when 1 or 2 of those happen, while we declare a victory in
| software if we manage to avoid 2 of them.
|
| > And then how much software is actually pretty great?
|
| And now I'm curious about your bar for "pretty great" software.
| This is obviously subjective, but most people wouldn't look at
| any practical software and call it great.
| wvenable wrote:
| > This is obviously subjective, but most people wouldn't look
| at any practical software and call it great.
|
| This what I think is insane. We interact with probably
| thousands of pieces of software every day (and I'm probably
| off by an order of magnitude) and none of it is great? What
| is the bar here?
|
| It's funny that something that was absolutely amazing
| yesterday is no longer even so much as great today. I can
| instantly connect to a remote server from my phone, type some
| text, and generate a pretty good poem about my cat and all of
| that is already "meh".
| bitwize wrote:
| Complaining about too many abstractions in software is like
| complaining about too many meetings in a Scrum shop. It's true.
| Everybody knows it. Or rather, it would be true if efficiency of
| the software/development process were the thing being optimized
| for. But it's ultimately a short-sighted perspective because
| you're not thinking of the _people_ involved.
|
| In the case of Scrum, Scrum is implemented because it gives
| managers and stakeholders some semblance of observability and
| control over the software development process. Granted, Scrum
| shops are merely cosplaying at implementing a rigorous,
| controlled methodology, but if you take that semblance away you
| will have angry, frustrated decision makers who resent the
| software teams for being opaque and unwilling to commit to
| budgets or schedules.
|
| In the case of abstractions... maybe there are a bunch of junior
| devs who just can't learn the complexities of SQL and _need_ an
| ORM layer in order to reckon with the database. (I worked at a
| software shop where _the most senior_ dev was like this; part of
| the reason I was brought on board was because I knew how to munge
| the PL /SQL scripts that formed part of their ETL process.) Maybe
| one part needs to be completely decoupled from another part in
| order to allow, for example, easy swapping of storage backends,
| or the creation of mocks for testing. Maybe some architect or
| staff SE is just empire-building, but at any rate they're way
| above your pay grade so the thing will be built _their_ way, with
| their favorite toolkit of abstractions and advocating for the use
| of fewer abstractions will get you nowhere.
|
| If you're working on a team of John Carmacks, great! You will be
| able to design and build a refined jewel of a software system
| that contains just the right amount of abstraction to enable your
| team of Carmacks to maintain and extend it easily while still
| running smoothly on a 386. Unfortunately, most software teams are
| not like that, most customers are not like that, so the systems
| they build will develop the abstractions needed to adjust to the
| peculiarities of that team.
| kleiba wrote:
| OT: anyone's got an idea why Firefox would not be able to display
| that page in reader mode (button in the URL bar not showing up)?
|
| For some reason, my eyes cannot cope with white text on black
| backgrounds, so I usually just go to reader mode in cases like
| this article. But here, this option does not exist, for some
| reason?
| wryl wrote:
| Thanks for the feedback regarding this, I've been trying to
| find a middle-ground between my authoring style (writing raw
| HTML with low overhead, instead of using a static site
| generator) and "portable readability".
|
| I have a feeling it's just me not annotating the markup
| correctly, which I am in the process of fixing and porting
| around!
| marcosdumay wrote:
| Personally, I've found that adding 3 lines of javascript to
| make a style switcher leads to great results. (You know you
| can simply load a CSS file in javascript and apply it, in a
| single command?)
| pauleee wrote:
| They didn't use proper html syntax like <article>, <h2>, etc.;
| only <div>.
|
| With this it's not possible for the browser to know what is the
| content of the article.
| kwhitefoot wrote:
| No idea. And I won't be reading it either.
|
| Someone wanting the world's attention because of a crisis
| really should not add extra friction.
| williamcotton wrote:
| Shallow and composable is something we all experience when using
| UNIX tooling.
|
| GUIs are where this all falls apart as they are literal islands
| that don't communicate with each other in a composable manner.
|
| I've been experimenting with some GUI-meets-shell-pipeline ideas
| with a tool I've been working on call guish.
|
| https://github.com/williamcotton/guish
|
| I'm curious to know if anyone knows of any similar tools or
| approaches to composable GUIs!
| wryl wrote:
| This is really, really cool! Definitely going to share this
| around, I love tools like this. They bring so much visibility
| into dark spaces.
| williamcotton wrote:
| Thanks! I started on it a week ago so it is just barely
| presentable at this point. I expect there are many glitches
| and bugs that I am completely unaware of!
|
| The parser is off-the-shelf and seems robust. The AST-to-bash
| aspect is a bit messy. Wrapping arguments with spaces in them
| with either " or ' is somewhat of an unsolved problem. Like,
| it's hard to tell if someone wants to embed a bash variable
| and thus wants " or if they are using something like awk that
| uses $var itself and thus need '. We'll see how it goes!
| wryl wrote:
| It's still really impressive! Comparisons were thrown
| around to nushell, which is also a really impressive
| project. I can see the vision and I hope to follow along!
| :)
| marcosdumay wrote:
| Yes, shallow, wide, and composable. That's how our abstractions
| should be.
|
| But the larger problem is not GUIs. GUIs are a problem, but
| they are necessarily at the top of the abstraction stack, so
| the problem doesn't compose any further. (What interestingly
| means they are so much of a problem that they aren't anymore.)
|
| The elephant in the room nowadays are distributed systems.
| rednafi wrote:
| It's absurd to complain about abstraction. Hardware will continue
| to leapfrog the pace of software innovation and things will keep
| getting better.
|
| Sure layers of abstraction is leaky and has their issues, but I
| don't want to write a hipster language in a hipster editor. If
| you enjoy that, great.
|
| Also, it's easy to look at the past with rose tinted glasses.
| Modern softwares are bloated mess but still a million times more
| productive.
| adamrezich wrote:
| > Hardware will continue to leapfrog the pace of software
| innovation and things will keep getting better.
|
| This is wishful thinking. Try running Windows 2000 on era-
| appropriate consumer hardware and tell me just how much better
| off we truly are now with bloated and unresponsive web apps.
|
| > Also, it's easy to look at the past with rose tinted glasses.
| Modern softwares are bloated mess but still a million times
| more productive.
|
| What metrics are you using to quantify "a million times more
| productive"?
| woah wrote:
| I hate when people try to promote things by inventing a crisis
| beryilma wrote:
| I don't think we have a software crisis. Millions of programmers
| are able to create more or less useful programs all over the
| word; everything including your toaster is running software on it
| succesfully enough; and the community was able to build programs
| that are accessible to all from a 5 year old kid all the way to
| your grandparents. Where is the crisis in it?
|
| However, we have a project management crisis, which is not only
| limited to software, where people in charge of planning are
| distanced from people in charge of the delivery. And we don't
| seem to be able to bridge the gap. Agile, Scrum, whatever are
| indicators of this gap where "gurus" take all of us as fools and
| we are not able to produce anything better ourselves.
|
| Commoditization of software development is also contributing to
| this mess because people of all skill levels can find a way to
| partake in this activity with results of varying success. This is
| not good or bad, but just the nature of ease of entry into the
| field. Not much different than food business where we have both
| Michelin star restaurants as well as MacDonalds of the world both
| of which have consumers. But we don't say we have a restaurant
| crisis.
| nicce wrote:
| > I don't think we have a software crisis. Millions of
| programmers are able to create more or less useful programs all
| over the word; everything including your toaster is running
| software on it succesfully enough; and the community was able
| to build programs that are accessible to all from a 5 year old
| kid all the way to your grandparents. Where is the crisis in
| it?
|
| About that toaster point; this is the actual, real, concrete
| example of the software crisis. Your toaster runs bad code. Bad
| code that is not designed with resiliency or security in mind
| but is connected to internet. It reduces your toaster's
| lifetime. In the past, you might have had a toaster which
| lasted 10 years. Now, because of the bad software and maybe
| even mandated Wifi or Bluetooth connection, your toaster is
| garbage after 2 years because the vendor stopped pushing
| updates. Or maybe there were no updates at all. But because we
| might not always directly see it, or because the current hyper
| consumption and never-ending buying of new products, the crisis
| is not always that visible.
|
| We might be even okay that toaster stopped working after 2
| years, and did not pay attention or knew why that happened. But
| maybe it was part of Mirai botnet
| https://www.cloudflare.com/learning/ddos/glossary/mirai-botn...
|
| Likely toaster was not part of that, as they use simpler chips,
| but who knows.
| beryilma wrote:
| But I wouldn't call a toaster running code an example of
| software crisis. Call it business crisis or crisis of
| capitalism/consumerism, but it is not software crisis.
|
| I agree wholeheartedly that an internet connected toaster is
| a very stupid idea, but our ability to build such a system
| shows, if anything, a triumph of software (and hardware), not
| a crisis. What is a crisis here is societal one; the fact
| that there is a demand, however artificially created, for
| such things. Software abstractions or layers of abstractions
| to be able to build such devices have nothing to do with this
| kind of crisis.
| nicce wrote:
| > I agree wholeheartedly that an internet connected toaster
| is a very stupid idea, but our ability to build such a
| system shows, if anything, a triumph of software (and
| hardware), not a crisis
|
| We have managed to build systems with software that
| "triumph" for a very long time. Look no further than
| Voyager 1. A technical masterpiece which still works.
|
| But fundamentally, about the toaster; we see these things
| in toasters because there are so many abstractions and
| possibilities. It is easy to embed software for the toaster
| without having a legion of software developers from
| different fields.
|
| And it adds up and is the main issue of "unintended"
| usecases for toasters. Many from these cases are not
| depending from the developer. Developers do not even know
| about them or were not thinking about them because there
| were abstracted away. Overall system of the toaster can be
| so complex in these days, that no single person can
| understand it completely alone.
|
| But yes, the consumer part and that it is okay if toaster
| breaks, is indeed social problem. But likelihood of
| toasters breaking or doing something else than supposed to,
| could be originating or increasing from the software
| crisis.
| fungiblecog wrote:
| Just watch all the Rich Hickey videos on YouTube and you'll
| understand where it's all gone wrong.
| wryl wrote:
| With a splash of Joe Armstrong!
| peter_d_sherman wrote:
| >"We developed methods of building nested layers of
| _abstractions_ , hiding information at _multiple levels_. We took
| the problem of constructing software and morphed it into
| _towering layers_. "
|
| https://www.joelonsoftware.com/2002/11/11/the-law-of-leaky-a...
|
| https://en.wikipedia.org/wiki/Tower_of_Babel
|
| https://en.wikipedia.org/wiki/Hierarchy
|
| https://en.wikipedia.org/wiki/Abstraction
|
| https://en.wikipedia.org/wiki/Abstraction_(computer_science)
|
| (Compare and contrast!)
___________________________________________________________________
(page generated 2024-07-05 23:01 UTC)