[HN Gopher] Glue: The Dark Matter of Software
___________________________________________________________________
Glue: The Dark Matter of Software
Author : mpweiher
Score : 71 points
Date : 2021-06-12 08:43 UTC (1 days ago)
(HTM) web link (blog.metaobject.com)
(TXT) w3m dump (blog.metaobject.com)
| ducharmdev wrote:
| Interesting observations; although by definition glue code is
| amorphous and hard to pin down, I would like to see more concrete
| examples of the problem discussed here.
|
| Is this a general problem related to how we build interfaces for
| our implementations? Or is this more a problem with particular
| kinds of module systems? (Or both)
| ryanackley wrote:
| I get frustrated with the layers between relational data and
| data structures you pass around in code.
|
| So for example, if you want a build a simple REST api for your
| data model, you have glue code to read/write the data and
| transfer it to/from DAO's (Data access objects). Then you have
| glue code to transfer the data into to/from object model you
| expose via the API. This feels like 90% of server side CRUD app
| programming these days.
|
| It's not quite boilerplate because it's a mix of ORM
| optimizations for your data model and data massaging to make it
| more digestable to end users of your API.
| sdenton4 wrote:
| It's all about compartmentalized code... Glue happens whenever
| compartments interact with one another. Examples, thinking of
| an app I used to work on:
|
| a) App frontend (UI logic) is compartmentalized from the app
| backend. Communication is handled by an async messaging system,
| requiring bundling up messages in both directions.
|
| b) The app backend talks to a few different servers, each with
| their on expected message set. glue glue glue.
|
| c) The app backend talks to the app database,
| inserting/altering info, and extracting database rows into
| objects which can be passed around.
|
| d) (We also had peer-to-peer communication over local wifi,
| which required its own massive pile of communication code...)
|
| e) Even different objects/methods in the backend code end up
| requiring small amounts of glue code to arrange+prepare
| argument sets to call one another.
|
| So, every time we create a division between two pieces of code,
| we introduce boundaries, which introduces glue.
|
| ---
|
| One way to mitigate the complexity is by making the passed
| messages first class. In our app, there was some 'AppThing'
| object which represented an instance of the fundamental thing
| the app was about. You then stuff that object with every piece
| of info about AppThings you could ever want, and pass it around
| freely between components. Invariably it gets kinda bloated and
| fractures, when someone doesn't want to bother building up the
| WHOLE thing with the necessary DB and server calls to ensure
| the data is all accurately filled in, when you just want to
| pass one or two fields to a neighboring UI element. And then
| you find yourself with lots of logic checking that certain
| fields are present amongst the million fields in your universal
| object...
|
| Another approach is to aggressively prune the graph of possible
| component interactions. Functionality should be well-
| encapsulated, with absolutely minimal APIs for external
| interactions. Actually draw the 'cell membranes' between
| different parts of the app, map out the interactions between
| cells, and find ways to cut it back.
| dgb23 wrote:
| I think it's interfaces and protocols at a granular level.
|
| If every interface is bespoke while also being explicitly
| described top down, then we end up with this quadratic
| complexity.
|
| Uniform, extensible interfaces and generic data structures are
| the solutions to this, as well as shifting complexity to data
| and away from code. Several modern languages, techniques and
| protocols address this in varying degrees and from different
| angles. An old language that does this very well is SQL.
| kaycebasques wrote:
| > Several modern languages, techniques and protocols address
| this in varying degrees and from different angles.
|
| Examples? (Asking as a curious beginner who doesn't know
| where to start.)
| dgb23 wrote:
| Go's implicit interfaces, Rust's traits, Clojure's
| sequences, transducers, multimethods and spec, Julia's
| multiple dispatch, HTTP REST, JSON and json-schema.
|
| These are all attempts to enable generic, simple code over
| complex data structures, while respecting and incorporating
| extensibility and change.
|
| It's an old idea but I think more importance is given to it
| in more recent years.
| mettamage wrote:
| I wrote a non-programmer summary to some of my friends, because
| my mind is blown. Here it is (warning: it's very informal).
|
| This is some next level 80/20 stuff, I'd argue it's even 99/1
| (consultant speech for "being efficient" by seeing if you can
| exploit Pareto's principle [0]).
|
| A guy on YT [1] simulates in 7 minutes how the two biggest
| operating systems were built in the sixties (Multics by IBM and
| Unix by 2 guys in a garage). Nowadays we only have descendants of
| Unix, this simulation shows IMO two things:
|
| 1. How a chain of command screws up productivity (I doubt that
| anyone at IBM had the power/autonomy to change complete structure
| of the product. This was needed, but such change management is
| insane).
|
| 2. How the pipe command in Unix changed the mathematical nature
| of developing an operating system (in fact, it's even better than
| 99/1, when you think about it). In normal human language, the
| difference is: IBM programmed modular blocks. Unix programmed
| modular blocks. IBM then programmed how those modular blocks
| should interact. Unix created a pipe command and allowed computer
| users themselves to implement how these modular blocks should
| interact. At the time, computer users were all savvy enough like
| that.
|
| Result: when IBM created a feature, it needed to integrate it
| with all the other features that came before. When those 2 guys
| in a garage (Unix) created a feature, all they needed to do was
| create that feature.
|
| So mathematically you could say (where each term is the amount of
| work required for one new feature + integrating it with the
| operating system): IBM/Multics: (0 features_before + 1
| new_feature) + (1 features_before + 1 new_feature) + (2
| features_before + 1 new_feature) + (3 features_before + 1
| new_feature) + (4 features_before + 1 new_feature) + (5
| features_before + 1 new_feature) + (6 features_before + 1
| new_feature) ... + (n_features_before + 1 new_feature) -->
| simplifies to my favorite formula which is: n(n-1)/2 --> you can
| also see it as --> 0.5n^2 - 0.5n --> So, roughly this is n^2.
|
| Just count the amount of numbers, it comes down to: 28 units at
| the 7th term/timestep.
|
| Unix/2 garage guys: 1 new_feature + 1 new_feature + 1 new_feature
| + 1 new_feature --> simplifies to n_new_features
|
| At term/timestep, you'd get 7.
|
| At the 100th feature, you'd literally 100^2 amount of coding work
| at IBM whereas at the Unix place you'd simply need 1 unit ...
|
| This isn't 80/20 or 99/1. This is n^2/n (meaning: what one needs
| n^2 time for, the other could do in n time).
|
| [0] https://en.wikipedia.org/wiki/Pareto_principle
|
| [1] https://www.youtube.com/watch?v=3Ea3pkTCYx4&ab_channel=FOAM
| convolvatron wrote:
| open standards. there must be one of those game-theory economic
| analyses.
|
| but frankly i don't see how its _ever_ a good idea to structure
| yourself around closed worlds unless the seller really has
| something that novel and indispensable.
| kgr wrote:
| I'm not advocating a closed system. Just like UNIX doesn't
| lock you in to any hardware or CPU vendor. FOAM is open
| source and most of the functionality is provided by existing
| languages, libraries and databases.A universal adapter, not a
| universal provider.
| https://www.youtube.com/watch?v=PsFLlgrzn2E
| kgr wrote:
| https://www.youtube.com/watch?v=S4LbUv5FsGQ covers this topic. As
| I say in the video, no point solution (ie. some new API) can
| solve the problem, when the need to integrate (glue) between so
| many point solutions, is the problem.
| kgr wrote:
| Both the video from the article and the one I posted above are
| from this longer talk of mine:
| https://www.youtube.com/watch?v=PsFLlgrzn2E
|
| I think this the most important problem in software development
| to be given the least amount of attention, or even awareness.
| Another new MVC framework, build tool, or even programming
| language, isn't going to solve this problem. First we need to
| acknowledge the problem.
| kuharich wrote:
| Past comments: https://news.ycombinator.com/item?id=27486706
| PaulDavisThe1st wrote:
| I do a lot of stuff with my house. As a result I have on hand, at
| various points in time: * contact cement *
| superglue * mastic * wood glue * gorilla glue
| * liquid nails * various caulks
|
| Why so much glue? Because the right glue to use depends on the
| materials being joined, and the context in which the joint will
| need to survive (moisture, temperature, duration).
|
| As with construction and art, so with software. The Unix pipe is
| powerful because it enforced "the only thing you can connect
| together is line-oriented byte streams". But when you are
| building something where that's not the substrates in question,
| it's the wrong tool, and you need a different kind of glue.
|
| Expand that to all the possible different things you may
| wish/need to "glue" together in software, and it becomes fairly
| apparent why there's a huge diversity in the glue being used, and
| why dreaming of a world in which you get by with just the
| software analog of gorilla glue is the wrong thing to do.
| pimlottc wrote:
| > Why so much glue? Because the right glue to use depends on
| the materials being joined, and the context in which the joint
| will need to survive (moisture, temperature, duration).
|
| On that topic, I am obligated to link to a wonderful no-
| nonsense guide choosing on the best glue for your particular
| materials:
|
| http://www.thistothat.com/
| mpweiher wrote:
| Unix pipes and filters were not presented as "the" solution,
| but as an example for the properties we want from our glue,
| i.e. we should be able to glue our pieces together without
| having to write much additional code.
|
| And yes, in order for this to work, we need lots of different
| kinds of glue: https://blog.metaobject.com/2019/02/why-
| architecture-oriente...
|
| However, for most programming tasks, we have one kind of glue
| (with minor sub-variants): the procedure call. And so instead
| of having something that snaps together with a bit of glue, we
| have to write enormous amounts of boiler-plate in order to make
| the pieces fit. (The analogy only goes so far...)
| PaulDavisThe1st wrote:
| I'm been a compiled-language programmer for 35 or so years. I
| don't view procedure calls as glue, I perceive them as a
| fundamental mechanism for organizing code. At the level of
| code that I normally operate at, if you don't write
| everything in-line, then you've got procedure calls. Not
| glue, at least, not primarily.
|
| Glue is a way to move data (and perhaps other stuff too)
| across code boundaries that are imposed by design, language
| or other constraints. So, sure, stuffing a bunch of stuff on
| the stack, jumping to a new address and doing something with
| whatever is on the stack is, in some sense, a sort of glue.
|
| But your TFA seems to be talking about a much higher level of
| glue than this, I think. The sort of glue that's represented
| by things like (random list): protobufs, RPC, serialized
| JSON, even ye olde CORBA/OLE models. Questions like "I've got
| this customer data, I need to give it to some code that will
| munge it in some way... how do I do that?"
|
| In that sense, procedure calls are low level mechanism that
| will support a higher level solution. Again, if it's not all
| just inline execution of machine code, then at some point,
| there's going to be a procedure call no matter what
| abstraction level you're working at.
| PaulDavisThe1st wrote:
| Also, note that the use of a pipe from a Unix shell is very
| different from the use of a pipe in most (all?) languages. So
| the property you're interested in is not really the
| fundamental mechanism (which is quite a complicated object,
| with error conditions and required logic to handle i/o), but
| the abstraction presented by the shell.
| p0nce wrote:
| Like gas, software naturally gets bigger up until it fills up any
| added value that might have been produced.
| maltalex wrote:
| Very interesting post.
|
| It feels very applicable to the functional vs imperative
| programming debate, and I say that as someone who's mostly on the
| imperative side. The top row of the "perimeter" is very
| "functional".
|
| It's interesting to note that people from the same school of
| thought as those who built Unix (or even the same actual people)
| ended up creating Golang, which is anything but a functional
| language.
___________________________________________________________________
(page generated 2021-06-13 23:01 UTC)