https://blog.metaobject.com/2021/06/glue-dark-matter-of-software.html?m=1

metablog

Wednesday, June 9, 2021

Glue: the Dark Matter of Software

"Software seems 'large' and 'complicated' for what it does". I keep
coming back to this quote by Alan Kay.

The same feeling has been nagging me pretty me much ever since I
started writing software. On the one hand, there is the magic, almost
literally: we write some text (spells) and the machine does things in
the real world. On the other hand, it seems just way too much work to
make the machine do anything more complex than:

    10 PRINT "Hello"
    20 GOTO 10

Almost like threading a needle with boxing gloves. And that's even if
we are careful, if we avoid unnecessary complexity.

And the numbers appear to back that up, Alan Kay mentions Microsoft
office at several hundred million lines of code. From my personal
experience, the Wunderlist iOS client was not quite 200 KLOC. For the
latter, I can attest to the attention given by the team to not
introduce unnecessary bloat, and even to actively reduce it. (For
example, we cut our core code by around 30KLOC thanks to some of the
architectural mechanisms such as Storage Combinators). I am fairly
sure I am not the only one with this experience.

    hexagonal architecture has enabled me to extract the business
    logic in the product i'm building and currently it's less than 5%
    of all code

    -- 3. life out of balance (@infinitary) November 25, 2017

So why so much code? After all Wunderlist was just a To Do List,
albeit a really nice one. I can't really say much about Office, I
don't think anyone can, because 400 MLOC is just way too much code to
comprehend. I think the answer is:

Glue Code.

It's the unglamorous, invisible code that connects two pieces of
software, makes sure that data that's in location A reaches location
B unscathed (from the datbase to the UI, from the UI to the model,
from the model to the backend and so on...). And like Dark Matter, it
is invisible and massive.

Why do I say it is "invisible"? After all, the code is right there,
isn't it? As far as I can tell, there are several related reasons:

 1. Glue code is deemed not important. It's just a couple of lines
    here, and another couple of lines over there ... and soon enough
    you're talking real MLOCs!
 2. We cannot directly express glue code. Most of our languages are
    what I call "DSLs for Algorithms" (See ALGOL, the ALGOrithmic
    Language), so glue can not be expressed intentionally, but only
    by describing algorithms for implementing the glue.

That's why it is invisible, and also partly why it is massive: not
being able to express it directly means we cannot abstract and
encapsulate it, we keep repeating slight variations of that glue.
There is another reason why it's massive:

 3. Glue is quadratic. If you have N features that interact with each
    other, you have O(N2) pieces of glue to get them to talk to each
    other.

This last point was illustrated quite nicely by Kevin Greer in a
video comparing Multics and Unix development, with the crucial
insight being that you need to "program the perimeter, not the area":

For him, the key difference is that Unix had the pipe, and I would
agree. The pipe is one-character glue: "|". This is absolutely
crucial.

If you have to write even a little custom code every time you connect
two modules, you will be in quadratic complexity, meaning that as
your features grow your glue code will overwhelm the core
functionality. And you will only notice this when it's far too late
to do anything about it, because the initial growth rate will be low.

So what can we do about it? I think we need to make glue first class
so we can actually write down the glue itself, and not the algorithms
that implement the glue. Once we have that, we can and hopefully will
create better kinds of glue, ones like the Unix pipe in that they can
connect components generically, without requiring custom glue per
component pair.

Marcel Weiher at 1:43 AM
Share

1 comment:

 1. [S]
    SinicaJune 9, 2021 at 10:41 AM

    There ideas like executable choreographies (swarm communication)
    intended to be glue code at the integration level of the systems.
    Unfortunatly systems start small and dont need choroegraphies and
    when they become big is too late...

    ReplyDelete
    Replies
        Reply

Add comment
Load more...

 

>
Home
View web version

About Me

Marcel Weiher

View my complete profile
#
Powered by Blogger.
#