[HN Gopher] Steel Threads are a powerful but obscure software de...
___________________________________________________________________
Steel Threads are a powerful but obscure software design approach
Author : jaderubick
Score : 143 points
Date : 2023-03-10 06:09 UTC (16 hours ago)
(HTM) web link (www.rubick.com)
(TXT) w3m dump (www.rubick.com)
| angarg12 wrote:
| I hope people will excuse my cynicism
|
| Step 1: Rehash a bunch of existing ideas together (PoC, vertical
| slice, strangler pattern).
|
| Step 2: Give it a flashy new name.
|
| Step 3: Market your consulting services as an expert for flashy
| new name. Sell to companies how they can build high performing
| organizations with this new technique.
| goostavos wrote:
| Why are you calling out Uncle Bob like this?
| mhuffman wrote:
| JAMSTACK! or pre-materialized if you coded in the early 2000s
| or pre-rendered if you coded in the late 90s or pre-generated
| static files if you coded in the early 90s
| asplake wrote:
| See also Walking Skeleton https://wiki.c2.com/?WalkingSkeleton
| andrewshawcare wrote:
| This is the strangler (fig) pattern under a different name:
| https://martinfowler.com/bliki/StranglerFigApplication.html
| d--b wrote:
| This reminds me of something John Carmack tweeted once (can't
| find the tweet).
|
| In the tweet, he said that when coding, he'd start by the
| smallest possible PoC, and code it entirely front to back. That'd
| give the general structure, and then he'd build upon that. (this
| is what I remember of it fwiw).
|
| I do this all the time too, which I think is a vastly superior
| approach to TDD, which assumes how an API is going to be used,
| without actually writing the actual thing that's going to use it.
| andonisus wrote:
| I take a similar but somewhat orthogonal approach.
|
| Most of the time, any major features that require refactoring
| are usually around the data model and its representation in
| code (the existing control flow and overall flow of a request
| through the system is generally fine).
|
| I will build out what I believe the new data model should be,
| and then just work front-to-back, updating any references and
| refactoring the shared state and responsibility into the new
| data model, clearly separating out concerns and encapsulating
| responsibility.
|
| This method has proved itself time and again, and I recommend
| it to anyone who needs to make large changes to and existing
| code base. That is, start with how the kernels of data, state,
| and responsibility should look, and everything grows from
| there.
| layer8 wrote:
| That's the vertical slicing mentioned in the article.
|
| https://en.wikipedia.org/wiki/Vertical_slice
| thom wrote:
| You can (and perhaps should!) do this with TDD, you just start
| with functional tests instead. Only commit to unit and
| integration tests when you know you're not going to be throwing
| lots of stuff away through refactorings.
| manmal wrote:
| That reminds me of a quote from John Gall:
|
| A complex system that works is invariably found to have evolved
| from a simple system that works.
| radicalbyte wrote:
| This is how I do TDD - first I build out the smallest possible
| thing (usually with code, sometimes with comments) then iterate
| / play around a little until I have the structure right. Then
| start putting tests in.
|
| Then later in the cycle you have high-level structure so it's
| easier to start with the test.
|
| We teach TDD and a lot of Software Engineering practises
| largely to beginners to make them productive for the
| maintenance of software - as that is about 95% of or work. So
| the flows that are stressed are those suitable to
| maturing/mature code not to completely new systems.
| lordnacho wrote:
| That works if you are experienced in architecting these things.
| As with anything, if you're already pretty close in structure,
| you won't have as many problems adapting the PoC to the final
| form. If you were further away that you thought you'll end up
| with a bunch of inelegant hacks to reach the working state.
|
| The real problem is getting to the point where the initial
| solution you imagine is close to the initial PoC.
| q7xvh97o2pDhNrh wrote:
| > The real problem is getting to the point where the initial
| solution you imagine is close to the initial PoC.
|
| IMHO Carmack's point is closer to the standard advice given
| to writers (which is also true and good):
|
| Just get _something_ on paper that you know is _somewhat_
| workable, and then reshape it from there.
|
| Especially in team situations (as opposed to solo coding),
| the effect is magical. Endless meetings and weeks of
| technical flailing can be skipped by just having _something_
| instead of _nothing._
|
| Although, lately I've been finding I like this approach for
| solo coding too. Last night I opened up a Terminal, along
| with a browser tab for my new coding buddy, ChatGPT. The code
| I got from ChatGPT was absolutely horrendous for my needs,
| but at least it was something -- and, an hour later, I'd
| scrapped and rewritten everything except for a couple
| function names.
|
| There's something just plain _nice_ about keeping things
| moving along -- about getting some more clay on the table,
| some more paint on the canvas, and not being shy about
| reworking it after that.
|
| I think TDD _tries_ to capture this (especially in its pair-
| programming ping-pong-style implementations) -- but, in its
| haste to come up with a one-size-fits-all system, TDD glosses
| over the soul of what we actually _do._ You 're getting at
| exactly why -- it over-constrains the freedom to reshape
| things, and it slaps those constraints in too early.
| tekknolagi wrote:
| In Lessons from Writing a Compiler, there is a great section on
| implementation strategies: https://borretti.me/article/lessons-
| writing-compiler
| mindcrime wrote:
| I was working with a newer developer once, and we started
| working on some app that was going to be something moderately
| complex. I literally started with a single class that looked
| like: public class Foo { public
| static void main( String[] args ) {
| System.out.println( "Done" ); } }
|
| And he was really struck at first, like "Why would you write
| Hello World when we're building a $WHATEVER? My response was
| that the very first thing I always want to see, in any new
| project, is something compiling, packaging and executing. It's
| my way of ensuring that at a minimum the environment is set up,
| cosmic radiation hasn't fried my CPU or RAM, etc. And once I
| have that trivial program running, I just start building up
| from there.
|
| I more or less follow that same pattern for everything I write
| to this day. At most, the slightly more complex version I start
| with is something like a "template" or "skeleton" project. For
| example, I keep a sample Spring Boot project around that as a
| pom file, the directory structure, a package named something
| like org.fogbeam.example, and a simple controller that just
| returns "Hello World". Once I can build and run that, and hit
| localhost:8080 and see my "Hello World" page I start iterating
| from there.
|
| I can't tell you exactly how I developed this habit over the
| years, but it's worked well for me.
| chris_j wrote:
| On the subject of TDD, the way I've done TDD has been very
| similar to the technique described in the article. Work in very
| small increments, try to get a very thin vertical slice of
| functionality working through the system as soon as possible
| and, whatever you build, try to get it working end-to-end as
| soon as you can. Of course, I use tests to drive the work but I
| find it very helpful to use tests to drive those thin vertical
| slices of functionality.
|
| The book that really helped me to start working in this way is
| Growing Object Oriented Software, Guided by Tests by Steve
| Freeman and Nat Pryce [0]. The book has been around for a few
| years at this point and tech has moved on since then, as have
| some of the techniques, but it's still a very interesting read.
| (Disclosure: I was lucky enough to briefly work with one of the
| authors a few years ago but I was a fan of the book long before
| then.)
|
| [0] http://www.growing-object-oriented-software.com/
| bsenftner wrote:
| I don't remember where I heard this, but the method was
| described to me using the story of how a suspension bridge is
| built: first an arrow with twine tied to it is shot from one
| side of the canyon to the other. Then that twine is used to
| pull thicker twine, then rope, then steel cables, and so on.
| From a string to a suspension bridge, with the gap between
| the canyons conquered the entire time. To achieve large scale
| software projects, start with the thinnest logical twine that
| goes from start to finish for the project at hand, and build
| out with a start to finish operating environment as soon as
| possible and throughout the duration of the project.
| pbronez wrote:
| I use this metaphor all the time!
| eikenberry wrote:
| IMO this style isn't at odds with the spirit of TDD. TDD is
| mainly a technique to teach people about loose coupling and
| designing for maintenance. The main takeaway has always been
| that you should be able to run/exercise your code at every step
| and that you should use code to do that. No matter if it's an
| API or the whole program, that code you use to exercise it is
| what is key as it not only exercises the code it also helps you
| understand the problem.
| mpweiher wrote:
| > TDD, which assumes how an API is going to be used,
|
| I think you misunderstand TDD.
| d--b wrote:
| If you write tests first, you don't write a complete front to
| back "thread" first, right?
|
| If you write tests first, you assume that the test is going
| to match how you're going to use the stuff in a real
| situation, which you generally don't know.
| zidad wrote:
| That complete front-to-back 'happy path' test is exactly
| what I would normally start with as a first test, and it
| should of course be representative of how you are going to
| use it in a real situation.
|
| Not sure what kind of tests you write if they don't
| represent the actual expected behavior?
| mpweiher wrote:
| You don't write production code unless there is a need for
| it.
|
| The need is documented in a failing test case.
|
| Where does the failing test case come from?
|
| Hopefully from some other part of the code needing that
| code to be there. Or are you just conjuring up test cases
| out of thin air? If you're doing that, I'd venture you're
| not doing TDD.
|
| And certainly doing a spike to get the lay of the land is
| very mach part of XP where TDD came from.
|
| As is slicing your system vertically, so _complete_ units
| of functionality within an incomplete system.
|
| Rather than slicing horizontally, which is what you seem to
| be doing.
| StrangeATractor wrote:
| You see this pattern with almost anyone who is proficient at
| almost anything. Start with something in the simplest, smallest
| or most general way you can and master it, then build from
| there.
|
| Mathematicians and physicists make a career out of this, but it
| works with almost anything, including sports.
| robertlagrant wrote:
| > I think is a vastly superior approach to TDD, which assumes
| how an API is going to be used
|
| I think this is a function of how much you're designing up
| front, not of TDD itself. The stuff you design, you write tests
| for and then build. How much you choose to design up front
| (maybe almost nothing) is up to you.
| thefourthchime wrote:
| I've nice to hear that. This is also how I approach building
| software. My goal after at the end of a V1 is that if anyone
| looked at the code, they would wonder where it all is,
| shouldn't there be more code?
|
| I try to make the features, structure, everything as simple as
| possible. As you do this you'll see things that should be
| probably be abstracted, things you'll want to do as soon as
| this next part is in.
|
| Don't do it yet, wait for that actual feature to be written,
| then you refactor, make a system, etc.. Don't do it
| prematurely. You want to wait because the longer you wait, the
| chance the features parameters or use case will be different,
| or it won't even exist. Half the requirements they think they
| need at the start will be side thoughts by the end.
| nailer wrote:
| Similar here. I pick the one thing the system depends on, I
| call it 'hello world', and implement it with no UI at all and
| one command line test. The test exists to prove to myself and
| my team that it works, and know when it breaks, not to
| demonstrate an API. You can call it TDD or not.
|
| So I when someone suggests "we can build X" I say yes but first
| we need to build "hello world". The discussion about what
| constitutes Hello World is often valuable, but often ends up
| with examples like the article, eg "can we write a message and
| the recipient gets it?" "can we do one trade?" etc. These
| sometimes seems like trivial goals but implementing them can be
| surprising.
| kramerger wrote:
| > I think is a vastly superior approach to TDD
|
| Read the fibonaci example at the end of Becks book. He does in
| fact start with the smallest possible PoC.
| carterschonwald wrote:
| This sounds like monad transformers in Haskell. But fuzzier
| chaboud wrote:
| The article starts with stating a desire to bring the term "Steel
| thread" back to Wikipedia, formerly removed for the term's lack
| of use in the industry.
|
| After reading the linked article, I'm actually more convinced
| that removal was the correct course of action.
| [deleted]
| moomin wrote:
| aka Tracer Bullets aka Walking Skeleton
|
| and probably a bunch more.
| fedeb95 wrote:
| I didn't know this way to operate was called such, but it's the
| only way a sane person (i.e. someone who has to replace a
| monolith with smaller services AND do maintenance on the result)
| would.
| iamflimflam1 wrote:
| Never heard it called this before. We used to describe this as a
| tracer bullet through the system.
|
| https://flylib.com/books/en/1.315.1.25/1/
| voiper1 wrote:
| Indeed, tracer bullet is immediately what I thought of from The
| Pragmatic Programmer.
| tpoacher wrote:
| Sounds like a better name would have been "the Theseus Ship
| approach".
| smusamashah wrote:
| Found it on C2 wiki. It's not exactly the same definition as in
| the linked article though. Also, referred to as a
| "steel thread". Runs the length of the application architecture
| (front-to-back, top-to-bottom, whatever) of what you are
| building. Each new top-to-bottom feature is a new thread. Steel
| threads wrapped together incrementally form a cable stronger than
| an equivalent diameter solid cable extruded all at once. Thread
| akin to string, as in string testing. - NormanECarpenter
|
| https://wiki.c2.com/?SpikeSolution (too bad the c2.wiki has
| modernized the UI, its now almost unusable)
| DeathArrow wrote:
| This reminds me of Vertical Slice Architecture by Jimmy Bogard.
|
| https://jimmybogard.com/vertical-slice-architecture/
| MauranKilom wrote:
| Pro: Quick feedback cycle, particularly with how the new software
| fits the requirements. Some would call it agile.
|
| Con: Early design decisions are the hardest to change. For
| example, retro-fitting security (or parallelism) onto a system
| that was not designed with it in mind is a fool's errand.
| jdonaldson wrote:
| The problem with steel threads is that they take only a single
| path through a series of components. Rather than building
| something robust, you're more or less wearing a rut through the
| implementation strategy for a single service, and piling on use
| cases as you go. This is more or less how you end up with b2b
| "get 'er done" software, and you have to ask yourself if that's
| what you want.
| hidelooktropic wrote:
| The author states Wikipedia removed the term in 2013 because it's
| not notable.
|
| I'll join others here by saying I haven't heard of it as well and
| it would seem "tracer bullet" did just fine in The Pragmatic
| Programmer published much earlier.
|
| The author doesn't state who came up with the term "steel thread"
| and I'm suspecting it was the author.
| distcs wrote:
| > The author doesn't state who came up with the term "steel
| thread" and I'm suspecting it was the author.
|
| That's a weird type of suspicion. A simple search yields some
| prior usage of this term:
|
| https://dl.acm.org/doi/10.5555/2608547.2608553
|
| https://simplicable.com/IT/steel-thread
| shireboy wrote:
| Microsoft calls this the strangler fig pattern and recommends it
| for large migrations:
| https://martinfowler.com/bliki/StranglerFigApplication.html
|
| They make it somewhat easy to do using Yet Another Reverse Proxy
| (YARP). I'm in the middle of it for a .NET 4 to 6 migration. The
| challenge for me is that it introduces complexity. Developers
| have to think "do I fix this bug in v1 code or v2?" We have to
| host two backends instead of one. They both touch the same db, so
| that adds complexity- what if v2 does something in data that
| breaks v1? All solvable problems but just thought I'd share a
| from the trenches take. I do still think it is the right approach
| for this scenario.
| jollyllama wrote:
| Agreed. I've seen steel thread used to refer to a technique in
| developing _new_ applications. In this context, it means
| building one feature to completion before starting others. For
| example in web development, build a screen that uses a route
| and an api endpoint to fetch data from your datastore before
| building other screens using only mocks.
|
| Edit: the advantage of this is that any systemic problems will
| become apparent quicker; your earlier tasks become a proof of
| concept for the viability of the project as a whole.
| robertlagrant wrote:
| That sounds like vertical slices.
| jollyllama wrote:
| I hadn't heard of that term before but yes it is accurate.
| paulrpotts wrote:
| I think the concept is good, but the name is pretty bad - I mean,
| we have "green threads," "POSIX threads," "kernel threads," "user
| threads," etc. Given that this concept from software engineering
| has absolutely nothing to do with execution threads, it needs a
| less confusing name.
| pshirshov wrote:
| > Let's say you're building a new service to replace a part of
| your monolithic codebase.
|
| Why would I do that instead of building good configurable
| monoliths?
| regularfry wrote:
| Because you don't have access to a time machine, usually.
| girafffe_i wrote:
| [dead]
| foepys wrote:
| As much as I am a fan of monoliths, too many configuration
| parameters will land you in a world of hurt.
|
| You may start with 20 options but people will demand more and
| more options if you don't stop them very early. Those options
| will have side effects that other options are supposed to fix
| and those will also have side effects and in the end you have
| 1000 options, all doing various things nobody knows. The worst
| part was: since every option was global, people used options
| for things they weren't supposed to be used for across multiple
| modules.
|
| I worked at a company with over 15,000 options in their on-prem
| monolith. Nobody can know about all of them and each consultant
| demanded more and more customer-specific options.
|
| It was a nightmare and we tried to get the mess sorted with a
| plugin system where each plugin had their own options that
| couldn't pollute the global configuration. But it was very
| hard, very painful, and in the end we could only eliminate a
| few thousand global options.
| jsrcout wrote:
| I think the developers of the system I work on used your
| product:-). Although I don't think it has plugins. They said
| that in the early days they brought in a vendor engineer for
| a week to configure it. Currently the main config file is
| over 3K lines and some variants are close to 10K.
| pshirshov wrote:
| This is not correct.
|
| Not all the options are equal.
|
| It's possible to soundly define, verify and instantiate big
| modular contexts.
|
| https://github.com/7mind/izumi/
| sokoloff wrote:
| Things that are possible are frequently not the default
| condition.
| pshirshov wrote:
| What forces you to stick to the suboptimal "defaults"?
| grandinj wrote:
| Sounds like XP's (eXtreme Programming) Spike Solution
| mpweiher wrote:
| Hmm...
|
| http://wiki.c2.com/?SpikeDescribed
| sveme wrote:
| Sounds like a very common approach - build a small atomic feature
| front to end, test it, expand on top of that. This is, for
| example, the suggested approach in the Shape Up process. Works
| quite well for my team.
| t344344 wrote:
| How is that different from agile, TDD and refactoring?
|
| > A steel thread is a very thin slice of functionality that
| threads through a software system. They are called a "thread"
| because they weave through the various parts of the software
| system and implement an important use case.
|
| This sounds awfully like spaghetti code.
| fwlr wrote:
| It seems possible to do agile, TDD, and/or refactoring all
| without this practice of "following a single use case from
| start to finish throughout the entire application". That's all
| this is, an evocative name for the suggestion that taking a
| single use case through the entire system from beginning to
| end, implementing just what's necessary at each step, is a good
| way to program.
|
| I think it has benefits but you also hit on the biggest risk,
| if you aren't careful you'll end up writing spaghetti code,
| except now your spaghetti is made of steel which is way harder
| to untangle.
| vkou wrote:
| It's not spaghetti, it forces you to think about how the
| various layers are going to integrate, with a real-world
| testcase before you've written so much code that making
| corrections necessary to fix any abstraction errors you have
| made is painful.
| miceeatnicerice wrote:
| Or - it sounds like spaghetti, but ordered into a sounder
| structure
| jmull wrote:
| It's saying switchovers, when refactoring a system, can be hard,
| with a big risk of unforeseen complications.
|
| So identify as narrow a case as you can, implement that first,
| and once that's good, build out from there.
|
| That is, break down your problem into manageable chunks. Nothing
| new...
|
| the part that might not be obvious (there are many ways to break
| down a problem, after all) is the idea _to fully deliver_ a
| narrow case.
|
| Seems pretty reasonable to me.
| sb8244 wrote:
| A small tweak to the "old style" plan that I'd look at is running
| the new service in parallel with the old but not actually taking
| customer facing action. For example, send all writes to the new
| microservice when the write happens in the monolith.
|
| Pros: Gives a real work indicator of performance with very low
| risk. Data could be truncated and then backfilled before the
| final release.
|
| Cons: not always possible depending on complexity or feature.
| Requires implementing the parallel path which carries some risk
| in itself.
| TaylorAlexander wrote:
| As an ex-machinist this thread title really threw me!
| samatman wrote:
| I always wonder how these sorts of things happen.
|
| MBA: "we have a new technique, you'll love it, it's called a
| 'steel thread'"
|
| "So like, on a bolt?"
|
| "..."
|
| "Or do you mean more like,,, a wire?"
|
| "...Steel. Thread."
|
| "Excellent sir. I shall be adding the term 'steel thread' to my
| next quarterly report. Give my best to the missus."
| ochoseis wrote:
| "Marketing driven development"
| mikewarot wrote:
| Thread forming instead of thread cutting gets you some of the
| strongest results.
| tonetheman wrote:
| [dead]
| dnh44 wrote:
| I've written software like this for as long as I can remember but
| I don't remember ever learning it or being taught. It's always
| just seemed like easiest way to compartmentalise complexity.
| kdazzle wrote:
| Isnt this just the strangler pattern?
| https://martinfowler.com/bliki/StranglerFigApplication.html
|
| Not sure I agree with the steel thread metaphor
| legulere wrote:
| How is this different from the relatively well-known concept of a
| minimum viable product?
|
| > A minimum viable product (MVP) is a version of a product with
| just enough features to be usable by early customers who can then
| provide feedback for future product development.
|
| https://en.wikipedia.org/wiki/Minimum_viable_product
| chubot wrote:
| I call this a "vertical slice", rather than horizontal layers
|
| Not sure steel threads is a very good name
| black_13 wrote:
| [dead]
| PaulHoule wrote:
| There's the general agile principle that you implement complete
| features end-to-end on a regular basis. (e.g. a "user story")
|
| It's arguable, but I'd say the definition of a good software
| design is that it makes the above straightforward (e.g. testing,
| DRY, ... are means to that end)
| sampo wrote:
| The Pragmatic Programmer book, published 1999, calls this "tracer
| bullets". Topic 12 in the book.
| xmcqdpt2 wrote:
| The example in the article doesn't make sense to me. It basically
| says "You want to switch out a piece of a monolith to a new
| service, you do it with feature flags etc. That's hard." Which is
| true.
|
| But then it says "Steel Threads is better. Instead of switching
| out a part of your monolith, you... switch out a part of your
| monolith but it's a smaller part. That's not as hard". Which is
| true but isn't that the same thing? It's not clear to me what the
| qualitative difference is that requires the introduction of new
| terminology?
| TeeMassive wrote:
| The main difference between the two is that the cutter approach
| means that the two features must coexist at more or less the
| same place in the code and that the architecture must be
| adapted to accommodate this half-dead half-alive chimera.
|
| In the end you have three states two deal with with the code
| before the new feature, the chimeric code and then the code
| with the new code enabled.
|
| With the steel thread approach the feature exists on its own
| and is used and tested in production at the very beginning,
| although with limited traffic first. Important to note that the
| author seems to assume a micro-service architecture.
| prmph wrote:
| Exactly, not sure what new insight this post is supposed to
| provide. On the other hand, sometimes switching over gradually,
| one small part at a time, actually increases the complexity of
| the whole migration.
| ta988 wrote:
| Especially when you now have to sync two data stores and one
| only has a limited feature set.
| wwilim wrote:
| It provides a useful metaphor
| iandanforth wrote:
| Agree, when I'm looking to pull out a service I'm often looking
| for state boundaries. Is there some part of state or a data
| model which is separable? If so I can abstract around that and
| pull it out. If I try to pull out something smaller then I end
| up in trying to run a service with a split-brain backing
| datastore, which is far more problematic IME.
___________________________________________________________________
(page generated 2023-03-10 23:02 UTC)