[HN Gopher] Layered Design in Go
___________________________________________________________________
Layered Design in Go
Author : misonic
Score : 117 points
Date : 2025-04-20 01:58 UTC (21 hours ago)
(HTM) web link (jerf.org)
(TXT) w3m dump (jerf.org)
| __s wrote:
| Kind of reminds me of the concept of spheres in randomizers
| breadchris wrote:
| great blog post! also this website has a lot of incredible posts,
| if you like learning about functional programming, you should
| check out
|
| https://jerf.org/iri/blogbooks/functional-programming-lesson...
| rapidlua wrote:
| > Packages may not circularly reference each other.
|
| Actually possible with go:linkname.
| jerf wrote:
| Fair, but I think we can classify that under "unsafe" and
| ignore it under normal circumstances. I can also say things
| like "Go doesn't have pointer arithmetic" with a straight face,
| even though unsafe permits pointer arithmetic just fine. If
| you're programming with that routinely, you're out of the
| bounds of my advice for architecture anyhow. Whether for good
| or bad reasons would be left as an exercise for the architect
| in question.
| ncruces wrote:
| Looking at the diagram for the SQLite VFS page I didn't think
| I was going overboard with designing my driver around 3
| packages: https://sqlite.org/vfs.html
|
| One layer wraps SQLite C API, below it lives a pure Go VFS,
| and above all that database/sql driver.
|
| Even this coarse split (with a collection internal packages,
| the bigger one of them "utils") is enough to need various
| band-aids to accommodate the impossibility of circular
| dependencies.
|
| I honestly don't think it helps much.
|
| At the module level, there are obvious benefits from the
| impossibility. Just like all the pain around v2 modules can
| be justified, even if I find it annoying.
|
| When packages are _also_ the only layer at which you can
| enforce visibility, it becomes worse.
| pjmlp wrote:
| Looks like I am reading a book about Yourdon structured method.
| jerf wrote:
| Since I meant this just as a "how I do it" post I suppose I
| forgot the disclaimer that I'm not particularly claiming to
| have invented anything or to be the first. Indeed to a large
| degree I consider myself just to be following the grain of Go
| and hardly doing anything myself.
|
| That said, after some quick googling around, I don't think I
| feel bad not knowing what Yourdon design is, as it seems to be
| somewhat proprietary and behind paywalls, so it's hard for me
| to tell if there's much similarity. Certainly it has a lot of
| stuff I tend to eschew; lots of references to diagrams and
| state charts and such. I tend to prefer a more "agile but wait
| before you panic I mean 'original' agile not 'scrum' or
| whatever other abomination it was turned into", my formal
| method is more based around exploring the design space with
| extensive unit tests and code rather than that sort of up-front
| design.
| pjmlp wrote:
| It was a common way to structure enterprise C code during the
| 1990's, and the snarky remark is how the anti-enterprise
| culture from Go ends up adopting the same big corporation
| principles, given enough wind behind its sails.
|
| Yourdon is the big wave of enterprise methodologies
| immediately predating the OOP wave with Booch, UML, GoF and
| friends.
|
| I can gladly bet there are some Go pattern books around the
| corner as well.
|
| As for the book paywall, it is certainly available in many
| libraries, given its age.
| jerf wrote:
| Nobody has accused my code bases of being "too enterprise"
| yet.
|
| Edit: I should probably elaborate on that before my edit
| window closes. Other than pervasive use of dependency
| injection, done directly with no framework simply by
| passing values around, there are effectively no
| "Enterprise" structures in sight in my code base. That's
| what I mean by "this design is sufficient for me". The only
| thing that resembles a "factory" is in the precise place I
| need to construct values from a type specified by an input
| string. No patterns put in place "just in case". No top-
| level frameworks used for 3% of their functionality. I use
| a process monitor but the interface that requires is
| "Serve(context.Context)", which is just the minimum you
| need to be able to monitor a service.
|
| There are what I'd call "patterns", but they're there to do
| their job to the full, not guesses about what maybe I'll
| need later.
|
| I've actually got a half-written pattern book for Go I've
| been trying to figure out what to do with, but the
| introduction is basically "why pattern books shouldn't just
| be a recitation of the original GoF patterns", because that
| is, well, stupid. Even the original book bit off too much
| trying to straddle Smalltalk and C++ in one shot. They
| require different patterns. My pattern book is how to solve
| Go problems in Go, not to give people words to slap in
| their code in case someday their code _might_ grow enough
| to need it.
| kubb wrote:
| Cool description of how jerf thinks about packages and how he
| deals with circular dependencies!
| nickcw wrote:
| In my opinion, not allowing circular dependencies is a great
| design choice for building large programs. It forces you to
| separate your concerns properly.
|
| If you get a circular dependency something is wrong with your
| design and the article does a good job on how to fix them.
|
| I sometimes use function pointers which other packages override
| to fix circular dependencies which I don't think was mentioned in
| the article.
|
| My only wish is that the go compiler gave more helpful output
| when you make a circular dependency. Currently it gives a list of
| all the packages involved in the loop which can be quite long,
| though generally it is the last thing you changed which caused
| the problem.
| ternaryoperator wrote:
| In the abstract, I think I agree with you. But in reality, what
| I see is that go projects use far fewer packages than, say,
| programs in Java. Many go projects use one or two omnibus
| packages--principally, I expect, to avoid having to worry about
| circularity issues.
|
| By forcing this design pattern on developers (something no
| other language does), I think the result has been overall worse
| code rather than better.
|
| Perhaps a warning, rather than stop-the-compiler error would
| have been a better choice. Not sure.
|
| Either way though, I wholly agree that the compiler gives too
| little information, which is curious because it knows the
| needed data and should easily be able to present it in a useful
| way.
| jen20 wrote:
| I see the opposite in most Java programs: poor organization
| with packages based on a type of thing (eg models,
| controllers) rather than related behaviors.
|
| Go doesn't have warnings, which is great - if something is
| worth warning about, it is also worth erroring about. It
| never ceases to amaze me when a brand new JavaScript project
| spits out dozens of errors after pulling in a common library,
| and everyone thinks that is ok.
| ncruces wrote:
| Of course Go has warnings.
|
| It has go vet, which is not a linter, and according to the
| authors doesn't need comments to ignore checks, because the
| checks are always correct about you having written shitty
| code. Except where it might warn you about something
| completely outside your control.
| wavemode wrote:
| > If you get a circular dependency something is wrong with your
| design
|
| Packages not being able import from each other circularly is
| purely a compiler limitation. It says nothing about the
| realities of software development.
|
| This notion stems from the idea that software design is
| inherently hierarchical, and that there is always a clear
| "higher level" and "lower level" between every possible
| software module.
|
| What I've found in practice is that this is a fictional
| concept. Circularity between modules is very common and natural
| (especially as business requirements change over time). The
| workarounds people invent to avoid circularity literally always
| result in a codebase that is harder to understand and maintain,
| rather than easier.
|
| > It forces you to separate your concerns properly.
|
| Nah. It's not separation of concerns, it's separation of
| implementation. Two functions that in every other way shape and
| form deal directly with the same concepts, end up needing to be
| in separate modules purely because they differ in the
| functionality they import. And if later their imports change,
| they may need to be moved again. Which means implementation
| details are leaking into your design, which makes code less
| discoverable (since you now need to know implementation details
| in order to reasonably predict where a given function might be
| defined).
| politician wrote:
| I prefer extremely fast compile times.
| LtWorf wrote:
| It's not fast if it fails every time you commented a line
| because a variable becomes unused though.
| homebrewer wrote:
| Other languages (Pascal, Ocaml, maybe Zig) have proven that
| it's possible to implement a very fast compiler that emits
| efficient machine code while not dumbing down the language
| to complete brain death.
| 9rx wrote:
| _> Circularity between modules is very common and natural_
|
| In theory, but in practice it isn't because there are very
| few language that do not see software design as being
| hierarchical. That stems primarily from most languages being
| based on a hierarchical filesystem, which imposes a
| hierarchical view of the world at the very core. Circular
| references in languages that are hierarchical end up being
| very awkward.
|
| There are a small handful of languages that reject all things
| hierarchical, including the filesystem, but they are few and
| far between and most probably have never heard of them and
| they certainly aren't what you are going to find in
| production. For better or worse, we've settled on a
| hierarchical model.
| wavemode wrote:
| > in practice it isn't because there are very few language
| that do not see software design as being hierarchical
|
| It has been possible since the days of C (via forward
| declarations / header files) for two compilation units to
| call functionality in each other circularly. Java and many
| other languages have followed suit. I don't buy the
| argument that it is some sort of new or esoteric thing for
| a compiler to allow this.
| 9rx wrote:
| Is there a reason you decided to reply after only reading
| the first sentence? This off-topic straw man you have
| imaged doesn't exist.
| wavemode wrote:
| You're going to have to elaborate in words what your
| actual problem with my comment is. It's not clear to me
| why you believe I only read the first sentence, nor in
| what way I am addressing a strawman. It is perhaps you
| who have misunderstood my point, rather than the other
| way around. But, again, you've provided not enough
| details for me to ascertain this.
| 9rx wrote:
| Your point is understood, and nobody would disagree with
| it, but your point is towards a straw man. Nobody ever in
| the history of computing has made this "argument" you
| have imagined. If you honestly believe that you didn't
| make it up arbitrarily, where did you get it from?
| wavemode wrote:
| The argument I mentioned:
|
| > I don't buy the argument that it is some sort of new or
| esoteric thing for a compiler to allow this
|
| Where I got it from:
|
| > there are very few language that do not see software
| design as being hierarchical
|
| > Circular references in languages that are hierarchical
| end up being very awkward.
|
| > There are a small handful of languages that reject all
| things hierarchical, including the filesystem, but they
| are few and far between and most probably have never
| heard of them and they certainly aren't what you are
| going to find in production
|
| My point being that it is not "awkward" at all for a
| language to allow circular references between modules. It
| is only awkward in languages that make it awkward (like
| Go does). In C (a language over 50 years old) and Java
| (one of the most widely used languages in the world) it's
| the most straightforward and natural thing in the world.
| So it's certainly not correct to say that it is "awkward"
| in all languages except ones nobody has ever heard of
| (aka "esoteric", the term I used in my comment).
|
| So yeah I'm still not fully following how I failed to
| address your comment, or attacked a strawman. Based on
| what you're saying now, it's clear that you feel I have
| misunderstood the point you were trying to make. But I
| can say that it certainly was not based on a single
| sentence, or a lack of reading on my part. Probably just
| a confusion of terminology.
|
| So, what did you actually mean?
| 9rx wrote:
| _> Where I got it from:_
|
| Which must mean that you only read the first sentence and
| dreamt up the rest, or didn't read it at all, else you'd
| know there was nothing said about circular imports being
| esoteric. In fact, the comment you originally replied to
| wouldn't work if they were esoteric.
|
| _> It is only awkward in languages that make it awkward
| (like Go does)._
|
| How could it be awkward in Go? It doesn't support
| circular references. It cannot be awkward - it isn't
| possible at all! This was already told in earlier
| comments, so how did you manage to get here not knowing
| that other than by not reading the comments?
| wavemode wrote:
| > there was nothing said about circular imports being
| esoteric
|
| If you aren't saying that circular imports are not common
| and natural, then don't reply to the statement
| "circularity between modules is very common and natural"
| with the statement "In theory, but in practice it isn't"
|
| I have no quarrel with you. If there has been a
| misunderstanding, then simply correct it and elaborate.
| That could have been a 2-second interaction - I would
| have replied "oh, my mistake, I misunderstood what you
| were saying" and this conversation would be over.
|
| On the other hand, if your goal is just to look smart or
| superior, or make me look dumb because you believe I'm
| incapable of reading, then feel free. I have better
| things to do with my time.
|
| I won't be replying further.
|
| > How could it be awkward in Go? It doesn't support
| circular references. It cannot be awkward - it isn't
| possible at all!
|
| There are workarounds. That was the whole point of the
| beginning of this entire comment chain - me pointing out
| that these workarounds are worse quality code than if the
| compiler just supported doing things in a more
| straightforward way.
| layer8 wrote:
| [The following is intended as language-agnostic.]
|
| It's useful to distinguish between interface and implementation
| dependencies. I agree that there shouldn't be circular
| interface dependencies between modules. The absence of circular
| interface dependencies allows separate compilation of modules.
| It also means that at least in principle, the implementations
| can be made non-circular (can be refactored to non-circular
| without breaking any of the existing interfaces). But it's
| often okay for the implementation of A to depend on the
| interface of B, and at the same time the implementation of B to
| depend on the interface of A, as long as there is no mutual
| dependency between the interfaces of A and B.
| euroderf wrote:
| > In my opinion, not allowing circular dependencies is a great
| design choice for building large programs.
|
| I have a hobby project with maybe 20 packages involved.
| Circular dependencies were getting harder and harder to solve.
|
| What worked in this situation was to separate the whole
| hairball into two layers. The packages in the app layer could
| import packages from the utilities layer, but not vice-versa.
|
| This introduced enough structure to simplify the removal of
| circularities and prevent new outbreaks.
| ncruces wrote:
| But, don't you know you can't have a utils package either?
|
| A package must implement one functionality, and it must be
| clear from the short import name what that functionality is.
|
| A package is also the only way to enforce visibility (and
| often, with that, other useful properties like immutability).
|
| So packages can't be too big, or too generic, but they can't
| also be too small, or too specific.
|
| I also have 2 (or 3?) layers, _and_ a utils package, _and_
| the packages are huge, _and_ yet I need various cheats to
| allow cyclic dependencies, _and_ have some repetition with
| _other_ tricks to avoid some of it.
|
| It's the real world, and guess what, the very well designed
| standard library does... all of the above too.
| shizcakes wrote:
| One bonus technique related to the "move to a third package"
| advice: generating many of your model structures (SQL, Protobuf,
| graphql, etc) allows you to set up obvious directionality between
| generated layers and to provide all generated code as "base
| packages" to your application code, which then composes
| everything together.
|
| Prior to this technique we often had "models importing models
| circularly" as an issue but that's entirely disappeared due to
| the introduction of the structural additional layer.
| darioush wrote:
| A funny quirk about golang is you cannot have circular
| dependencies at the package level, but you can have circular
| dependencies in go.mod
|
| The tl;dr is don't do that either.
___________________________________________________________________
(page generated 2025-04-20 23:01 UTC)