[HN Gopher] The way we're thinking about breaking changes
___________________________________________________________________
The way we're thinking about breaking changes
Author : luu
Score : 50 points
Date : 2025-03-27 19:59 UTC (3 days ago)
(HTM) web link (welltypedwitch.bearblog.dev)
(TXT) w3m dump (welltypedwitch.bearblog.dev)
| AnthonBerg wrote:
| Hear hear!
|
| --and--
|
| From the hip: I think this is encompassed by dependent types.
|
| The stuff that just falls naturally out of dependent types
| includes tools that grasp... this. And other concerns.
|
| If I put it like so?:
|
| Software is automation.
|
| Dependent types is the automation of the automation.
| Guthur wrote:
| If one believes that migrations have solved this for databases
| one is very sorely mistaken. Sure, you can migrate the data using
| some often brittle form of SQL migration log, but this doesn't
| help if some code happens to be reading that table and expected
| the previous data shape, and this is rarely caught before the
| caller tries to access the data. This might be solved in some
| language that is some how taking full control of the data model
| such as an ORM, but that then pretty much breaks and additional
| language interoperability.
| oftenwrong wrote:
| This is a typical use case for views and stored procedures in
| RDBMSs. They can be used to provide a stable API for a given
| client even as the underlying tables change. To your point,
| however, these still do not solve the problem.
| lblume wrote:
| Good article, although the wider applications should likely be
| considered more. One small mistake seems to be defining Maybe
| (like in Haskell) as Nothing | Just but then still using Some
| (like in Rust) to initiate it.
| lolinder wrote:
| It's not perfect, but Kotlin already has a limited form of this
| with its ReplaceWith annotation [0]. It allows you to mark
| something as @Deprecated and to specify what should be used
| instead. IDEs and other tooling can pick that up and apply it
| automatically.
|
| This assumes that you're approaching the breaking change problem
| from the perspective of immutable names and deprecations: don't
| make breaking changes to existing names, create a new name for
| the new behavior and communicate that the old behavior is no
| longer going to be supported.
|
| [0] https://kotlinlang.org/api/core/kotlin-
| stdlib/kotlin/-replac...
| chromanoid wrote:
| OpenRewrite is quite nice and has a great collection of recipes
| for migrating, e.g. from javax to jakarta:
| https://docs.openrewrite.org/recipes/java/migrate
|
| There are some quirks you have to work around for big projects
| since the free tooling has some limitations.
| blixt wrote:
| I think interactions between many types and functions would be
| harder to migrate, especially if you go so granular as to target
| individual types independently. Maybe if it was a migration from
| some checkpoint (timestamp / version / commit / whatever you like
| that describes an entire state of the code) to another, then you
| could migrate indirect code patterns that need to change as well.
|
| I think most of all one must have discipline to thoroughly think
| of old code and how it should behave now. I found that one can
| already do this by versioning code and types, similar to how an
| API endpoint might go from /v2/... to /v3/.... But again, it
| requires discipline and it's not something I'd do for anything
| but very critical code.
| ivan_gammel wrote:
| This post has clickbait vibe unfortunately and is really shallow
| regarding the analysis of the problem. No, it is not silly how we
| think about breaking changes. The idea of using some change log
| isn't new and there are reasons why it's not implemented by most
| build systems.
|
| The elephant in the room is changing business requirements. If
| the change requires the code to consume new parameter without
| reasonable defaults, no automatic migration would solve it
| reliably. Let's say you deal with the money and you introduce
| currency field. The compiler of the client code cannot just
| assume the currency, neither it can take it from anywhere by just
| applying the migration. It is a manual operation to fix the
| broken code.
| TimTheTinker wrote:
| I think the idea is the kernel of something very interesting.
| TA is right - in typed languages especially, why not add some
| extra metadata what would render many changes non-breaking?
|
| I think part of our hang-up is that we tend to think of code
| and data in very separate categories (except for literals, of
| course)--and only allow treating code as data after a lexer
| starts processing it (or via reflection or first-class
| functions at run time).
|
| Correcting this category error and adding metadata (not just
| types) could yield all sorts of interesting ideas.
|
| Like - as TA suggests, why not encode the notion of _how_ a
| function signature has changed over time to enable automatic
| migrations?
| ivan_gammel wrote:
| It's not going to work. You cannot capture the meaning of the
| change with metadata.
|
| Let's say you have this change: BigDecimal
| doSomething();
|
| to Money doSomething();
|
| In theory metadata can capture the migration from a plain
| number to Money#amount() and compiler will generate a facade
| for the client code. But simply discarding the currency will
| have disastrous effects at some later point. Even relatively
| simple migration String->Number may go wrong without
| semantics. Language feature allowing such metadata and
| migrations will be a minefield.
| TimTheTinker wrote:
| Sure, there are lots of examples of lossy migrations, even
| in databases.
|
| That doesn't mean migrations in general aren't a good idea.
|
| And it doesn't invalidate the notion of metadata for code
| in general and all the possibilities that opens up.
| ivan_gammel wrote:
| When I was writing about shallow analysis, I was meaning
| that we need to see those possibilities beyond trivial
| cases. It sounds cool on the surface, but the devil is in
| details. There are lots of examples I can come up with in
| few minutes of thinking that show how this idea may not
| work. I would really like to see how this may work in a
| way where it will do more good than harm.
| AtlasBarfed wrote:
| A lot of that kernel is that dependency information is
| grossly coarse. For the vast majority of langs I ve used it
| is simply the library name: not even a library version (build
| files like gradle will).
|
| The node approach is very fine grained libraries, but that
| makes its own headaches
|
| Really every function or interface invocation has an implicit
| version number. It should be explicit in the code, and that
| assumes major minor breaking change versioning.
| bluGill wrote:
| That only works if your tools have planned ahead and so when an
| 'old call' it somehow goes to a translation system now. It also
| assumes you have good translation for the changee function and
| the performance and memory costs of it are acceptable.
|
| none of the above as always true.
| oftenwrong wrote:
| Alternatively, we could use a model in which functions are
| immutable, and therefore make such breaking changes impossible.
| This is the approach taken by Unison:
|
| https://www.unison-lang.org/docs/the-big-idea/
| naasking wrote:
| This also has issues with security fixes, unfortunately.
| lolinder wrote:
| How so? You can always deprecate the old functions and
| encourage people to use the new ones, it just doesn't
| automatically force everyone to do so immediately by breaking
| their compilation.
| jcelerier wrote:
| but that's the crux of the issue: some people think that if
| software is insecure, it is better for it to be actually
| unuseably broken. see for instance how many valid usecases
| of software that "spies" on global key input are broken by
| X11 -> wayland for the sake of better security: stuff like
| autohotkey, global recording in apps like OBS Studio,
| global key displays, yaquake-style terminals..
| ssivark wrote:
| Why is it a good model to allow some software engineer
| "at a distance" to enforce whether some downstream user
| must drop every other priority and upgrade?
|
| I agree that attitudes towards security are generally
| very poor, but breaking working infrastructure sounds
| like a crazy practice. Like any sensible system, a
| good/robust design should allow staged upgrades / hot
| reloading for anything but a very tiny core of critical
| functionality. Erlang/BEAM is a great example; it just
| requires software engineering to adopt a different
| mindset.
| naasking wrote:
| > I agree that attitudes towards security are generally
| very poor, but breaking working infrastructure sounds
| like a crazy practice.
|
| Yes, breaking infrastructure is bad. But letting already
| broken infrastructure continue can be worse.
|
| The point is that we want a better way to detect when
| breaking changes happen so that security fixes can be
| applied without breaking anything, while permitting
| optional upgrades on our own schedule for other features.
| There doesn't seem to be a great solution yet, it's
| either "it never breaks but you're possibly vulnerable to
| security issues that can't be easily patched", or "things
| can break at any time due to updates so we have to
| manually verify this doesn't happen".
| TeMPOraL wrote:
| Yeah, though 'jcelerier brought up a case where
| "insecure" behavior is a _feature_ , and the more
| "secure" design is directly incompatible with it. These
| cases of breakage can't easily be solved though better
| coding, and are not random mistakes - there's a
| fundamental incompatibility that needs to be resolved.
| steego wrote:
| I think this proposal ultimately introduces more complexity than
| it solves. By automatically inserting migrations (whether at
| compile time or via "migration files"), you end up with a form of
| hidden control flow that's arguably worse than traditional
| overloading or type coercion. In normal type coercion, the
| transformation is explicit and visible in the language syntax,
| whereas these "migrations" happen magically behind the scenes.
|
| Second, database migrations are notoriously tricky for developers
| to manage correctly. They often require significant domain
| knowledge to avoid breaking assumptions further down the line.
| Applying that paradigm directly to compiler-managed code changes
| feels like it would amplify the same problems--especially in
| languages that rely on strong type inference. The slightest
| mismatch in inferred types could ripple through a large codebase
| in ways that are far from obvious.
|
| While it's an interesting idea in theory, I think the "fix your
| old code with a macro-like script" approach just shifts
| maintenance costs elsewhere. We'd still be chasing edge cases,
| except now they're tucked away behind code-generation layers and
| elaborate type transformations. It may reduce immediate breakage,
| but at the expense of clarity and predictable behavior in the
| long run.
| lolinder wrote:
| > In normal type coercion, the transformation is explicit and
| visible in the language syntax, whereas these "migrations"
| happen magically behind the scenes.
|
| I was assuming that the migration would actually alter the call
| site in a way that is reviewable and committable, not
| implicitly do it every time it's needed at compile time. If
| that's the idea, it doesn't alter anything behind the scenes,
| it does so explicitly and visibly one time to change everything
| to meet the new requirements.
|
| The larger problem I see is that the applications for this
| would be extremely limited. There's a reason why they used a
| simple s/a/Some a/ as an illustration--once you get beyond that
| the migrations will become a pain to write, a pain to validate,
| and likely to break tons of code in subtle ways. And since most
| library code changes aren't this simple kind, it's likely that
| it's not worth the effort of building and maintaining this
| migration syntax for something with so narrow an application.
| esafak wrote:
| Products could version each interface as well as their aggregate,
| so you know if the parts you are using are breaking. It does
| introduce complexity, I know.
| tags2k wrote:
| I'm entirely unsure how database migrations aren't breaking
| changes - you migrate to a new version of your schema, queries
| that use an older schema aren't going to work. Database server
| functions can be changed through migrations too.
| wavemode wrote:
| The code equivalent of a "migration" is that you just write a new
| function that takes your new desired argument types, and then
| have the old function perform whatever type conversion is
| necessary and call the new.
|
| If writing such a function is not possible (because there is no
| automated way to convert from the old function call style to the
| new) then a database-style automated migration isn't going to be
| possible either.
| morsecodist wrote:
| I am a bit confused by this. Typically when you make a breaking
| change it is because the call site has to make a new sort of
| decision so you may not want all calls to be refactored the same
| way and there is no way of determining which you want at runtime.
|
| The example the author uses is modifying a function to return an
| int or null instead of an int. Let's say you implemented the
| function a bit naively and in the null case your function would
| crash the program. Now you are going to refactor your codebase so
| the caller gets to decide what happens in the null case. Some
| callers may be unable to handle the situation and will need to
| implement the crash/exception some may use some sort of fallback
| behavior.
|
| I think haskell's type system probably would probably prevent
| someone from having the issue I described above but the problem
| still stands. Let's say you had a function that returns a union
| type and you have areas in the code that are supposed to handle
| every case of that type. If you had a case you need to handle it
| everywhere and the compiler can't know how you want to handle it.
| A lot of type systems will catch the missing case which is
| awesome but you still need to handle each one.
| immibis wrote:
| If the null case crashed the program before, and still crashes
| the program after but at the call site instead, then
| compatibility has not been sacrificed. It still crashes in the
| null case, but now it runs at all in the non-null case, whereas
| if you don't have this migration feature, your "fix" made it
| crash in both cases.
| kmeisthax wrote:
| The way you handle breaking changes in SQL is _views_ , not
| migrations; the latter is only appropriate single applications
| abusing SQL as a data store[0].
|
| In code, if you want to handle a breaking change transparently,
| you don't need the moral equivalent of a linker fix-up table. You
| need to keep the old definition around and redirect it to the new
| function. This can be as simple as having, say, the old version
| of your code wrap something in a list or closure and then call
| the new version. In languages with overloading, this can be done
| transparently; in others you'd have to give the new function a
| new name. Maybe API versioned symbols and a syntax for them is
| what you would want?
|
| [0] The original idea with SQL is that multiple applications
| would store data in the same place in a common format you could
| meaningfully interchange.
| samus wrote:
| Databases have empathically _not_ solved this problem. I have
| never heard of an DBS with first-class support of migrations;
| they are usually layered on top using something like Flyway or
| Liquibase. A query still working after a migration is still
| either an accident or due to a deliberate backwards compatibility
| effort.
|
| The proposal essentially expects the dependency to provide a
| migration path. However, to the extent described this is only
| necessary in languages with no overloading. And it doesn't help
| for cases when the change is _not_ as simple as providing a few
| rewrite rules or things like that.
|
| Also, the jab about static type systems is misplaced. Their
| purpose is specifically to reject certain invalid programs, at
| the cost of rejecting a few valid. You can't have one without the
| other. With dynamic typing, backwards compatibility breakages fly
| under the radar until they gleefully blow up in production.
| AndrewDucker wrote:
| In order to do this the compiler needs to know what the previous
| compilation did so that it can apply these changes purely to
| things that changed between then and now. Which doesn't seem
| impossible, but does seem like an interesting problem to solve.
___________________________________________________________________
(page generated 2025-03-30 23:01 UTC)