[HN Gopher] The golden rule of software distributions
___________________________________________________________________
The golden rule of software distributions
Author : todsacerdoti
Score : 43 points
Date : 2022-05-09 15:38 UTC (7 hours ago)
(HTM) web link (www.haskellforall.com)
(TXT) w3m dump (www.haskellforall.com)
| Strilanc wrote:
| > _Stackage works by publishing a set of blessed package versions
| for all of the packages vetted by Stackage and these packages are
| guaranteed to all build together. Periodically, Stackage
| publishes an updated set of blessed package versions._
|
| All the packages being intercompatible sounds extremely valuable,
| but how is it actually done? For example, how does stackage allow
| package A to continue updating despite some never-updating
| package B requiring an old version of package A? Drop B? Don't
| let A change? Ignore B's declared dependency constraints? Take
| over development of B? Require every package to have a
| contactable maintainer who responds to "you gotta update B"
| emails or else the package gets dropped within a month?
| zokier wrote:
| I do have to point out that what Stackage is not doing
| something super novel here; it is essentially the same work
| that traditional distros like Debian etc have been doing since
| the dawn of time, especially for C libraries.
| Gabriel439 wrote:
| All of the above to some degree. I can name some specific
| examples of each of these occurring:
|
| * For an example of "Drop B", at one point the haskell-lsp
| package was deprecated in favor of the lsp package, so all
| downstream packages had to drop the haskell-lsp package as a
| dependency and migrate to the lsp package (I personally had to
| do this)
|
| * For an example of "Don't let A change", that might happen for
| some period of time, although not indefinitely. The most
| obvious example is holding back the compiler version. For
| example, Stackage was on GHC 8.10 for a while, even after GHC
| 9.2 was released, due to breakage introduced in GHC 9.0.
|
| * For an example of "Ignored B's declared dependency
| constraints", this is extremely common, especially when the
| `base` package is upgraded (since many packages have
| conservative upper bounds on their `base` dependency which can
| often be trivially bumped without issues).
|
| * For an example of "Take over development of B" the `aeson`
| package for JSON support is one example of this. More
| generally, this happens when abandoned packages get adopted by
| the Haskell organization.
|
| And Stackage does require contactable maintainers for supported
| packages. There are some exceptions to this rule, though. For
| example, sometimes a package gets added where the maintainer is
| available, but a dependency for that package was not yet on
| Stackage. I believe you can either add that maintainer to be
| the contact for that dependency, too, or it can be an orphan
| package. There are a bunch of people who fill in the
| maintenance gaps in the ecosystem by fixing these packages that
| don't have official contacts or active maintainers.
| Diggsey wrote:
| > If a package manager only permits installing or depending on
| one version of each of package then ...
|
| So just allow installing more than one version of a given
| package? Problem solved :)
|
| > A locally coherent package manager requires a globally coherent
| software distribution.
|
| Ah, so the implication is that "local incoherency" is bad? There
| are certainly some specific packages where you don't want
| multiple versions in use at the same time, but for most packages
| it's fine. For the sake of discussion, let's call those packages
| where having multiple versions is a problem "key" packages.
|
| The problem with the solution in the article is that it requires
| some third party (ie. Stackage maintainers) to do a ton of extra
| curation work, plus every package that might depend on a "key"
| package needs to do work to make sure they're using the same
| version as everyone else if they want their package to be
| included in the next curated set.
|
| An ideal solution would only require extra work for the
| maintainers of those "key" packages. This can be achieved by
| splitting the "key" package into two parts:
|
| 1) The "key" part of the package - only one of these can be in
| use, but has no public API. 2) The adapter package - depends on
| the "key" package and exposes a public API.
|
| Since the adapter package is not a "key" package, it is fine to
| have multiple versions, and since no other packages will directly
| depend on the "key" package, there's no extra work for anyone
| else.
|
| The maintainers of the "key" package will continue to release
| patches to older versions of the adapter package to ensure that
| all adapter versions are compatible with the latest version of
| the "key" package.
| froh wrote:
| How would we easily quantify the coherence of a software
| platform, pre release? So we can state some subsystem is
| completely coherent or the overall platform is slightly
| incoherent?
|
| In that terminology, Linux distributions are "mostly coherent" in
| their rolling beta release and the strive to become globally
| coherent for their release forks, right?
| dsr_ wrote:
| In an ideal universe, Debian Stable is globally coherent minus
| declared Conflicts. That is, there are packages that are
| mutually exclusive, but they are expected to perform similar
| functions such that they are reasonable alternatives for each
| other.
|
| What do we call NixOS, where different users can have
| incompatible package versions but the whole system allows for
| that? 'Tolerance' rather than 'coherence', perhaps?
| Mathnerd314 wrote:
| NixOS is globally coherent: the packages are specified using
| global identifiers like "python3", "vlc", etc. and each of
| these identifiers corresponds to a "blessed" version. But it
| is not locally coherent, because the hashes allow installing
| multiple versions of a package at once. So NixOS would be
| "locally tolerant", to use your terminology.
| gmfawcett wrote:
| "Joyfully incoherent" comes to mind. :)
| Mathnerd314 wrote:
| A rolling release is not a "pre release" but rather a
| continuous succession of releases, one for every package
| update. To the extent the rolling release has no broken
| packages it is completely globally coherent. And I don't think
| a failing package is necessarily a failure in coherence either
| - it just means the blessed version is broken. Incoherence is
| when you have A depending on C-1.0 and B depending on C-2.0 and
| the two versions of C not simultaneously installable (local
| coherence of C) - it is only the complex constraint-solving
| package managers that would be able to function with such a
| dependency tree. Such a situation is unlikely to occur in a
| rolling release - it is more likely to have two C package
| C1-1.0 and C2-2.0 which do not conflict.
| cosmic_quanta wrote:
| In the case of the Haskell ecosystem, I think that achieving this
| golden rule is hard specifically because not every package
| conforms to semantic versioning.
|
| For example, the newly released `mtl`, which is a dependency of
| _many_ other packages, breaks backwards compatibility from
| version 2.2 to 2.3. Personally, I 'd expect to be able to have a
| dependency requirement like `mtl = 2.*` and forget about it
| forever.
|
| If package versions were 'updated' to follow semantic versioning
| (e.g. `mtl` is not implicitly a version 3), we might have a
| better view of the current level of coherence of the Haskell
| ecosystem.
| Gabriel439 wrote:
| That's actually correct versioning in Haskell. The Haskell
| ecosystem uses a different versioning convention from other
| ecosystems where the first two components of the version number
| signal breaking changes. For more details, see:
| https://pvp.haskell.org/
| paskozdilar wrote:
| I don't know how controversial this will be - but I'm wondering
| whether packages which break semantic versioning should be re-
| versioned by distributions to 0.X.Y versions (perhaps with a
| X.Y.? -> 0.X.Y transformation), until they prove themselves
| willing to maintain API stability enough for an official,
| community-accepted, non-zero major version.
___________________________________________________________________
(page generated 2022-05-09 23:01 UTC)