Post AoP5hm4RQLDOv2zhmS by kirschwipfel@nerdculture.de
(DIR) More posts by kirschwipfel@nerdculture.de
(DIR) Post #AoKe9PPsNewm8n4dUm by civodul@toot.aquilenet.fr
2024-11-22T16:56:03Z
0 likes, 0 repeats
Dear #Nix contributors,What do you think is key to making Nixpkgs the most up-to-date distro?
(DIR) Post #AoKeFQGC9jpiUmPfgu by picnoir@social.alternativebit.fr
2024-11-23T10:52:56.288031Z
0 likes, 0 repeats
@civodul all of the above plus the adoption of FOD-based dependencies fetchers for the Rust and Go toolchains. (wrote this in the survey a few days ago, I sound like a broken record!)It's definitely a tradeoff, patching a Rust or Go library for the whole packageset is definitly not an option for Nixpkgs. At least not a realistic one at the moment. But that's a tradeoff I personally think makes sense. The very little experience of packaging a not so complicated Rust package on Guix has been horrendeous. The importer is brittle, it generates a lot of code that you have to manually massage into the tree.This plus my very limited experience with email-based workflows has been the real show-stopper for me during my latest attempt to use Guix more.--------------------------------I also feel like we need a Guix counterpart of nixpkgs-review for reviewers. This is a really important tool for me, and last time I checked (granted a few months ago), there were no credible option in the Guix land.https://github.com/Mic92/nixpkgs-review
(DIR) Post #AoKeLEnHhUCufefBFQ by PuercoPop@mastodon.social
2024-11-23T00:48:02Z
0 likes, 0 repeats
@civodul And I doubt that I have the most exposure to #nixpkgs and #guix processes. But these are my observations:
(DIR) Post #AoKeLFmG2mypilBsHo by PuercoPop@mastodon.social
2024-11-23T00:48:29Z
0 likes, 0 repeats
@civodul # Activation Energy of mergingTo merge a change in nixpkgs one needs to press a button. For guix one needs to download the patch locally, rebuild to double check and then push. The extra steps can be done by a CI, it doesn't require a pull request flow.
(DIR) Post #AoKeLG3yyt9sbjU1ui by PuercoPop@mastodon.social
2024-11-23T00:48:46Z
0 likes, 0 repeats
@civodul # Merging _your own_ changes requires you to look at other people submissionsThis is a consequence of integration being the responsibility of the CI. #nixpkgs maintainers have to go through the same process as non-maintainers. They can't push to master. If #guix maintainers had to visit https://issues.guix.gnu.org/ my guess is that there would be more eyes on other people submissions.
(DIR) Post #AoKeLGqY4K0T2MCoc4 by PuercoPop@mastodon.social
2024-11-23T00:49:00Z
0 likes, 0 repeats
@civodul # Automated 'first response' triaging of a submissionThis includes adding a reviewer automatically. You don't even have to have acommit bit to be able to approve the change. This is important because it letsthe submitter know that who is they need to reach out in case their change isforgotten. It also lets maintainers check their 'backlog' of submissions.https://github.com/NixOS/nixpkgs/blob/7eb0c197fbdcedf2e29487b467fc743a1a4e0f0a/maintainers/README.md#how-to-become-a-maintainerf/e #guix issues could run guix lint automatically on each submission.
(DIR) Post #AoKeLHUFghlX1UmWUy by civodul@toot.aquilenet.fr
2024-11-23T10:31:22Z
0 likes, 0 repeats
@PuercoPop Teams in Guix are an attempt at addressing these very problems (we purposefully chose not to have per-package maintainers back in the day).https://guix.gnu.org/manual/devel/en/html_node/Teams.htmlSomeone who submits a patch automatically has members of the relevant team(s) Cc’d.But again, tooling is not making it as easy as it could, especially for team members.
(DIR) Post #AoKeLIy4BLCzcFjqbo by PuercoPop@mastodon.social
2024-11-23T00:49:22Z
0 likes, 0 repeats
@civodul# More trust/lower requirements to get commit accessThis is again not directly related to pull request flow, but one can give ownership to parts of the tree in most Forges. (ej. using CodeOwners file). I'm not suggesting that #guix adopt a CodeOwners file.
(DIR) Post #AoKeLLZ0WxwjfP2NSy by PuercoPop@mastodon.social
2024-11-23T00:49:47Z
0 likes, 0 repeats
@civodul # Neglecting triaging drives away potential contributorsOne might be less motivated to look at other people submissions than to work on your own patch. After all when one starts to write a patch they already have a goal they want to accomplish. Other people's submissions aren't guaranteed to align with your interests. But triaging grows the developer base.
(DIR) Post #AoKeLOSjkjiGep7ubo by PuercoPop@mastodon.social
2024-11-23T00:50:44Z
0 likes, 0 repeats
@civodul # documentation for _developers_ not usersnixpkgs have markdown files sprinkled in different parts of the codebase that give an overview of how things work. #guix does a good job at _user_ oriented documentation. But less so for developer oriented documentation. For example, an overview of how does each build-system works. Or what does the bootstrap process for node looks like. This would give newcomers necessary context on the work needed to be down to accomplish their goal.
(DIR) Post #AoKeLQZtt4dDDcUf3I by PuercoPop@mastodon.social
2024-11-23T00:51:38Z
0 likes, 0 repeats
@civodul Don't know if this is too cynic a take but my guess is that if there would be automatic pull requests for package updates for #guix the most likely out would be more open issues that are not acted on. Using my own submissions as a reference the average time to acknowledge a submission in #guix is around a month. There are a couple where the turnaround was one day. But those were outliers.
(DIR) Post #AoKeLT1GoHiGn5JXzU by PuercoPop@mastodon.social
2024-11-23T00:54:51Z
0 likes, 0 repeats
@civodul A lot of the things I wrote above could be addresses by improving cuirass. I've been meaning to look into it in more detail. And afaict nixpkgs spends way more $$$$ resources to their CI infrastructure.
(DIR) Post #AoO5QReuZoQQFtKmaO by nobody@mastodon.acm.org
2024-11-25T02:41:41Z
0 likes, 0 repeats
@civodul I'm just mixing other people's replies with various weights, but I'd say it's: automation, made possible by the store model and integration tests; the other key ingredients arguably have been opportunism and borderline cheating (aka pragmatism, scoping, and goal-setting...): upstream lockfiles, codegen, sometimes wheels from PyPi as temporary substitutes in the middle of an otherwise source-provisioned closure.
(DIR) Post #AoO6obtqFkUUv9P8Vs by nobody@mastodon.acm.org
2024-11-25T02:57:15Z
0 likes, 0 repeats
@civodul if we weren't comparing to Guix, I'd include a mandatory section on how the immutable deployment approach is so much more superior in terms of distributed and local decision making (as long as you still have tests) but in this thread it's a given
(DIR) Post #AoO7sthY6mkRFDGLxo by nobody@mastodon.acm.org
2024-11-25T03:09:14Z
0 likes, 0 repeats
@civodul Actually, scratch the first reply. I reckon "distributed and local decision-making" was the key, and everything else (automation, immutability, tests, monorepo) is what made that possible up until now. All of that is subject to change though
(DIR) Post #AoO8p6hfvQFmMi4uiO by nobody@mastodon.acm.org
2024-11-25T03:19:45Z
0 likes, 0 repeats
@civodul (propaganda mode on) a distributed community that empowers individuals is what makes contributing easy, and the tooling is what helps us not to break each other's stuff unknowingly too often and to reuse each other's work.Community: distributed and local.Computation: centralized and global.
(DIR) Post #AoO92l7SyFYvTgeSMy by nobody@mastodon.acm.org
2024-11-25T03:22:13Z
0 likes, 0 repeats
@civodul @civodul also about the tools, they do the job but they're not "great". Nixpkgs-review was mentioned: it's indispensable for certain kinds of updates, but when you touch pytorch its reports become a waste (of compute time). After eight hours of rebuilds you just get a 1-2k lines of noise which is no input for decision making. This could be fixed by integrating the tool with hydra's history, but point is the tools is overfitted to the common-enough use-case
(DIR) Post #AoO9IRk2mhxy0oAzRo by nobody@mastodon.acm.org
2024-11-25T03:25:03Z
0 likes, 0 repeats
@civodul Ofborg... ofborg isn't great. But as long as all of Nixpkgs' attributes are evaluated for all PRs we're, yet again, comparatively safe about breaking the stuff we may not even know other contributors have put in (which we still do)
(DIR) Post #AoO9TQ31QHBtwaHkR6 by nobody@mastodon.acm.org
2024-11-25T03:27:03Z
0 likes, 0 repeats
@civodul RE: pull requestsgithub is probably more of a hindrance than help, but it is a local optimum
(DIR) Post #AoOAql8dH6UxyAbRFw by nobody@mastodon.acm.org
2024-11-25T03:42:28Z
0 likes, 0 repeats
@civodul (tired rambling mode on) Honestly there's just so much stuff missing that could improve out QoL immensely... we don't test stuff outside NixOS (which has bad consequences e.g. for GL, ROCm and CUDA), we don't test on hardware other than CPUs (which... well, GL and CUDA), we don't test mixing Nixpkgs revisions...
(DIR) Post #AoOAyFpDa2GPAc603M by nobody@mastodon.acm.org
2024-11-25T03:43:49Z
0 likes, 0 repeats
@civodul And right now there isn't (yet) any infrastructure that would allow to implement these checks externally and integrate them into the existing PR-blocking and channel-blocking mechanisms
(DIR) Post #AoP4CPvRbotFJ2GnBI by civodul@toot.aquilenet.fr
2024-11-23T13:22:19Z
0 likes, 0 repeats
@picnoir Could you explain a bit the FOD-based dependencies fetchers (or send a link)? (I always read “FOD” is “FUD”; for those like me, we’re talking about fixed-output derivations. :-))‘nixpkgs-review’ looks pretty cool indeed; the recently-added ‘mumi am’ helps, but it only does a tiny part of the job.How does ‘nixpkgs-review’ figure out which packages (if any) need to be rebuilt to validate a PR?
(DIR) Post #AoP4IZs7XdD1tLeX2G by PuercoPop@mastodon.social
2024-11-23T14:31:30Z
0 likes, 0 repeats
@civodul @picnoir FOD stands for fixed output derivations. IIUC what PicNoir is suggesting that guix should use the lockfile of the Go/Rust package to define an intermediate package that contains all the dependencies for a package. This leads to less conflicts trying to find common versions that work for multiple crates at the expense of more storage requirements and longer build times
(DIR) Post #AoP5hm4RQLDOv2zhmS by kirschwipfel@nerdculture.de
2024-11-24T21:33:35Z
0 likes, 0 repeats
It also leads to package inflation. Esp. if lock files of included crates pick up different, yet sem-ver-same versions of another crate.Also build times for Rust packages are HUGE already since in rust there is no such thing like a static library. This means, every crate is build from source when used by another crate.@PuercoPop @civodul @picnoir
(DIR) Post #AoPCaduy2LGrHm6qPo by civodul@toot.aquilenet.fr
2024-11-25T15:30:20Z
0 likes, 0 repeats
@picnoir Thanks for explaining!The changed path computation is similar to what the Guix Data Service does, and it’s indeed expensive (and the main bottleneck for qa.guix right now).
(DIR) Post #AoPTfGWE9xeLNqKMm8 by octorine@fosstodon.org
2024-11-25T18:23:04Z
0 likes, 0 repeats
@PuercoPop @civodul @picnoir If I'm understanding this correctly, the dependencies would only exist in the FOD and wouldn't need to be explicitly packaged and included in nixpkgs, right?
(DIR) Post #AoPh8sf4iw8NU0uyJs by civodul@toot.aquilenet.fr
2024-11-25T21:05:50Z
0 likes, 0 repeats
@picnoir It’s a FOD whose output hash cannot be known in advance too, so I suppose one just builds it, looks at the hash mismatch error, and feeds the “expected” hash back in the package definition?@PuercoPop @octorine
(DIR) Post #AoQaNowuIsfvnAOa80 by Parnikkapore@mastodon.social
2024-11-26T00:13:01Z
0 likes, 0 repeats
@picnoir @octorine @PuercoPop @civodul TIL that the lockfile has to live in nixpkgs. I thought it's computed on-demand from the source. (Guix actually can't do this because the list of inputs is used quite often, even though it's also computed-on-demand)
(DIR) Post #AoQaPifGdkg8sd3A12 by PuercoPop@mastodon.social
2024-11-26T03:14:39Z
0 likes, 0 repeats
@picnoir @octorine @civodul One thing to add is that it is trade-off that might make sense in the context of distributing software to end users, but not for development. The amount of rust desktop _applications_ is probably around ~100.
(DIR) Post #AoQaPkHEe4dxsfozpo by PuercoPop@mastodon.social
2024-11-26T03:17:15Z
0 likes, 0 repeats
@picnoir @octorine @civodul For development the nix ecosystem has other alternatives like https://github.com/ipetkov/crane or https://github.com/oxalica/nocargo. I'm mostly familiar with crane, that although uses the cargo.lock as the source of truth for dependencies it stores each crate separately in the store so that changing one dependency doesn't require a full recompilation of dependencies.
(DIR) Post #AoQaPmXGFSeQsxKp5k by PuercoPop@mastodon.social
2024-11-26T03:21:40Z
0 likes, 0 repeats
@picnoir @octorine @civodul That said, it looks there is a push away from the current approach https://github.com/NixOS/nixpkgs/issues/333702