Post AxjTKLMypvENPUI5jM by raito@nixos.paris
(DIR) More posts by raito@nixos.paris
(DIR) Post #AxjTKFGjWEfWUNK1A0 by agowa338@chaos.social
2025-08-31T12:20:00Z
0 likes, 0 repeats
googling for how to update the checksum in a #nixpkg is surprisingly difficult. You get more incorrect solutions than search results.Why can't you just use a normal sha256 of the exact file downloaded from the url that is literally right above it?#nix #nixos
(DIR) Post #AxjTKGQhCgES5YzUKO by raito@nixos.paris
2025-08-31T13:47:52Z
0 likes, 0 repeats
@agowa338 how do you sha256 a directory?
(DIR) Post #AxjTKHL3p7JouNMVBQ by raito@nixos.paris
2025-08-31T13:47:59Z
0 likes, 0 repeats
@agowa338 how do you sha256 the permission bits of a file?
(DIR) Post #AxjTKHzpNXvcwoR3j6 by agowa338@chaos.social
2025-08-31T13:50:20Z
0 likes, 0 repeats
@raito For both by doing it on the non-unpacked file you downloaded...
(DIR) Post #AxjTKIi8inNFAFARnM by raito@nixos.paris
2025-08-31T13:52:10Z
0 likes, 0 repeats
@agowa338 you just assumed that the file was packed, that's not always true?I get you get the idea with where I'm going, the SHA256 is not the one of the file because in the case the target is not a simple inode, it needs to have a wrapper structure to enclose its metadata and that's the responsibility of the NAR file format which is a Nix archive. For many reasons, other formats cannot be used to that purpose.The sha256 that is usually computed is the sha256 of the NAR serialization.
(DIR) Post #AxjTKJSDxSElTAjFcu by agowa338@chaos.social
2025-08-31T13:53:45Z
0 likes, 0 repeats
@raito So far I've only seen nixpkgs that:1st line url for file to be downloaded2nd url the hash in a non-default format claiming to be sha256...There is more?
(DIR) Post #AxjTKK2jlhRbIPoPXU by raito@nixos.paris
2025-08-31T13:55:27Z
0 likes, 0 repeats
@agowa338 the "non default format" is literally a W3C spec: https://www.w3.org/TR/sri/ FYI.Well, there's as many as possible usecases. :)
(DIR) Post #AxjTKKoEv5RRfk2La4 by agowa338@chaos.social
2025-08-31T14:04:46Z
0 likes, 0 repeats
@raito I didn't say "non standardised". I said "non default". As it is neither what everyone providing a sha256sum next to their download uses nor what sha256sum without additional parameters outputs.(Also it's quite hard to google for that if you don't literally already know the answer, too. And it was kinda irrelevant for making my point to differentiate if it is just another standard or custom...)
(DIR) Post #AxjTKLMypvENPUI5jM by raito@nixos.paris
2025-08-31T14:07:09Z
0 likes, 0 repeats
@agowa338 I mean sure, I never heard about sha256sum being more a default than anything else, but I'm not going to debate that. I just thought it made sense to make a remark about the fact that it is a standard.(I totally agree.)
(DIR) Post #AxjTKMCNkoLbyuL8qm by agowa338@chaos.social
2025-08-31T14:12:40Z
0 likes, 0 repeats
@raito Thanks for pointing out which standard it is. Would have been hard to find it otherwise as I said.Also it may depend on which area we're in. But for literally everything (except for S3 buckets) I've only seen the format from sha256sum so far.Like go to any download site that provides it. Like e.g.:* https://archlinux.org/download/* https://alpinelinux.org/downloads/* ...
(DIR) Post #AxjTKMcy9xcBJMmNI8 by agowa338@chaos.social
2025-08-31T12:22:26Z
0 likes, 0 repeats
The most disappointing thing was finding https://github.com/NixOS/nixpkgs/issues/191128where you at first basically get the feeling that nobody knows what the correct way is...
(DIR) Post #AxjTKMlpd0hhkqvS6a by raito@nixos.paris
2025-08-31T14:14:27Z
0 likes, 0 repeats
@agowa338 Right, I think none of these things attempts to solve the same problem as Nixpkgs though.Put it in another way, I am aware that downloading an ISO or a tarball package will get you a SHA256SUMS with a GPG signature from time to time, but it's not a scaled solution for the problem that Nixpkgs solves, therefore, I usually encounter NAR-style solutions in these areas as well.(This is true of many things in the NixOS ecosystem, FWIW.)
(DIR) Post #AxjTKNbaWa6WLN8mmG by raito@nixos.paris
2025-08-31T14:14:40Z
0 likes, 0 repeats
@agowa338 (anyway, won't take much more of your time, YMMV!)
(DIR) Post #AxjTKOG06KQkMi33lg by agowa338@chaos.social
2025-08-31T14:17:02Z
0 likes, 0 repeats
@raito Nah don't worry, I appreciate your feedback. There probably was some thought put into it I'd assume. I just fail to see it right now. For example if it can download some unpacked folder or something that would explain it. But I haven't seen any such usages within nixpkgs up until now. Only ones where it downloads single files. And well for that even people providing a sha256 on their github release pages so far appear to use the sha256sum format exclusively.Just want to understand this
(DIR) Post #AxjTKP2ZBlHKnKlqT2 by raito@nixos.paris
2025-08-31T14:18:30Z
0 likes, 0 repeats
@agowa338 Ah, right, let me explain then.Nix supports two hashing mode:- flat → sha256 of the file- recursive → sha256 of the NAR serializationThe reason for why Nix doesn't apply flat hashing to lets say GitHub release tarballs is that GitHub release tarballs or any generic tarball does not offer sha256 stability guarantees.For example, in the past, GitHub changed ALL the hashes of ALL their tarballs. Imagine the impact this would had on nixpkgs if it had used flat hashing.
(DIR) Post #AxjTKPhgisAiqs0gYy by agowa338@chaos.social
2025-08-31T14:21:44Z
0 likes, 0 repeats
@raito wtf?!? I'm kinda glad that chaos went unnoticed for me then. That sounds like a real clusterfuck. How would you even ensure that stuff afterwards is the same as before and not tampered with? (Also them doing something that changes the hash is basically already an act of tampering with the files though).Did that also apply to stuff developers uploaded within the release section themselves or just the automatic ones?
(DIR) Post #AxjTKQOw84lb10FDyS by raito@nixos.paris
2025-08-31T14:23:08Z
0 likes, 0 repeats
@agowa338 Yes, this sucks.Fortunately, with Nix, you don't care: fetchurl is allowed to use network temporarily to download anything as long as it writes only what it declared in the parameters.So you can DL the GitHub new shite, unpack it and Nix will check: yep, that's the contents I expected.So GitHub changes that DO NOT tamper the tarball went unnoticed WHILE being still verified.
(DIR) Post #AxjTKQy21aq6lqfFg0 by agowa338@chaos.social
2025-08-31T14:26:41Z
0 likes, 0 repeats
@raito One of the points on my todo list is finding a way to tell it to not. But I assume that's an advanced configuration thing, so I accepted it for now. My end goal is having it only depend on stuff I control and being able to support air-gapped environments.That way I may be able to actually use it for work stuff in the end...
(DIR) Post #AxjTKRfzOA08yBEMC0 by raito@nixos.paris
2025-08-31T14:28:30Z
0 likes, 0 repeats
@agowa338 You do not need to tell Nix "not to use the network" to use it in airgapped controlled environment.Cut the network, put a `cache.myairgapped.network.com`.Nix will download all artifacts from there only.If you want to have the "local rebuild capability", mirror all the source code too and your system will act as if it was a parallel universe.You are responsible for two things:(a) updating the sources bucket(b) updating the cache (if you want to avoid client rebuilds)
(DIR) Post #AxjTKSRUXXzzLVSIEa by agowa338@chaos.social
2025-08-31T14:33:18Z
0 likes, 0 repeats
@raito Well but then I'd also have to have setup that server first. And getting to know all the things I'd have to account for and version properly is probably more complicated.I currently imagine for a good solution having to do something with ipfs and git or otherwise it also would prevent rollbacks as I wouldn't have the older versions in e.g. 6 months from now anymore.And I'm sure there will be a lot more issues I'm not even thinking about right now to discover and work out then...
(DIR) Post #AxjTKT9Ru7A1Xq1Oka by raito@nixos.paris
2025-08-31T14:34:25Z
0 likes, 0 repeats
@agowa338 I'd strongly recommend to keep it simple, e.g. an HTTP server, you put all the files there (or a S3 bucket, you put all the files there).Nix is low technology on the caching mechanisms, it's *dumb* (a blessing and a curse).
(DIR) Post #AxjTKTb6FJHKvaxTqi by agowa338@chaos.social
2025-08-31T12:23:48Z
0 likes, 0 repeats
(Especially when you've a regular sha256sum already. It could be as simple as just c'n'p-ing but yet...)
(DIR) Post #AxjTKToDSXlpaH5xIG by agowa338@chaos.social
2025-08-31T14:38:05Z
0 likes, 0 repeats
@raito the ipfs would be backend, like a git repo. git-lfs would be another way. git-lfs would have the benefit of being able to more easily bind things to specific git commits of the repo to make rollbacks easier.For ipfs I'd have to move the git repo into it.The actual clients at the end would just get a HTTP server and download it from there. Most of this ipfs/git thing would just be for managing and keeping track of all of the files on-disk and such.
(DIR) Post #AxjTKUpfecWol4mdCS by raito@nixos.paris
2025-08-31T14:38:49Z
0 likes, 0 repeats
@agowa338 I don't know what is your operational experience of IPFS but I know that I am unhappy with it personally.Again, here, your mileage may vary.
(DIR) Post #AxjTKV0ezlJpJ9vPKS by agowa338@chaos.social
2025-08-31T14:18:31Z
0 likes, 0 repeats
@raito Also just in case we're crosstalking I'm referring to this within fetchurl:https://github.com/NixOS/nixpkgs/blob/master/pkgs/by-name/bi/biglybt/package.nix#L16
(DIR) Post #AxjTKVa6rxfv56ViaG by agowa338@chaos.social
2025-08-31T14:41:49Z
0 likes, 0 repeats
@raito Yea, but I haven't seen anything better so far. And putting GBs or TBs into git-lfs is also quite shitty...Using an S3 just adds the need for something custom to keep track of everything, where as with ipfs I would only have to make a pin for each version I'd like to keep and when I move my fork of the git repo into it as well then that pin will also contain the exact version of the git.(Tbh ipfs and the ipfs community was better before they tried to chaise the crypto hype)
(DIR) Post #AxjTKW9YkA20r361q4 by raito@nixos.paris
2025-08-31T14:42:57Z
0 likes, 0 repeats
@agowa338 That's very fair, I prefer much more https://snix.dev/ which possess the same IPFS properties but better for data storage.Obviously, it doesn't have all the discoverability, P2P, stuff. Arguably, very easy to add on the top of it with libtorrent or anything.
(DIR) Post #AxjTKWUpT52rv130zY by raito@nixos.paris
2025-08-31T14:19:17Z
0 likes, 0 repeats
@agowa338 Instead of that, in many places, it's preferred to use recursive hashing and let Nix hash the actual contents through the container tarball or anything because NAR *is* guaranteed to hash the same *ALL THE TIME*.Therefore, it doesn't matter that the container you get from Internet itself changes, the actual contents will be the same.And the hash will remain the same.
(DIR) Post #AxjTKWvPsEJRFTUFQu by agowa338@chaos.social
2025-08-31T14:46:17Z
0 likes, 0 repeats
@raito Tbh even after looking at that website I've no clue what it is. If you hadn't mentioned it in this context right now I'd have assumed it is an alternative to nix similar to https://lix.systemsEsp. with their self description as:> Snix is a modern Rust re-implementation of the components of the Nix package manager.I would never have guessed that it is something for managing file storage in general...
(DIR) Post #AxjTKXVvgTWH4iZPLU by raito@nixos.paris
2025-08-31T14:47:41Z
0 likes, 0 repeats
@agowa338 snix has something called snix-store and it has a much better story than the classical Nix store for storage
(DIR) Post #AxjTKYJugdVBZjxKFs by raito@nixos.paris
2025-08-31T14:48:00Z
0 likes, 0 repeats
@agowa338 you can mix'n'match pieces of Nix implementations to achieve a stack, here, why it's interesting even if you keep using Lix or CppNix
(DIR) Post #AxjTKYzOCQg9eNMRu4 by agowa338@chaos.social
2025-08-31T14:58:18Z
0 likes, 0 repeats
@raito good to know. But without someone telling me that these pieces even exist and fit my use case I fear that I won't even be able to find them. And in case I do find them bounce off of them because of such descriptions.The nix ecosystem really isn't that great at discoverability and elevator pitches, or is it? I thought it was just an overstatement before but it appears to become a deeper and deeper rabbit hole the more I learn about Nix.
(DIR) Post #AxjTKZc1slaTaDRJ8C by nobody@mastodon.acm.org
2025-08-31T15:06:46Z
0 likes, 0 repeats
@agowa338 @raito surprisingly effective at elevator pitches, actually, and at most other things that involve talking to humans... it's the "discovering stuff on your own" that's so incredibly hard
(DIR) Post #AxjTKbNDKovP2qWVJg by agowa338@chaos.social
2025-08-31T14:28:38Z
0 likes, 0 repeats
@raito (also that'd be a requirement to get protection from some upstream dependency getting DMCA-ed and removed without prior notice. It is quite a cluster f* when you get a call in the middle of the night for production being down because one of the dependencies got DMCA take down removed [or because the developer decided to just delete it; or the repo got defaced by an attacker])