Post 9irMShEMRM3iHspqKm by jazzyeagle@mastodon.social
(DIR) More posts by jazzyeagle@mastodon.social
(DIR) Post #9iqtI8d7KgaFPP1njk by fribbledom@mastodon.social
2019-05-14T17:24:25Z
0 likes, 0 repeats
Thread:We need to talk about packaging, signatures, checksums and reproducible builds:On your system you have a keyring of packagers' GPG keys that you inherently trust.Releases get signed with a key, which verifies the packager as the author, and supposedly lets you and your system trust their contents.But do you really trust your packagers? How could you? Do you know them personally and monitor their packaging work?Would you even know if they release a package with malicious content?
(DIR) Post #9iqtI8nOiSo5vHq0lE by murks@social.tchncs.de
2019-05-14T18:45:00Z
0 likes, 0 repeats
@fribbledom Sure it would be nice to have, but even then you are only marginally safer from malicious intent. Can you trust all the code in your dependencies? Are you able to check it all yourself? Probably not. At some point it comes down to trusting someone else, no matter how you turn it.Is it good if you can reduce the number of people you need to trust? Yes, but don't expect to be able to get that number down to zero.
(DIR) Post #9iqtI8yk2HsgUT94RU by clacke@libranet.de
2019-05-15T05:31:50Z
0 likes, 0 repeats
@murks @fribbledom Without reproducible builds, reading the source code isn't even entirely useful.Reproducible builds is step zero in being able to verify your system from source.
(DIR) Post #9iqtI9KMjtB7ZXGL9E by murks@social.tchncs.de
2019-05-15T06:52:39Z
0 likes, 0 repeats
@clacke @fribbledom Provided you are not compiling yourself, right? If you read all source you depend on, including compiler etc. and assuming you can trust the hardware and build the software yourself then you could trust it.
(DIR) Post #9iqtI9aJmZwGN0j50q by fribbledom@mastodon.social
2019-05-15T07:01:22Z
1 likes, 0 repeats
@murks No, that's a collective effort. You don't have to do this alone. Trust the public consensus, not a single entity.
(DIR) Post #9iqteP1z2CRie9H95U by edheil@dice.camp
2019-05-14T19:13:01Z
0 likes, 0 repeats
@fribbledom my lack of familiarity with compiled languages led me to assume for many years that the same sources always produced the same binaries. It seems weird that they don't and an obvious source of possible evildoing.
(DIR) Post #9iqtePBYSc6P7pkn0S by clacke@libranet.de
2019-05-15T08:04:05Z
0 likes, 0 repeats
@edheil @fribbledom Indeed. But there are ways.www.schneier.com/blog/archives…
(DIR) Post #9iqvZGham1GBH8PvW4 by clacke@libranet.de
2019-05-15T08:28:11Z
0 likes, 0 repeats
@edheil @fribbledom The NetBSD post about their 100% reproducible CD-ROM has a long list of the perfectly normal, non-malicious, changes the can occur between people compiling the same source code:blog.netbsd.org/tnf/entry/netb…It includes things like what order files happen to end up on disk, what order your object files are built (which changed depending on how many fires you have, what performance your disk and CPU have, etc).
(DIR) Post #9iqvjnQpwKrHgGbGYi by clacke@libranet.de
2019-05-15T08:31:28Z
0 likes, 0 repeats
changes* depending on how many cores* 😁
(DIR) Post #9iqw36ufjWxbbQlUkS by clacke@libranet.de
2019-05-15T08:30:30Z
0 likes, 0 repeats
@edheil @fribbledom Compilers are not unique in this, it converns every form of data processing. For any non-trivial process, if you don't make deliberate efforts toward reproducibility you won't have it.I just fixed a bug in racket2nix where a dictionary I dump has different ordering on macOS and Linux, and potentially between minor versions of Racket, so I have to sort it by key when exporting it.
(DIR) Post #9iqyoHucr7yxEdMod6 by fribbledom@mastodon.social
2019-05-14T17:25:01Z
0 likes, 0 repeats
We need a system that lets us reproduce a packager's work and confirm that whatever they release was indeed built from a specific source tree without any unintended or even malicious changes.To achieve that we require reproducible builds, so we can correlate a build with its source tree(s) and dependencies.Whenever a new package gets released, this would allow independent systems and entities to verify that its contents really match the expectations.
(DIR) Post #9iqyoICLnEA07beyG0 by Xjs@chaos.social
2019-05-15T10:27:04Z
0 likes, 0 repeats
@fribbledom Did somebody call Bazel?
(DIR) Post #9iqyoIU4jKL30Zx7su by clacke@libranet.de
2019-05-15T15:48:19Z
0 likes, 0 repeats
@Xjs @fribbledom Bazel/Pants/Buck are part Make, part lightweight/partial Nix/Guix, but they don't do anything about bit-for-bit reproducibility as far as I'm aware.
(DIR) Post #9iqyoIg80Vynbxakfg by Xjs@chaos.social
2019-05-15T16:08:56Z
0 likes, 0 repeats
@clacke @fribbledom We use Bazel at my company to guarantee bit-by-bit identical builds. I have already made use of this for some regulatory checks we had to undergo. (Finance sector, many think the scrutiny here is high, I have little comparison to make)
(DIR) Post #9iqyoIsBHhcYDLENSS by Xjs@chaos.social
2019-05-15T16:13:02Z
1 likes, 0 repeats
@clacke @fribbledom Bazel is (as far as I'm aware) the only build system that uses content-based addressing of build artefacts based on hashes of the inputs. This, among other shenanigans like sandboxing and stripping of timestamps or other side effects, makes binaries bit-by-bit identical if built from the exact same sources. (Can give a few other nice things, too, like centralised artifact caching, distributed building…)
(DIR) Post #9iqyoJ4aXZXspp2HnU by clacke@libranet.de
2019-05-15T18:58:57Z
0 likes, 0 repeats
@Xjs @fribbledom Bazel, Pants, Buck, Nix/Guix. 😀But great to hear that there is work on physical reproducibility there too, not just logical reproducibility. Is that your work, or does Bazel have knowledge about stripping and normalizing data?
(DIR) Post #9irJ3KulyQDryQtAVk by Wolf480pl@niu.moe
2019-05-14T18:09:09Z
1 likes, 0 repeats
@fribbledom >But do you really trust your packagers?Yes, I do.>How could you?How could I not? I'd go crazy if I didn't trust them.
(DIR) Post #9irKXdBGj1KXGac8dU by clacke@libranet.de
2019-05-15T05:12:06Z
0 likes, 0 repeats
@fribbledom Yes!reproducible-builds.org/ is a must for systems a user can trust. "Debian's dirty secret", developer-supplied binaries, is no longer a thing. Over 90% of Debian, NixOS and I assume GuixSD are now bit-for-bit reproducible!
(DIR) Post #9irKkQOpYaMhOI3fN2 by felix@radical.town
2019-05-14T17:30:13Z
1 likes, 0 repeats
@fribbledom relevant links:https://reproducible-builds.org/https://wiki.debian.org/ReproducibleBuilds
(DIR) Post #9irKu429ZDXwv8302y by Wolf480pl@niu.moe
2019-05-14T18:08:58Z
0 likes, 0 repeats
@fribbledom >But do you really trust your packagers?Yes, I do.>How could you?How could I not?ecause I'd go crazy if I didn't trust them.
(DIR) Post #9irL3aBnNnpLFt61zM by alcinnz@floss.social
2019-05-14T17:26:20Z
0 likes, 0 repeats
@fribbledom Sounds like Guix!
(DIR) Post #9irL3aMmiwcLnyEo7M by jasper@mastodon.nl
2019-05-14T20:19:22Z
0 likes, 0 repeats
@alcinnz @fribbledom it registers stuff under the hash but the compilation process isn't necessarily reproducible?Do wonder if it might be useful to gather data on how compilations differ. Different people compiling it could also vouch for different outcomes.
(DIR) Post #9irL3aaxsDxaVws8Dg by clacke@libranet.de
2019-05-15T05:14:37Z
0 likes, 0 repeats
@jasper @alcinnz @fribbledom You are absolutely right, and this is now a thing!r13y.com/
(DIR) Post #9irLD41roBGSPMyP9E by jeffmc@pdx.social
2019-05-14T17:57:08Z
1 likes, 0 repeats
@fribbledom we do the same. We build everything from source, including the dependencies, and mirror most things so we know what we are building. That includes building the kernel with buildroot.
(DIR) Post #9irLrS4Fqx0WXtLfVo by fmartingr@mastodon.social
2019-05-14T22:51:46Z
0 likes, 0 repeats
@fribbledom We developers "inherently trust" all the time, not only in packaging but adding a lot of dependencies from other people to our projects just hoping that they don't do anything malicious.Truth is that when you install something, one person (usually) creates the package, but a lot more had contributed to the code in way the packager nor the author can expect.A sourcemap will verity that dependency A is effectivelly the same in github, but it wont check if the code is malicious.
(DIR) Post #9irLrSFbAm5774ejC4 by fribbledom@mastodon.social
2019-05-14T22:56:38Z
1 likes, 0 repeats
@fmartingrVerifying the sources is a separate (yet connected) issue indeed, but it's useless if you don't know whether the sources you're verifying are the ones that have been packaged.As a developer you should never blindly trust any dependencies. If you depend on some code, you will inherit all its flaws and issues, as well. That's your responsibility.Luckily you're not alone and it's a collective process. The same needs be achieved for verifying build integrity.
(DIR) Post #9irMShEMRM3iHspqKm by jazzyeagle@mastodon.social
2019-05-14T18:52:48Z
0 likes, 0 repeats
@fribbledom My $0.02 USD on the topic is that, unless you are fluent in multiple programming langauges and have taken the time to comb through every line of code in a package, you can never be 100% certain that a package is built as intended and will work without any maliciousness. I would say very few people have the expertise to be able to review all the code in the various languages, even less have the time to review hundreds of thousands of lines of code.
(DIR) Post #9irMShOHqRzymfTlo0 by fribbledom@mastodon.social
2019-05-14T18:55:36Z
0 likes, 0 repeats
@jazzyeagleYou're not supposed to do that yourself and alone. The point is that we need to set up systems that let us verify integrity collectively. Reproducible builds are just one step towards that. Only once we achieve that, we can actually rest assured, that the source code we're verifying is actually the code we're running. And again, that's a collective process with open source.
(DIR) Post #9irMShW5NSElAr7zxg by jazzyeagle@mastodon.social
2019-05-14T19:05:50Z
0 likes, 0 repeats
@fribbledom I guess I don't fully understand your assertion. The question is whether or not we can trust those who package the software in our repos. Setting up a system that verifies integrity is in itself another piece of software we would need to trust written by others. I'm not sure how this is any different than trusting the current packagers in our respective distros.
(DIR) Post #9irMShkGWjZzsplK40 by clacke@libranet.de
2019-05-15T05:33:48Z
0 likes, 0 repeats
@jazzyeagle @fribbledom It's turtles all the way down. But unlike the real universe, this one *does* have a bottom turtle.
(DIR) Post #9irMg8Quc63KQ2782y by fribbledom@mastodon.social
2019-05-14T19:11:50Z
0 likes, 0 repeats
@jazzyeagleIf just one entity did that, it would indeed be useless. The point is that this can be done collectively. Anyone can run such a system and verify the packages themselves.
(DIR) Post #9irMg8cxtHh51Pkkpk by roptat@framapiaf.org
2019-05-14T20:20:57Z
0 likes, 0 repeats
@fribbledomTo add some concrete examples, in guix there is this nice feature called `guix challenge` that allows you to compare local packages to one or more indipendent build farms (anyone can run there own independent build farm, and the project has two).There's also `guix build --check` that performs a local build and compares the result with what is currently installed.These features would not exist if packages were not reproducible.@jazzyeagle
(DIR) Post #9irMuHWZbogAfYVX84 by flussence@nulled.red
2019-05-14T17:47:00Z
1 likes, 0 repeats
@fribbledom >But do you really trust your packagers?nope>How could you?I trust the automated QA, marginally.>Do you know them personally and monitor their packaging work?yeah. that's why I don't trust them. also one is me
(DIR) Post #9irhLyeHNcnzJpazdg by Xjs@chaos.social
2019-05-16T05:40:54Z
1 likes, 0 repeats
@clacke @fribbledom Bazel does, and the contributed rules do, too. I think it's still possible to construct examples that circumvent it though. – Interesting, last time I checked (some years ago) there wasn't this much choice ;)
(DIR) Post #9irhLyxQES7MHCYHTc by clacke@libranet.de
2019-05-16T10:22:55Z
0 likes, 0 repeats
@Xjs @fribbledom If I understand and remember correctly, Bazel, Buck and Pants are all spiritual forks of Google's internal build system. Bazel by Googlers who stayed in Google, and the other two by the Google diaspora at LinkedIn/Facebook/whatever.Nix is a decade older than all of them, but has a stricter turtles-all-the-way-down attitude, and no make-like functionality -- it's a meta buildsystem more than a build system.But the greatest innovation the new 3 have is probably that they track changes via notifications rather than polling megabytes upon megabytes of directories on disk every time you ask if building is still a noop.
(DIR) Post #9irhePftdWFWLq8vLc by clacke@libranet.de
2019-05-16T10:27:09Z
0 likes, 0 repeats
@Xjs @fribbledom I was first made aware of these new build systems when I joined my previous-previous assignment back in 2016.I heard about Guix just before 0.9 came out in 2015 and that's how I learned about Nix.