tmake-your-own-distro.txt - monochromatic - monochromatic blog: http://blog.z3bra.org
(HTM) git clone git://z3bra.org/monochromatic
(DIR) Log
(DIR) Files
(DIR) Refs
---
tmake-your-own-distro.txt (6039B)
---
1 # Make your own distro
2
3 17 January, 2016
4
5 Happy new year everyone! For the new year I've decided to focus more on
6 projects, and less on "ricing" which bores me more and more...
7 So here is a fun project I'm working on: Making my own linux distro!
8
9 I consider building a linux distribution from scratch to be one of the coolest
10 things one could do with their computers. And I finally got into it!
11
12 I've been writing my "tales" in my
13 [journal](gopher://z3bra.org/0/distro/journal.txt). Be sure to read it for a
14 more "day-to-day" like approach.
15 This post is only meant to detail the tools used and the workflow I have when
16 I work on it.
17
18 First off all, I have never built an [LFS](https://linuxfromscratch.org) system,
19 and will probably never do (their documentation is great though).
20
21 ## Scope
22
23 So here I am, sitting in front of my computer, deciding what this "new distro"
24 should be. I've always liked the idea of [sta.li](http://sta.li), an similar
25 distributions ala [rlsd2](http://rlsd2.dimakrasner.com). Minimalist systems
26 trying to bring fresh new ideas on the table of linux distributions. So I
27 decided to list the things I wanted in my distro:
28
29 * statically compiled
30 * stripped down userland
31 * simplified file system
32 * port-tree featuring plan9's mk(1)
33 * featuring new/obscure softwares when possible
34
35 I chose each point because I though it would be and interresting challenge,
36 allowing me to learn more about linux distributions in the process. I'm not
37 trying to argue that X is better than Y, or anyhing amongst those lines.
38
39 ## Toolchains
40
41 There are, from my experience, three important toolchains when building a
42 distro:
43
44 * Compilation toolchain
45 * Package management toolchain
46 * Testing toolchain
47
48 Each one is as important as the two others, because they will define how
49 efficient you'll be for the whole project.
50
51 ### Compilation toolchain
52 I am using [pcc](http://pcc.ludd.ltu.se) as my complier of choice, and
53 [musl](http://musl-libc.org) as my libc.
54
55 To this day, I couldn't find a suitable replacement for GNU's binutils, but am
56 still looking forward to elfutils and elftoolchain.
57
58 To make it easier to work with this toolchain, I decided to build a
59 cross-compiler featuring both. I also had to create one using gcc, until I can
60 figure out a patch for softwares that can't compile with pcc.
61 The script used to generate the cross-compiler can be found
62 [here](http://git.z3bra.org/cross/file/cross-gcc.html).
63
64 In order to use it, add the path to the toolchain in your path. Calling
65 `x86_64-linux-musl-gcc` will automatically call libs/include from the cross
66 prefix.
67
68 <video controls>
69 <source src="http://pub.z3bra.org/monochromatic/vid/20160131-osdev-xcompiler.webm" type="video/webm">
70 </video>
71
72 *Using the cross-compiler to build a software statically against musl
73 libc.*
74
75
76 ### Package management toolchain
77 There are a lot of them. But the most challenging one will be the one I'll write
78 myself. This is why I wrote [pm](http://git.z3bra.org/pm/log.html), my pack
79 manager. It's still a work in progress, but it can install, delete and inspect
80 packs in a given directory, which is perfect for my needs.
81
82 For the packaging of softwares, I decided to go with an mk(1) based port-tree
83 ala BSD. Inspired by the [morpheus port-tree](http://git.2f30.org/ports/log.html),
84 it follows the way the *BSD handle their own port-tree, while using mk(1) from
85 plan9, which is, in my opinion, more predictable. For more information on mk,
86 check the link [maintaining files on plan9 with
87 mk](http://plan9.bell-labs.com/sys/doc/mk.html).
88
89 The ports are available [here](http://git.z3bra.org/mkports/log.html). To build
90 a port, you need to configure the `config.mk` file to match your environment, and
91 run `mk <port>`. This will build all the deps required to build you port, and
92 install them under your cross-compilation root directory.
93 To clean the tree from this deps, run `mk clean`. The tarball will be packed in the
94 port directory of your tool, eg `pm/pm#1.0.tar.bz`. Theses file are suitable for
95 installation via [pm](http://git.z3bra.org/pm/log.html).
96
97 <video controls>
98 <source src="http://pub.z3bra.org/monochromatic/vid/20160131-osdev-mkports.webm" type="video/webm">
99 </video>
100 *Show casing the creation of a port within the mk port-tree
101 and building/installing it under a specific directory*
102
103 ### Testing toolchain
104 To actually test the distro, you need a way to boot from it. I chose to use linux
105 containers (LXC) over traditionnal VMs (virtualbox, qemu) as I'm not playing with
106 different CPU architectures (yet!).
107 This way I can treat my distro as a simple chroot and modify it from outside in real
108 time. It also let me play with those containerization technologies I'm not
109 familiar with.
110
111 The first thing to do is to install the packs under your chroot. For this purpose, I
112 chose `/ns/pm/rootfs`. After installing a few packs, you can fire up a container
113 using this directory as its root to check wether the distro boots, and if the tools
114 you installed work as expected.
115
116 As of today, spawning gettys doesn't work. I'll need to figure out why, but otherwise,
117 it's been a pretty successful (and fun!) experience.
118
119 <video controls>
120 <source src="http://pub.z3bra.org/monochromatic/vid/20160131-osdev-containers.webm" type="video/webm">
121 </video>
122 *Demonstrating the whole testing process, from creating the
123 containers and saying "hello world!" from within this container*
124
125 In the process, I learnt how to spawn containers "the hard way". Here is a quick
126 peak at it:
127
128 ip link add veth0 type veth peer name eth0
129 ip netns add foo
130 ip link set eth0 netns foo
131 ip netns exec foo ip addr add 192.168.0.100/24 dev eth0 broadcast +
132 ip netns exec foo ip link set eth0 up
133 ip netns exec foo ip route add default via 192.168.0.254
134 brctl addif br0 veth0
135 ip netns exec unshare -fpium --mount-proc env -i chroot /ns/pm/rootfs /sbin/init
136
137 This is far from a final-run-everywhere solution of course, because you'll miss /dev
138 mounts, ttys and such, and won't be able to log within this container, or inject
139 processes in to it.
140
141 But.
142
143 That's neat.