kokada.dev.atom.xml - sfeed_tests - sfeed tests and RSS and Atom files
(HTM) git clone git://git.codemadness.org/sfeed_tests
(DIR) Log
(DIR) Files
(DIR) Refs
(DIR) README
(DIR) LICENSE
---
kokada.dev.atom.xml (494691B)
---
1 <?xml version="1.0" encoding="utf-8"?>
2 <feed xml:lang="en-us" xmlns="http://www.w3.org/2005/Atom"><title>kokada</title><link href="https://kokada.dev" rel="alternate"/><link href="https://kokada.dev/atom/" rel="self"/><id>https://kokada.dev</id><updated>2024-12-04T00:00:00+00:00</updated><subtitle/><entry><title>Praise to scdoc to generate man pages</title><link href="https://kokada.dev/blog/praise-to-scdoc-to-generate-man-pages/" rel="alternate"/><published>2024-12-04T00:00:00+00:00</published><id>https://kokada.dev/blog/praise-to-scdoc-to-generate-man-pages/</id><summary type="html"><p>Hey, its been a long time since my <a href="posts/2024-10-07/01-enabling-le-audio-lc3-in-wf-1000xm5.md">last blog
3 post</a>. It is mostly
4 because I ran out of things to write, but I expected this. This is probably
5 more likely how I am actually going to post from now. At least, it shows that
6 my plan to have a <a href="https://kokada.dev/blog/making-a-blog-for-the-next-10-years">blog for a long
7 time</a>, that is
8 easy to go back when I wanted is working fine, but I digress.</p>
9 <p>Going back to the theme of the today blog post, I needed to write a <a href="https://en.wikipedia.org/wiki/Man_page">man
10 page</a> for the first time in years. I
11 hate <a href="https://en.wikipedia.org/wiki/Troff">troff</a>, the typesetting system used
12 for man pages (similar to <a href="https://en.wikipedia.org/wiki/LaTeX">LaTeX</a> for
13 documents). It is one of the weirdest languages that I ever saw, and even the
14 example in Wikipedia shows that:</p>
15 <pre><code class="language-troff">.ND &quot;January 10, 1993&quot;
16 .AU &quot;Ms. Jane Smith&quot;
17 .AT &quot;Upcoming appointment&quot;
18 .MT 5
19 .DS
20 Reference #A12345
21 .sp 4
22 Mr. Samuel Jones
23 Field director, Bureau of Inspections
24 1010 Government Plaza
25 Capitoltown, ST
26 .sp 3
27 Dear Mr. Jones,
28 .sp 2
29 .P
30 Making reference to the noted obligation to submit for state inspection our newly created production process, we request that you consider the possible inappropriateness of subjecting the innovative technologies of tomorrow to the largely antiquated requirements of yesterday. If our great state is to prosper in the twenty-first century, we must take steps
31 .B now ,
32 in
33 .I this
34 year of
35 .I this
36 decade, to prepare our industrial base for the interstate and international competition that is sure to appear. Our new process does precisely that. Please do not let it be undone by a regulatory environment that is no longer apt.
37 .P
38 Thank you for your consideration of our position.
39 .FC Sincerely
40 .SG
41 </code></pre>
42 <p>Keep in mind that the break lines are necessary every time you introduce a
43 macro, like <code>.I this</code> (that I <em>think</em> it is for italics). Yes, this format is
44 as illegible as hell, and it is worse that the format lacks good tooling (or at
45 least I didn't find any good ones).</p>
46 <p>Most people when they need to write a man page nowadays ends up using some
47 other format that generates a man page. For example, in the past I used
48 <a href="https://pandoc.org/">Pandoc</a> to convert Markdown to a man page, but even if
49 Pandoc is a great project the result is sub-optimal at best: Markdowns are, at
50 the end, designed for generating HTML (and a subset of it), and not man pages,
51 so you basically ends up fighting the format for it to do what you want.
52 Also, Pandoc is a big project, with a ~200MB binary (at least it is the default
53 Pandoc binary in Nix).</p>
54 <p>For this specific project I needed something small. I am trying to replace one
55 of the most essential pieces inside NixOS, <code>nixos-rebuild</code>, written in Bash,
56 with a <a href="https://discourse.nixos.org/t/nixos-rebuild-ng-a-nixos-rebuild-rewrite/55606/">full rewritten in
57 Python</a>
58 (sorry Rust zealots!), called <code>nixos-rebuild-ng</code>.</p>
59 <p>Since this project will eventually (if successful) be in the critical path for
60 NixOS, I want to reduce the number of dependencies as much as possible, so
61 something as big as Pandoc is out. I could use
62 <a href="https://asciidoc.org/">AsciiDoc</a>, but it is a big complicated Python project
63 (this may seem ironic, but <code>nixos-rebuild-ng</code> has only one runtime dependency,
64 that is optional). And I also hated the last time I tried to use it to generate
65 man pages: it more flexible than Markdown, but still far from optimal.</p>
66 <p>Thanks to Drew DeVault (creator of <a href="https://swaywm.org/">SwayWM</a>) that seems it
67 had the same issues in the past and created
68 <a href="https://drewdevault.com/2018/05/13/scdoc.html"><code>scdoc</code></a>, a very simple man
69 page generator using a DSL inspired in Markdown, but specific to generate man
70 pages. The binary is written in C (and advantage in this case since it means it
71 is easier to bootstrap), is small (~1 Kloc) and has no dependencies, so it
72 fits the requirement.</p>
73 <p>While the language suffers from being a niche project for a niche segment, the
74 <a href="https://man.archlinux.org/man/scdoc.5.en">man page</a> for it is actually really
75 nice. It is terse though and lacks examples, and this is what this blog post
76 will try to accomplish.</p>
77 <p>To start, let's have a quick summary of the syntax, written in <code>scdoc</code> as
78 comments:</p>
79 <pre><code class="language-scdoc">; quick summary:
80 ; # new section
81 ; comments starts with ;
82 ; - this is a list
83 ; - sub-list
84 ; - *bold*: _underline_, force a line break++
85 ; - [tables], \[ can be used to force an actual [
86 ; . numbered list
87 ; please configure your editor to use hard tabs
88 ; see `man 5 scdoc` for more information about syntax
89 ; or https://man.archlinux.org/man/scdoc.5.en
90 </code></pre>
91 <p>I actually added this summary in the <code>.scd</code> (the <code>scdoc</code> extension) files that
92 I wrote, so it is easy for someone that never saw the format to start
93 collaborating.</p>
94 <p>And here an example of a (summarised) man page in <code>.scd</code> format:</p>
95 <pre class="chroma"><code><span class="line"><span class="cl">nixos-rebuild-ng(8)
96 </span></span><span class="line"><span class="cl">
97 </span></span><span class="line"><span class="cl"><span class="gh"># NAME
98 </span></span></span><span class="line"><span class="cl"><span class="gh"></span>
99 </span></span><span class="line"><span class="cl">nixos-rebuild - reconfigure a NixOS machine
100 </span></span><span class="line"><span class="cl">
101 </span></span><span class="line"><span class="cl"><span class="gh"># SYNOPSIS
102 </span></span></span><span class="line"><span class="cl"><span class="gh"></span>
103 </span></span><span class="line"><span class="cl"><span class="ge">_nixos-rebuild_</span> \[--upgrade] [--upgrade-all]++
104 </span></span><span class="line"><span class="cl"> \[{switch,boot}]
105 </span></span><span class="line"><span class="cl">
106 </span></span><span class="line"><span class="cl"><span class="gh"># DESCRIPTION
107 </span></span></span><span class="line"><span class="cl"><span class="gh"></span>
108 </span></span><span class="line"><span class="cl">This command has one required argument, which specifies the desired operation.
109 </span></span><span class="line"><span class="cl">It must be one of the following:
110 </span></span><span class="line"><span class="cl">
111 </span></span><span class="line"><span class="cl"><span class="ge">*switch*</span>
112 </span></span><span class="line"><span class="cl"> Build and activate the new configuration, and make it the boot default.
113 </span></span><span class="line"><span class="cl"> That is, the configuration is added to the GRUB boot menu as the
114 </span></span><span class="line"><span class="cl"> default menu entry, so that subsequent reboots will boot the system
115 </span></span><span class="line"><span class="cl"> into the new configuration. Previous configurations activated with
116 </span></span><span class="line"><span class="cl"> nixos-rebuild switch or nixos-rebuild boot remain available in the GRUB
117 </span></span><span class="line"><span class="cl"> menu.
118 </span></span><span class="line"><span class="cl">
119 </span></span><span class="line"><span class="cl"><span class="ge">*boot*</span>
120 </span></span><span class="line"><span class="cl"> Build the new configuration and make it the boot default (as with
121 </span></span><span class="line"><span class="cl"> <span class="ge">*nixos-rebuild switch*</span>), but do not activate it. That is, the system
122 </span></span><span class="line"><span class="cl"> continues to run the previous configuration until the next reboot.
123 </span></span><span class="line"><span class="cl">
124 </span></span><span class="line"><span class="cl"><span class="gh"># OPTIONS
125 </span></span></span><span class="line"><span class="cl"><span class="gh"></span>
126 </span></span><span class="line"><span class="cl"><span class="ge">*--upgrade, --upgrade-all*</span>
127 </span></span><span class="line"><span class="cl"> Update the root user&#39;s channel named &#39;nixos&#39; before rebuilding the
128 </span></span><span class="line"><span class="cl"> system.
129 </span></span><span class="line"><span class="cl">
130 </span></span><span class="line"><span class="cl"> In addition to the &#39;nixos&#39; channel, the root user&#39;s channels which have
131 </span></span><span class="line"><span class="cl"> a file named &#39;.update-on-nixos-rebuild&#39; in their base directory will
132 </span></span><span class="line"><span class="cl"> also be updated.
133 </span></span><span class="line"><span class="cl">
134 </span></span><span class="line"><span class="cl"> Passing <span class="ge">*--upgrade-all*</span> updates all of the root user&#39;s channels.
135 </span></span><span class="line"><span class="cl">
136 </span></span><span class="line"><span class="cl">See the Nix manual, <span class="ge">*nix flake lock --help*</span> or <span class="ge">*nix-build --help*</span> for details.
137 </span></span><span class="line"><span class="cl">
138 </span></span><span class="line"><span class="cl"><span class="gh"># ENVIRONMENT
139 </span></span></span><span class="line"><span class="cl"><span class="gh"></span>
140 </span></span><span class="line"><span class="cl">NIXOS_CONFIG
141 </span></span><span class="line"><span class="cl"> Path to the main NixOS configuration module. Defaults to
142 </span></span><span class="line"><span class="cl"> <span class="ge">_/etc/nixos/configuration.nix_</span>.
143 </span></span><span class="line"><span class="cl">
144 </span></span><span class="line"><span class="cl"><span class="gh"># FILES
145 </span></span></span><span class="line"><span class="cl"><span class="gh"></span>
146 </span></span><span class="line"><span class="cl">/etc/nixos/flake.nix
147 </span></span><span class="line"><span class="cl"> If this file exists, then <span class="ge">*nixos-rebuild*</span> will use it as if the
148 </span></span><span class="line"><span class="cl"> <span class="ge">*--flake*</span> option was given. This file may be a symlink to a
149 </span></span><span class="line"><span class="cl"> flake.nix in an actual flake; thus <span class="ge">_/etc/nixos_</span> need not be a
150 </span></span><span class="line"><span class="cl"> flake.
151 </span></span><span class="line"><span class="cl">
152 </span></span><span class="line"><span class="cl"><span class="gh"># AUTHORS
153 </span></span></span><span class="line"><span class="cl"><span class="gh"></span>
154 </span></span><span class="line"><span class="cl">Nixpkgs/NixOS contributors
155 </span></span></code></pre><p>And here is a screenshot of the result:</p>
156 <p><a href="https://github.com/thiagokokada/blog/raw/main/posts/2024-12-04/2024-12-04-230955_hyprshot.png"><img src="https://github.com/thiagokokada/blog/raw/main/posts/2024-12-04/2024-12-04-230955_hyprshot.png" alt="Man page rendered from scd
157 file"></a></p>
158 <p>One of nice things that I found is how looking at the plain text looks kind
159 like the man page result already. And if you know Markdown, you can basically
160 understand most things that is happening. There are a few differences, like
161 <code>*bold*</code> instead of <code>**bold**</code>, and while they're unfortunate they're not the
162 end of the world.</p>
163 <p>Now, the format has its quirks. The first line being the name of the program
164 and section in parenthesis is required, but this makes sense, since you need
165 this information for the corners. But for one, it requires the usage of hard
166 tabs to create indentation, and the error messages are awful, in a situation
167 that kind remembers me of <code>Makefile</code>. Also the choice of <code>[</code> to start a table
168 means that the traditional <code>app [command]</code> needs in many cases to be escaped as
169 <code>app \[command]</code>. I found this a strange choice since this is supposed to be a
170 format that is only used for man pages, and using <code>[command]</code> to indicate an
171 optional is common, but at least it is easy to escape.</p>
172 <p>In the end, I think all that matters is the result. And for the first time for
173 all those years trying to write a man page, I am satisfied with the result. The
174 man page looks exactly as I wanted once rendered, and the <code>.scd</code> file looks
175 reasonable good that it can work as a documentation for someone that for one
176 reason or another can't use the man page (can't say the same for the troff
177 version). Also, it is really easy for someone to just go there and update the
178 man page, even without experience in the format (except for maybe the
179 requirement of tabs). So all in all, I really liked the format, and will use it
180 again if I need to write another man page in the future.</p></summary></entry><entry><title>Enabling LE Audio/LC3 in WF-1000XM5</title><link href="https://kokada.dev/blog/enabling-le-audio-lc3-in-wf-1000xm5/" rel="alternate"/><published>2024-10-07T00:00:00+00:00</published><id>https://kokada.dev/blog/enabling-le-audio-lc3-in-wf-1000xm5/</id><summary type="html"><p>One of things that I hate the most about the fact that we are all using
181 wireless earbuds instead of wired earphones is the latency: it is bad, getting
182 up to seconds(!) depending on your particular combination of OS/earbuds/device.</p>
183 <p>There is a solution though: Bluetooth LE Audio, that is supposed to fix
184 multiple issues with the original design for Bluetooth Classic Audio, including
185 a much lower latency, improved efficiency (e.g.: less battery power) and even
186 multiple streams of audio. LE Audio also includes a new default codec for
187 improved audio quality, <a href="https://en.wikipedia.org/wiki/LC3_(codec)">LC3</a>, that
188 replaces the venerable <a href="https://en.wikipedia.org/wiki/SBC_(codec)">SBC</a> codec
189 for audio.</p>
190 <p>However, the standard is a mess right now: a few wireless headphones already
191 support it, but they're generally disabled by default and it is pretty messy to
192 enable. And even after enabling it, getting it to work can be a pain.</p>
193 <p>I have pretty much the best setup to use LE Audio right now: a recently
194 released Pixel 9 Pro with Sony's
195 <a href="https://www.sony.ie/headphones/products/wf-1000xm5">WF-1000XM5</a> earbuds, and
196 after lots of tries I got it to work. You can see below the versions of
197 everything I am using:</p>
198 <ul>
199 <li>Android: 14</li>
200 <li><a href="https://play.google.com/store/apps/details?id=com.sony.songpal.mdr">Sound
201 Connect</a>:
202 11.0.1</li>
203 <li>WM-1000XM5: 4.0.2</li>
204 </ul>
205 <p>The first thing you need to do is enable in &quot;Sound Connect&quot; app &quot;LE Audio
206 Priority&quot; in &quot;Device Settings -&gt; System&quot;:</p>
207 <p><a href="https://github.com/thiagokokada/blog/raw/main/posts/2024-10-07/photo_4909454744305642922_y.jpg"><img src="https://github.com/thiagokokada/blog/raw/main/posts/2024-10-07/photo_4909454744305642922_y.jpg" alt="LE Audio option inside Sound
208 Connect"></a></p>
209 <p>After this, you will need to pair your headset with the device again. You can
210 do this as same as always: press and hold the button in case for a few seconds
211 until a blue light starts to blink. However, this is where things starts to get
212 janky: I couldn't get the headset to pair with Android again.</p>
213 <p>A few of the things that I needed to do (in no specific order):</p>
214 <ul>
215 <li>Remove the previous paired headset</li>
216 <li>Restart the Android</li>
217 <li>Clean-up &quot;Sound Connect&quot; storage (Long press the app icon -&gt; &quot;App info&quot; -&gt;
218 &quot;Storage and Cache&quot; -&gt; &quot;Clear storage&quot;)</li>
219 </ul>
220 <p>If you can get the headset to connect, go to the &quot;Bluetooth&quot; settings in
221 Android, click in the gear icon for the headset and enable &quot;LE Audio&quot; option:</p>
222 <p><a href="https://github.com/thiagokokada/blog/raw/main/posts/2024-10-07/photo_4909454744305642937_y.jpg"><img src="https://github.com/thiagokokada/blog/raw/main/posts/2024-10-07/photo_4909454744305642937_y.jpg" alt="LE Audio option Bluetooth
223 Settings"></a></p>
224 <p>If you can't, you may want to <a href="https://helpguide.sony.net/mdr/2963/v1/en/contents/TP1000783925.html">restore the headset to factory
225 settings</a>
226 and try again from the start (that means pairing your device with &quot;Sound
227 Connect&quot; again, and you may want to try to clear the storage before doing so).</p>
228 <p>Yes, the process is extremely janky, but I think this is why both &quot;Sound
229 Connect&quot; and Android marks this feature as beta/experimental. And I still need
230 to test the latency, but from my initial testing there are some glitches when
231 the audio is only used for a short period of time (e.g.: Duolingo only enables
232 the audio when the character is speaking). So I only recommend this if you want
233 to test how LE Audio will behave, since it is clear that this needs more
234 polish.</p></summary></entry><entry><title>Quick bits: where does Nix store Flake's trusted settings?</title><link href="https://kokada.dev/blog/quick-bits-where-does-nix-store-flakes-trusted-settings/" rel="alternate"/><published>2024-09-30T00:00:00+00:00</published><id>https://kokada.dev/blog/quick-bits-where-does-nix-store-flakes-trusted-settings/</id><summary type="html"><p>Have you ever run a <code>nix build</code> command and had this prompt?</p>
235 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> nix run .#darwinActivations/Sekai-MacBook-Pro
236 </span></span><span class="line"><span class="cl"><span class="go">do you want to allow configuration setting &#39;extra-substituters&#39; to be set to &#39;https://nix-community.cachix.org https://thiagokokada-nix-configs.cachix.org&#39; (y/N)? y
237 </span></span></span><span class="line"><span class="cl"><span class="go">do you want to permanently mark this value as trusted (y/N)? y
238 </span></span></span></code></pre><p>And realise that you did/didn't want to mark this value as trusted? But where
239 is this stored? Well, I had to look at the Nix source code to find the answer,
240 since I can't find this anywhere in the web or in ChatGPT (but I am sure that
241 now that I posted this it will eventually be searchable), and the answer can be
242 found
243 <a href="https://github.com/NixOS/nix/blob/c116030605bf7fecd232d0ff3b6fe066f23e4620/src/libflake/flake/config.cc#L13-L16">here</a>:</p>
244 <pre class="chroma"><code><span class="line"><span class="cl"><span class="n">Path</span> <span class="nf">trustedListPath</span><span class="p">()</span>
245 </span></span><span class="line"><span class="cl"><span class="p">{</span>
246 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="n">getDataDir</span><span class="p">()</span> <span class="o">+</span> <span class="s">&#34;/trusted-settings.json&#34;</span><span class="p">;</span>
247 </span></span><span class="line"><span class="cl"><span class="p">}</span>
248 </span></span></code></pre><p>Where is <code>getDataDir()</code> though? I found the answer
249 <a href="https://github.com/NixOS/nix/blob/c116030605bf7fecd232d0ff3b6fe066f23e4620/src/libutil/users.cc#L52-L65">here</a>:</p>
250 <pre class="chroma"><code><span class="line"><span class="cl"><span class="n">Path</span> <span class="nf">getDataDir</span><span class="p">()</span>
251 </span></span><span class="line"><span class="cl"><span class="p">{</span>
252 </span></span><span class="line"><span class="cl"> <span class="k">auto</span> <span class="n">dir</span> <span class="o">=</span> <span class="n">getEnv</span><span class="p">(</span><span class="s">&#34;NIX_DATA_HOME&#34;</span><span class="p">);</span>
253 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="p">(</span><span class="n">dir</span><span class="p">)</span> <span class="p">{</span>
254 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="o">*</span><span class="n">dir</span><span class="p">;</span>
255 </span></span><span class="line"><span class="cl"> <span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
256 </span></span><span class="line"><span class="cl"> <span class="k">auto</span> <span class="n">xdgDir</span> <span class="o">=</span> <span class="n">getEnv</span><span class="p">(</span><span class="s">&#34;XDG_DATA_HOME&#34;</span><span class="p">);</span>
257 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="p">(</span><span class="n">xdgDir</span><span class="p">)</span> <span class="p">{</span>
258 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="o">*</span><span class="n">xdgDir</span> <span class="o">+</span> <span class="s">&#34;/nix&#34;</span><span class="p">;</span>
259 </span></span><span class="line"><span class="cl"> <span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
260 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="n">getHome</span><span class="p">()</span> <span class="o">+</span> <span class="s">&#34;/.local/share/nix&#34;</span><span class="p">;</span>
261 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
262 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
263 </span></span><span class="line"><span class="cl"><span class="p">}</span>
264 </span></span></code></pre><p>So we solved the mystery:</p>
265 <ul>
266 <li>If <code>NIX_DATA_HOME</code> is set, the file will be in
267 <code>$NIX_DATA_HOME/trusted-settings.json</code></li>
268 <li>If <code>XDG_DATA_HOME</code> is set, the file will be in
269 <code>$XDG_DATA_HOME/nix/trusted-settings.json</code></li>
270 <li>Otherwise Nix will fallback to <code>$HOME/.local/share/nix/trusted-settings.json</code></li>
271 </ul>
272 <p>By the way, if you don't know why you got this prompt, if <code>flake.nix</code> has a
273 <code>nixConfig</code> attribute inside <code>outputs</code> <strong>and</strong> this <code>nixConfig</code> is an unsafe
274 attribute (like <code>extra-substituters</code>) you will get this prompt, unless you set
275 <a href="https://nix.dev/manual/nix/2.23/command-ref/conf-file#conf-accept-flake-config"><code>accept-flake-config = true</code></a>
276 in your Nix configuration (please <strong>do not do this</strong>, it is dangerous because
277 it may allow running possible unsafe options without asking you first).</p>
278 <p>You can inspect the JSON file or delete it and Nix will prompt you again the
279 next time you run a <code>nix</code> command. And yes, saving this preference is
280 controversial considering this is Nix, but having the power to set <code>nixConfig</code>
281 attributes is really powerful, and with great powers comes great
282 responsibilities.</p></summary></entry><entry><title>My coding experience with LLM</title><link href="https://kokada.dev/blog/my-coding-experience-with-llm/" rel="alternate"/><published>2024-09-02T00:00:00+00:00</published><id>https://kokada.dev/blog/my-coding-experience-with-llm/</id><summary type="html"><p>I don't want to talk about AI. AI fatigue is real, with (garbage) AI content
283 everywhere you go. However, I like to talk about tools, and AIs, or more
284 specifically, <a href="https://en.wikipedia.org/wiki/Large_language_model">Large Language
285 Models</a> (LLM), can be
286 really useful tools. Maybe someday we will have our AI overlords dominating
287 everything (including my job), but right now I don't feel particularly
288 threatened given my experience with AIs.</p>
289 <p>Given that <a href="https://mashable.com/article/google-search-low-quality-research">Google Search is getting
290 worse</a>, I am
291 slowly relying in LLMs more and more. This is the focus of this post, basically
292 <a href="https://www.rottentomatoes.com/m/the_good_the_bad_and_the_ugly">the good, the bad and the
293 ugly</a> (by the
294 way, never saw this film) of LLMs, especially focused for solving coding
295 problems. I will take a look at the my chat history in
296 <a href="https://chatgpt.com/">ChatGPT</a> and see where it was a good tool, where it
297 shined, and where it just completely wasted my time.</p>
298 <p>A disclaimer, I never bothered to buy a ChatGPT subscription because I don't
299 feel that the tool is useful enough (at least yet), so most of the examples
300 here are either with <a href="https://openai.com/index/chatgpt/">GPT-3.5</a> or <a href="https://openai.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/">GPT-4o
301 mini</a>,
302 since those where the models offered by free in ChatGPT.</p>
303 <p>An addendum, I have some experience with <a href="https://github.com/features/copilot">GitHub
304 Copilot</a> (another LLM, but this one runs
305 inside your IDE or favorite text editor) in <code>$PREVIOUS_JOB</code>. But my experience
306 with it is even more underwhelming than what I am going to describe in this
307 post: it was basically really good for generating fixture for tests and between
308 ok-ish to plain bad for everything else. Yes, that's would fit in a tweet (or
309 xeet?).</p>
310 <h2>The good</h2>
311 <p><a href="https://chatgpt.com/share/c58a1432-2761-4954-acac-cade29942c34">Chat history</a>.</p>
312 <blockquote>
313 <p>Is there any way that I can pass a context for a net.conn to cancel its
314 Read()?</p>
315 </blockquote>
316 <p>This is one of the good cases where ChatGPT felt to me that I was talking with
317 an expert in Go. To start, I didn't even mention in my initial prompt that I
318 was talking about Go, but it correctly inferred so. Also, the solution overview
319 was great:</p>
320 <blockquote>
321 <ol>
322 <li><strong>Use a separate goroutine</strong> to handle the context cancellation.</li>
323 <li><strong>Set a deadline</strong> on the connection using <code>SetReadDeadline()</code> to unblock
324 the <code>Read()</code> when the context is canceled.</li>
325 </ol>
326 </blockquote>
327 <p>Before asking ChatGPT about this issue I had already coded my own naive
328 solution that was something similar to this:</p>
329 <pre class="chroma"><code><span class="line"><span class="cl"><span class="kd">func</span> <span class="nf">foo</span><span class="p">(</span><span class="nx">ctx</span> <span class="nx">context</span><span class="p">.</span><span class="nx">Context</span><span class="p">)</span> <span class="p">{</span>
330 </span></span><span class="line"><span class="cl"> <span class="c1">// ...
331 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="nx">done</span> <span class="o">:=</span> <span class="nb">make</span><span class="p">(</span><span class="kd">chan</span> <span class="kd">struct</span><span class="p">{})</span>
332 </span></span><span class="line"><span class="cl"> <span class="kd">var</span> <span class="nx">n</span> <span class="kt">int</span>
333 </span></span><span class="line"><span class="cl"> <span class="kd">var</span> <span class="nx">err</span> <span class="kt">error</span>
334 </span></span><span class="line"><span class="cl">
335 </span></span><span class="line"><span class="cl"> <span class="k">go</span> <span class="kd">func</span><span class="p">()</span> <span class="p">{</span>
336 </span></span><span class="line"><span class="cl"> <span class="nx">n</span><span class="p">,</span> <span class="nx">err</span> <span class="p">=</span> <span class="nx">conn</span><span class="p">.</span><span class="nf">Read</span><span class="p">(</span><span class="nx">buf</span><span class="p">)</span>
337 </span></span><span class="line"><span class="cl"> <span class="nx">done</span> <span class="o">&lt;-</span><span class="kd">struct</span><span class="p">{}{}</span>
338 </span></span><span class="line"><span class="cl"> <span class="p">}()</span>
339 </span></span><span class="line"><span class="cl">
340 </span></span><span class="line"><span class="cl"> <span class="k">select</span> <span class="p">{</span>
341 </span></span><span class="line"><span class="cl"> <span class="k">case</span> <span class="o">&lt;-</span><span class="nx">ctx</span><span class="p">.</span><span class="nf">Done</span><span class="p">():</span>
342 </span></span><span class="line"><span class="cl"> <span class="nx">conn</span><span class="p">.</span><span class="nf">Close</span><span class="p">()</span>
343 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">ctx</span><span class="p">.</span><span class="nf">Err</span><span class="p">()</span>
344 </span></span><span class="line"><span class="cl"> <span class="k">case</span> <span class="o">&lt;-</span><span class="nx">done</span><span class="p">:</span>
345 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="nx">n</span><span class="p">,</span> <span class="nx">err</span>
346 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
347 </span></span><span class="line"><span class="cl"> <span class="c1">// ...
348 </span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="p">}</span>
349 </span></span></code></pre><p>Compared this solution with the one from ChatGPT:</p>
350 <pre class="chroma"><code><span class="line"><span class="cl"><span class="kd">func</span> <span class="nf">readWithContext</span><span class="p">(</span><span class="nx">ctx</span> <span class="nx">context</span><span class="p">.</span><span class="nx">Context</span><span class="p">,</span> <span class="nx">conn</span> <span class="nx">net</span><span class="p">.</span><span class="nx">Conn</span><span class="p">,</span> <span class="nx">buf</span> <span class="p">[]</span><span class="kt">byte</span><span class="p">)</span> <span class="p">(</span><span class="kt">int</span><span class="p">,</span> <span class="kt">error</span><span class="p">)</span> <span class="p">{</span>
351 </span></span><span class="line"><span class="cl"> <span class="nx">done</span> <span class="o">:=</span> <span class="nb">make</span><span class="p">(</span><span class="kd">chan</span> <span class="kd">struct</span><span class="p">{})</span>
352 </span></span><span class="line"><span class="cl"> <span class="kd">var</span> <span class="nx">n</span> <span class="kt">int</span>
353 </span></span><span class="line"><span class="cl"> <span class="kd">var</span> <span class="nx">err</span> <span class="kt">error</span>
354 </span></span><span class="line"><span class="cl">
355 </span></span><span class="line"><span class="cl"> <span class="c1">// Start a goroutine to perform the read
356 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="k">go</span> <span class="kd">func</span><span class="p">()</span> <span class="p">{</span>
357 </span></span><span class="line"><span class="cl"> <span class="nx">n</span><span class="p">,</span> <span class="nx">err</span> <span class="p">=</span> <span class="nx">conn</span><span class="p">.</span><span class="nf">Read</span><span class="p">(</span><span class="nx">buf</span><span class="p">)</span>
358 </span></span><span class="line"><span class="cl"> <span class="nb">close</span><span class="p">(</span><span class="nx">done</span><span class="p">)</span>
359 </span></span><span class="line"><span class="cl"> <span class="p">}()</span>
360 </span></span><span class="line"><span class="cl">
361 </span></span><span class="line"><span class="cl"> <span class="k">select</span> <span class="p">{</span>
362 </span></span><span class="line"><span class="cl"> <span class="k">case</span> <span class="o">&lt;-</span><span class="nx">ctx</span><span class="p">.</span><span class="nf">Done</span><span class="p">():</span> <span class="c1">// Context was canceled or timed out
363 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="c1">// Set a short deadline to unblock the Read()
364 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="nx">conn</span><span class="p">.</span><span class="nf">SetReadDeadline</span><span class="p">(</span><span class="nx">time</span><span class="p">.</span><span class="nf">Now</span><span class="p">())</span>
365 </span></span><span class="line"><span class="cl"> <span class="o">&lt;-</span><span class="nx">done</span> <span class="c1">// Wait for the read to finish
366 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="k">return</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">ctx</span><span class="p">.</span><span class="nf">Err</span><span class="p">()</span>
367 </span></span><span class="line"><span class="cl"> <span class="k">case</span> <span class="o">&lt;-</span><span class="nx">done</span><span class="p">:</span> <span class="c1">// Read finished successfully
368 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="k">return</span> <span class="nx">n</span><span class="p">,</span> <span class="nx">err</span>
369 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
370 </span></span><span class="line"><span class="cl"><span class="p">}</span>
371 </span></span></code></pre><p>The ChatGPT version is much better because:</p>
372 <ol>
373 <li>It doesn't close the connection socket allowing it to be reused</li>
374 <li>It waits until the goroutine is finished to avoid leaks</li>
375 <li>It closes the channel after it finishes</li>
376 </ol>
377 <p>Now, I don't claim to be a Go expert (far from it), but this was one of those
378 cases where I felt that I was being guided by someone that is coding in Go for
379 years.</p>
380 <p>But it didn't end there, I asked:</p>
381 <blockquote>
382 <p>If I want to reuse the connection, do I need to reset the SetReadDeadline?</p>
383 </blockquote>
384 <p>Since in this particular case I wanted the underlying connection to be reused,
385 and ChatGPT correctly amended the code:</p>
386 <pre class="chroma"><code><span class="line"><span class="cl"><span class="kd">func</span> <span class="nf">readWithContext</span><span class="p">(</span><span class="nx">ctx</span> <span class="nx">context</span><span class="p">.</span><span class="nx">Context</span><span class="p">,</span> <span class="nx">conn</span> <span class="nx">net</span><span class="p">.</span><span class="nx">Conn</span><span class="p">,</span> <span class="nx">buf</span> <span class="p">[]</span><span class="kt">byte</span><span class="p">)</span> <span class="p">(</span><span class="kt">int</span><span class="p">,</span> <span class="kt">error</span><span class="p">)</span> <span class="p">{</span>
387 </span></span><span class="line"><span class="cl"> <span class="c1">// Same as before
388 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="k">select</span> <span class="p">{</span>
389 </span></span><span class="line"><span class="cl"> <span class="k">case</span> <span class="o">&lt;-</span><span class="nx">ctx</span><span class="p">.</span><span class="nf">Done</span><span class="p">():</span> <span class="c1">// Context was canceled or timed out
390 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="c1">// Set a short deadline to unblock the Read()
391 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="nx">conn</span><span class="p">.</span><span class="nf">SetReadDeadline</span><span class="p">(</span><span class="nx">time</span><span class="p">.</span><span class="nf">Now</span><span class="p">())</span>
392 </span></span><span class="line"><span class="cl"> <span class="o">&lt;-</span><span class="nx">done</span> <span class="c1">// Wait for the read to finish
393 </span></span></span><span class="line"><span class="cl"><span class="c1"></span>
394 </span></span><span class="line"><span class="cl"> <span class="c1">// Reset the deadline so the connection can be reused
395 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="nx">conn</span><span class="p">.</span><span class="nf">SetReadDeadline</span><span class="p">(</span><span class="nx">time</span><span class="p">.</span><span class="nx">Time</span><span class="p">{})</span>
396 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">ctx</span><span class="p">.</span><span class="nf">Err</span><span class="p">()</span>
397 </span></span><span class="line"><span class="cl"> <span class="k">case</span> <span class="o">&lt;-</span><span class="nx">done</span><span class="p">:</span> <span class="c1">// Read finished successfully
398 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="k">return</span> <span class="nx">n</span><span class="p">,</span> <span class="nx">err</span>
399 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
400 </span></span><span class="line"><span class="cl"><span class="p">}</span>
401 </span></span></code></pre><p>Also the explanation on why this happens was good:</p>
402 <blockquote>
403 <p>When you set a read deadline with <code>conn.SetReadDeadline(time.Now())</code> to unblock
404 the <code>Read()</code> call, this deadline remains in effect for future <code>Read()</code>
405 operations. This means that if you attempt another read operation after this,
406 it could immediately fail with a timeout error if the deadline is still in
407 effect.
408 To reuse the connection, you'll need to remove the deadline or set a new one.
409 You can do this by calling <code>conn.SetReadDeadline(time.Time{})</code>, which clears
410 any existing deadline.</p>
411 </blockquote>
412 <p>I had one last question that I kind already knew the answer:</p>
413 <blockquote>
414 <p>Do we need to defer the clearing?</p>
415 </blockquote>
416 <p>And ChatGPT correctly returned:</p>
417 <blockquote>
418 <p>Whether you need to defer the clearing of the read deadline depends on how
419 your code is structured. If you want to ensure that the read deadline is
420 always cleared after a context cancellation, even if the code panics or exits
421 early, then deferring the <code>conn.SetReadDeadline(time.Time{})</code> call is a good
422 idea.</p>
423 </blockquote>
424 <p>This whole interaction was one of the best ones I ever had in ChatGPT, one that
425 it gave me an actual functional code with good explanation on what was
426 happening. But still, it didn't talked at all about the gotcha of setting the
427 read deadline and reusing the underlying connection, needing another prompt to
428 give me this information.</p>
429 <p>Also, there is one issue in ChatGPT solution though: we are discarding the
430 values of the buffer once the context is done by returning <code>n = 0</code> instead of
431 the number of bytes read. In this particular case this is what I wanted, but
432 this is a possible issue that ChatGPT didn't talk.</p>
433 <p>For comparison,
434 <a href="https://stackoverflow.com/questions/61637359/how-to-use-context-context-with-tcp-connection-read">this</a>
435 is the best that I got searching in both DuckDuckGo and Google, that is where I
436 kind based my first solution on. I still not sure where ChatGPT could give me
437 such a good answer for this question, because I just couldn't find anything
438 remotely similar during my searches.</p>
439 <h2>The amazing</h2>
440 <p><a href="https://chatgpt.com/share/0c9a148f-4cfd-4b4e-aa84-7486e185125b">Chat history</a>.</p>
441 <blockquote>
442 <p>I want to write an extension to <a href="https://github.com/yuin/goldmark/">https://github.com/yuin/goldmark/</a> that allows
443 me to rewrite <code>[link](../2024-07-30/post.md)</code> to
444 <code>[link](https://kokada.capivaras.dev/slug)</code></p>
445 </blockquote>
446 <p>Here is the generated code:</p>
447 <pre class="chroma"><code><span class="line"><span class="cl"><span class="kn">package</span> <span class="nx">linkrewriter</span>
448 </span></span><span class="line"><span class="cl">
449 </span></span><span class="line"><span class="cl"><span class="kn">import</span> <span class="p">(</span>
450 </span></span><span class="line"><span class="cl"> <span class="s">&#34;fmt&#34;</span>
451 </span></span><span class="line"><span class="cl"> <span class="s">&#34;path&#34;</span>
452 </span></span><span class="line"><span class="cl"> <span class="s">&#34;strings&#34;</span>
453 </span></span><span class="line"><span class="cl">
454 </span></span><span class="line"><span class="cl"> <span class="s">&#34;github.com/yuin/goldmark&#34;</span>
455 </span></span><span class="line"><span class="cl"> <span class="s">&#34;github.com/yuin/goldmark/ast&#34;</span>
456 </span></span><span class="line"><span class="cl"> <span class="s">&#34;github.com/yuin/goldmark/parser&#34;</span>
457 </span></span><span class="line"><span class="cl"> <span class="s">&#34;github.com/yuin/goldmark/renderer&#34;</span>
458 </span></span><span class="line"><span class="cl"> <span class="s">&#34;github.com/yuin/goldmark/util&#34;</span>
459 </span></span><span class="line"><span class="cl"><span class="p">)</span>
460 </span></span><span class="line"><span class="cl">
461 </span></span><span class="line"><span class="cl"><span class="c1">// LinkRewriter is the main struct for your extension
462 </span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="kd">type</span> <span class="nx">LinkRewriter</span> <span class="kd">struct</span><span class="p">{}</span>
463 </span></span><span class="line"><span class="cl">
464 </span></span><span class="line"><span class="cl"><span class="c1">// NewLinkRewriter returns a new instance of LinkRewriter
465 </span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="kd">func</span> <span class="nf">NewLinkRewriter</span><span class="p">()</span> <span class="o">*</span><span class="nx">LinkRewriter</span> <span class="p">{</span>
466 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="o">&amp;</span><span class="nx">LinkRewriter</span><span class="p">{}</span>
467 </span></span><span class="line"><span class="cl"><span class="p">}</span>
468 </span></span><span class="line"><span class="cl">
469 </span></span><span class="line"><span class="cl"><span class="c1">// Extend will be called by Goldmark to add your extension
470 </span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="kd">func</span> <span class="p">(</span><span class="nx">e</span> <span class="o">*</span><span class="nx">LinkRewriter</span><span class="p">)</span> <span class="nf">Extend</span><span class="p">(</span><span class="nx">m</span> <span class="nx">goldmark</span><span class="p">.</span><span class="nx">Markdown</span><span class="p">)</span> <span class="p">{</span>
471 </span></span><span class="line"><span class="cl"> <span class="nx">m</span><span class="p">.</span><span class="nf">Parser</span><span class="p">().</span><span class="nf">AddOptions</span><span class="p">(</span><span class="nx">parser</span><span class="p">.</span><span class="nf">WithASTTransformers</span><span class="p">(</span><span class="nx">util</span><span class="p">.</span><span class="nf">Prioritized</span><span class="p">(</span><span class="nx">e</span><span class="p">,</span> <span class="mi">0</span><span class="p">)))</span>
472 </span></span><span class="line"><span class="cl"><span class="p">}</span>
473 </span></span><span class="line"><span class="cl">
474 </span></span><span class="line"><span class="cl"><span class="c1">// Transform is the method that modifies the AST
475 </span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="kd">func</span> <span class="p">(</span><span class="nx">e</span> <span class="o">*</span><span class="nx">LinkRewriter</span><span class="p">)</span> <span class="nf">Transform</span><span class="p">(</span><span class="nx">node</span> <span class="o">*</span><span class="nx">ast</span><span class="p">.</span><span class="nx">Document</span><span class="p">,</span> <span class="nx">reader</span> <span class="nx">text</span><span class="p">.</span><span class="nx">Reader</span><span class="p">,</span> <span class="nx">pc</span> <span class="nx">parser</span><span class="p">.</span><span class="nx">Context</span><span class="p">)</span> <span class="p">{</span>
476 </span></span><span class="line"><span class="cl"> <span class="nx">ast</span><span class="p">.</span><span class="nf">Walk</span><span class="p">(</span><span class="nx">node</span><span class="p">,</span> <span class="kd">func</span><span class="p">(</span><span class="nx">n</span> <span class="nx">ast</span><span class="p">.</span><span class="nx">Node</span><span class="p">,</span> <span class="nx">entering</span> <span class="kt">bool</span><span class="p">)</span> <span class="p">(</span><span class="nx">ast</span><span class="p">.</span><span class="nx">WalkStatus</span><span class="p">,</span> <span class="kt">error</span><span class="p">)</span> <span class="p">{</span>
477 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="p">!</span><span class="nx">entering</span> <span class="p">{</span>
478 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="nx">ast</span><span class="p">.</span><span class="nx">WalkContinue</span><span class="p">,</span> <span class="kc">nil</span>
479 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
480 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="nx">link</span><span class="p">,</span> <span class="nx">ok</span> <span class="o">:=</span> <span class="nx">n</span><span class="p">.(</span><span class="o">*</span><span class="nx">ast</span><span class="p">.</span><span class="nx">Link</span><span class="p">);</span> <span class="nx">ok</span> <span class="p">{</span>
481 </span></span><span class="line"><span class="cl"> <span class="nf">rewriteLink</span><span class="p">(</span><span class="nx">link</span><span class="p">)</span>
482 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
483 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="nx">ast</span><span class="p">.</span><span class="nx">WalkContinue</span><span class="p">,</span> <span class="kc">nil</span>
484 </span></span><span class="line"><span class="cl"> <span class="p">})</span>
485 </span></span><span class="line"><span class="cl"><span class="p">}</span>
486 </span></span><span class="line"><span class="cl">
487 </span></span><span class="line"><span class="cl"><span class="c1">// rewriteLink modifies the link URL
488 </span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="kd">func</span> <span class="nf">rewriteLink</span><span class="p">(</span><span class="nx">link</span> <span class="o">*</span><span class="nx">ast</span><span class="p">.</span><span class="nx">Link</span><span class="p">)</span> <span class="p">{</span>
489 </span></span><span class="line"><span class="cl"> <span class="nx">url</span> <span class="o">:=</span> <span class="nb">string</span><span class="p">(</span><span class="nx">link</span><span class="p">.</span><span class="nx">Destination</span><span class="p">)</span>
490 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="nx">strings</span><span class="p">.</span><span class="nf">HasPrefix</span><span class="p">(</span><span class="nx">url</span><span class="p">,</span> <span class="s">&#34;../&#34;</span><span class="p">)</span> <span class="p">{</span>
491 </span></span><span class="line"><span class="cl"> <span class="c1">// Modify this part according to your slug generation logic
492 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="nx">slug</span> <span class="o">:=</span> <span class="nf">generateSlug</span><span class="p">(</span><span class="nx">url</span><span class="p">)</span>
493 </span></span><span class="line"><span class="cl"> <span class="nx">link</span><span class="p">.</span><span class="nx">Destination</span> <span class="p">=</span> <span class="p">[]</span><span class="nb">byte</span><span class="p">(</span><span class="nx">fmt</span><span class="p">.</span><span class="nf">Sprintf</span><span class="p">(</span><span class="s">&#34;https://kokada.capivaras.dev/%s&#34;</span><span class="p">,</span> <span class="nx">slug</span><span class="p">))</span>
494 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
495 </span></span><span class="line"><span class="cl"><span class="p">}</span>
496 </span></span><span class="line"><span class="cl">
497 </span></span><span class="line"><span class="cl"><span class="c1">// generateSlug generates a slug from the given URL
498 </span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="kd">func</span> <span class="nf">generateSlug</span><span class="p">(</span><span class="nx">url</span> <span class="kt">string</span><span class="p">)</span> <span class="kt">string</span> <span class="p">{</span>
499 </span></span><span class="line"><span class="cl"> <span class="c1">// Assuming the slug is the last part of the URL without the extension
500 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="k">return</span> <span class="nx">strings</span><span class="p">.</span><span class="nf">TrimSuffix</span><span class="p">(</span><span class="nx">path</span><span class="p">.</span><span class="nf">Base</span><span class="p">(</span><span class="nx">url</span><span class="p">),</span> <span class="nx">path</span><span class="p">.</span><span class="nf">Ext</span><span class="p">(</span><span class="nx">url</span><span class="p">))</span>
501 </span></span><span class="line"><span class="cl"><span class="p">}</span>
502 </span></span></code></pre><pre class="chroma"><code><span class="line"><span class="cl"><span class="kn">package</span> <span class="nx">main</span>
503 </span></span><span class="line"><span class="cl">
504 </span></span><span class="line"><span class="cl"><span class="kn">import</span> <span class="p">(</span>
505 </span></span><span class="line"><span class="cl"> <span class="s">&#34;github.com/yuin/goldmark&#34;</span>
506 </span></span><span class="line"><span class="cl"> <span class="s">&#34;github.com/yuin/goldmark/renderer/html&#34;</span>
507 </span></span><span class="line"><span class="cl"> <span class="nx">linkrewriter</span> <span class="s">&#34;path/to/your/linkrewriter&#34;</span>
508 </span></span><span class="line"><span class="cl"><span class="p">)</span>
509 </span></span><span class="line"><span class="cl">
510 </span></span><span class="line"><span class="cl"><span class="kd">func</span> <span class="nf">main</span><span class="p">()</span> <span class="p">{</span>
511 </span></span><span class="line"><span class="cl"> <span class="nx">md</span> <span class="o">:=</span> <span class="nx">goldmark</span><span class="p">.</span><span class="nf">New</span><span class="p">(</span>
512 </span></span><span class="line"><span class="cl"> <span class="nx">goldmark</span><span class="p">.</span><span class="nf">WithExtensions</span><span class="p">(</span>
513 </span></span><span class="line"><span class="cl"> <span class="nx">linkrewriter</span><span class="p">.</span><span class="nf">NewLinkRewriter</span><span class="p">(),</span>
514 </span></span><span class="line"><span class="cl"> <span class="p">),</span>
515 </span></span><span class="line"><span class="cl"> <span class="nx">goldmark</span><span class="p">.</span><span class="nf">WithRendererOptions</span><span class="p">(</span>
516 </span></span><span class="line"><span class="cl"> <span class="nx">html</span><span class="p">.</span><span class="nf">WithHardWraps</span><span class="p">(),</span>
517 </span></span><span class="line"><span class="cl"> <span class="p">),</span>
518 </span></span><span class="line"><span class="cl"> <span class="p">)</span>
519 </span></span><span class="line"><span class="cl">
520 </span></span><span class="line"><span class="cl"> <span class="c1">// Use `md.Convert` to convert your markdown to HTML as usual
521 </span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="p">}</span>
522 </span></span></code></pre><p>This one will need some context. <a href="https://github.com/yuin/goldmark/">Goldmark</a>,
523 the Markdown renderer that <a href="https://kokada.dev/blog/making-a-blog-for-the-next-10-years">powers this
524 blog</a>, does not
525 have good examples or documentation (at least I didn't found neither).</p>
526 <p>So I was not really hopeful I would get any answers in ChatGPT, but not only it
527 put me in the correct path, the code was almost correct. It did had some unused
528 and missing imports, and there were some compilation errors that were easily
529 fixed. And when it registered the extension, it also added a
530 <code>html.WithHardWraps()</code> renderer for no reason at all.</p>
531 <p>But this is one of the experiences in LLM that most impressed me. I had no idea
532 at all on how to solve this problem, and ChatGPT basically gave me the
533 solution. A <a href="https://github.com/thiagokokada/blog/blob/3a2ea58a0638d1005617d25eabeae0626933db46/link_rewriter.go">modified version of the
534 code</a>
535 from this chat is still used to power this blog to this day, that also
536 demonstrates that the code is not garbage (since now I understand what it does
537 and I can even extend it).</p>
538 <p>I am still not sure how the license situation of this code is though. Does
539 licenses even make sense in AI generated code?</p>
540 <h2>The bad</h2>
541 <p><a href="https://chatgpt.com/share/95fcd777-b03c-4cfe-8d1d-12e53f5a4dfe">Chat history</a>.</p>
542 <blockquote>
543 <p>I want to configure VM options inside IntelliJ by project</p>
544 </blockquote>
545 <blockquote>
546 <p>My project uses sbt, and I want to set this in a way that I can share with my
547 colleagues</p>
548 </blockquote>
549 <blockquote>
550 <p>I am using a build.sbt file, but IntelliJ is not loading it's JVM options and
551 I need to set this up manually. I want to automate this</p>
552 </blockquote>
553 <p>This is an example of my most common interaction with ChatGPT: it generally
554 gives some options that could work, but not in my specific case. So I start to
555 ask for more information until it eventually starts to
556 <a href="https://en.wikipedia.org/wiki/Hallucination_(artificial_intelligence)">hallucinate</a>.</p>
557 <p>In this particular case, the hallucination was in:</p>
558 <blockquote>
559 <p>The sbt-idea-plugin can generate IntelliJ IDEA project files based on your
560 SBT configuration, including JVM options. This allows you to ensure that
561 IntelliJ loads the correct settings from the SBT configuration.</p>
562 <p>Steps:</p>
563 <ol>
564 <li>Add the <code>sbt-idea-plugin</code> to your <code>project/plugins.sbt</code> file:</li>
565 </ol>
566 <pre class="chroma"><code><span class="line"><span class="cl"><span class="n">addSbtPlugin</span><span class="o">(</span><span class="s">&#34;com.github.mwz&#34;</span> <span class="o">%</span> <span class="s">&#34;sbt-idea&#34;</span> <span class="o">%</span> <span class="s">&#34;2.4.0&#34;</span><span class="o">)</span>
567 </span></span></code></pre><ol start="2">
568 <li>Run the following command in your terminal to regenerate IntelliJ IDEA
569 project files:</li>
570 </ol>
571 <pre class="chroma"><code><span class="line"><span class="cl"><span class="go">sbt gen-idea
572 </span></span></span></code></pre></blockquote>
573 <p>Yes, there is a
574 <a href="https://github.com/JetBrains/sbt-idea-plugin"><code>sbt-idea-plugin</code></a>, but no, it
575 doesn't generate project files based in my SBT configuration. It is not even
576 from the repository that it is supposed to be pointing too.</p>
577 <p>I will just not say that this interaction was a waste of time because some of
578 the other ideas can be applied, but this was the kind of case where I ended up
579 just being disappointed because the thing that seemed to be a possible solution
580 for my problem ended up not existing at all.</p>
581 <h2>The ugly</h2>
582 <p>There are lots of examples where ChatGPT just plain disappointed me. Here is a
583 few:</p>
584 <blockquote>
585 <p>What is the language that is a fork of Go that has extra features like map?</p>
586 </blockquote>
587 <p><a href="https://chatgpt.com/share/2b854bbd-d376-4049-97fe-b47321c1ef07">Chat history</a>.</p>
588 <p>This is one of those cases where I saw something once and was trying to
589 remember. ChatGPT just gave me lots of answers that were not a fork of Go
590 (<a href="https://vlang.io/">V</a>, <a href="https://github.com/gluon-lang/gluon">Gluon</a>,
591 <a href="https://www.ponylang.io">Pony</a>), hallucinated a programming language (Gomini),
592 and said that <a href="https://ziglang.org/">Zig</a> is focused in STEM.</p>
593 <p>By the way, I eventually remember what the language was:
594 <a href="https://goplus.org/">Go+</a>. To be fair with ChatGPT, I misremembered that the
595 Go+ had map (as the <a href="https://en.wikipedia.org/wiki/Map_(higher-order_function)">high-order
596 function</a>, not a
597 hashmap that both Go and Go+ have). But still, Go+ explicit announces that it
598 is a language for STEM everywhere in their website, and even them ChatGPT
599 couldn't answer.</p>
600 <blockquote>
601 <p>In IntelliJ, I need to pass some VM options to the run configuration for my
602 application to run tests, e.g.:</p>
603 <pre><code>--add-exports=java.base/sun.security.x509=ALL-UNNAMED
604 --add-opens=java.base/sun.security.ssl=ALL-UNNAMED
605 -Dconfig.resource=api.test.conf
606 </code></pre>
607 <p>However, the <code>api.test.conf</code> may be <code>jobs.api.conf</code> or something else
608 depending on the subproject I am running inside Idea. Can I parametrize this
609 in some way?</p>
610 </blockquote>
611 <p><a href="https://chatgpt.com/share/aa9bd051-65de-49d9-8d56-0e28de94722c">Chat history</a></p>
612 <p>So quick answer: you can't. IntelliJ macros are not powerful enough for this.
613 ChatGPT itself said so when I asked about the IntelliJ macros
614 <a href="https://chatgpt.com/share/7a057bf5-bcec-458b-9432-98e3f551ad28">directly</a>.</p>
615 <p>But in this particular case, ChatGPT happily said &quot;yes, you can parameterize VM
616 options in IntelliJ IDEA to make them more flexible depending on the subproject
617 or other context&quot;. Thanks for wasting my time, I guess.</p>
618 <h2>Conclusion</h2>
619 <p>I do think that LLMs, in particular ChatGPT since this is the one I have more
620 experience with, are improving. But still, if there is anything that I
621 concluded after putting all those experiences in a post is that you still need
622 someone that knows at least as much about whatever you're asking ChatGPT to
623 actually validate the answer.</p>
624 <p>Not only because LLMs hallucinates a lot, but also because even in the &quot;good&quot;
625 example the LLM needed extra input to figure out about the possible drawback of
626 the solution. The only case where I felt that ChatGPT outsmarted me was the
627 &quot;amazing&quot; case, but this was literally the only example that I could find in
628 all my chat history with ChatGPT that was like this.</p>
629 <p>So yes, at least for now I still don't think AI will be our overlords. Maybe
630 someday.</p></summary></entry><entry><title>Making a blog for the next 10 years</title><link href="https://kokada.dev/blog/making-a-blog-for-the-next-10-years/" rel="alternate"/><published>2024-08-24T00:00:00+00:00</published><id>https://kokada.dev/blog/making-a-blog-for-the-next-10-years/</id><summary type="html"><p>So one thing that I realise after starting writing this blog is that I care
631 more about it than some of my other projects. For some reason or another I want
632 to make sure that this blog will continue with me for a long time. This is one
633 of the reasons why I use <a href="https://kokada.dev/blog/using-github-as-a-bad-blog-platform">GitHub as mirror
634 blog</a> and why I
635 created a <a href="https://kokada.dev/blog/quick-bits-why-you-should-automate-everything">bunch of
636 automation</a>
637 to make sure I never forget how to maintain this blog.</p>
638 <p>Still, there are a bunch of dependencies that I need to make sure they're
639 working so I can publish blog posts:</p>
640 <ul>
641 <li>Markdown</li>
642 <li>A text editor</li>
643 <li>Hosting and blog platform</li>
644 <li>Shell script and
645 <a href="https://kokada.dev/blog/abusing-makefiles-for-fun-and-profit">Makefile</a></li>
646 <li>Go toolchain and modules</li>
647 </ul>
648 <p>Let's start with the most important one: the texts themselves, they're all
649 written in <a href="https://www.markdownguide.org/">Markdown</a>. The nice thing about
650 Markdown is that it is just plain text files with some special notation that
651 allow you to format text, but the text itself is still legible even if you have
652 no support to render Markdowns. So it means as long as I can read a plain text
653 file, I can still read the posts, no issue there. Closely related issue here is
654 the text editor, but the nice thing about Markdown is that if tomorrow I decide
655 to change from <code>neovim</code> to, say, <a href="https://zed.dev/">Zed</a>, I can still write it
656 Markdown files without issue. I also use a standardised Markdown implementation
657 called <a href="https://commonmark.org/">CommonMark</a>, that fixes a bunch of ambiguities
658 of the original Markdown spec.</p>
659 <p>The next point is the host (<a href="capivaras.dev">capivaras.dev</a>) and the blog
660 platform (<a href="https://github.com/mataroa-blog/mataroa">Mataroa</a>). One of the nice
661 things about Mataroa is that it avoids platform lock-in by having multiple ways
662 to <a href="https://mataroa.blog/export/">export your posts</a>. I could export everything
663 to <a href="https://gohugo.io/">Hugo</a>, for example, and serve the blog from <a href="https://pages.github.com/">GitHub
664 Pages</a>.</p>
665 <p>This is especially nice consider how small <a href="capivaras.dev">capivaras.dev</a> is,
666 so it is good to know that if I eventually have issues I could move to
667 somewhere else. I also have an e-mail backup every month with all posts made
668 (another <a href="https://hey.mataroa.blog/blog/monthly-auto-exports-via-email/">Mataroa
669 feature</a>), and
670 of course I have a <a href="https://github.com/thiagokokada/blog">Git repo</a> that also
671 acts as a <a href="https://kokada.dev/blog/using-github-as-a-bad-blog-platform">mirror of this
672 blog</a>. So I would
673 say the chance of losing access to the content is slim.</p>
674 <p>One other issue is the URL of the posts that are indexed in many different
675 places, like <a href="https://news.ycombinator.com/">Hacker News</a>,
676 <a href="https://lobste.rs/">Lobte.rs</a>, etc. This is why I finally decide to bite the
677 bullet and purchase a proper domain, and this website should now be available
678 in <a href="kokada.dev">kokada.dev</a>. This means that in my eventual exit from
679 <a href="capivaras.dev">capivaras.dev</a>, I can just point my new blog location to my own
680 domain if needed (it is not as easy since I also need to preserve the post
681 URLs, but shouldn't be difficult to fix this if I ever need to do so).</p>
682 <p>Now for the tools that I use to publish from the original Markdown files to
683 everything else. Let's start with shell script(s) and Makefile: I decided that
684 they're less of an issue if they eventually stop working: they're only used to
685 make my life easier, but I can still publish files manually if needed. Still, I
686 tried to rewrite both the
687 <a href="https://github.com/thiagokokada/blog/commit/a0d421ca90f3da059998295c5e3c6c7a6a3f0688">shell</a>
688 and
689 <a href="https://github.com/thiagokokada/blog/commit/074580065b21fbdaf930aa51968e69f015d49505">Makefile</a>
690 to avoid GNUisms, so in the eventual case that I decide to e.g.: stop using a
691 GNU/Linux system like NixOS and use a *BSD system instead, I am covered.</p>
692 <p>In other projects I would probably add a <code>shell.nix</code> file and add the GNU
693 versions of the tools that I am using, but I am trying to avoid Nix while
694 building this blog as much as I can, mainly because it is not available
695 everywhere (e.g.: *BSD systems).</p>
696 <p>Go is the more important part: the tooling used to <a href="https://kokada.dev/blog/quick-bits-why-you-should-automate-everything">publish this blog is
697 written in
698 Go</a>. Go
699 is a good language when you want to ensure that things will work for a long
700 time because of its <a href="https://go.dev/blog/compat">backwards compatibility
701 guarantee</a>. Also I don't expect Google dropping Go
702 development soon, but even if this happen (<a href="https://killedbygoogle.com/">&quot;killed by
703 Google&quot;</a> is a thing after all), it is very likely
704 some other group or company would adopt its development quickly, considering
705 <a href="https://www.tiobe.com/tiobe-index/go/">how popular the language</a> is.</p>
706 <p>However, the <a href="https://github.com/thiagokokada/blog/blob/main/go.mod">Go
707 modules</a> that I depend
708 are another story:</p>
709 <ul>
710 <li><a href="https://github.com/elliotchance/orderedmap/">elliotchance/orderedmap</a>: an
711 ordered map implementation that I use until Go adds it in the <a href="https://kokada.dev/blog/an-unordered-list-of-things-i-miss-in-go">standard
712 library</a></li>
713 <li><a href="https://github.com/gorilla/feeds">gorilla/feeds</a>: a RSS generator library</li>
714 <li><a href="https://github.com/gosimple/slug">gosimple/slug</a>: a
715 <a href="https://developer.mozilla.org/en-US/docs/Glossary/Slug">slug</a> generator
716 library</li>
717 <li><a href="https://github.com/yuin/goldmark">yuin/goldmark</a>: a CommonMark parser and
718 renderer</li>
719 <li><a href="https://github.com/teekennedy/goldmark-markdown">teekennedy/goldmark-markdown</a>:
720 a renderer for Goldmark to render back to Markdown (since Goldmark itself
721 doesn't have this capacity)</li>
722 </ul>
723 <p>In common for all those modules are that they're clearly small projects
724 maintained mostly by one developer. They're all very good, don't get me wrong,
725 but they're still an reliability issue in the future. There is no guarantee
726 those repositories will not be deleted tomorrow, for example.</p>
727 <p>Yes, <a href="https://proxy.golang.org/">Go Proxy</a> exist, but from what I understood
728 reading its page is that while it caches modules contents, this is not
729 guarantee:</p>
730 <blockquote>
731 <p>proxy.golang.org does not save all modules forever. There are a number of
732 reasons for this, but one reason is if proxy.golang.org is not able to detect
733 a suitable license. In this case, only a temporarily cached copy of the
734 module will be made available, and may become unavailable if it is removed
735 from the original source and becomes outdated. The checksums will still
736 remain in the checksum database regardless of whether or not they have become
737 unavailable in the mirror.</p>
738 </blockquote>
739 <p>This is why this is the first project that made sense to me to use <a href="https://go.dev/ref/mod#go-mod-vendor"><code>go mod vendor</code></a>. Now I have a copy of the source
740 code of all modules inside the
741 <a href="https://github.com/thiagokokada/blog/tree/0b97630d6b30551ffe05b5d8124305b1065f729d/vendor">vendor</a>
742 directory in the repository, avoiding the risks I commented above. This allows
743 me to ensure that this blog will still be publishable in the future, as long as
744 I have a working Go toolchain (and Go toolchain makes this
745 <a href="https://kokada.dev/blog/quick-bits-go-automatically-downloads-a-newer-toolchain-if-needed">easy</a>).</p>
746 <p>There are a few other things that can bitrot this blog, for example links going
747 nowhere. I always try to use
748 <a href="https://en.wikipedia.org/wiki/Permalink">permalinks</a> when it makes sense, but
749 the only actual way to ensure those links would work in the future would be to
750 point them to <a href="https://archive.org/">archive.org</a> (but even archive.org may not
751 exist forever). Maybe something to fix in the future, hope not in the far
752 future, before things start to break.</p></summary></entry><entry><title>An unordered list of hidden gems inside NixOS</title><link href="https://kokada.dev/blog/an-unordered-list-of-hidden-gems-inside-nixos/" rel="alternate"/><published>2024-08-21T00:00:00+00:00</published><id>https://kokada.dev/blog/an-unordered-list-of-hidden-gems-inside-nixos/</id><summary type="html"><p>After using <a href="https://nixos.org/">NixOS</a> for the last 5+ years as my main OS, I
753 end up with a <a href="https://github.com/thiagokokada/nix-configs/">configuration</a>
754 with many things that are interesting for one reason or another, but it is not
755 listed anywhere (well, except if you are the kind of person that reads <code>man 5 configuration.nix</code> or the release notes in every release).</p>
756 <p>So kind in the same spirit as my <a href="https://kokada.dev/blog/an-unordered-list-of-things-i-miss-in-go">list of things that I miss in
757 Go</a>, here is
758 a list of modules that I find neat in NixOS and are not default already. Again,
759 the list is unordered since this makes it easier to update in the future if I
760 find something else, but also I don't want to think too hard about an order
761 here.</p>
762 <p>With all above, let's start.</p>
763 <h2><a href="https://github.com/NixOS/nixpkgs/blob/6afb255d976f85f3359e4929abd6f5149c323a02/nixos/modules/services/networking/nftables.nix"><code>networking.nftables</code></a></h2>
764 <p><a href="https://www.nftables.org/">nftables</a> is, accordingly to Wikipedia:</p>
765 <blockquote>
766 <p>nftables is a subsystem of the Linux kernel providing filtering and
767 classification of network packets/datagrams/frames.</p>
768 </blockquote>
769 <p>It is basically a replacement of the venerable
770 <a href="https://en.wikipedia.org/wiki/Iptables">iptables</a>, that still exist and is the
771 default program to configure the famous <code>networking.firewall</code>, the declarative
772 <a href="https://wiki.nixos.org/wiki/Firewall">Firewall</a> that NixOS enable by default.</p>
773 <p>To enable, it is simple, just add to your configuration:</p>
774 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
775 </span></span><span class="line"><span class="cl"> <span class="n">networking</span><span class="o">.</span><span class="n">nftables</span><span class="o">.</span><span class="n">enable</span> <span class="o">=</span> <span class="no">true</span><span class="p">;</span>
776 </span></span><span class="line"><span class="cl"><span class="p">}</span>
777 </span></span></code></pre><p>And thanks to the fact that NixOS's Firewall is declarative, everything should
778 still work as expect: any service that you set <code>openFirewall = true</code> will still
779 have its ports open, if you set <code>networking.firewall.allowPing = false</code> it will
780 still disable pings like before, etc.</p>
781 <p>If you look at the documentation of the above option, you will find the
782 following warning:</p>
783 <blockquote>
784 <p>Note that if you have Docker enabled you will not be able to use nftables
785 without intervention. Docker uses iptables internally to setup NAT for
786 containers. This module disables the ip_tables kernel module, however Docker
787 automatically loads the module. Please see
788 <a href="https://github.com/NixOS/nixpkgs/issues/24318#issuecomment-289216273">https://github.com/NixOS/nixpkgs/issues/24318#issuecomment-289216273</a> for
789 more information.</p>
790 </blockquote>
791 <p>I don't use Docker (switched to Podman instead for quite a long time), so I
792 don't know how bad the situation is. Also keep in mind that <code>nftables</code> does
793 offer <code>iptables-compat</code> for compatibility with old iptables scripts, so it is
794 most likely Docker doing something weird here.</p>
795 <p>Now, the actual advantage from the user here is not clear: the main advantage
796 from my point of view (and the reason I used to use in other distros like Arch)
797 is the improved syntax, however if you are using the declarative NixOS's
798 Firewall you are not interacting with either <code>iptables</code> or <code>nftables</code> directly
799 anyway. <code>nftables</code> is supposed to be more efficient, but not sure most users
800 will care about this.</p>
801 <p>However if you are the kind of person that needs custom rules, switching to
802 <code>nftables</code> does bring a few benefits, including
803 <code>networking.nftables.checkRuleset</code> (enabled by default), that checks if your
804 ruleset has syntax errors during build time. Really valuable to avoid issues
805 only after switch.</p>
806 <p>Anyway, this is one of those options that I think it should be the default for
807 a long time, since most of the new development in NixOS firewall seems to be
808 focusing <code>nftables</code> for a while.</p>
809 <h2><a href="https://github.com/NixOS/nixpkgs/blob/877d19523edcac81b167e8fd716ad2658da2adca/nixos/modules/system/activation/switchable-system.nix#L30-L38"><code>system.switch.enableNg</code></a></h2>
810 <p><a href="https://github.com/NixOS/nixpkgs/pull/308801">This one</a> I just discovered
811 today, but it has been available for a while (~2 months if you're using
812 <code>nixos-unstable</code>). Finally someone is rewriting
813 <a href="https://github.com/NixOS/nixpkgs/blob/b1eff03c35aa7c90ab3a4d9f6ef297dae5fba37b/nixos/modules/system/activation/switch-to-configuration.pl"><code>switch-to-configuration.pl</code></a>,
814 the Perl script that is called everytime you run <code>nixos-rebuild switch</code>.</p>
815 <p>Now, I am not one of those &quot;rewrite in Rust&quot; zealots, but in this case this is
816 definitely worth it: <code>switch-to-configuration.pl</code> is one of those pieces of
817 code in NixOS that most people avoid touching at the fear of breaking
818 something. There is a reason why
819 <a href="https://github.com/NixOS/nixpkgs/commit/eeb2588a59c938042b74183ce1da7052a6ef7e59"><code>nixos-rebuild</code></a>
820 is as convoluted as it is, because even if it is a messy shell script, most
821 people preferred to workaround issues from the <code>switch-to-configuration.pl</code>
822 inside it than trying to understand the mess that <code>switch-to-configuration.pl</code>
823 is.</p>
824 <p>Trying this one is easy:</p>
825 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
826 </span></span><span class="line"><span class="cl"> <span class="n">system</span><span class="o">.</span><span class="n">switch</span> <span class="o">=</span> <span class="p">{</span>
827 </span></span><span class="line"><span class="cl"> <span class="n">enable</span> <span class="o">=</span> <span class="no">false</span><span class="p">;</span>
828 </span></span><span class="line"><span class="cl"> <span class="n">enableNg</span> <span class="o">=</span> <span class="no">true</span><span class="p">;</span>
829 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
830 </span></span><span class="line"><span class="cl"><span class="p">}</span>
831 </span></span></code></pre><p>Yes, you need to explicit set <code>system.switch.enable = false</code>, since the default
832 is <code>true</code>.</p>
833 <p>By the way, what is the reason you would want to set <code>system.switch.enable = false</code> before the <code>enableNg</code> appeared you ask? For systems that are immutable
834 and updated by e.g.: image upgrades instead of modifying root.</p>
835 <p>Enabling <code>switch-to-configuration-ng</code> right now is mostly for testing purposes,
836 but one of the advantages that I saw is that system switches are (slightly)
837 faster:</p>
838 <pre><code>$ hyperfine &quot;sudo nixos-rebuild switch&quot; # switch-to-configuration.pl
839 Benchmark 1: sudo nixos-rebuild switch
840 Time (mean ± σ): 3.576 s ± 0.035 s [User: 0.004 s, System: 0.014 s]
841 Range (min … max): 3.522 s … 3.645 s 10 runs
842 </code><p><code>$ hyperfine &quot;sudo nixos-rebuild switch&quot; # switch-to-configuration-ng
843 Benchmark 1: sudo nixos-rebuild switch
844 Time (mean ± σ): 3.394 s ± 0.080 s [User: 0.004 s, System: 0.013 s]
845 Range (min … max): 3.325 s … 3.608 s 10 runs
846 </code></p></pre><p></p>
847 <p>But yes, the difference is not enough to make a significant impact, and it is
848 not the objective anyway. The real reason for the rewrite is to make it easier
849 to collaborate. I hope one day we also have someone brave enough to rewrite the
850 <code>nixos-rebuild</code> script in something saner.</p>
851 <h2><a href="https://github.com/NixOS/nixpkgs/blob/cce9aef6fd8f010d288d685b9d2a38f3b6ac47e9/nixos/modules/system/boot/systemd/initrd.nix"><code>boot.initrd.systemd</code></a></h2>
852 <p>A quick recap on how a modern Linux distro generally boots: the first thing
853 that the bootloader (say <a href="https://www.gnu.org/software/grub/">GRUB</a> or
854 <a href="https://systemd.io/BOOT/">systemd-boot</a>) loads is <code>initrd</code> (<em>initial
855 ramdisk</em>), a small image that runs from RAM and includes the Linux kernel and
856 some utilities that are responsible for setting up the main system. For
857 example, one of the responsibilities of the <code>initrd</code> is to mount the disks and
858 start init system (<code>systemd</code>).</p>
859 <p>It may surprising that this <code>initrd</code> image does <strong>not</strong> generally include
860 <code>systemd</code>. Traditionally <code>initrd</code> is composed by a bunch of shell scripts and a
861 minimal runtime (e.g.: <a href="https://www.busybox.net/">busybox</a>), however <code>systemd</code>
862 can also do this job since a long time ago. It is just the role of the distros
863 to integrate <code>systemd</code> inside the <code>initrd</code>.</p>
864 <p>This is what <code>boot.initrd.systemd</code> does: enable <code>systemd</code> inside the <code>initrd</code>.
865 It make a few subtle changes:</p>
866 <ul>
867 <li>If you are using <a href="https://wiki.nixos.org/wiki/Full_Disk_Encryption">Full Disk Encryption via
868 LUKS</a>, you will get a
869 different password prompt at login</li>
870 <li>You will get <code>initrd</code> time information if using <code>systemd-analyze</code> to measure
871 boot time
872 <ul>
873 <li>You can get even more information (bootloader) if you also use
874 <code>systemd-boot</code></li>
875 </ul>
876 </li>
877 <li>You will also get <code>systemd</code> style status about services during <code>initrd</code> (not
878 only afterwards)</li>
879 </ul>
880 <p>But I think the main reason is that since <code>systemd</code> is event-driven, it should
881 make boot more reliable, especially in challenging situations (like booting
882 from network). I can't say that I have any system like this to test if it is
883 actually more reliable or not, but I don't remember having any issues since I
884 set <code>boot.initrd.systemd.enable = true</code>, so there is that.</p>
885 <h2><a href="https://github.com/NixOS/nixpkgs/blob/b4a09f1f9d1599478afadffa782a02690550447c/pkgs/development/libraries/pipewire/default.nix"><code>services.pipewire</code></a></h2>
886 <p>If there is something in that list that has a good chance that you're using
887 already, it is this one, especially if you're using
888 <a href="https://wayland.freedesktop.org/">Wayland</a>. Still, I think it is interesting
889 to include in this list since <a href="https://www.pipewire.org/">PipeWire</a> is great.</p>
890 <p>The experience with PipeWire until now for me was seamless: I never had any
891 issues with it, all my applications still work exactly as it always worked. I
892 also didn't had any issues with
893 <a href="https://www.freedesktop.org/wiki/Software/PulseAudio/">PulseAudio</a> for a
894 while, but I still remember when I first tried PulseAudio during the 0.x in
895 Fedora and having tons of issues. So bonus points for PipeWire developers for
896 polishing the experience of enough that most people will feel no difference.</p>
897 <p>To enable PipeWire, I would recommend:</p>
898 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
899 </span></span><span class="line"><span class="cl"> <span class="n">services</span><span class="o">.</span><span class="n">pipewire</span> <span class="o">=</span> <span class="p">{</span>
900 </span></span><span class="line"><span class="cl"> <span class="n">enable</span> <span class="o">=</span> <span class="no">true</span><span class="p">;</span>
901 </span></span><span class="line"><span class="cl"> <span class="n">alsa</span><span class="o">.</span><span class="n">enable</span> <span class="o">=</span> <span class="no">true</span><span class="p">;</span>
902 </span></span><span class="line"><span class="cl"> <span class="n">pulse</span><span class="o">.</span><span class="n">enable</span> <span class="o">=</span> <span class="no">true</span><span class="p">;</span>
903 </span></span><span class="line"><span class="cl"> <span class="c1"># jack.enable = true;</span>
904 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
905 </span></span><span class="line"><span class="cl"> <span class="n">security</span><span class="o">.</span><span class="n">rtkit</span><span class="o">.</span><span class="n">enable</span> <span class="o">=</span> <span class="no">true</span><span class="p">;</span>
906 </span></span><span class="line"><span class="cl"><span class="p">}</span>
907 </span></span></code></pre><p>This enables both ALSA and PulseAudio emulation support in PipeWire for maximum
908 compatibility with desktop applications (you can also enable
909 <a href="https://jackaudio.org/"><code>jack</code></a> if you use professional audio applications).
910 It also enables <a href="https://github.com/heftig/rtkit"><code>rtkit</code></a>, allowing PipeWire
911 to get (soft) realtime, helping avoiding cracks during high CPU load.</p>
912 <p>I also recommend taking a look at the <a href="https://wiki.nixos.org/wiki/PipeWire">Wiki
913 article</a>, that has multiple interesting
914 configurations that can be added for low-latency setups or improved codecs for
915 Bluetooth devices.</p>
916 <h2><a href="https://github.com/NixOS/nixpkgs/blob/c9ec8289781a3c4ac4dd5c42c8d50dd65360e79c/nixos/modules/services/networking/networkmanager.nix#L264-L271"><code>networking.networkmanager.wifi.backend = &quot;iwd&quot;</code></a></h2>
917 <p>There is a good change that you're using
918 <a href="https://www.networkmanager.dev/"><code>NetworkManager</code></a> to manage network,
919 especially for Wi-Fi. And if that is the case, I can't recommend enough
920 changing the backend from the default <code>wpa_supplicant</code> to
921 <a href="https://iwd.wiki.kernel.org/"><code>iwd</code></a>.</p>
922 <p>If you think that your Wi-Fi takes a long time to connect/re-connect, it may be
923 because <code>wpa_supplicant</code>. <code>iwd</code> seems much more optimised in this regard, and
924 since switching to it I never felt that my Wi-Fi was worse than other OSes (and
925 generally slightly better than Windows, but keep in mind that this is a
926 non-scientific comparison).</p>
927 <p>Not saying that I never had Wi-Fi issues since switching to <code>iwd</code>, however
928 switching back to <code>wpa_supplicant</code> in those cases never fixed the issue (it was
929 the same or worse), so I assume either bad hardware or drivers in those cases.</p>
930 <h2><a href="https://github.com/NixOS/nixpkgs/blob/d5badef1e0416160298aad6dd8604eaf8b6e7e52/nixos/modules/system/boot/tmp.nix#L25-L32"><code>boot.tmp.useTmpfs</code></a></h2>
931 <p><em>Added in 2024-08-22</em></p>
932 <p>Most Linux distro nowadays mount <code>/tmp</code> inside
933 <a href="https://en.wikipedia.org/wiki/Tmpfs">tmpfs</a>, storing the files inside RAM (and
934 making them actually temporary, e.g.: does not persist between reboots). There
935 is a <a href="https://lwn.net/Articles/499410/">long discussion</a> whether this makes
936 sense or not, but it is a fact that using <code>/tmp</code> as an storage for small files
937 generally makes sense.</p>
938 <p>However, NixOS still stores <code>/tmp</code> inside <code>/</code> by default. The main reason for
939 this is because Nix, by default, still builds everything in <code>TMPDIR</code> (that
940 defaults to <code>/tmp</code> if not set). To fix the situation you can do:</p>
941 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
942 </span></span><span class="line"><span class="cl"> <span class="n">boot</span><span class="o">.</span><span class="n">tmp</span><span class="o">.</span><span class="n">useTmpfs</span> <span class="o">=</span> <span class="no">true</span><span class="p">;</span>
943 </span></span><span class="line"><span class="cl"> <span class="n">systemd</span><span class="o">.</span><span class="n">services</span><span class="o">.</span><span class="n">nix-daemon</span> <span class="o">=</span> <span class="p">{</span>
944 </span></span><span class="line"><span class="cl"> <span class="n">environment</span><span class="o">.</span><span class="n">TMPDIR</span> <span class="o">=</span> <span class="s2">&#34;/var/tmp&#34;</span><span class="p">;</span>
945 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
946 </span></span><span class="line"><span class="cl"><span class="p">}</span>
947 </span></span></code></pre><p>This ensures that Nix will build in <code>/var/tmp</code>, that is the temporary directory
948 for large files.</p>
949 <p>If you don't want to use <code>/tmp</code> inside tmpfs for some reason, I recommend at
950 least setting <code>boot.tmp.cleanOnBoot = true</code>. This is to avoid issues with e.g.:
951 mistakenly writing a program that depends in <code>/tmp</code> being persistent, because
952 this is the behavior that most other distros expects.</p>
953 <h2><a href="https://github.com/NixOS/nixpkgs/blob/68142254d20256e617f596a5d50f9950f98fa36a/nixos/modules/config/zram.nix"><code>zramSwap</code></a></h2>
954 <p><em>Added in 2024-08-22</em></p>
955 <p>Have a system with small amounts of RAM? <del>You can download more RAM for free,
956 just click this <a href="https://downloadmoreram.com/">link</a>.</del> Well, just kidding, but
957 you can actually improve RAM usage by compressing it. This is what
958 <a href="https://en.wikipedia.org/wiki/Zram"><code>zram</code></a> does: it will create a block
959 device using part of your RAM that will be compressed.</p>
960 <p>While this means that whatever is in the compressed part can't be used as-is
961 (it needs to be decompressed first), it means you can store a lot more
962 information. And since RAM is much faster than disk, this generally is a good
963 trade-off. It is still recommended that you have some swap space though, e.g.:
964 to swap pages that are unused.</p>
965 <p>To enable it, you can do:</p>
966 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
967 </span></span><span class="line"><span class="cl"> <span class="n">zramSwap</span> <span class="o">=</span> <span class="p">{</span>
968 </span></span><span class="line"><span class="cl"> <span class="n">enable</span> <span class="o">=</span> <span class="no">true</span><span class="p">;</span>
969 </span></span><span class="line"><span class="cl"> <span class="n">algorithm</span> <span class="o">=</span> <span class="s2">&#34;zstd&#34;</span><span class="p">;</span>
970 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
971 </span></span><span class="line"><span class="cl"><span class="p">}</span>
972 </span></span></code></pre><p>You can play a little with the <code>zramSwap.algorithm</code> parameter, but I recommend
973 either <code>lz4</code> or <code>zstd</code>. It is important for the compression algorithm to be
974 fast, since you probably want to minimise CPU usage. Well, unless you have a
975 very good CPU and small amounts of RAM, them it may make sense to use an
976 algorithm that compress better but uses more CPU.</p>
977 <h2><a href="https://github.com/NixOS/nixpkgs/blob/5356420466c4d7901b63acc5e337c5bf30573f8a/nixos/modules/services/misc/fstrim.nix"><code>services.fstrim</code></a></h2>
978 <p><em>Added in 2024-08-22</em></p>
979 <p>Do you have a SSD? If so, you probably want to
980 <a href="https://en.wikipedia.org/wiki/Trim_(computing)">trim</a> it regularly to keep a
981 good performance. This should be as easy as:</p>
982 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
983 </span></span><span class="line"><span class="cl"> <span class="n">services</span><span class="o">.</span><span class="n">fstrim</span><span class="o">.</span><span class="n">enable</span> <span class="o">=</span> <span class="no">true</span><span class="p">;</span>
984 </span></span><span class="line"><span class="cl"><span class="p">}</span>
985 </span></span></code></pre><p>One caveat though: if you are using Full Disk Encryption with LUKS, you also
986 need to enable <code>boot.initrd.luks.devices.&lt;name&gt;.allowDiscards</code>, otherwise
987 <code>fstrim</code> will not work. The reason that this is not enabled by default is
988 because there are some <a href="https://asalor.blogspot.com/2011/08/trim-dm-crypt-problems.html">security
989 implications</a>.
990 It shouldn't be an issue for most people, but if you are paranoid please assess
991 your risks first.</p>
992 <h2><a href="https://github.com/NixOS/nixpkgs/blob/e83fc891118e5e5660ccca000040e33b0a0a3080/nixos/modules/system/boot/binfmt.nix"><code>boot.binfmt.emulatedSystems</code></a></h2>
993 <p><em>Added in 2024-08-22</em></p>
994 <p>This is one of my favorites. Do you want to have the power to run binaries from
995 other architectures like
996 <a href="https://en.wikipedia.org/wiki/ARM_architecture_family">ARM</a> or
997 <a href="https://en.wikipedia.org/wiki/RISC-V">RISC-V</a>? In NixOS it is easy:</p>
998 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
999 </span></span><span class="line"><span class="cl"> <span class="n">boot</span><span class="o">.</span><span class="n">binfmt</span><span class="o">.</span><span class="n">emulatedSystems</span> <span class="o">=</span> <span class="p">[</span> <span class="s2">&#34;aarch64-linux&#34;</span> <span class="s2">&#34;riscv64-linux&#34;</span> <span class="p">];</span>
1000 </span></span><span class="line"><span class="cl"><span class="p">}</span>
1001 </span></span></code></pre><p>This will install a QEMU emulator for the target architectures, and configure
1002 <a href="https://en.wikipedia.org/wiki/Binfmt_misc"><code>binfmt</code></a> so that the kernel will
1003 automatically detect when a binary from another architecture is run,
1004 running it with the QEMU emulator instead.</p>
1005 <p>Now, keep in mind that you still need e.g.: libraries for the target
1006 architecture. However this is where the Nix magic comes in, you can easily
1007 compile something to another architecture as:</p>
1008 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> uname -a
1009 </span></span><span class="line"><span class="cl"><span class="go">Linux sankyuu-nixos 6.10.3 #1-NixOS SMP PREEMPT_DYNAMIC Sat Aug 3 07:01:09 UTC 2024 x86_64 GNU/Linux
1010 </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="gp">$</span> <span class="nb">cd</span> nixpkgs
1011 </span></span><span class="line"><span class="cl"><span class="gp">$</span> nix build .#hello --system aarch64-linux
1012 </span></span><span class="line"><span class="cl"><span class="gp">$</span> file ./result/bin/hello
1013 </span></span><span class="line"><span class="cl"><span class="go">./result/bin/hello: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /nix/store/kz7xglxzhad64v667wwpn8vrxhjwcbna-glibc-2.39-52/lib/ld-linux-aarch64.so.1, for GNU/Linux 3.10.0, not stripped
1014 </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="gp">$</span> ./result/bin/hello
1015 </span></span><span class="line"><span class="cl"><span class="go">Hello, world!
1016 </span></span></span></code></pre><p>And since the binary will be linked with the libraries from the target
1017 architecture, everything works as expect.</p>
1018 <p>You also need to temper your expectations: QEMU emulation is slow. It is
1019 sufficient fast to build small programs, but if you need to build something
1020 bigger, I would recommend you to do it via <a href="https://kokada.dev/blog/building-static-binaries-in-nix">cross-compilation
1021 instead</a>.</p>
1022 <h2><a href="https://github.com/NixOS/nixpkgs/blob/d9cf6ea6f809b862b73766705bdadb1236eeeff8/nixos/modules/services/system/dbus.nix#L40-L50"><code>services.dbus.implementation = &quot;broker&quot;</code></a></h2>
1023 <p><em>Added in 2024-08-22</em></p>
1024 <p>Someone in
1025 <a href="https://lobste.rs/s/ymmale/unordered_list_hidden_gems_inside_nixos#c_dz1hck">Lobste.rs</a>
1026 remembered me of this gem: <a href="https://github.com/bus1/dbus-broker">dbus-broker</a>,
1027 a high performance implementation of
1028 <a href="https://en.wikipedia.org/wiki/D-Bus">D-Bus</a>.</p>
1029 <p>For those who doesn't know, D-Bus is the <a href="https://en.wikipedia.org/wiki/Inter-process_communication">Inter-Process
1030 Communication</a>
1031 protocol used mostly in Linux desktop. And distros like
1032 <a href="https://archlinux.org/news/making-dbus-broker-our-default-d-bus-daemon/">Arch</a>
1033 and
1034 <a href="https://fedoraproject.org/wiki/Changes/DbusBrokerAsTheDefaultDbusImplementation">Fedora</a>
1035 already use it to default, so there aren't lots of reason to not use it.</p>
1036 <h2><a href="https://github.com/NixOS/nixpkgs/blob/6afb255d976f85f3359e4929abd6f5149c323a02/nixos/modules/services/hardware/irqbalance.nix"><code>services.irqbalance</code></a></h2>
1037 <p><em>Added in 2024-08-23</em></p>
1038 <p>Here is one that I have no experience but got curious after this comment in
1039 <a href="https://lobste.rs/s/ymmale/unordered_list_hidden_gems_inside_nixos#c_aghpwy">Lobte.rs</a>:</p>
1040 <blockquote>
1041 <p>One more option to add is maybe <code>services.irqbalance.enable</code>. For my old
1042 laptop it’s a life savior: no more freezes of the DE or video playback during
1043 heavy operations like e.g. compilation.</p>
1044 </blockquote>
1045 <p>Never tried it before but will give a try!</p></summary></entry><entry><title>Quick bits: using forks with Go modules</title><link href="https://kokada.dev/blog/quick-bits-using-forks-with-go-modules/" rel="alternate"/><published>2024-08-20T00:00:00+00:00</published><id>https://kokada.dev/blog/quick-bits-using-forks-with-go-modules/</id><summary type="html"><p>There are 2 types of blog posts: the ones you write for others, and the ones
1046 that you write for yourself. This blog post is the latter kind. What I am going
1047 to talk here is probably something most people know, but I didn't, and the
1048 documentation is all scattered in the internet. So I am writing mostly to
1049 remember myself, in future, if I have the same kind of issue what I need to do.</p>
1050 <p>The context: <a href="https://mataroa.blog/">Mataroa</a>, the blog platform
1051 <a href="https://capivaras.dev">capivaras.dev</a> is hosted on, relies in
1052 <a href="https://python-markdown.github.io/">python-markdown</a>, predates
1053 <a href="https://commonmark.org/">CommonMark</a> specification and as such, has some
1054 incompatibilities. One of those incompatibilities with CommonMark is the way
1055 sub lists are handled. From the documentation:</p>
1056 <blockquote>
1057 <p>The syntax rules clearly state that when a list item consists of multiple
1058 paragraphs, “each subsequent paragraph in a list item must be indented by
1059 either 4 spaces or one tab” (emphasis added). However, many implementations
1060 do not enforce this rule and allow less than 4 spaces of indentation. The
1061 implementers of Python-Markdown consider it a bug to not enforce this rule.</p>
1062 </blockquote>
1063 <p>CommonMark <a href="https://spec.commonmark.org/0.31.2/#lists">relax those
1064 restrictions</a>, allowing a sublist to
1065 be defined with just 2 spaces of indentation.</p>
1066 <p>So I have automated all posts from this blog <a href="https://kokada.dev/blog/quick-bits-why-you-should-automate-everything">using
1067 Go</a> and
1068 a CommonMark renderer called <a href="https://github.com/yuin/goldmark/">Goldmark</a>. I
1069 them re-render the Markdown to Mataroa using a <a href="https://github.com/teekennedy/goldmark-markdown">Markdown
1070 renderer</a> before publising to
1071 <a href="https://capivaras.dev">capivaras.dev</a>, because this allow me to do some
1072 transformations in the original Markdown. It mostly works fine except for sub
1073 lists, thanks to the fact that the Markdown renderer I am using renders sub
1074 lists with 2 spaces.</p>
1075 <p>The only reason sub lists are working right now is because
1076 <a href="https://gluer.org/">@ratsclub</a> fixed this issue in the fork that
1077 <a href="https://capivaras.dev">capivaras.dev</a> runs. But I want to be compatible with
1078 the official instance if I ever need to migrate.</p>
1079 <p>The solution? Let's fix this in a
1080 <a href="https://github.com/teekennedy/goldmark-markdown/pull/21">PR</a>. However now that
1081 I have code to fix the issue, how can I use it without waiting upstream to
1082 merge my code?</p>
1083 <p>If you are using Go modules it is easy, you just need to use the <a href="https://go.dev/ref/mod#go-mod-file-replace"><code>replace</code>
1084 directive</a>:</p>
1085 <pre class="chroma"><code><span class="line"><span class="cl"><span class="nx">module</span> <span class="nx">github</span><span class="p">.</span><span class="nx">com</span><span class="o">/</span><span class="nx">thiagokokada</span><span class="o">/</span><span class="nx">blog</span>
1086 </span></span><span class="line"><span class="cl">
1087 </span></span><span class="line"><span class="cl"><span class="k">go</span> <span class="mf">1.23</span>
1088 </span></span><span class="line"><span class="cl">
1089 </span></span><span class="line"><span class="cl"><span class="nf">require</span> <span class="p">(</span>
1090 </span></span><span class="line"><span class="cl"> <span class="nx">github</span><span class="p">.</span><span class="nx">com</span><span class="o">/</span><span class="nx">elliotchance</span><span class="o">/</span><span class="nx">orderedmap</span><span class="o">/</span><span class="nx">v2</span> <span class="nx">v2</span><span class="mf">.4.0</span>
1091 </span></span><span class="line"><span class="cl"> <span class="nx">github</span><span class="p">.</span><span class="nx">com</span><span class="o">/</span><span class="nx">gorilla</span><span class="o">/</span><span class="nx">feeds</span> <span class="nx">v1</span><span class="mf">.2.0</span>
1092 </span></span><span class="line"><span class="cl"> <span class="nx">github</span><span class="p">.</span><span class="nx">com</span><span class="o">/</span><span class="nx">gosimple</span><span class="o">/</span><span class="nx">slug</span> <span class="nx">v1</span><span class="mf">.14.0</span>
1093 </span></span><span class="line"><span class="cl"> <span class="nx">github</span><span class="p">.</span><span class="nx">com</span><span class="o">/</span><span class="nx">teekennedy</span><span class="o">/</span><span class="nx">goldmark</span><span class="o">-</span><span class="nx">markdown</span> <span class="nx">v0</span><span class="mf">.3.0</span>
1094 </span></span><span class="line"><span class="cl"> <span class="nx">github</span><span class="p">.</span><span class="nx">com</span><span class="o">/</span><span class="nx">yuin</span><span class="o">/</span><span class="nx">goldmark</span> <span class="nx">v1</span><span class="mf">.7.4</span>
1095 </span></span><span class="line"><span class="cl"> <span class="nx">github</span><span class="p">.</span><span class="nx">com</span><span class="o">/</span><span class="nx">yuin</span><span class="o">/</span><span class="nx">goldmark</span><span class="o">-</span><span class="nx">highlighting</span> <span class="nx">v0</span><span class="mf">.0.0</span><span class="o">-</span><span class="mi">20220208100518</span><span class="o">-</span><span class="mi">594</span><span class="nx">be1970594</span>
1096 </span></span><span class="line"><span class="cl"><span class="p">)</span>
1097 </span></span><span class="line"><span class="cl">
1098 </span></span><span class="line"><span class="cl"><span class="nf">require</span> <span class="p">(</span>
1099 </span></span><span class="line"><span class="cl"> <span class="nx">github</span><span class="p">.</span><span class="nx">com</span><span class="o">/</span><span class="nx">alecthomas</span><span class="o">/</span><span class="nx">chroma</span> <span class="nx">v0</span><span class="mf">.10.0</span> <span class="c1">// indirect
1100 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="nx">github</span><span class="p">.</span><span class="nx">com</span><span class="o">/</span><span class="nx">dlclark</span><span class="o">/</span><span class="nx">regexp2</span> <span class="nx">v1</span><span class="mf">.11.4</span> <span class="c1">// indirect
1101 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="nx">github</span><span class="p">.</span><span class="nx">com</span><span class="o">/</span><span class="nx">gosimple</span><span class="o">/</span><span class="nx">unidecode</span> <span class="nx">v1</span><span class="mf">.0.1</span> <span class="c1">// indirect
1102 </span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="p">)</span>
1103 </span></span><span class="line"><span class="cl">
1104 </span></span><span class="line"><span class="cl"><span class="nx">replace</span> <span class="nx">github</span><span class="p">.</span><span class="nx">com</span><span class="o">/</span><span class="nx">teekennedy</span><span class="o">/</span><span class="nx">goldmark</span><span class="o">-</span><span class="nx">markdown</span> <span class="p">=&gt;</span> <span class="nx">github</span><span class="p">.</span><span class="nx">com</span><span class="o">/</span><span class="nx">thiagokokada</span><span class="o">/</span><span class="nx">goldmark</span><span class="o">-</span><span class="nx">markdown</span> <span class="nx">v0</span><span class="mf">.0.0</span><span class="o">-</span><span class="mi">20240820111219</span><span class="o">-</span><span class="nx">f30775d8ed15</span>
1105 </span></span></code></pre><p>This will replace all usages of <code>github.com/teekennedy/goldmark-markdown</code> to my
1106 fork in <code>github.com/thiagokokada/goldmark-markdown</code>. You even get all the
1107 reproducibility of modules since Go automatically pins the commit.</p>
1108 <p>Since the Go format for versions is quite unique, you can just set to the
1109 desired branch (e.g.: instead of <code>v0.0.0-20240820111219-f30775d8ed15</code>, you can
1110 use <code>add-sublist-length-opt</code> that is the branch name) and run <code>go mod tidy</code> to
1111 fix the format.</p></summary></entry><entry><title>Quick bits: Go automatically downloads a newer toolchain if needed</title><link href="https://kokada.dev/blog/quick-bits-go-automatically-downloads-a-newer-toolchain-if-needed/" rel="alternate"/><published>2024-08-19T00:00:00+00:00</published><id>https://kokada.dev/blog/quick-bits-go-automatically-downloads-a-newer-toolchain-if-needed/</id><summary type="html"><p>I am using
1112 <a href="https://github.com/elliotchance/orderedmap/">elliotchance/orderedmap</a> as my
1113 choice of ordered maps (since Go <a href="https://kokada.dev/blog/an-unordered-list-of-things-i-miss-in-go">doesn't have
1114 one</a> in
1115 standard library yet). I recently did a
1116 <a href="https://github.com/elliotchance/orderedmap/pull/41">PR</a> to implement Go 1.23
1117 iterators, because I find them neat, however I was avoiding to use it in the
1118 code that generates this <a href="https://github.com/thiagokokada/blog">blog</a> since Go
1119 1.23 was just released and is still not the default Go in
1120 <a href="https://github.com/NixOS/nixpkgs">nixpkgs</a>.</p>
1121 <p>I decided that I would create a
1122 <a href="https://github.com/thiagokokada/blog/pull/2">branch</a> and leave there for a few
1123 months, until I decided to try to run the code locally and got this:</p>
1124 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> go build
1125 </span></span><span class="line"><span class="cl"><span class="go">go: downloading go1.23.0 (darwin/arm64)
1126 </span></span></span></code></pre><p>Nice. And before you ask, yes, the compiled binary works perfectly:</p>
1127 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> make
1128 </span></span><span class="line"><span class="cl"><span class="go">./blog &gt; README.md
1129 </span></span></span><span class="line"><span class="cl"><span class="go">./blog -rss &gt; rss.xml
1130 </span></span></span></code></pre><p>So how does this work? Take a look at the documentation in the official <a href="https://tip.golang.org/doc/toolchain">Golang
1131 page</a>:</p>
1132 <blockquote>
1133 <p>Starting in Go 1.21, the Go distribution consists of a go command and a
1134 bundled Go toolchain, which is the standard library as well as the compiler,
1135 assembler, and other tools. The go command can use its bundled Go toolchain
1136 as well as other versions that it finds in the local PATH or downloads as
1137 needed.</p>
1138 </blockquote>
1139 <p>There are a bunch of rules here that I am not going to enter in detail (I
1140 recommend you to read the official documentation), but a quick summary:</p>
1141 <ul>
1142 <li>
1143 <p>Go will download a toolchain when either <code>go</code> or <code>toolchain</code> lines <code>go.mod</code>
1144 is set to a Go version higher than your current <code>go</code> binary</p>
1145 <ul>
1146 <li>But only if your <code>go</code> binary is at least version 1.21, since this is the
1147 version that introduces this behavior</li>
1148 </ul>
1149 </li>
1150 <li>
1151 <p>You can force a specific toolchain with <code>GOTOOLCHAIN</code> environment setting,
1152 e.g.: <code>GOTOOLCHAIN=1.23</code></p>
1153 <ul>
1154 <li>The default value for <code>GOTOOLCHAIN</code> is <code>auto</code>, that basically has the
1155 behavior described in this post</li>
1156 <li>You can also set to <code>local</code> to always use the current <code>go</code> binary, or the
1157 previous behaviour pre-1.21 Go</li>
1158 <li>There is also <code>&lt;name&gt;+auto</code> and <code>path</code> options, that can be seen in the
1159 docs</li>
1160 </ul>
1161 </li>
1162 <li>
1163 <p>The downloaded toolchains go to whatever your <code>GOPATH</code> is, inside
1164 <code>golang.org/toolchain</code> module, and version <code>v0.0.1-goVERSION.GOOS-GOARCH</code>, for
1165 example:</p>
1166 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> ls -lah <span class="nv">$GOPATH</span>/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.darwin-arm64
1167 </span></span><span class="line"><span class="cl"><span class="go">total 64
1168 </span></span></span><span class="line"><span class="cl"><span class="go">dr-xr-xr-x 14 user staff 448B 19 Aug 12:01 .
1169 </span></span></span><span class="line"><span class="cl"><span class="go">drwxr-xr-x 4 user staff 128B 19 Aug 12:01 ..
1170 </span></span></span><span class="line"><span class="cl"><span class="go">-r--r--r-- 1 user staff 1.3K 19 Aug 12:01 CONTRIBUTING.md
1171 </span></span></span><span class="line"><span class="cl"><span class="go">-r--r--r-- 1 user staff 1.4K 19 Aug 12:01 LICENSE
1172 </span></span></span><span class="line"><span class="cl"><span class="go">-r--r--r-- 1 user staff 1.3K 19 Aug 12:01 PATENTS
1173 </span></span></span><span class="line"><span class="cl"><span class="go">-r--r--r-- 1 user staff 1.4K 19 Aug 12:01 README.md
1174 </span></span></span><span class="line"><span class="cl"><span class="go">-r--r--r-- 1 user staff 426B 19 Aug 12:01 SECURITY.md
1175 </span></span></span><span class="line"><span class="cl"><span class="go">-r--r--r-- 1 user staff 35B 19 Aug 12:01 VERSION
1176 </span></span></span><span class="line"><span class="cl"><span class="go">dr-xr-xr-x 4 user staff 128B 19 Aug 12:01 bin
1177 </span></span></span><span class="line"><span class="cl"><span class="go">-r--r--r-- 1 user staff 52B 19 Aug 12:01 codereview.cfg
1178 </span></span></span><span class="line"><span class="cl"><span class="go">-r--r--r-- 1 user staff 505B 19 Aug 12:01 go.env
1179 </span></span></span><span class="line"><span class="cl"><span class="go">dr-xr-xr-x 3 user staff 96B 19 Aug 12:01 lib
1180 </span></span></span><span class="line"><span class="cl"><span class="go">dr-xr-xr-x 4 user staff 128B 19 Aug 12:01 pkg
1181 </span></span></span><span class="line"><span class="cl"><span class="go">dr-xr-xr-x 77 user staff 2.4K 19 Aug 12:02 src
1182 </span></span></span></code></pre></li>
1183 </ul>
1184 <p>By the way, this only works well because Go toolchain binaries are static.</p>
1185 <p>While I don't like a program downloading random binaries from the internet, I
1186 like what Go is doing here. It makes the whole bootstrapping process for a Go
1187 project much easier: as long as you have a reasonable up-to-date <code>go</code> binary in
1188 your <code>PATH</code>, you should be ready to go (pun intended). And Go modules are
1189 already reasonable secure, ensuring that each module have a proper checksum. As
1190 long as nobody else can publish modules in <code>golang.org/toolchain</code> namespace I
1191 can't see much of a security issue here, but I am not a security expert.</p>
1192 <p>But if you don't like this behavior, you can always disable it by setting
1193 <code>GOTOOLCHAIN=local</code>. And just do not forget to set this in your
1194 <a href="https://brandur.org/fragments/go-version-matrix">CI</a>, unless you don't care
1195 about Go versions.</p></summary></entry><entry><title>An unordered list of things I miss in Go</title><link href="https://kokada.dev/blog/an-unordered-list-of-things-i-miss-in-go/" rel="alternate"/><published>2024-08-17T00:00:00+00:00</published><id>https://kokada.dev/blog/an-unordered-list-of-things-i-miss-in-go/</id><summary type="html"><p>I like Go. I think it is a <a href="https://kokada.dev/blog/go-a-reasonable-good-language">reasonable good
1196 language</a>, and has some
1197 good qualities that makes up for its flaws. However, this doesn't mean I think
1198 the language couldn't be better, far from it.</p>
1199 <p>This blog post is a list of things that I miss from Go from other languages.
1200 Some of the things here could probably be implemented soon, some other would
1201 probably need a major revision of the language. The list is unordered, because
1202 this makes it easier for me to update in the future if I found something else,
1203 but also because I don't want to think too hard about giving each point here a
1204 rank.</p>
1205 <p>With all above, let's start.</p>
1206 <h2>Ordered maps in standard library</h2>
1207 <p>When I first learned about
1208 <a href="https://docs.python.org/3/library/stdtypes.html#typesmapping">dictionaries</a> in
1209 Python it quickly became one of my favorite data structures ever. They're
1210 extremely versatile, and most modern programming languages have something
1211 similar in its standard library. Go isn't different, it has
1212 <a href="https://go.dev/blog/maps"><code>map</code></a>, that is Go implementation of a <a href="https://en.wikipedia.org/wiki/Hash_table">hash
1213 table</a>. However <code>map</code> in Go are
1214 quirky, for example:</p>
1215 <pre class="chroma"><code><span class="line"><span class="cl"><span class="kn">package</span> <span class="nx">main</span>
1216 </span></span><span class="line"><span class="cl">
1217 </span></span><span class="line"><span class="cl"><span class="kd">func</span> <span class="nf">main</span><span class="p">()</span> <span class="p">{</span>
1218 </span></span><span class="line"><span class="cl"> <span class="nx">m</span> <span class="o">:=</span> <span class="kd">map</span><span class="p">[</span><span class="kt">string</span><span class="p">]</span><span class="kt">bool</span><span class="p">{</span><span class="s">&#34;foo&#34;</span><span class="p">:</span> <span class="kc">true</span><span class="p">,</span> <span class="s">&#34;bar&#34;</span><span class="p">:</span> <span class="kc">false</span><span class="p">,</span> <span class="s">&#34;baz&#34;</span><span class="p">:</span> <span class="kc">true</span><span class="p">,</span> <span class="s">&#34;qux&#34;</span><span class="p">:</span> <span class="kc">false</span><span class="p">,</span> <span class="s">&#34;quux&#34;</span><span class="p">:</span> <span class="kc">true</span><span class="p">}</span>
1219 </span></span><span class="line"><span class="cl">
1220 </span></span><span class="line"><span class="cl"> <span class="k">for</span> <span class="nx">k</span> <span class="o">:=</span> <span class="k">range</span> <span class="nx">m</span> <span class="p">{</span>
1221 </span></span><span class="line"><span class="cl"> <span class="nb">println</span><span class="p">(</span><span class="nx">k</span><span class="p">)</span>
1222 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
1223 </span></span><span class="line"><span class="cl"><span class="p">}</span>
1224 </span></span></code></pre><pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> go run ./test.go
1225 </span></span><span class="line"><span class="cl"><span class="go">bar
1226 </span></span></span><span class="line"><span class="cl"><span class="go">baz
1227 </span></span></span><span class="line"><span class="cl"><span class="go">qux
1228 </span></span></span><span class="line"><span class="cl"><span class="go">quux
1229 </span></span></span><span class="line"><span class="cl"><span class="go">foo
1230 </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="err">
1231 </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="gp">$</span> go run ./test.go
1232 </span></span><span class="line"><span class="cl"><span class="go">foo
1233 </span></span></span><span class="line"><span class="cl"><span class="go">bar
1234 </span></span></span><span class="line"><span class="cl"><span class="go">baz
1235 </span></span></span><span class="line"><span class="cl"><span class="go">qux
1236 </span></span></span><span class="line"><span class="cl"><span class="go">quux
1237 </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="err">
1238 </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="gp">$</span> go run ./test.go
1239 </span></span><span class="line"><span class="cl"><span class="go">qux
1240 </span></span></span><span class="line"><span class="cl"><span class="go">quux
1241 </span></span></span><span class="line"><span class="cl"><span class="go">foo
1242 </span></span></span><span class="line"><span class="cl"><span class="go">bar
1243 </span></span></span><span class="line"><span class="cl"><span class="go">baz
1244 </span></span></span></code></pre><p>Now, I don't expect any hash table implementation to keep the order of the
1245 elements, but Go actually <a href="https://victoriametrics.com/blog/go-map/">randomise each map
1246 instance</a>:</p>
1247 <blockquote>
1248 <p>But here’s the deal, while the hash function used for maps in Go is
1249 consistent across all maps with <strong>the same key type</strong>, the <code>seed</code> used by
1250 that hash function is different for each map instance. So, when you create a
1251 new map, Go generates a random seed just for that map.</p>
1252 </blockquote>
1253 <p>While I understand the reason for this (i.e.: to avoid developers relying in a
1254 specific iteration order), I still find it weird, and I think this is something
1255 unique for Go. This decision means that even if you don't care about a specific
1256 order, you will still need to sort the map before doing something else if you
1257 want reproducibility, something that I care a lot.</p>
1258 <p>The fix for this? Go could offer an ordered map implementation inside the
1259 standard library. An ordered map ensure that the iteration order of the map is
1260 the same as the insertion order (that is, by the way, a powerful property that
1261 allow maps to be used in other contexts, not just my pet peeve above).</p>
1262 <p>Python actually does this for any dictionaries since <a href="https://stackoverflow.com/a/39980744">Python
1263 3.6</a>, but it offered an
1264 <a href="https://docs.python.org/3/library/collections.html#collections.OrderedDict">OrderedDict</a>
1265 before it (and <code>OrderedDict</code> still has some methods that normal <code>dict</code> doesn't,
1266 that maybe useful in specific cases).</p>
1267 <p>Before generics it would be impossible to have a type-safe API for such data
1268 structure without introducing a new data type in the language (like <code>slices</code>),
1269 but now Go has generics so it is not an issue anymore. The other issue is that
1270 you would be forced to iterate manually in this new data structure, but thanks
1271 to the new <a href="https://tip.golang.org/doc/go1.23#language"><code>range-over-func</code></a> in
1272 Go 1.23, it means we can iterate in an ordered map as a library almost exactly
1273 like we can do as a <code>map</code>:</p>
1274 <pre class="chroma"><code><span class="line"><span class="cl"><span class="kn">import</span> <span class="s">&#34;orderedmap&#34;</span>
1275 </span></span><span class="line"><span class="cl">
1276 </span></span><span class="line"><span class="cl"><span class="kd">func</span> <span class="nf">main</span><span class="p">()</span> <span class="p">{</span>
1277 </span></span><span class="line"><span class="cl"> <span class="nx">m</span> <span class="o">:=</span> <span class="nx">orderedmap</span><span class="p">.</span><span class="nx">New</span><span class="p">[</span><span class="kt">string</span><span class="p">,</span> <span class="kt">bool</span><span class="p">]()</span>
1278 </span></span><span class="line"><span class="cl"> <span class="nx">m</span><span class="p">.</span><span class="nf">Set</span><span class="p">(</span><span class="s">&#34;foo&#34;</span><span class="p">,</span> <span class="kc">true</span><span class="p">)</span>
1279 </span></span><span class="line"><span class="cl"> <span class="nx">m</span><span class="p">.</span><span class="nf">Set</span><span class="p">(</span><span class="s">&#34;bar&#34;</span><span class="p">,</span> <span class="kc">false</span><span class="p">)</span>
1280 </span></span><span class="line"><span class="cl"> <span class="nx">m</span><span class="p">.</span><span class="nf">Set</span><span class="p">(</span><span class="s">&#34;baz&#34;</span><span class="p">,</span> <span class="kc">true</span><span class="p">)</span>
1281 </span></span><span class="line"><span class="cl">
1282 </span></span><span class="line"><span class="cl"> <span class="k">for</span> <span class="nx">k</span> <span class="o">:=</span> <span class="k">range</span> <span class="nx">m</span><span class="p">.</span><span class="nf">Iterator</span><span class="p">()</span> <span class="p">{</span>
1283 </span></span><span class="line"><span class="cl"> <span class="nb">println</span><span class="p">(</span><span class="nx">k</span><span class="p">)</span> <span class="c1">// Order always will be: foo, bar, baz
1284 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="p">}</span>
1285 </span></span><span class="line"><span class="cl"><span class="p">}</span>
1286 </span></span></code></pre><p>Now, of course the lack of Ordered Map in the standard library can be filled
1287 with third party implementations, e.g.: I am using this
1288 <a href="https://github.com/elliotchance/orderedmap">one</a> in one of my projects. But
1289 being in standard library reduces the friction: if there was some
1290 implementation in standard library, I would generally prefer it unless I have
1291 some specific needs. However when the standard library doesn't offer what I
1292 need, I need to find it myself a suitable library, and this ends up taking time
1293 since generally there are lots of alternatives.</p>
1294 <h2>Keyword and default arguments for functions</h2>
1295 <p>Something that comes straight from Python that I miss sometimes in Go is that
1296 you can do things like this when declaring a function:</p>
1297 <pre class="chroma"><code><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">hello</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s2">&#34;World&#34;</span><span class="p">):</span>
1298 </span></span><span class="line"><span class="cl"> <span class="nb">print</span><span class="p">(</span><span class="sa">f</span><span class="s2">&#34;Hello, </span><span class="si">{</span><span class="n">name</span><span class="si">}</span><span class="s2">&#34;</span><span class="p">)</span>
1299 </span></span><span class="line"><span class="cl">
1300 </span></span><span class="line"><span class="cl"><span class="n">hello</span><span class="p">(</span><span class="s2">&#34;Foo&#34;</span><span class="p">)</span> <span class="c1"># &#34;normal&#34; function call</span>
1301 </span></span><span class="line"><span class="cl"><span class="n">hello</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s2">&#34;Bar&#34;</span><span class="p">)</span> <span class="c1"># calling with keyword arguments</span>
1302 </span></span><span class="line"><span class="cl"><span class="n">hello</span><span class="p">()</span> <span class="c1"># calling with default arguments</span>
1303 </span></span></code></pre><pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> python hello.py
1304 </span></span><span class="line"><span class="cl"><span class="go">Hello, Foo
1305 </span></span></span><span class="line"><span class="cl"><span class="go">Hello, Bar
1306 </span></span></span><span class="line"><span class="cl"><span class="go">Hello, World
1307 </span></span></span></code></pre><p>The lack of default arguments especially affects even some of the API decisions
1308 for Go standard library, for example, <code>string.Replace</code>:</p>
1309 <blockquote>
1310 <p><code>func Replace(s, old, new string, n int) string</code></p>
1311 <p>Replace returns a copy of the string s with the first n non-overlapping
1312 instances of old replaced by new. If old is empty, it matches at the
1313 beginning of the string and after each UTF-8 sequence, yielding up to k+1
1314 replacements for a k-rune string. If n &lt; 0, there is no limit on the number
1315 of replacements.</p>
1316 </blockquote>
1317 <p>If Go had default arguments, <code>Replace</code> could have e.g.: <code>func Replace(s, old, new string, n int = -1)</code> signature, that would mean we don't need
1318 <code>strings.ReplaceAll</code> (since this is basically what this function does, call
1319 <code>strings.Replace(s, old, new, -1)</code>).</p>
1320 <h2>Nullability (or nillability)</h2>
1321 <p>I talked I little about this in <a href="https://kokada.dev/blog/go-a-reasonable-good-language">my previous post about
1322 Go</a>, but I want to
1323 expand here.</p>
1324 <p>First, I don't think the language needs to support the generic solution for
1325 nullability, that would be either having proper Union or Sum types. Kotlin
1326 AFAIK doesn't support neither, but my 2 years experience with Kotlin showed
1327 that just having nullable types already helped a lot in ensuring type safety.</p>
1328 <p>Second, I do feel that Go has less issues with <code>nil</code> values, than say, Java,
1329 because its decision of using zero values instead of <code>nil</code> in many cases. So
1330 for example, a string can never be <code>nil</code>, however a string pointer can be. This
1331 means that this is fine:</p>
1332 <pre class="chroma"><code><span class="line"><span class="cl"><span class="kd">func</span><span class="p">(</span><span class="nx">s</span> <span class="kt">string</span><span class="p">)</span> <span class="p">{</span>
1333 </span></span><span class="line"><span class="cl"> <span class="c1">// do something with s
1334 </span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="p">}</span>
1335 </span></span></code></pre><p>However:</p>
1336 <pre class="chroma"><code><span class="line"><span class="cl"><span class="kd">func</span><span class="p">(</span><span class="nx">s</span> <span class="o">*</span><span class="kt">string</span><span class="p">)</span> <span class="p">{</span>
1337 </span></span><span class="line"><span class="cl"> <span class="c1">// s maybe nil here, better check first
1338 </span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="p">}</span>
1339 </span></span></code></pre><p>Still, I get more <code>panic</code> for <code>nil</code> pointer deference than I get in other
1340 languages that offer nullables (heck, even Python with
1341 <a href="https://www.mypy-lang.org/"><code>mypy</code></a> is safer).</p>
1342 <p>Sadly this is the change in this post that is more likely to need a completely
1343 new revision of the language.
1344 <a href="https://github.com/golang/go/issues/49202">nillability</a> was proposed before,
1345 but it is really unlikely it can be done without breaking backwards
1346 compatibility.</p>
1347 <p>It could be done the Java way by adding a <code>nullable</code> type to the standard
1348 library (<a href="https://jcp.org/en/jsr/detail?id=305">JSR305</a>), but the fact that
1349 <a href="https://stackoverflow.com/questions/2289694/what-is-the-status-of-jsr-305">JSR305 is considerd
1350 dead</a>
1351 by many shows how difficult it is to do something like this without a major
1352 change in the language. Dart is the only language that I know that <a href="https://dart.dev/null-safety/understanding-null-safety">did this
1353 successfully</a>, but
1354 definitely it was not without its pains. And the fact that most people that
1355 program in Dart probably does because of Flutter (that eventually required
1356 newer versions with null-safety) is not a good sign.</p>
1357 <h2>Lambdas</h2>
1358 <p><em>Added in 2024-08-18</em></p>
1359 <p>Go is a surprising good language for some functional code, thanks to having
1360 first class functions and closures. Sadly the syntax doesn't help, since the
1361 only way you can use anonymous functions in Go is using <code>func</code>. Especially if
1362 the types are complex, this can result in some convoluted code. Take the
1363 example from the <a href="https://go.dev/wiki/RangefuncExperiment"><code>range-over-func</code>
1364 experiment</a>:</p>
1365 <pre class="chroma"><code><span class="line"><span class="cl"><span class="kn">package</span> <span class="nx">slices</span>
1366 </span></span><span class="line"><span class="cl">
1367 </span></span><span class="line"><span class="cl"><span class="kd">func</span> <span class="nx">Backward</span><span class="p">[</span><span class="nx">E</span> <span class="nx">any</span><span class="p">](</span><span class="nx">s</span> <span class="p">[]</span><span class="nx">E</span><span class="p">)</span> <span class="kd">func</span><span class="p">(</span><span class="kd">func</span><span class="p">(</span><span class="kt">int</span><span class="p">,</span> <span class="nx">E</span><span class="p">)</span> <span class="kt">bool</span><span class="p">)</span> <span class="p">{</span>
1368 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="kd">func</span><span class="p">(</span><span class="nx">yield</span> <span class="kd">func</span><span class="p">(</span><span class="kt">int</span><span class="p">,</span> <span class="nx">E</span><span class="p">)</span> <span class="kt">bool</span><span class="p">)</span> <span class="p">{</span>
1369 </span></span><span class="line"><span class="cl"> <span class="k">for</span> <span class="nx">i</span> <span class="o">:=</span> <span class="nb">len</span><span class="p">(</span><span class="nx">s</span><span class="p">)</span><span class="o">-</span><span class="mi">1</span><span class="p">;</span> <span class="nx">i</span> <span class="o">&gt;=</span> <span class="mi">0</span><span class="p">;</span> <span class="nx">i</span><span class="o">--</span> <span class="p">{</span>
1370 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="p">!</span><span class="nf">yield</span><span class="p">(</span><span class="nx">i</span><span class="p">,</span> <span class="nx">s</span><span class="p">[</span><span class="nx">i</span><span class="p">])</span> <span class="p">{</span>
1371 </span></span><span class="line"><span class="cl"> <span class="k">return</span>
1372 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
1373 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
1374 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
1375 </span></span><span class="line"><span class="cl"><span class="p">}</span>
1376 </span></span></code></pre><p>If Go had a syntax for lambdas, especially if we could elide the types, this
1377 could be simplified a lot:</p>
1378 <pre class="chroma"><code><span class="line"><span class="cl"><span class="kn">package</span> <span class="nx">slices</span>
1379 </span></span><span class="line"><span class="cl">
1380 </span></span><span class="line"><span class="cl"><span class="kd">func</span> <span class="nx">Backward</span><span class="p">[</span><span class="nx">E</span> <span class="nx">any</span><span class="p">](</span><span class="nx">s</span> <span class="p">[]</span><span class="nx">E</span><span class="p">)</span> <span class="kd">func</span><span class="p">(</span><span class="kd">func</span><span class="p">(</span><span class="kt">int</span><span class="p">,</span> <span class="nx">E</span><span class="p">)</span> <span class="kt">bool</span><span class="p">)</span> <span class="p">{</span>
1381 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="p">(</span><span class="nx">yield</span><span class="p">)</span> <span class="p">=&gt;</span> <span class="p">{</span>
1382 </span></span><span class="line"><span class="cl"> <span class="k">for</span> <span class="nx">i</span> <span class="o">:=</span> <span class="nb">len</span><span class="p">(</span><span class="nx">s</span><span class="p">)</span><span class="o">-</span><span class="mi">1</span><span class="p">;</span> <span class="nx">i</span> <span class="o">&gt;=</span> <span class="mi">0</span><span class="p">;</span> <span class="nx">i</span><span class="o">--</span> <span class="p">{</span>
1383 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="p">!</span><span class="nf">yield</span><span class="p">(</span><span class="nx">i</span><span class="p">,</span> <span class="nx">s</span><span class="p">[</span><span class="nx">i</span><span class="p">])</span> <span class="p">{</span>
1384 </span></span><span class="line"><span class="cl"> <span class="k">return</span>
1385 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
1386 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
1387 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
1388 </span></span><span class="line"><span class="cl"><span class="p">}</span>
1389 </span></span></code></pre><p>Or even something like this would already help, no special syntax but allowing
1390 the types to be elided in an unnamed function:</p>
1391 <pre class="chroma"><code><span class="line"><span class="cl"><span class="kn">package</span> <span class="nx">slices</span>
1392 </span></span><span class="line"><span class="cl">
1393 </span></span><span class="line"><span class="cl"><span class="kd">func</span> <span class="nx">Backward</span><span class="p">[</span><span class="nx">E</span> <span class="nx">any</span><span class="p">](</span><span class="nx">s</span> <span class="p">[]</span><span class="nx">E</span><span class="p">)</span> <span class="kd">func</span><span class="p">(</span><span class="kd">func</span><span class="p">(</span><span class="kt">int</span><span class="p">,</span> <span class="nx">E</span><span class="p">)</span> <span class="kt">bool</span><span class="p">)</span> <span class="p">{</span>
1394 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="kd">func</span><span class="p">(</span><span class="nx">yield</span><span class="p">)</span> <span class="p">{</span>
1395 </span></span><span class="line"><span class="cl"> <span class="k">for</span> <span class="nx">i</span> <span class="o">:=</span> <span class="nb">len</span><span class="p">(</span><span class="nx">s</span><span class="p">)</span><span class="o">-</span><span class="mi">1</span><span class="p">;</span> <span class="nx">i</span> <span class="o">&gt;=</span> <span class="mi">0</span><span class="p">;</span> <span class="nx">i</span><span class="o">--</span> <span class="p">{</span>
1396 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="p">!</span><span class="nf">yield</span><span class="p">(</span><span class="nx">i</span><span class="p">,</span> <span class="nx">s</span><span class="p">[</span><span class="nx">i</span><span class="p">])</span> <span class="p">{</span>
1397 </span></span><span class="line"><span class="cl"> <span class="k">return</span>
1398 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
1399 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
1400 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
1401 </span></span><span class="line"><span class="cl"><span class="p">}</span>
1402 </span></span></code></pre><p>This feature I am still somewhat hopeful that may become a reality in some
1403 future version of the language, since they didn't close the
1404 <a href="https://github.com/golang/go/issues/21498">issue</a> yet, and the discussion
1405 about the possibility of this feature is still ongoing.</p>
1406 <h2>Error on unused return values</h2>
1407 <p><em>Added in 2024-09-12</em></p>
1408 <p>In a <a href="https://kokada.dev/blog/my-coding-experience-with-llm">recent post about
1409 LLMs</a>, I talked about
1410 this function that I asked ChatGPT to generate to create a socket that accepted
1411 a <a href="https://pkg.go.dev/context">context</a>:</p>
1412 <pre class="chroma"><code><span class="line"><span class="cl"><span class="kd">func</span> <span class="nf">readWithContext</span><span class="p">(</span><span class="nx">ctx</span> <span class="nx">context</span><span class="p">.</span><span class="nx">Context</span><span class="p">,</span> <span class="nx">conn</span> <span class="nx">net</span><span class="p">.</span><span class="nx">Conn</span><span class="p">,</span> <span class="nx">buf</span> <span class="p">[]</span><span class="kt">byte</span><span class="p">)</span> <span class="p">(</span><span class="kt">int</span><span class="p">,</span> <span class="kt">error</span><span class="p">)</span> <span class="p">{</span>
1413 </span></span><span class="line"><span class="cl"> <span class="nx">done</span> <span class="o">:=</span> <span class="nb">make</span><span class="p">(</span><span class="kd">chan</span> <span class="kd">struct</span><span class="p">{})</span>
1414 </span></span><span class="line"><span class="cl"> <span class="kd">var</span> <span class="nx">n</span> <span class="kt">int</span>
1415 </span></span><span class="line"><span class="cl"> <span class="kd">var</span> <span class="nx">err</span> <span class="kt">error</span>
1416 </span></span><span class="line"><span class="cl">
1417 </span></span><span class="line"><span class="cl"> <span class="c1">// Start a goroutine to perform the read
1418 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="k">go</span> <span class="kd">func</span><span class="p">()</span> <span class="p">{</span>
1419 </span></span><span class="line"><span class="cl"> <span class="nx">n</span><span class="p">,</span> <span class="nx">err</span> <span class="p">=</span> <span class="nx">conn</span><span class="p">.</span><span class="nf">Read</span><span class="p">(</span><span class="nx">buf</span><span class="p">)</span>
1420 </span></span><span class="line"><span class="cl"> <span class="nb">close</span><span class="p">(</span><span class="nx">done</span><span class="p">)</span>
1421 </span></span><span class="line"><span class="cl"> <span class="p">}()</span>
1422 </span></span><span class="line"><span class="cl">
1423 </span></span><span class="line"><span class="cl"> <span class="k">select</span> <span class="p">{</span>
1424 </span></span><span class="line"><span class="cl"> <span class="k">case</span> <span class="o">&lt;-</span><span class="nx">ctx</span><span class="p">.</span><span class="nf">Done</span><span class="p">():</span> <span class="c1">// Context was canceled or timed out
1425 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="c1">// Set a short deadline to unblock the Read()
1426 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="nx">conn</span><span class="p">.</span><span class="nf">SetReadDeadline</span><span class="p">(</span><span class="nx">time</span><span class="p">.</span><span class="nf">Now</span><span class="p">())</span>
1427 </span></span><span class="line"><span class="cl"> <span class="o">&lt;-</span><span class="nx">done</span> <span class="c1">// Wait for the read to finish
1428 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="k">return</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">ctx</span><span class="p">.</span><span class="nf">Err</span><span class="p">()</span>
1429 </span></span><span class="line"><span class="cl"> <span class="k">case</span> <span class="o">&lt;-</span><span class="nx">done</span><span class="p">:</span> <span class="c1">// Read finished successfully
1430 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="k">return</span> <span class="nx">n</span><span class="p">,</span> <span class="nx">err</span>
1431 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
1432 </span></span><span class="line"><span class="cl"><span class="p">}</span>
1433 </span></span></code></pre><p>However while reading <a href="https://joeduffyblog.com/2016/02/07/the-error-model/">this blog
1434 post</a> about &quot;The Error
1435 Model&quot;, I realised that this function is lacking error check during
1436 <code>conn.SetReadDeadline()</code> calls:</p>
1437 <pre class="chroma"><code><span class="line"><span class="cl"><span class="kd">func</span> <span class="nf">readWithContext</span><span class="p">(</span><span class="nx">ctx</span> <span class="nx">context</span><span class="p">.</span><span class="nx">Context</span><span class="p">,</span> <span class="nx">conn</span> <span class="nx">net</span><span class="p">.</span><span class="nx">Conn</span><span class="p">,</span> <span class="nx">buf</span> <span class="p">[]</span><span class="kt">byte</span><span class="p">)</span> <span class="p">(</span><span class="nx">n</span> <span class="kt">int</span><span class="p">,</span> <span class="nx">err</span> <span class="kt">error</span><span class="p">)</span> <span class="p">{</span>
1438 </span></span><span class="line"><span class="cl"> <span class="nx">done</span> <span class="o">:=</span> <span class="nb">make</span><span class="p">(</span><span class="kd">chan</span> <span class="kd">struct</span><span class="p">{})</span>
1439 </span></span><span class="line"><span class="cl">
1440 </span></span><span class="line"><span class="cl"> <span class="c1">// Start a goroutine to perform the read
1441 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="k">go</span> <span class="kd">func</span><span class="p">()</span> <span class="p">{</span>
1442 </span></span><span class="line"><span class="cl"> <span class="nx">n</span><span class="p">,</span> <span class="nx">err</span> <span class="p">=</span> <span class="nx">conn</span><span class="p">.</span><span class="nf">Read</span><span class="p">(</span><span class="nx">buf</span><span class="p">)</span>
1443 </span></span><span class="line"><span class="cl"> <span class="nb">close</span><span class="p">(</span><span class="nx">done</span><span class="p">)</span>
1444 </span></span><span class="line"><span class="cl"> <span class="p">}()</span>
1445 </span></span><span class="line"><span class="cl">
1446 </span></span><span class="line"><span class="cl"> <span class="k">select</span> <span class="p">{</span>
1447 </span></span><span class="line"><span class="cl"> <span class="k">case</span> <span class="o">&lt;-</span><span class="nx">done</span><span class="p">:</span>
1448 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="nx">n</span><span class="p">,</span> <span class="nx">err</span>
1449 </span></span><span class="line"><span class="cl"> <span class="k">case</span> <span class="o">&lt;-</span><span class="nx">ctx</span><span class="p">.</span><span class="nf">Done</span><span class="p">():</span>
1450 </span></span><span class="line"><span class="cl"> <span class="c1">// Set a short deadline to unblock the Read()
1451 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="nx">err</span> <span class="p">=</span> <span class="nx">conn</span><span class="p">.</span><span class="nf">SetReadDeadline</span><span class="p">(</span><span class="nx">time</span><span class="p">.</span><span class="nf">Now</span><span class="p">())</span>
1452 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="nx">err</span> <span class="o">!=</span> <span class="kc">nil</span> <span class="p">{</span>
1453 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">err</span>
1454 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
1455 </span></span><span class="line"><span class="cl"> <span class="c1">// Reset read deadline
1456 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="k">defer</span> <span class="kd">func</span><span class="p">()</span> <span class="p">{</span>
1457 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="nx">e</span> <span class="o">:=</span> <span class="nx">conn</span><span class="p">.</span><span class="nf">SetReadDeadline</span><span class="p">(</span><span class="nx">time</span><span class="p">.</span><span class="nx">Time</span><span class="p">{});</span> <span class="nx">e</span> <span class="o">!=</span> <span class="kc">nil</span> <span class="p">{</span>
1458 </span></span><span class="line"><span class="cl"> <span class="nx">err</span> <span class="p">=</span> <span class="nx">errors</span><span class="p">.</span><span class="nf">Join</span><span class="p">(</span><span class="nx">err</span><span class="p">,</span> <span class="nx">e</span><span class="p">)</span>
1459 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
1460 </span></span><span class="line"><span class="cl"> <span class="p">}()</span>
1461 </span></span><span class="line"><span class="cl"> <span class="c1">// Make sure that the goroutine is done to avoid leaks
1462 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="o">&lt;-</span><span class="nx">done</span>
1463 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">errors</span><span class="p">.</span><span class="nf">Join</span><span class="p">(</span><span class="nx">err</span><span class="p">,</span> <span class="nx">ctx</span><span class="p">.</span><span class="nf">Err</span><span class="p">())</span>
1464 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
1465 </span></span><span class="line"><span class="cl"><span class="p">}</span>
1466 </span></span></code></pre><p>I could blame the LLM on this, but this is the type of mistake that I could see
1467 happening even in a Pull Request done by an actual human.</p>
1468 <p>Ignoring errors is bad, especially because once the error is ignored it is
1469 gone, forever. You may have strange issues in the code that are impossible to
1470 debug because you can't know about something that doesn't exist. This is one of
1471 the reasons I think exceptions are better, since this is the kind of error that
1472 would be impossible to ignore if the language had exceptions, and the exception
1473 would eventually propagate in your stack until it is captured or you get a
1474 crash (with a stack trace to debug the issue).</p>
1475 <p>Now, I don't think this is a fault of using error as values instead of
1476 exceptions. Go has the tools to encapsulate the errors and propagate them
1477 properly, like it is shown in the fixed code. But the fact that someone can do
1478 this mistake without any warning or error from the compiler is bad.</p>
1479 <p>If Go had a warning or error for unused return values, this would be different:</p>
1480 <pre class="chroma"><code><span class="line"><span class="cl"><span class="kd">func</span> <span class="nf">foo</span><span class="p">(</span><span class="nx">conn</span> <span class="o">*</span><span class="nx">net</span><span class="p">.</span><span class="nx">Conn</span><span class="p">)</span> <span class="p">{</span>
1481 </span></span><span class="line"><span class="cl"> <span class="c1">// ...
1482 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="nx">_</span> <span class="p">=</span> <span class="nx">conn</span><span class="p">.</span><span class="nf">SetReadDeadline</span><span class="p">(</span><span class="nx">time</span><span class="p">.</span><span class="nf">Now</span><span class="p">())</span>
1483 </span></span><span class="line"><span class="cl"><span class="p">}</span>
1484 </span></span></code></pre><p>Now in this case it is clear: I am ignoring the error explicitly, probably for
1485 a good reason. I can ask during a Pull Request why the committer is ignoring if
1486 it is lacking appropriate context, and maybe even ask for some comments to be
1487 added why this would be safe. What can't happen is this being ignored by
1488 mistake.</p>
1489 <p>I am not completely sure if I want this only for errors or for any unused value
1490 though. There are lots of cases where ignoring non-error values is ok, but I
1491 also don't see the issue of having something like:</p>
1492 <pre class="chroma"><code><span class="line"><span class="cl"><span class="kd">func</span> <span class="nf">foo</span><span class="p">()</span> <span class="p">{</span>
1493 </span></span><span class="line"><span class="cl"> <span class="c1">// ...
1494 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="nx">_</span><span class="p">,</span> <span class="nx">_</span> <span class="p">=</span> <span class="nf">FuncThatReturnIntAndError</span><span class="p">()</span>
1495 </span></span><span class="line"><span class="cl"><span class="p">}</span>
1496 </span></span></code></pre><p>If anything, it is clear for whoever is reading this code later that you're
1497 only calling this function only for its side-effects.</p>
1498 <p>By the way, yes, <a href="https://github.com/kisielk/errcheck">errcheck</a> linter exists,
1499 but if the language itself doesn't enforce this it means that there are lots of
1500 libraries that are probably mistakenly ignoring errors. And if the library is
1501 ignoring errors by mistake, you can't do much when you actually have them.</p></summary></entry><entry><title>Quick bits: NixOS's boot.kernelPackages is a attrset</title><link href="https://kokada.dev/blog/quick-bits-nixoss-boot-kernelpackages-is-a-attrset/" rel="alternate"/><published>2024-08-15T00:00:00+00:00</published><id>https://kokada.dev/blog/quick-bits-nixoss-boot-kernelpackages-is-a-attrset/</id><summary type="html"><p>I don't know if this is just something that got me by surprise or not, but
1502 <a href="https://github.com/NixOS/nixpkgs/blob/bb16119a4a7639ebbc91ad0f516b324b0f7c9b68/nixos/modules/system/boot/kernel.nix#L40-L71"><code>boot.kernelPackages</code></a>
1503 option does not receive a derivation like most other packages, but instead
1504 receives a function and returns an attribute set with all packages. Here is the
1505 documentation:</p>
1506 <blockquote>
1507 <p>This option allows you to override the Linux kernel used by NixOS. Since
1508 things like external kernel module packages are tied to the kernel you’re
1509 using, it also overrides those. This option is a function that takes Nixpkgs
1510 as an argument (as a convenience), and returns an attribute set containing at
1511 the very least an attribute kernel. Additional attributes may be needed
1512 depending on your configuration. For instance, if you use the NVIDIA X
1513 driver, then it also needs to contain an attribute <code>nvidia_x11</code>.</p>
1514 </blockquote>
1515 <p>The kernel package itself is referenced by the
1516 <a href="https://github.com/NixOS/nixpkgs/blob/bb16119a4a7639ebbc91ad0f516b324b0f7c9b68/nixos/modules/system/boot/kernel.nix#L331-L332"><code>kernel</code></a>
1517 derivation inside this attribute set:</p>
1518 <pre class="chroma"><code><span class="line"><span class="cl"><span class="go">nix-repl&gt; nixosConfigurations.sankyuu-nixos.config.boot.kernelPackages.kernel
1519 </span></span></span><span class="line"><span class="cl"><span class="go">«derivation /nix/store/5zyjvf3qgfk52qmgxh36l4dkr9lf100x-linux-6.10.3.drv»
1520 </span></span></span></code></pre><p>The reason for this is because it ensure that things like modules are built
1521 with the same kernel version you are booting.</p>
1522 <p>However one less obvious consequence about this is that if you want packages
1523 that come from <code>linuxPackages</code>, say for example
1524 <a href="https://github.com/NixOS/nixpkgs/blob/nixos-unstable/pkgs/os-specific/linux/cpupower/default.nix"><code>cpupower</code></a>
1525 or
1526 <a href="https://github.com/NixOS/nixpkgs/blob/nixos-unstable/pkgs/os-specific/linux/cpupower/default.nix"><code>turbostat</code></a>,
1527 it is better to do:</p>
1528 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span> <span class="n">config</span><span class="o">,</span> <span class="o">...</span> <span class="p">}:</span>
1529 </span></span><span class="line"><span class="cl"><span class="p">{</span>
1530 </span></span><span class="line"><span class="cl"> <span class="n">environment</span><span class="o">.</span><span class="n">systemPackages</span> <span class="o">=</span> <span class="p">[</span>
1531 </span></span><span class="line"><span class="cl"> <span class="n">config</span><span class="o">.</span><span class="n">boot</span><span class="o">.</span><span class="n">kernelPackages</span><span class="o">.</span><span class="n">cpupower</span>
1532 </span></span><span class="line"><span class="cl"> <span class="n">config</span><span class="o">.</span><span class="n">boot</span><span class="o">.</span><span class="n">kernelPackages</span><span class="o">.</span><span class="n">turbostat</span>
1533 </span></span><span class="line"><span class="cl"> <span class="p">];</span>
1534 </span></span><span class="line"><span class="cl"><span class="p">}</span>
1535 </span></span></code></pre><p>Instead of:</p>
1536 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span> <span class="n">pkgs</span><span class="o">,</span> <span class="o">...</span> <span class="p">}:</span>
1537 </span></span><span class="line"><span class="cl"><span class="p">{</span>
1538 </span></span><span class="line"><span class="cl"> <span class="n">environment</span><span class="o">.</span><span class="n">systemPackages</span> <span class="o">=</span> <span class="k">with</span> <span class="n">pkgs</span><span class="p">;</span> <span class="p">[</span>
1539 </span></span><span class="line"><span class="cl"> <span class="n">linuxPackages</span><span class="o">.</span><span class="n">cpupower</span>
1540 </span></span><span class="line"><span class="cl"> <span class="n">linuxPackages</span><span class="o">.</span><span class="n">turbostat</span>
1541 </span></span><span class="line"><span class="cl"> <span class="p">];</span>
1542 </span></span><span class="line"><span class="cl"><span class="p">}</span>
1543 </span></span></code></pre><p>Now, I used the later for a long time and never had issues. But technically
1544 those packages depends in a specific kernel version for a reason, so getting
1545 them from <code>config.boot.kernelPackages</code> reduces the chance of you having some
1546 compatibility issue in the future.</p></summary></entry><entry><title>Abusing Makefiles for fun and profit</title><link href="https://kokada.dev/blog/abusing-makefiles-for-fun-and-profit/" rel="alternate"/><published>2024-08-13T00:00:00+00:00</published><id>https://kokada.dev/blog/abusing-makefiles-for-fun-and-profit/</id><summary type="html"><p>If you are following this blog for a while, it should be no surprise that most
1547 of the workflow in this blog is <a href="https://kokada.dev/blog/quick-bits-why-you-should-automate-everything">automated using
1548 Go</a>. I
1549 basically write Markdown files with some special rules inside the
1550 <a href="https://github.com/thiagokokada/blog">repository</a>, commit and push it. In
1551 seconds, the CI (currently <a href="https://github.com/thiagokokada/blog/blob/4e3f25485c6682f3e066b219df2290934bc0d256/.github/workflows/go.yml">GitHub
1552 Actions</a>)
1553 will take the latest commit, generate some files (since I use the <a href="https://kokada.dev/blog/using-github-as-a-bad-blog-platform">repository
1554 itself</a> as a
1555 backup blog) and publish to the <a href="https://kokada.capivaras.dev/">capivaras.dev
1556 website</a>.</p>
1557 <p>Now, considering how much about <a href="https://nixos.org/">Nix</a> I talk in this blog,
1558 it should be a surprise that the workflow above has <strong>zero</strong> Nix code inside
1559 it. I am not saying this blog will never have it, but I am only going to add if
1560 this is necessary, for example if I start using a tool to build this blog that
1561 I generally don't expect it to be installed by the machine I am currently
1562 using. Go is an exception of this rule since it is relatively straightfoward to
1563 install (just download the <a href="https://go.dev/doc/install">binary</a>) and because
1564 its <a href="https://go.dev/doc/go1compat">stability guarantee</a> means (hopefully) no
1565 breakage. But most other things I consider moving targets, and I wouldn't be
1566 comfortable to use unless I have Nix to ensure reproducibility.</p>
1567 <p>This is why the other tool that this blog (ab)uses during its workflow is
1568 <a href="https://en.wikipedia.org/wiki/Make_(software)"><code>Make</code></a>, one of the oldest
1569 build automation tool that exist. It is basically available in any *nix (do not
1570 confuse with <a href="https://nixos.org/">Nix</a>) system, from most Linux distros to
1571 macOS, by default. So it is the tool I choose to automatise some tasks in this
1572 blog, even if I consider writing a <code>Makefile</code> (the domain-specific language
1573 that <code>Make</code> uses) kind of a lost, dark art.</p>
1574 <p>To be clear, the idea of this post is not to be a <code>Makefile</code> tutorial. I will
1575 explain some basic concepts, but if you want an actual tutorial a good one can
1576 be found <a href="https://makefiletutorial.com/">here</a>. Also, while I am using <code>Make</code>
1577 thanks to the reasons above, you can use many other tools for a similar
1578 objective, like <a href="https://github.com/casey/just">Justfiles</a>,
1579 <a href="https://taskfile.dev/">Taskfiles</a> (sadly it uses
1580 <a href="https://kokada.dev/blog/generating-yaml-files-with-nix">YAML</a>), or even a
1581 small script written in any language you want. The reason that I am writing
1582 this post is why you should do it, not how.</p>
1583 <p>A quick recap on how this blog works: inside the
1584 <a href="https://github.com/thiagokokada/blog">repository</a>, a post is basically a
1585 Markdown post following the directory structure below
1586 (<a href="https://github.com/thiagokokada/blog/tree/894a388c61ca3a38dfc9d4cbe88dc684fd964bb7">permalink</a>
1587 for the current version of this blog):</p>
1588 <pre class="chroma"><code><span class="line"><span class="cl"><span class="go">.
1589 </span></span></span><span class="line"><span class="cl"><span class="go">&lt;...&gt;
1590 </span></span></span><span class="line"><span class="cl"><span class="go">├── 2024-08-07
1591 </span></span></span><span class="line"><span class="cl"><span class="go">│ ├── 01-quick-bits-is-crostini-a-microvm.md
1592 </span></span></span><span class="line"><span class="cl"><span class="go">│ └── 02-meta-are-quick-bits-really-quick.md
1593 </span></span></span><span class="line"><span class="cl"><span class="go">├── 2024-08-11
1594 </span></span></span><span class="line"><span class="cl"><span class="go">│ └── 01-building-static-binaries-in-nix.md
1595 </span></span></span><span class="line"><span class="cl"><span class="go">├── 2024-08-12
1596 </span></span></span><span class="line"><span class="cl"><span class="go">│ ├── 01-things-i-dont-like-in-my-chromebook-duet-3.md
1597 </span></span></span><span class="line"><span class="cl"><span class="go">│ └── Screenshot_2024-08-12_20.50.42.png
1598 </span></span></span><span class="line"><span class="cl"><span class="go">├── 2024-08-13
1599 </span></span></span><span class="line"><span class="cl"><span class="go">│ ├── 01-abusing-makefiles-for-fun-and-profit.md &lt;-- this file
1600 </span></span></span><span class="line"><span class="cl"><span class="go">├── .github
1601 </span></span></span><span class="line"><span class="cl"><span class="go">│ └── workflows
1602 </span></span></span><span class="line"><span class="cl"><span class="go">│ └── go.yml
1603 </span></span></span><span class="line"><span class="cl"><span class="go">├── .gitignore
1604 </span></span></span><span class="line"><span class="cl"><span class="go">├── go.mod
1605 </span></span></span><span class="line"><span class="cl"><span class="go">├── go.sum
1606 </span></span></span><span class="line"><span class="cl"><span class="go">├── LICENSE
1607 </span></span></span><span class="line"><span class="cl"><span class="go">├── link_rewriter.go
1608 </span></span></span><span class="line"><span class="cl"><span class="go">├── Makefile
1609 </span></span></span><span class="line"><span class="cl"><span class="go">├── mataroa.go
1610 </span></span></span><span class="line"><span class="cl"><span class="go">├── README.md
1611 </span></span></span><span class="line"><span class="cl"><span class="go">├── rss.xml
1612 </span></span></span><span class="line"><span class="cl"><span class="go">└── .scripts
1613 </span></span></span><span class="line"><span class="cl"><span class="go"> └── gen-post.sh
1614 </span></span></span></code></pre><p>So I just create a new Markdown file following the
1615 <code>YYYY-MM-DD/XX-title-slug.md</code> format. It <strong>must</strong> start with a <code>h1</code> header,
1616 that will be automatically extract to be used as the post title, but otherwise
1617 there is no other formatting rules. It is a highly optionated structure, but
1618 the nice thing about being optionated is that we can extract lots of
1619 information just from how the files are organised in the filesystem.</p>
1620 <p>Most of the magic that converts those Markdown files to actual blog posts are
1621 in the Go files that you can see above: <code>blog.go</code> is the main logic that walks
1622 in the repository and extracts the necessary information, <code>mataroa.go</code> is
1623 responsible for the <a href="https://capivaras.dev/">capivaras.dev</a> integration (that
1624 uses <a href="https://mataroa.blog/">Mataroa</a> platform), while <code>link_rewriter.go</code> is
1625 responsible to do some transformations in the Markdown files before posting.</p>
1626 <p>While I could manage everything by just using <code>go</code> CLI and a few other *nix
1627 commands, to make it easier to manager everything I have the following
1628 <a href="https://github.com/thiagokokada/blog/blob/527466a2a7c8baae532281bff5db3f0695f018cb/Makefile"><code>Makefile</code></a>:</p>
1629 <pre class="chroma"><code><span class="line"><span class="cl"><span class="nv">MARKDOWN</span> <span class="o">:=</span> <span class="k">$(</span>wildcard ./**/*.md<span class="k">)</span>
1630 </span></span><span class="line"><span class="cl">
1631 </span></span><span class="line"><span class="cl"><span class="nf">.PHONY</span><span class="o">:</span> <span class="n">all</span>
1632 </span></span><span class="line"><span class="cl"><span class="nf">all</span><span class="o">:</span> <span class="n">README</span>.<span class="n">md</span> <span class="n">rss</span>.<span class="n">xml</span>
1633 </span></span><span class="line"><span class="cl">
1634 </span></span><span class="line"><span class="cl"><span class="nf">blog</span><span class="o">:</span> *.<span class="n">go</span> <span class="n">go</span>.*
1635 </span></span><span class="line"><span class="cl"> go build
1636 </span></span><span class="line"><span class="cl">
1637 </span></span><span class="line"><span class="cl"><span class="nf">README.md</span><span class="o">:</span> <span class="n">blog</span> <span class="k">$(</span><span class="nv">MARKDOWN</span><span class="k">)</span>
1638 </span></span><span class="line"><span class="cl"> ./blog &gt; README.md
1639 </span></span><span class="line"><span class="cl">
1640 </span></span><span class="line"><span class="cl"><span class="nf">rss.xml</span><span class="o">:</span> <span class="n">blog</span> <span class="k">$(</span><span class="nv">MARKDOWN</span><span class="k">)</span>
1641 </span></span><span class="line"><span class="cl"> ./blog -rss &gt; rss.xml
1642 </span></span><span class="line"><span class="cl">
1643 </span></span><span class="line"><span class="cl"><span class="nf">.PHONY</span><span class="o">:</span> <span class="n">publish</span>
1644 </span></span><span class="line"><span class="cl"><span class="nf">publish</span><span class="o">:</span> <span class="n">blog</span>
1645 </span></span><span class="line"><span class="cl"> ./blog -publish
1646 </span></span><span class="line"><span class="cl">
1647 </span></span><span class="line"><span class="cl"><span class="nv">DAY</span> <span class="o">:=</span> <span class="k">$(</span>shell date<span class="k">)</span>
1648 </span></span><span class="line"><span class="cl"><span class="nv">_PARSED_DAY</span> <span class="o">:=</span> <span class="k">$(</span>shell date <span class="s1">&#39;+%Y-%m-%d&#39;</span> -d <span class="s1">&#39;$(DAY)&#39;</span><span class="k">)</span>
1649 </span></span><span class="line"><span class="cl"><span class="nf">.PHONY</span><span class="o">:</span> <span class="n">day</span>
1650 </span></span><span class="line"><span class="cl"><span class="nf">day</span><span class="o">:</span>
1651 </span></span><span class="line"><span class="cl"> mkdir -p <span class="s1">&#39;$(_PARSED_DAY)&#39;</span>
1652 </span></span><span class="line"><span class="cl">
1653 </span></span><span class="line"><span class="cl"><span class="nv">TITLE</span> <span class="o">=</span> <span class="k">$(</span>error TITLE is not defined<span class="k">)</span>
1654 </span></span><span class="line"><span class="cl"><span class="nf">.PHONY</span><span class="o">:</span> <span class="n">post</span>
1655 </span></span><span class="line"><span class="cl"><span class="nf">post</span><span class="o">:</span> <span class="n">blog</span> <span class="n">day</span>
1656 </span></span><span class="line"><span class="cl"> ./.scripts/gen-post.sh <span class="s1">&#39;$(_PARSED_DAY)&#39;</span> <span class="s1">&#39;$(TITLE)&#39;</span>
1657 </span></span><span class="line"><span class="cl">
1658 </span></span><span class="line"><span class="cl"><span class="nv">FILE</span> <span class="o">=</span> <span class="k">$(</span>error FILE is not defined<span class="k">)</span>
1659 </span></span><span class="line"><span class="cl"><span class="nf">.PHONY</span><span class="o">:</span> <span class="n">draft</span>
1660 </span></span><span class="line"><span class="cl"><span class="nf">draft</span><span class="o">:</span>
1661 </span></span><span class="line"><span class="cl"> <span class="nb">cd</span> <span class="s1">&#39;$(dir $(FILE))&#39;</span> <span class="o">&amp;&amp;</span> mv <span class="s1">&#39;$(notdir $(FILE))&#39;</span> <span class="s1">&#39;.$(notdir $(FILE))&#39;</span>
1662 </span></span><span class="line"><span class="cl">
1663 </span></span><span class="line"><span class="cl"><span class="nf">.PHONY</span><span class="o">:</span> <span class="n">undraft</span>
1664 </span></span><span class="line"><span class="cl"><span class="nf">undraft</span><span class="o">:</span>
1665 </span></span><span class="line"><span class="cl"> <span class="nb">cd</span> <span class="s1">&#39;$(dir $(FILE))&#39;</span> <span class="o">&amp;&amp;</span> mv <span class="s1">&#39;$(notdir $(FILE))&#39;</span> <span class="s1">&#39;$(patsubst .%,%,$(notdir $(FILE)))&#39;</span>
1666 </span></span><span class="line"><span class="cl">
1667 </span></span><span class="line"><span class="cl"><span class="nf">.PHONY</span><span class="o">:</span> <span class="n">clean</span>
1668 </span></span><span class="line"><span class="cl"><span class="nf">clean</span><span class="o">:</span>
1669 </span></span><span class="line"><span class="cl"> rm -rf blog
1670 </span></span></code></pre><p>For those unfamiliar with <code>Makefile</code>, a quick explanation on how it works from
1671 <a href="https://en.wikipedia.org/wiki/Make_(software)#Makefile">Wikipedia</a>:</p>
1672 <blockquote>
1673 <p>Each rule begins with a <em>dependency line</em> which consists of the rule's target
1674 name followed by a colon (:) and optionally a list of targets on which the
1675 rule's target depends, its prerequisites.</p>
1676 </blockquote>
1677 <p>So if we look for example at the <code>blog</code> binary, the dependencies are all the
1678 <code>.go</code> files and Go module files like <code>go.mod</code> and <code>go.sum</code>. We can make the
1679 <code>blog</code> binary by running:</p>
1680 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> make blog
1681 </span></span><span class="line"><span class="cl"><span class="go">go build
1682 </span></span></span></code></pre><p>One nice thing about <code>Makefile</code> is that they track if any of the source files
1683 has a newer timestamp than the target file, and only trigger the build again if
1684 there are changes, for example:</p>
1685 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> make blog
1686 </span></span><span class="line"><span class="cl"><span class="go">make: &#39;blog&#39; is up to date.
1687 </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="err">
1688 </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="gp">$</span> touch blog.go
1689 </span></span><span class="line"><span class="cl"><span class="err">
1690 </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="gp">$</span> make blog
1691 </span></span><span class="line"><span class="cl"><span class="go">go build
1692 </span></span></span></code></pre><p>But sometimes this property is undesirable. In those cases we can declare a
1693 target as <code>.PHONY</code>, that basically instructs <code>Makefile</code> to always make the
1694 target. One classic example is <code>clean</code> target, that removes build artifacts:</p>
1695 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> make clean
1696 </span></span><span class="line"><span class="cl"><span class="go">rm -rf blog
1697 </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="err">
1698 </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="gp">$</span> make clean
1699 </span></span><span class="line"><span class="cl"><span class="go">rm -rf blog
1700 </span></span></span></code></pre><p>By the way, it is better to declare a target as <code>.PHONY</code> than declaring
1701 dependencies incorrectly, especially in languages that has fast build times
1702 like e.g.: Go. The worst thing that can happen is something not being rebuild
1703 when it needs to. So my recomendation if you are writing your first <code>Makefile</code>
1704 is to just declare everything as <code>.PHONY</code>. You can always improve it later.</p>
1705 <p>One last basic concept that I want to explain about <code>Makefile</code> is the default
1706 target: it is the target that is run if you just run <code>make</code> without arguments
1707 inside the directory that contains a <code>Makefile</code>. The default target is
1708 generally the first target in the <code>Makefile</code>. It is common to have an <code>all</code>
1709 target (that is also marked as <code>.PHONY</code>) that has as dependencies all the
1710 targets that you want to build by default. In this particular case I declare
1711 the <code>README.md</code> and <code>rss.xml</code> files to be build by default, and they themselves
1712 depends in <code>blog</code> binary being build. So once I run <code>make</code> you get as result:</p>
1713 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> make
1714 </span></span><span class="line"><span class="cl"><span class="go">go build
1715 </span></span></span><span class="line"><span class="cl"><span class="go">./blog &gt; README.md
1716 </span></span></span><span class="line"><span class="cl"><span class="go">./blog -rss &gt; rss.xml
1717 </span></span></span></code></pre><p>And this result above highlights the first reason I think you should have a
1718 <code>Makefile</code> or something similar in your projects: you don't need to remember
1719 the exactly steps that you need to get things working. If I see one project of
1720 mine having a <code>Makefile</code>, I can be reasonably confident that I can get it
1721 working by just running <code>make</code>.</p>
1722 <p>But now let's focus in the other targets that I have in the <code>Makefile</code> that are
1723 not related to the build process but are there to help me manage my blog posts.
1724 Remember the rules I explained above? Maybe not, but it should be no problem,
1725 because:</p>
1726 <pre><code>$ make post TITLE=&quot;My new blog post&quot;
1727 mkdir -p &quot;2024-08-13&quot;
1728 ./.scripts/gen-post.sh &quot;2024-08-13&quot; &quot;My new blog post&quot;
1729 Creating file: 2024-08-13/02-my-new-blog-post.md
1730 <p>$ cat 2024-08-13/02-my-new-blog-post.md</p>
1731 <h1>My new blog post</h1>
1732 </code><p><code></code></p></pre><p></p>
1733 <p>This command, <code>make post</code>, is responsible for:</p>
1734 <ol>
1735 <li>Create a new directory for today, if it doesn't exist</li>
1736 <li>Run the
1737 <a href="https://github.com/thiagokokada/blog/blob/6a3b06970729f7650e5bee5fb0e1f9f2541ffea8/.scripts/gen-post.sh"><code>gen-post.sh</code></a>
1738 script, that:
1739 <ol>
1740 <li>Enumerates all posts from the day, so we can number the new post correctly
1741 <ul>
1742 <li>We already had this post planned for 2024-08-13, so the new post is 02</li>
1743 </ul>
1744 </li>
1745 <li>Slugify the title, so we can create each Markdown file with the correct
1746 filename</li>
1747 <li>Creates a new Markdown file with the title as a <code>h1</code> header</li>
1748 </ol>
1749 </li>
1750 </ol>
1751 <p>The steps above may or may not seen trivial, and for a while I was doing them
1752 manually. But not having to think what is the current date or if I already
1753 posted that day or what is the slug is for the title make (pun intended) my
1754 like much easier.</p>
1755 <p>Yes, the code is ugly. The way variables works in <code>Make</code> is that you can
1756 declare then inside the <code>Makefile</code>, but they can be overwritten in the terminal
1757 if you pass them. I used this to allow <code>make post</code> to also work for future
1758 posts:</p>
1759 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> make post <span class="nv">TITLE</span><span class="o">=</span><span class="s2">&#34;Another new blog post&#34;</span> <span class="nv">DAY</span><span class="o">=</span>2024-12-12
1760 </span></span><span class="line"><span class="cl"><span class="go">mkdir -p &#34;2024-12-12&#34;
1761 </span></span></span><span class="line"><span class="cl"><span class="go">./.scripts/gen-post.sh &#34;2024-12-12&#34; &#34;Another new blog post&#34;
1762 </span></span></span><span class="line"><span class="cl"><span class="go">Creating file: 2024-12-12/01-another-new-blog-post.md
1763 </span></span></span></code></pre><p>So in the above case, <code>DAY</code> is filled with the value passed in the terminal
1764 instead of default (that would be the current day), and <code>_PARSED_DAY</code> is the
1765 day we use to actually create the directory. We can actually pass any date
1766 format recognised by
1767 <a href="https://www.gnu.org/software/coreutils/manual/html_node/Examples-of-date.html"><code>date</code></a>,
1768 not just <code>YYYY-MM-DD</code>.</p>
1769 <p>I have 2 other phony targets that I want to talk, <code>draft</code> and <code>undraft</code>. They
1770 expect a <code>FILE</code> to be passed, and I use them to either hide or unhide a file:</p>
1771 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> make draft <span class="nv">FILE</span><span class="o">=</span>2024-12-12/01-another-new-blog-post.md
1772 </span></span><span class="line"><span class="cl"><span class="go">mv &#34;2024-12-12/01-another-new-blog-post.md&#34; &#34;2024-12-12/.01-another-new-blog-post.md&#34;
1773 </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="err">
1774 </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="gp">$</span> make undraft <span class="nv">FILE</span><span class="o">=</span>2024-12-12/.01-another-new-blog-post.md
1775 </span></span><span class="line"><span class="cl"><span class="go">mv &#34;2024-12-12/.01-another-new-blog-post.md&#34; &#34;2024-12-12/01-another-new-blog-post.md&#34;
1776 </span></span></span></code></pre><p>Why? Because hidden files are <a href="https://github.com/thiagokokada/blog/blob/894a388c61ca3a38dfc9d4cbe88dc684fd964bb7/blog.go#L101-L104">explicit
1777 ignored</a>
1778 during my directory parser to mean they're a draft post and not ready to be
1779 published. And the reason I created those targets is because I was tired of
1780 trying to hide or unhide a file manually.</p>
1781 <p>So that's it, for the same reason you <a href="https://kokada.dev/blog/quick-bits-why-you-should-automate-everything">should probably automate
1782 everything</a>,
1783 you also need to have some way to automate your tasks. <code>Makefile</code> is one way to
1784 do it, maybe not the best way to do it, but it works and it is available
1785 anywhere.</p></summary></entry><entry><title>Things I don't like in my Chromebook Duet 3</title><link href="https://kokada.dev/blog/things-i-dont-like-in-my-chromebook-duet-3/" rel="alternate"/><published>2024-08-12T00:00:00+00:00</published><id>https://kokada.dev/blog/things-i-dont-like-in-my-chromebook-duet-3/</id><summary type="html"><p>So this is kind of a continuation from my <a href="https://kokada.dev/blog/my-favorite-device-is-a-chromebook">previous
1786 post</a> talking why
1787 my favorite device is a Chromebook. In this post I want to talk about what
1788 makes me this device unhappy, and comment about things that if changed would
1789 make it a much better device.</p>
1790 <p>But before talking about the negative aspects, let me talk about a positive
1791 aspect that I just briefly talked in the previous post: the screen. It is a
1792 HiDPI screen (2000x1200 resolution in 10.95''), that is unexpected bright (400
1793 nits according to the
1794 <a href="https://www.lenovo.com/us/en/p/laptops/lenovo/lenovo-edu-chromebooks/ideapad-duet-3-chromebook-11-inch,-qlc/len101i0034">specs</a>).
1795 It is difficult to find laptops at the same price with a screen that good. At
1796 10.95'' in its default resolution I find it too small (like 1250x750), but I
1797 find the font size acceptable at 115% scale (1087x652). Yes, it result in a
1798 small workspace, but this is not a big issue for what I do in this device. It
1799 is also only 60Hz, but I thought I would miss high refresh rate more than I
1800 actually miss in this device.</p>
1801 <p>Update: I forgot to say one thing about the screen: it scratches really easy. I
1802 got my screen scratched after the first day of usage, and considering the price
1803 I don't think the screen has a hardened glass. I bought a cheap glass screen
1804 protector and this did the trick though, even hiding the previous scratch, and
1805 I have zero issues with the screen now.</p>
1806 <p>Now the first aspect that I don't like: the speakers. They sound tiny and even
1807 at maximum volume it is not really loud. The speakers is the only reason why I
1808 still keep my <a href="https://www.gsmarena.com/xiaomi_pad_5-11042.php">Xiaomi Pad 5</a>,
1809 because I like to watch animes/videos before sleep and having good speakers is
1810 a must.</p>
1811 <p>The keyboard has that issue that I mentioned in the previous post: sometimes
1812 the key get stuck, and I get duplicated characters. But it also has some minor
1813 issues that I didn't talked about: the first one is the UK layout that has some
1814 extra keys that I have no use for, but this also makes the keys that I use
1815 smaller. Very much a &quot;me&quot; problem, since if I had got a US version I wouldn't
1816 have those issues, but an issue nonetheless that gets worse considering how
1817 small the keyboard is. I am actually suprised how fast I can type considering
1818 how many issues this keyboard has, so maybe this is a testament that this
1819 keyboard is not actually that bad.</p>
1820 <p>The other keyboard issue is a problem that affects all Chromebooks: its custom
1821 layout. Google replaced a few keys like Fn keys with shortcuts and replaced the
1822 Caps Lock with a
1823 <a href="https://chromeunboxed.com/chromebook-launcher-now-everything-button">&quot;Everything&quot;</a>
1824 key (that is similar to the Windows Key), while removing Windows Key from its
1825 place. I actually have less issue with this than I initially though: I don't
1826 care too much about Fn keys (except when using IntelliJ, but that is something
1827 that I only use at <code>$CURRENT_JOB</code>), and ChromeOS is surprisingly powerful in
1828 its customisation, allowing you to swap key functionality. I remap Everything
1829 key with Esc, and Esc for the Everything key, and I can get productive in my
1830 <code>neovim</code> setup.</p>
1831 <p>And finally, let me talk more about the performance: yes, it is bad, but
1832 bearable once you get used to. The issue is both the CPU and IO. While the CPU,
1833 a <a href="https://www.qualcomm.com/products/mobile/snapdragon/laptops-and-tablets/snapdragon-mobile-compute-platforms/snapdragon-7c-gen-2-compute-platform">Snapdragon 7c Gen
1834 2</a>
1835 is octa-core, it has only 2 high performance CPU cores vs 6 low performance
1836 ones (2xARM Cortex A76 vs 6xARM Cortex A55). If it was something like 4x4, it
1837 would be much better. The fact that the cores are old doesn't help either.</p>
1838 <p>But the worst part is the IO. Not only it uses a eMMC module, it is slow:</p>
1839 <p><a href="https://github.com/thiagokokada/blog/raw/main/posts/2024-08-12/Screenshot_2024-08-12_20.50.42.png"><img src="https://github.com/thiagokokada/blog/raw/main/posts/2024-08-12/Screenshot_2024-08-12_20.50.42.png" alt="CPDT Benchmark results from Chromebook Duet 3."></a></p>
1840 <p>I don't know how much more expensive it would be to put a
1841 <a href="https://en.wikipedia.org/wiki/Universal_Flash_Storage">UFS</a> instead of eMMC in
1842 this device, but this is probably the choice that would most increase
1843 performance in this device, especially considering how aggressive Chromebooks
1844 use (z)swap.</p>
1845 <p>Update 2: I forgot to talk about the fact that the exterior of the device is
1846 covered in cloth. I thought I would hate this at first, but nowadays I kind
1847 like it. And it is also nice that it will never get scratched, I don't care too
1848 much about the exterior of this device and it is the only device that I have at
1849 home that doesn't have additional protection (except the screen protector
1850 mentioned above).</p></summary></entry><entry><title>Building static binaries in Nix</title><link href="https://kokada.dev/blog/building-static-binaries-in-nix/" rel="alternate"/><published>2024-08-11T00:00:00+00:00</published><id>https://kokada.dev/blog/building-static-binaries-in-nix/</id><summary type="html"><p>I recently had laser eye surgery for my near-sightedness, and while evaluating
1851 if I could have the surgery I discovered that I was suffering from dry eyes.
1852 Thanks to this, my ophthalmologist recommended that every 20 minutes while
1853 using screens, I look somewhere 20 feet away for 20 seconds, a technique known
1854 as <a href="https://www.healthline.com/health/eye-health/20-20-20-rule">20-20-20 rule</a>.</p>
1855 <p>I had issues following this rule because I never remembered to do the pauses. I
1856 initially tried to setup an alarm, but this became annoying, so I decided to
1857 try to find a program. I found
1858 <a href="https://tonyh4156.github.io/20-20-20.github.io/">this</a> one for macOS that
1859 worked fine, but it bothered me that it was compiled for x86_64 while I was
1860 using a M1 MacBook Pro at the time, and also I needed something that worked in
1861 Linux.</p>
1862 <p>Since I never found a good multi-platform alternative, I decided to write my
1863 own. This became
1864 <a href="https://github.com/thiagokokada/twenty-twenty-twenty/">twenty-twenty-twenty</a>,
1865 the first <a href="https://kokada.dev/blog/go-a-reasonable-good-language">Go</a> program
1866 that I ever wrote. I wrote it in Go because I wanted to learn the language, but
1867 also because Go made it easy to build static binaries. And the first version I
1868 could build static binaries without issues because I was using
1869 <a href="https://github.com/gen2brain/beeep">beeep</a>, that uses pure Go code in all
1870 supported platforms. However, it also meant that the notifications in macOS
1871 looked ugly, since it used
1872 <a href="https://github.com/gen2brain/beeep/blob/master/beep_darwin.go#L20">osascript</a>.</p>
1873 <p>I wanted better integration with macOS, so this meant switching libraries.
1874 After searching for a while, the
1875 <a href="https://pkg.go.dev/gioui.org/x/notify">notify</a> library from
1876 <a href="https://gioui.org/">GioUI</a> is the one that seemed to work better. It
1877 implements notification in macOS using its native framework, so it works much
1878 better, but sadly it meant losing static binaries because it depends in CGO.</p>
1879 <p>Not a big loss initially, because I am only depending in Foundation inside
1880 macOS (that should always be available), and in Linux I could still statically
1881 compile. However I eventually added more features like sound
1882 (via <a href="https://github.com/gopxl/beep">beep</a>) and tray icon (via
1883 <a href="https://github.com/fyne-io/systray">systray</a>), that meant I needed CGO in both
1884 macOS and Linux.</p>
1885 <p>Losing static binaries in Linux is a much bigger deal, since Linux is a moving
1886 target. The general recommendation for building CGO binaries statically is
1887 using
1888 <a href="https://eli.thegreenplace.net/2024/building-static-binaries-with-go-on-linux/">musl</a>,
1889 but this also means building all dependencies that we need using musl (e.g.:
1890 <a href="https://github.com/ebitengine/oto?tab=readme-ov-file#linux"><code>ALSA</code></a> for
1891 [beep/oto]). This generally means pain, but Nix makes it easy.</p>
1892 <p>Let's start by creating a <a href="https://github.com/thiagokokada/twenty-twenty-twenty/blob/main/twenty-twenty-twenty.nix">Nix
1893 file</a>
1894 that builds our Go module (simplified below for brevity):</p>
1895 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span> <span class="n">lib</span>
1896 </span></span><span class="line"><span class="cl"><span class="o">,</span> <span class="n">stdenv</span>
1897 </span></span><span class="line"><span class="cl"><span class="o">,</span> <span class="n">alsa-lib</span>
1898 </span></span><span class="line"><span class="cl"><span class="o">,</span> <span class="n">buildGoModule</span>
1899 </span></span><span class="line"><span class="cl"><span class="o">,</span> <span class="n">pkg-config</span>
1900 </span></span><span class="line"><span class="cl"><span class="o">,</span> <span class="n">withStatic</span> <span class="o">?</span> <span class="no">false</span>
1901 </span></span><span class="line"><span class="cl"><span class="p">}:</span>
1902 </span></span><span class="line"><span class="cl">
1903 </span></span><span class="line"><span class="cl"><span class="n">buildGoModule</span> <span class="p">{</span>
1904 </span></span><span class="line"><span class="cl"> <span class="n">pname</span> <span class="o">=</span> <span class="s2">&#34;twenty-twenty-twenty&#34;</span><span class="p">;</span>
1905 </span></span><span class="line"><span class="cl"> <span class="n">version</span> <span class="o">=</span> <span class="s2">&#34;1.0.0&#34;</span><span class="p">;</span>
1906 </span></span><span class="line"><span class="cl"> <span class="n">src</span> <span class="o">=</span> <span class="n">lib</span><span class="o">.</span><span class="n">cleanSource</span> <span class="sr">./.</span><span class="p">;</span>
1907 </span></span><span class="line"><span class="cl"> <span class="n">vendorHash</span> <span class="o">=</span> <span class="s2">&#34;sha256-NzDhpJRogIfL2IYoqAUHoPh/ZdNnvnhEQ+kn8A+ZyBw=&#34;</span><span class="p">;</span>
1908 </span></span><span class="line"><span class="cl">
1909 </span></span><span class="line"><span class="cl"> <span class="n">CGO_ENABLED</span> <span class="o">=</span> <span class="s2">&#34;1&#34;</span><span class="p">;</span>
1910 </span></span><span class="line"><span class="cl">
1911 </span></span><span class="line"><span class="cl"> <span class="n">nativeBuildInputs</span> <span class="o">=</span> <span class="n">lib</span><span class="o">.</span><span class="n">optionals</span> <span class="p">(</span><span class="n">stdenv</span><span class="o">.</span><span class="n">hostPlatform</span><span class="o">.</span><span class="n">isLinux</span><span class="p">)</span> <span class="p">[</span>
1912 </span></span><span class="line"><span class="cl"> <span class="n">pkg-config</span>
1913 </span></span><span class="line"><span class="cl"> <span class="p">];</span>
1914 </span></span><span class="line"><span class="cl">
1915 </span></span><span class="line"><span class="cl"> <span class="n">buildInputs</span> <span class="o">=</span> <span class="n">lib</span><span class="o">.</span><span class="n">optionals</span> <span class="p">(</span><span class="n">stdenv</span><span class="o">.</span><span class="n">hostPlatform</span><span class="o">.</span><span class="n">isLinux</span><span class="p">)</span> <span class="p">[</span>
1916 </span></span><span class="line"><span class="cl"> <span class="n">alsa-lib</span>
1917 </span></span><span class="line"><span class="cl"> <span class="p">];</span>
1918 </span></span><span class="line"><span class="cl">
1919 </span></span><span class="line"><span class="cl"> <span class="n">ldflags</span> <span class="o">=</span> <span class="p">[</span> <span class="s2">&#34;-X=main.version=</span><span class="si">${</span><span class="n">version</span><span class="si">}</span><span class="s2">&#34;</span> <span class="s2">&#34;-s&#34;</span> <span class="s2">&#34;-w&#34;</span> <span class="p">]</span>
1920 </span></span><span class="line"><span class="cl"> <span class="o">++</span> <span class="n">lib</span><span class="o">.</span><span class="n">optionals</span> <span class="n">withStatic</span> <span class="p">[</span> <span class="s2">&#34;-linkmode external&#34;</span> <span class="s1">&#39;&#39;-extldflags &#34;-static&#34;&#39;&#39;</span> <span class="p">];</span>
1921 </span></span><span class="line"><span class="cl">
1922 </span></span><span class="line"><span class="cl"> <span class="n">meta</span> <span class="o">=</span> <span class="k">with</span> <span class="n">lib</span><span class="p">;</span> <span class="p">{</span>
1923 </span></span><span class="line"><span class="cl"> <span class="n">description</span> <span class="o">=</span> <span class="s2">&#34;Alerts every 20 minutes to look something at 20 feet away for 20 seconds&#34;</span><span class="p">;</span>
1924 </span></span><span class="line"><span class="cl"> <span class="n">homepage</span> <span class="o">=</span> <span class="s2">&#34;https://github.com/thiagokokada/twenty-twenty-twenty&#34;</span><span class="p">;</span>
1925 </span></span><span class="line"><span class="cl"> <span class="n">license</span> <span class="o">=</span> <span class="n">licenses</span><span class="o">.</span><span class="n">mit</span><span class="p">;</span>
1926 </span></span><span class="line"><span class="cl"> <span class="n">mainProgram</span> <span class="o">=</span> <span class="s2">&#34;twenty-twenty-twenty&#34;</span><span class="p">;</span>
1927 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
1928 </span></span><span class="line"><span class="cl"><span class="p">}</span>
1929 </span></span></code></pre><p>And we can build it with the following <code>flake.nix</code>:</p>
1930 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
1931 </span></span><span class="line"><span class="cl"> <span class="n">description</span> <span class="o">=</span> <span class="s2">&#34;twenty-twenty-twenty&#34;</span><span class="p">;</span>
1932 </span></span><span class="line"><span class="cl">
1933 </span></span><span class="line"><span class="cl"> <span class="n">inputs</span> <span class="o">=</span> <span class="p">{</span>
1934 </span></span><span class="line"><span class="cl"> <span class="n">nixpkgs</span><span class="o">.</span><span class="n">url</span> <span class="o">=</span> <span class="s2">&#34;nixpkgs/nixpkgs-unstable&#34;</span><span class="p">;</span>
1935 </span></span><span class="line"><span class="cl"> <span class="n">flake-compat</span><span class="o">.</span><span class="n">url</span> <span class="o">=</span> <span class="s2">&#34;github:edolstra/flake-compat&#34;</span><span class="p">;</span>
1936 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
1937 </span></span><span class="line"><span class="cl">
1938 </span></span><span class="line"><span class="cl"> <span class="n">outputs</span> <span class="o">=</span> <span class="p">{</span> <span class="n">self</span><span class="o">,</span> <span class="n">nixpkgs</span><span class="o">,</span> <span class="o">...</span> <span class="p">}:</span>
1939 </span></span><span class="line"><span class="cl"> <span class="k">let</span>
1940 </span></span><span class="line"><span class="cl"> <span class="n">supportedSystems</span> <span class="o">=</span> <span class="p">[</span> <span class="s2">&#34;x86_64-linux&#34;</span> <span class="s2">&#34;aarch64-linux&#34;</span> <span class="p">];</span>
1941 </span></span><span class="line"><span class="cl">
1942 </span></span><span class="line"><span class="cl"> <span class="c1"># Helper function to generate an attrset &#39;{ x86_64-linux = f &#34;x86_64-linux&#34;; ... }&#39;.</span>
1943 </span></span><span class="line"><span class="cl"> <span class="n">forAllSystems</span> <span class="o">=</span> <span class="n">nixpkgs</span><span class="o">.</span><span class="n">lib</span><span class="o">.</span><span class="n">genAttrs</span> <span class="n">supportedSystems</span><span class="p">;</span>
1944 </span></span><span class="line"><span class="cl">
1945 </span></span><span class="line"><span class="cl"> <span class="c1"># Nixpkgs instantiated for supported system types.</span>
1946 </span></span><span class="line"><span class="cl"> <span class="n">nixpkgsFor</span> <span class="o">=</span> <span class="n">forAllSystems</span> <span class="p">(</span><span class="n">system</span><span class="p">:</span> <span class="kn">import</span> <span class="n">nixpkgs</span> <span class="p">{</span> <span class="k">inherit</span> <span class="n">system</span><span class="p">;</span> <span class="p">});</span>
1947 </span></span><span class="line"><span class="cl"> <span class="k">in</span>
1948 </span></span><span class="line"><span class="cl"> <span class="p">{</span>
1949 </span></span><span class="line"><span class="cl"> <span class="n">packages</span> <span class="o">=</span> <span class="n">forAllSystems</span> <span class="p">(</span><span class="n">system</span><span class="p">:</span>
1950 </span></span><span class="line"><span class="cl"> <span class="k">let</span> <span class="n">pkgs</span> <span class="o">=</span> <span class="n">nixpkgsFor</span><span class="o">.</span><span class="si">${</span><span class="n">system</span><span class="si">}</span><span class="p">;</span>
1951 </span></span><span class="line"><span class="cl"> <span class="k">in</span> <span class="p">{</span>
1952 </span></span><span class="line"><span class="cl"> <span class="n">default</span> <span class="o">=</span> <span class="n">self</span><span class="o">.</span><span class="n">packages</span><span class="o">.</span><span class="si">${</span><span class="n">system</span><span class="si">}</span><span class="o">.</span><span class="n">twenty-twenty-twenty</span><span class="p">;</span>
1953 </span></span><span class="line"><span class="cl"> <span class="n">twenty-twenty-twenty</span> <span class="o">=</span> <span class="n">pkgs</span><span class="o">.</span><span class="n">callPackage</span> <span class="sr">./twenty-twenty-twenty.nix</span> <span class="p">{</span> <span class="p">};</span>
1954 </span></span><span class="line"><span class="cl"> <span class="n">twenty-twenty-twenty-static</span> <span class="o">=</span> <span class="n">pkgs</span><span class="o">.</span><span class="n">pkgsStatic</span><span class="o">.</span><span class="n">callPackage</span> <span class="sr">./twenty-twenty-twenty.nix</span> <span class="p">{</span>
1955 </span></span><span class="line"><span class="cl"> <span class="n">withStatic</span> <span class="o">=</span> <span class="no">true</span><span class="p">;</span>
1956 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
1957 </span></span><span class="line"><span class="cl"> <span class="p">});</span>
1958 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
1959 </span></span><span class="line"><span class="cl"><span class="p">}</span>
1960 </span></span></code></pre><p>I think this shows how powerful Nix is: the only difference between the normal
1961 build and a static build the usage of <code>pkgs.pkgsStatic</code> instead of <code>pkgs</code>. This
1962 automatically builds all packages statically with <code>musl</code>. Also we need pass
1963 some <a href="https://honnef.co/articles/statically-compiled-go-programs-always-even-with-cgo-using-musl/">extra
1964 flags</a>
1965 to the Go compiler (i.e.: <code>-linkmode external -extldflags &quot;-static&quot;</code>), but this
1966 is a requirement from Go.</p>
1967 <p>So, does it work? Let's test:</p>
1968 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> nix build .#twenty-twenty-twenty-static
1969 </span></span><span class="line"><span class="cl"><span class="err">
1970 </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="gp">$</span> file result/bin/twenty-twenty-twenty
1971 </span></span><span class="line"><span class="cl"><span class="go">result/bin/twenty-twenty-twenty: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, stripped
1972 </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="err">
1973 </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="gp">$</span> ./result/bin/twenty-twenty-twenty
1974 </span></span><span class="line"><span class="cl"><span class="go">ALSA lib pcm.c:2712:(snd_pcm_open_conf) Either /nix/store/dhn51w2km4fyf9ivi00rz03qs8q4mpng-pipewire-1.2.1/lib/alsa-lib/libasound_module_pcm_pipewire.so cannot be opened or _snd_pcm_pipewire_open was not defined inside
1975 </span></span></span><span class="line"><span class="cl"><span class="go">ALSA lib pcm.c:2712:(snd_pcm_open_conf) Either /nix/store/ly9d7llymzjyf6gi1455qzayqipk2kab-pipewire-1.2.1/lib/alsa-lib/libasound_module_pcm_pipewire.so cannot be opened or _snd_pcm_pipewire_open was not defined inside
1976 </span></span></span><span class="line"><span class="cl"><span class="go">ALSA lib pcm.c:2712:(snd_pcm_open_conf) Either /nix/store/dhn51w2km4fyf9ivi00rz03qs8q4mpng-pipewire-1.2.1/lib/alsa-lib/libasound_module_pcm_pipewire.so cannot be opened or _snd_pcm_pipewire_open was not defined inside
1977 </span></span></span><span class="line"><span class="cl"><span class="go">ALSA lib pcm.c:2712:(snd_pcm_open_conf) Either /nix/store/ly9d7llymzjyf6gi1455qzayqipk2kab-pipewire-1.2.1/lib/alsa-lib/libasound_module_pcm_pipewire.so cannot be opened or _snd_pcm_pipewire_open was not defined inside
1978 </span></span></span><span class="line"><span class="cl"><span class="go">ALSA lib pcm.c:2712:(snd_pcm_open_conf) Either /nix/store/dhn51w2km4fyf9ivi00rz03qs8q4mpng-pipewire-1.2.1/lib/alsa-lib/libasound_module_pcm_pipewire.so cannot be opened or _snd_pcm_pipewire_open was not defined inside
1979 </span></span></span><span class="line"><span class="cl"><span class="go">ALSA lib pcm.c:2712:(snd_pcm_open_conf) Either /nix/store/ly9d7llymzjyf6gi1455qzayqipk2kab-pipewire-1.2.1/lib/alsa-lib/libasound_module_pcm_pipewire.so cannot be opened or _snd_pcm_pipewire_open was not defined inside
1980 </span></span></span><span class="line"><span class="cl"><span class="go">2024-08-11T19:26:33+01:00 INFO Running twenty-twenty-twenty every 20.0 minute(s), with 20 second(s) duration and sound set to true
1981 </span></span></span></code></pre><p>There are some warns and sadly the sound doesn't work. I think the issue is
1982 related because of my usage of PipeWire and the binary may work in a pure ALSA
1983 system, but I don't have access to one. Maybe adding <code>pipewire</code> to
1984 <code>buildInputs</code> would fix this issue, but I can't get <code>pipewire</code> to be compiled
1985 statically (because of its dependencies). I think this is a good show how easy
1986 it is to statically compilation is in Nix, but also how complex static binaries
1987 are to get correctly.</p>
1988 <p>Bonus points for
1989 <a href="https://nix.dev/tutorials/cross-compilation.html">cross-compilation</a>. We can
1990 easily cross-compile by using <code>pkgsCross</code>:</p>
1991 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
1992 </span></span><span class="line"><span class="cl"> <span class="c1"># ...</span>
1993 </span></span><span class="line"><span class="cl"> <span class="n">outputs</span> <span class="o">=</span> <span class="p">{</span> <span class="n">self</span><span class="o">,</span> <span class="n">nixpkgs</span><span class="o">,</span> <span class="o">...</span> <span class="p">}:</span>
1994 </span></span><span class="line"><span class="cl"> <span class="k">let</span>
1995 </span></span><span class="line"><span class="cl"> <span class="c1"># ...</span>
1996 </span></span><span class="line"><span class="cl"> <span class="k">in</span>
1997 </span></span><span class="line"><span class="cl"> <span class="p">{</span>
1998 </span></span><span class="line"><span class="cl"> <span class="n">packages</span> <span class="o">=</span> <span class="n">forAllSystems</span> <span class="p">(</span><span class="n">system</span><span class="p">:</span>
1999 </span></span><span class="line"><span class="cl"> <span class="k">let</span> <span class="n">pkgs</span> <span class="o">=</span> <span class="n">nixpkgsFor</span><span class="o">.</span><span class="si">${</span><span class="n">system</span><span class="si">}</span><span class="p">;</span>
2000 </span></span><span class="line"><span class="cl"> <span class="k">in</span> <span class="p">{</span>
2001 </span></span><span class="line"><span class="cl"> <span class="n">twenty-twenty-twenty-static-aarch64</span> <span class="o">=</span> <span class="n">pkgs</span><span class="o">.</span><span class="n">pkgsCross</span><span class="o">.</span><span class="n">aarch64-multiplatform</span><span class="o">.</span><span class="n">pkgsStatic</span><span class="o">.</span><span class="n">callPackage</span> <span class="sr">./twenty-twenty-twenty.nix</span> <span class="p">{</span>
2002 </span></span><span class="line"><span class="cl"> <span class="n">withStatic</span> <span class="o">=</span> <span class="no">true</span><span class="p">;</span>
2003 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
2004 </span></span><span class="line"><span class="cl"> <span class="p">});</span>
2005 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
2006 </span></span><span class="line"><span class="cl"><span class="p">}</span>
2007 </span></span></code></pre><p>The idea of <code>pkgsCross</code> is to select a target platform (e.g.:
2008 <code>aarch64-multiplatform</code>) and use it as any other <code>pkgs</code>. We can even chain
2009 <code>pkgsStatic</code> to statically cross compile binaries:</p>
2010 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> nix build .#twenty-twenty-twenty-static-aarch64
2011 </span></span><span class="line"><span class="cl"><span class="err">
2012 </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="gp">$</span> file result/bin/twenty-twenty-twenty
2013 </span></span><span class="line"><span class="cl"><span class="go">result/bin/twenty-twenty-twenty: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, stripped
2014 </span></span></span></code></pre></summary></entry><entry><title>Meta: are quick bits really quick?</title><link href="https://kokada.dev/blog/meta-are-quick-bits-really-quick/" rel="alternate"/><published>2024-08-07T00:00:00+00:00</published><id>https://kokada.dev/blog/meta-are-quick-bits-really-quick/</id><summary type="html"><p>When I wrote my first <a href="https://kokada.dev/blog/quick-bits-nix-shell-is-cursed">quick
2015 bits</a> post in this
2016 blog, I gave that title prefix without much thought: I knew it was supposed to
2017 be a quicker post than my previous one, but I never thought too deeply about
2018 it. But after:</p>
2019 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> ls -lah **/*quick-bits*.md <span class="p">|</span> wc -l
2020 </span></span><span class="line"><span class="cl"><span class="go">4
2021 </span></span></span></code></pre><p>Well, 4 blog posts starting with the same prefix, I was curious: are those
2022 quick bits really quick, or at least quicker? Let's see:</p>
2023 <pre><code>$ wc -w **/*.md
2024 1107 2024-07-26/01-writing-nixos-tests-for-fun-and-profit.md
2025 1220 2024-07-26/02-using-github-as-a-bad-blog-platform.md
2026 286 2024-07-27/01-quick-bits-nix-shell-is-cursed.md
2027 387 2024-07-29/01-quick-bits-why-you-should-automate-everything.md
2028 1060 2024-07-29/02-go-a-reasonable-good-language.md
2029 1380 2024-07-30/01-first-impressions-fpgbc.md
2030 1238 2024-07-31/01-generating-yaml-files-with-nix.md
2031 2308 2024-08-01/01-troubleshooting-zsh-lag-and-solutions-with-nix.md
2032 504 2024-08-01/02-quick-bits-realise-nix-symlinks.md
2033 834 2024-08-04/01-make-nixd-modules-completion-work-anywhere-with-flakes.md
2034 1147 2024-08-05/01-my-favorite-device-is-a-chromebook.md
2035 394 2024-08-07/01-quick-bits-is-crostini-a-microvm.md
2036 120 README.md
2037 11985 total
2038 </code></pre>
2039 <p>While using <code>wc</code> is probably not the best way to measure word count (especially
2040 in this blog, since I tend to write lots of code snippets), I think this at
2041 least it gives me a good insight: yes, quick bits are quicker, and they're
2042 basically posts with a soft limit around 500 words. So expect in future this
2043 limit to be used.</p>
2044 <p>By the way, at:</p>
2045 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> wc -w 2024-08-07/02-meta-are-quick-bits-really-quick.md
2046 </span></span><span class="line"><span class="cl"><span class="go">220 2024-08-07/02-meta-are-quick-bits-really-quick.md
2047 </span></span></span></code></pre><p>This post is also technically a quick bits post, but &quot;quick bits meta&quot; would be
2048 too much. And yes, that last block of code is also meta ;).</p></summary></entry><entry><title>Quick bits: is Crostini a micro VM?</title><link href="https://kokada.dev/blog/quick-bits-is-crostini-a-micro-vm/" rel="alternate"/><published>2024-08-07T00:00:00+00:00</published><id>https://kokada.dev/blog/quick-bits-is-crostini-a-micro-vm/</id><summary type="html"><p><strong>Disclaimer</strong>: I am not a Virtual Machine specialist, this post is mainly the
2049 conclusion I got after some searching.</p>
2050 <p>Someone asked me in my <a href="https://kokada.dev/blog/my-favorite-device-is-a-chromebook">previous
2051 post</a> about my
2052 Chromebook if Crostini could be considered a micro VM. This was a interesting
2053 question, so I decided to do another post.</p>
2054 <p>To start, it is really difficult to get a good definition of what a &quot;micro VM&quot;
2055 is. <a href="https://firecracker-microvm.github.io/">Firecracker</a> defines itself as a
2056 micro VM, and describes itself in its website as:</p>
2057 <blockquote>
2058 <p>Firecracker is a virtual machine monitor (VMM) that uses the Linux
2059 Kernel-based Virtual Machine (KVM) to create and manage microVMs. Firecracker
2060 has a minimalist design. It excludes unnecessary devices and guest
2061 functionality to reduce the memory footprint and attack surface area of each
2062 microVM. This improves security, decreases the startup time, and increases
2063 hardware utilization.</p>
2064 </blockquote>
2065 <p>Now looking at Crostini, its heart is a VMM called
2066 <a href="https://crosvm.dev/">crosvm</a>. It is described in its
2067 <a href="https://chromium.googlesource.com/chromiumos/platform/crosvm/+/HEAD/README.md">README</a>
2068 as:</p>
2069 <blockquote>
2070 <p>crosvm is a virtual machine monitor (VMM) based on Linux’s KVM hypervisor,
2071 with a focus on simplicity, security, and speed. crosvm is intended to run
2072 Linux guests, originally as a security boundary for running native
2073 applications on the ChromeOS platform. Compared to QEMU, crosvm doesn’t
2074 emulate architectures or real hardware, instead concentrating on
2075 paravirtualized devices, such as the virtio standard.</p>
2076 </blockquote>
2077 <p>Similar descriptions right? Actually Firecracker website says it &quot;started from
2078 Chromium OS's Virtual Machine Monitor, crosvm, an open source VMM written in
2079 Rust&quot;. So I would say it is safe to say crosvm itself is a micro VM.</p>
2080 <p>But
2081 <a href="https://www.chromium.org/chromium-os/developer-library/guides/containers/containers-and-vms/">Crostini</a>
2082 itself is a combination of virtualization AND containerization. Basically
2083 inside the VM it runs a Linux kernel and <a href="https://linuxcontainers.org/">LXC</a>,
2084 that can start arbitrary containers inside it. From the Crostini documentation
2085 this choice seems to be to keep startup times down, and also to increase
2086 security (e.g.: in case of a security issue inside the container).</p>
2087 <p>This is definitely an interesting choice, since containers allow the overhead
2088 of each distro that you run inside Crostini to be low, and the main VM itself
2089 (called
2090 <a href="https://chromium.googlesource.com/chromiumos/overlays/board-overlays/+/HEAD/project-termina/">Termina</a>)
2091 should have low overhead too thanks to crosvm.</p>
2092 <p>By the way, if you want to learn more how &quot;devices&quot; works inside a micro VM
2093 like crosvm, I recommend <a href="https://prilik.com/blog/post/crosvm-paravirt/">this blog
2094 post</a> talking about
2095 paravirtualized devices in crosvm.</p></summary></entry><entry><title>My favorite device is a Chromebook</title><link href="https://kokada.dev/blog/my-favorite-device-is-a-chromebook/" rel="alternate"/><published>2024-08-05T00:00:00+00:00</published><id>https://kokada.dev/blog/my-favorite-device-is-a-chromebook/</id><summary type="html"><p>Most of the posts in this blog (including this one) and most of I would call
2096 &quot;personal computing&quot; that I do nowadays is mostly done in one of the most
2097 unremarkable devices that I own: a cheap <a href="https://www.lenovo.com/us/en/p/laptops/lenovo/lenovo-edu-chromebooks/ideapad-duet-3-chromebook-11-inch,-qlc/len101i0034">Chromebook Duet
2098 3</a>,
2099 that I bought for around EUR300. I was thinking why, because it is woefully
2100 underpowered: a <a href="https://www.anandtech.com/show/16696/qualcomm-announces-snapdragon-7c-gen-2-entrylevel-pc-and-chromebook-refresh">Snapdragon 7c Gen
2101 2</a>,
2102 a CPU that was already considered slow 3 years ago, coupled with an eMMC for
2103 storage, that is not much faster than a HDD. At least I have the 8GB RAM
2104 version instead of the 4GB one.</p>
2105 <p>It is a hybrid device, one that can be used as either a tablet or laptop, but
2106 it is compromised experience in both cases: as a tablet, it lacks the better
2107 touch optimised interface from iOS or Android; as a laptop, you have to depend
2108 on the stand to adjust the screen, and the detachable keyboard is worse than
2109 any laptop I have ever owned: getting keys stucked and characters being
2110 duplicated as a result is a common occurence. It is not so bad that I can't get
2111 things done though. About the trackpad: its biggest quality is that I never
2112 feel the need to use the touchscreen in laptop mode, that is to say that it is
2113 acceptable. Just crank up the pointer speed in ChromeOS settings, otherwise you
2114 never get anywhere since the trackpad is so small. There is also an active
2115 stylus, that helped me sometimes when I needed to sign something but otherwise
2116 I can't comment too much.</p>
2117 <p>But I really love this device. It is generally the only device that I bring in
2118 trips nowadays, because while it is compromised it works well enough: I can use
2119 to consume media in tablet mode (the fact that ChromeOS supports Android apps
2120 is a plus in those cases), browse the web and even do Linux stuff (more about
2121 this later). The fact that it is small (the size remembers me of a
2122 <a href="https://en.wikipedia.org/wiki/Netbook">netbook</a>), lightweight (~1KG, including
2123 the keyboard), has a good screen (that is bright and HiDPI) and good battery
2124 life (I don't have numbers but I almost never worry about it) is what makes
2125 this device the perfect companion to trips.</p>
2126 <p>Also, it has 2 USB-C ports and supports DisplayPort alt-mode, so it means you
2127 can charge it, connect to a external display and peripherals, all at the same
2128 time. Sadly, the maximum output resolution I got was 1080p (2560x1080),
2129 although some people at Reddit <a href="https://www.reddit.com/r/chromeos/comments/zh27tg/comment/izku724/?utm_source=share&amp;utm_medium=web3x&amp;utm_name=web3xcss&amp;utm_term=1&amp;utm_content=share_button">seems to have
2130 success</a>
2131 at 1440p, and the specs suggests it supports 4k. It may be my Dell S3423DWC
2132 monitor being wonky, the fact that it is Ultrawide or the cable, who knows? I
2133 even tried to change the monitor to &quot;High Resolution&quot; mode in settings, but to
2134 no avail.</p>
2135 <p><em>Update:</em> looking at the <a href="https://www.qualcomm.com/content/dam/qcomm-martech/dm-assets/documents/prod_brief_qcom_sd7c_gen2.pdf">product
2136 brief</a>
2137 for Snapdragon 7c Gen 2, it seems it supports up to QHD@60Hz (1440p) for the
2138 external display, and not 4k. This explains why it doesn't work at maximum
2139 resolution in my Dell S3423DWC, since while it is 1440p it is Ultrawide, so the
2140 resolution is bigger than QHD (3440x1440 vs 2560x1440).</p>
2141 <p>ChromeOS is also really interesting nowadays. To start, it is designed from the
2142 ground up to be a <a href="https://support.google.com/chromebook/answer/3438631">secure computing
2143 environment</a>, probably
2144 the most secure OS for consumers right now. Being a Chrome-first OS makes it a
2145 compromised experience, for example, it is the only device that I use Chrome as
2146 my main browser (since I personally prefer Firefox). But having a OS that boots
2147 fast is great: I never worry about OS updates because I know the device will be
2148 ready in seconds after a reboot. And the whole desktop experience inside the
2149 ChromeOS desktop is good, having shortcuts for many operations so you can get
2150 things done fast, and support for virtual desktops (ChromeOS call it &quot;desks&quot;)
2151 means you can organise your windows as much as you want.</p>
2152 <p>And what I think makes ChromeOS really powerful is
2153 <a href="https://chromeos.dev/en/linux">Crostini</a>, a full Linux VM that you can run
2154 inside ChromeOS. It runs Debian (it seems you can <a href="https://www.reddit.com/r/Crostini/wiki/howto/run-other-distros/">run other
2155 distros</a>
2156 though) with a deep integration with ChromeOS, so you can run even graphical
2157 programs without issues (including OpenGL!):</p>
2158 <p><a href="https://github.com/thiagokokada/blog/raw/main/posts/2024-08-05/Screenshot_2024-08-05_21.22.29.png"><img src="https://github.com/thiagokokada/blog/raw/main/posts/2024-08-05/Screenshot_2024-08-05_21.22.29.png" alt="Fastfetch inside Crostini with gitk running side-by-side."></a></p>
2159 <p><a href="https://github.com/thiagokokada/blog/raw/main/posts/2024-08-05/Screenshot_2024-08-05_21.39.58.png"><img src="https://github.com/thiagokokada/blog/raw/main/posts/2024-08-05/Screenshot_2024-08-05_21.39.58.png" alt="Running glxgears inside Crostini."></a></p>
2160 <p>This is all thanks to
2161 <a href="https://chromium.googlesource.com/chromiumos/platform2/+/HEAD/vm_tools/sommelier/README.md">Sommelier</a>,
2162 a nested Wayland compositor that runs inside Crostini and allow both Wayland
2163 and X11 applications to be forwarded to ChromeOS. The integration is so good
2164 that I can run Firefox inside Crostini and works well enough, but sadly Firefox
2165 is too slow in this device (I am not sure if the issue is ChromeOS or Firefox,
2166 but I suspect the later since Google does some optimisation per device).</p>
2167 <p>One interesting tidbit about the OpenGL situation in this device: this seems to
2168 be the first Chromebook to ship with open source drivers, thanks to Freedreno.
2169 There is <a href="https://www.youtube.com/watch?v=8mnjSmN03VM">this</a> very interesting
2170 presentation done by Rob Clark in XDC 2021, that I recommended anyone
2171 interested in free drivers to watch (the reference design of Duet 3 is
2172 <a href="https://chromeunboxed.com/chromebook-tablet-snapdragon-7c-homestar-coachz-strongbad">Strongbad</a>).</p>
2173 <p>The Crostini integration is probably the best VM integration with Linux I ever
2174 saw in an OS: you can manage files inside the VM, share directories between the
2175 OS and VM, copy and paste works between the two, GUI applications installed
2176 inside the VM appear in the ChromeOS menu, memory allocation inside the VM is
2177 transparent, etc. Even the themes for Linux GUI applications are customised to
2178 match ChromeOS. It is unironically one of the best Linux desktop experiences I
2179 ever had.</p>
2180 <p>Of course I am using Nix, but since the Crostini integration depends on some
2181 services being configured and installed, I decided to run Nix inside Debian
2182 instead of NixOS and run <a href="https://nix-community.github.io/home-manager/index.xhtml#sec-install-standalone">Home-Manager
2183 standalone</a>.
2184 I recommend checking the official <a href="https://wiki.nixos.org/wiki/Installing_Nix_on_Crostini">NixOS Wiki article about
2185 Crostini</a>, that details
2186 how to register applications in ChromeOS (so desktop applications appear in
2187 menu) and use <a href="https://github.com/nix-community/nixGL">nixGL</a> to make OpenGL
2188 applications work.</p>
2189 <p>Like I said at the start of the article, the device is woefully slow thanks to
2190 its CPU and eMMC. It does mean that, for example, activating my Home-Manager
2191 configuration takes a while (around 1 minute, vs a few seconds in my laptop).
2192 But it is much faster than say,
2193 <a href="https://github.com/nix-community/nix-on-droid-app">nix-on-droid</a>, that the
2194 last time I tried in a much more powerful device (<a href="https://www.gsmarena.com/xiaomi_pad_5-11042.php">Xiaomi Pad
2195 5</a>), took 30 minutes until I
2196 just decided to cancel the operation. Having a proper VM instead of
2197 <a href="https://wiki.termux.com/wiki/PRoot">proot</a> makes all the difference here.</p>
2198 <p>I can even do some light programming here: using my
2199 <a href="https://kokada.dev/blog/troubleshoting-zsh-lag-and-solutions-with-nix">ZSH</a>
2200 and neovim configuration (including LSP for coding) is reasonable fast. For
2201 example, I did most of the code that <a href="https://kokada.dev/blog/quick-bits-why-you-should-automate-everything">powers this
2202 blog</a>
2203 using this Chromebook. If I need more power, I can use the <a href="https://tailscale.com/kb/1267/install-chromebook">Tailscale app for
2204 Android</a> to connect to any
2205 other of my hosts via SSH. Yes, the Tailscale app works in Crostini, sadly
2206 without MagicDNS, so you need to use the internal Tailscale IPs instead.</p>
2207 <p>Until Google decides to give us a proper VM or user namespaces in Android or
2208 release a hybrid Chromebook device with better specs, this small Chromebook
2209 will probably stay as my travel companion, and is one of my favorite devices.</p></summary></entry><entry><title>Make nixd module completion to work anywhere (with Flakes)</title><link href="https://kokada.dev/blog/make-nixd-module-completion-to-work-anywhere-with-flakes/" rel="alternate"/><published>2024-08-04T00:00:00+00:00</published><id>https://kokada.dev/blog/make-nixd-module-completion-to-work-anywhere-with-flakes/</id><summary type="html"><p>If you want the TL;DR, go to the bottom of the post (search for &quot;final
2210 result&quot;).</p>
2211 <p>I recently switched from <a href="https://github.com/oxalica/nil">nil</a> to
2212 <a href="https://github.com/nix-community/nixd">nixd</a> as my LSP of choice for Nix. I
2213 was curious in <code>nixd</code> for a long time since the fact that it can eval Nix code
2214 means it can offer much more powerful completion than the <code>nil</code>'s static
2215 analysis, however it used to be difficult to setup. Nowadays it is much easier,
2216 basically doing the right thing as long as you have <code>NIX_PATH</code> setup, and you
2217 get both package and NixOS modules completion.</p>
2218 <p>Getting Home-Manager modules though needs some setup. The recommended way to
2219 setup accordingly to the <a href="https://github.com/nix-community/nixd/blob/fe202307eaf7e89c4366ed927af761482a6065c8/nixd/docs/configuration.md">official
2220 documentation</a>
2221 is to use the following for Flake based configurations (using neovim
2222 configuration here, but it should be easy to adapt to other editors):</p>
2223 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
2224 </span></span><span class="line"><span class="cl"> <span class="n">nixpkgs</span> <span class="o">=</span> <span class="p">{</span>
2225 </span></span><span class="line"><span class="cl"> <span class="n">expr</span> <span class="o">=</span> <span class="s2">&#34;import &lt;nixpkgs&gt; { }&#34;</span><span class="p">,</span>
2226 </span></span><span class="line"><span class="cl"> <span class="p">},</span>
2227 </span></span><span class="line"><span class="cl"> <span class="n">options</span> <span class="o">=</span> <span class="p">{</span>
2228 </span></span><span class="line"><span class="cl"> <span class="n">nixos</span> <span class="o">=</span> <span class="p">{</span>
2229 </span></span><span class="line"><span class="cl"> <span class="n">expr</span> <span class="o">=</span> <span class="s1">&#39;(builtins.getFlake (&#34;git+file://&#34; + toString ./.)).nixosConfigurations.miku-nixos.options&#39;</span><span class="p">,</span>
2230 </span></span><span class="line"><span class="cl"> <span class="p">},</span>
2231 </span></span><span class="line"><span class="cl"> <span class="n">home_manager</span> <span class="o">=</span> <span class="p">{</span>
2232 </span></span><span class="line"><span class="cl"> <span class="n">expr</span> <span class="o">=</span> <span class="s1">&#39;(builtins.getFlake (&#34;git+file://&#34; + toString ./.)).homeConfigurations.home-linux.options&#39;</span><span class="p">,</span>
2233 </span></span><span class="line"><span class="cl"> <span class="p">},</span>
2234 </span></span><span class="line"><span class="cl"> <span class="p">},</span>
2235 </span></span><span class="line"><span class="cl"> <span class="c1">-- ...</span>
2236 </span></span><span class="line"><span class="cl"><span class="p">}</span>
2237 </span></span></code></pre><p>This works, but it should be pretty clear the limitations of using <code>./.</code>: this
2238 will only work if you open your editor in your <a href="https://github.com/thiagokokada/nix-configs">Nix configuration
2239 repository</a>. For any other
2240 repository, <code>nixosConfigurations.miku-nixos</code> or <code>homeConfigurations.home-linux</code>
2241 will not exist and the completion will not work.</p>
2242 <p>It may look like this is easy to fix if you have
2243 <a href="https://nixos.org/manual/nixos/unstable/options#opt-_module.args"><code>specialArgs</code></a>
2244 (or <code>extraSpecialArgs</code> in Home-Manager) set to have your
2245 Flakes inputs, but:</p>
2246 <pre class="chroma"><code><span class="line"><span class="cl"><span class="c1"># By the way, ${self} does not exist in the Flake output by default, you need</span>
2247 </span></span><span class="line"><span class="cl"><span class="c1"># to explicit add `inherit self` to your outputs:</span>
2248 </span></span><span class="line"><span class="cl"><span class="c1"># https://discourse.nixos.org/t/who-is-self-in-flake-outputs/31859/4</span>
2249 </span></span><span class="line"><span class="cl"><span class="n">nix-repl</span><span class="o">&gt;</span> <span class="p">(</span><span class="nb">builtins</span><span class="o">.</span><span class="n">getFlake</span> <span class="s2">&#34;git+file://</span><span class="si">${</span><span class="n">self</span><span class="si">}</span><span class="s2">&#34;</span><span class="p">)</span><span class="o">.</span><span class="n">nixosConfigurations</span><span class="o">.</span><span class="n">miku-linux</span><span class="o">.</span><span class="n">options</span>
2250 </span></span><span class="line"><span class="cl"><span class="n">error</span><span class="p">:</span>
2251 </span></span><span class="line"><span class="cl"> <span class="err">…</span> <span class="n">while</span> <span class="n">calling</span> <span class="n">the</span> <span class="err">&#39;</span><span class="n">getFlake&#39;</span> <span class="n">builtin</span>
2252 </span></span><span class="line"><span class="cl"> <span class="n">at</span> <span class="err">«</span><span class="n">string</span><span class="err">»</span><span class="p">:</span><span class="mi">1</span><span class="p">:</span><span class="mi">2</span><span class="p">:</span>
2253 </span></span><span class="line"><span class="cl"> <span class="mi">1</span><span class="err">|</span> <span class="p">(</span><span class="nb">builtins</span><span class="o">.</span><span class="n">getFlake</span> <span class="s2">&#34;git+file://</span><span class="si">${</span><span class="n">self</span><span class="si">}</span><span class="s2">&#34;</span><span class="p">)</span>
2254 </span></span><span class="line"><span class="cl"> <span class="err">|</span> <span class="err">^</span>
2255 </span></span><span class="line"><span class="cl">
2256 </span></span><span class="line"><span class="cl"> <span class="err">…</span> <span class="n">while</span> <span class="n">evaluating</span> <span class="n">the</span> <span class="n">argument</span> <span class="n">passed</span> <span class="n">to</span> <span class="nb">builtins</span><span class="o">.</span><span class="n">getFlake</span>
2257 </span></span><span class="line"><span class="cl">
2258 </span></span><span class="line"><span class="cl"> <span class="n">error</span><span class="p">:</span> <span class="n">the</span> <span class="n">string</span> <span class="err">&#39;</span><span class="sd">git+file:///nix/store/avr1lcmznj8ghynh5vj1kakgfdf0zrxx-source&#39;</span> <span class="n">is</span> <span class="n">not</span> <span class="n">allowed</span> <span class="n">to</span> <span class="n">refer</span> <span class="n">to</span> <span class="n">a</span> <span class="n">store</span> <span class="n">path</span> <span class="p">(</span><span class="n">such</span> <span class="n">as</span> <span class="err">&#39;</span><span class="n">avr1lcmznj8ghynh5vj1kakgfdf0zrxx-source&#39;</span><span class="p">)</span>
2259 </span></span></code></pre><p>Well, it was worth a try. Another option would be to:</p>
2260 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">(</span><span class="nb">builtins</span><span class="o">.</span><span class="n">getFlake</span> <span class="s2">&#34;github:thiagokokada/nix-configs&#34;</span><span class="p">)</span><span class="o">.</span><span class="n">nixosConfigurations</span><span class="o">.</span><span class="n">miku-linux</span><span class="o">.</span><span class="n">options</span>
2261 </span></span><span class="line"><span class="cl"><span class="c1"># Or even something like this</span>
2262 </span></span><span class="line"><span class="cl"><span class="c1"># However, using ${rev} means this wouldn&#39;t work in dirty Flake repos, since</span>
2263 </span></span><span class="line"><span class="cl"><span class="c1"># ${rev} is not set in those cases</span>
2264 </span></span><span class="line"><span class="cl"><span class="p">(</span><span class="nb">builtins</span><span class="o">.</span><span class="n">getFlake</span> <span class="s2">&#34;github:thiagokokada/nix-configs/</span><span class="si">${</span><span class="n">rev</span><span class="si">}</span><span class="s2">&#34;</span><span class="p">)</span><span class="o">.</span><span class="n">nixosConfigurations</span><span class="o">.</span><span class="n">miku-linux</span><span class="o">.</span><span class="n">options</span>
2265 </span></span></code></pre><p>But while it works, it is slow, because it needs network to evaluate (and it is
2266 impure, since there is no <code>flake.lock</code>).</p>
2267 <p>The default configuration for <code>nixd</code> makes NixOS completion work even outside
2268 of my configuration repo, and it is fast. How? I decided to take a look at the
2269 <code>nixd</code> source code and found
2270 <a href="https://github.com/nix-community/nixd/blob/d938026c55c7c36a6e79afd9627459160b4924ed/nixd/lib/Controller/LifeTime.cpp#L33C11-L35C76">this</a>
2271 (formatted here for legibility):</p>
2272 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">(</span>
2273 </span></span><span class="line"><span class="cl"> <span class="k">let</span>
2274 </span></span><span class="line"><span class="cl"> <span class="n">pkgs</span> <span class="o">=</span> <span class="kn">import</span> <span class="sr">&lt;nixpkgs&gt;</span> <span class="p">{</span> <span class="p">};</span>
2275 </span></span><span class="line"><span class="cl"> <span class="k">in</span>
2276 </span></span><span class="line"><span class="cl"> <span class="p">(</span><span class="n">pkgs</span><span class="o">.</span><span class="n">lib</span><span class="o">.</span><span class="n">evalModules</span> <span class="p">{</span>
2277 </span></span><span class="line"><span class="cl"> <span class="n">modules</span> <span class="o">=</span> <span class="p">(</span><span class="kn">import</span> <span class="sr">&lt;nixpkgs/nixos/modules/module-list.nix&gt;</span><span class="p">)</span> <span class="o">++</span> <span class="p">[</span>
2278 </span></span><span class="line"><span class="cl"> <span class="p">({</span> <span class="o">...</span> <span class="p">}:</span> <span class="p">{</span> <span class="n">nixpkgs</span><span class="o">.</span><span class="n">hostPlatform</span> <span class="o">=</span> <span class="nb">builtins</span><span class="o">.</span><span class="n">currentSystem</span><span class="p">;</span> <span class="p">})</span>
2279 </span></span><span class="line"><span class="cl"> <span class="p">];</span>
2280 </span></span><span class="line"><span class="cl"> <span class="p">})</span>
2281 </span></span><span class="line"><span class="cl"><span class="p">)</span><span class="o">.</span><span class="n">options</span>
2282 </span></span></code></pre><p>Interesting, so they're manually loading the modules using <code>evalModules</code>. As I
2283 said above, it depends in <code>NIX_PATH</code> being correctly set. Can we fix this to
2284 use our Flake inputs instead? After some tries in the Nix REPL, I got the
2285 following:</p>
2286 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">(</span>
2287 </span></span><span class="line"><span class="cl"> <span class="k">let</span>
2288 </span></span><span class="line"><span class="cl"> <span class="n">pkgs</span> <span class="o">=</span> <span class="kn">import</span> <span class="s2">&#34;</span><span class="si">${</span><span class="n">inputs</span><span class="o">.</span><span class="n">nixpkgs</span><span class="si">}</span><span class="s2">&#34;</span> <span class="p">{</span> <span class="p">};</span>
2289 </span></span><span class="line"><span class="cl"> <span class="k">in</span>
2290 </span></span><span class="line"><span class="cl"> <span class="p">(</span><span class="n">pkgs</span><span class="o">.</span><span class="n">lib</span><span class="o">.</span><span class="n">evalModules</span> <span class="p">{</span>
2291 </span></span><span class="line"><span class="cl"> <span class="n">modules</span> <span class="o">=</span> <span class="p">(</span><span class="kn">import</span> <span class="s2">&#34;</span><span class="si">${</span><span class="n">inputs</span><span class="o">.</span><span class="n">nixpkgs</span><span class="si">}</span><span class="s2">/nixos/modules/module-list.nix&#34;</span><span class="p">)</span> <span class="o">++</span> <span class="p">[</span>
2292 </span></span><span class="line"><span class="cl"> <span class="p">({</span> <span class="o">...</span> <span class="p">}:</span> <span class="p">{</span> <span class="n">nixpkgs</span><span class="o">.</span><span class="n">hostPlatform</span> <span class="o">=</span> <span class="nb">builtins</span><span class="o">.</span><span class="n">currentSystem</span><span class="p">;</span> <span class="p">})</span>
2293 </span></span><span class="line"><span class="cl"> <span class="p">];</span>
2294 </span></span><span class="line"><span class="cl"> <span class="p">})</span>
2295 </span></span><span class="line"><span class="cl"><span class="p">)</span><span class="o">.</span><span class="n">options</span>
2296 </span></span></code></pre><p>So we can adapt this to the neovim configuration:</p>
2297 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
2298 </span></span><span class="line"><span class="cl"> <span class="n">options</span> <span class="o">=</span> <span class="p">{</span>
2299 </span></span><span class="line"><span class="cl"> <span class="n">nixos</span> <span class="o">=</span> <span class="p">{</span>
2300 </span></span><span class="line"><span class="cl"> <span class="n">expr</span> <span class="o">=</span> <span class="s1">&#39;(let pkgs = import &#34;${inputs.nixpkgs}&#34; { }; in (pkgs.lib.evalModules { modules = (import &#34;${inputs.nixpkgs}/nixos/modules/module-list.nix&#34;) ++ [ ({...}: { nixpkgs.hostPlatform = builtins.currentSystem;} ) ] ; })).options&#39;</span><span class="p">,</span>
2301 </span></span><span class="line"><span class="cl"> <span class="p">},</span>
2302 </span></span><span class="line"><span class="cl"> <span class="p">},</span>
2303 </span></span><span class="line"><span class="cl"><span class="p">}</span>
2304 </span></span></code></pre><p>This was easy. But the main issue is Home-Manager. How can we fix it? I needed
2305 to take a look at the Home-Manager <a href="https://github.com/nix-community/home-manager/blob/master/docs/default.nix#L161-L169">source
2306 code</a>
2307 to find the answer:</p>
2308 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">(</span>
2309 </span></span><span class="line"><span class="cl"> <span class="k">let</span>
2310 </span></span><span class="line"><span class="cl"> <span class="n">pkgs</span> <span class="o">=</span> <span class="kn">import</span> <span class="s2">&#34;</span><span class="si">${</span><span class="n">inputs</span><span class="o">.</span><span class="n">nixpkgs</span><span class="si">}</span><span class="s2">&#34;</span> <span class="p">{</span> <span class="p">};</span>
2311 </span></span><span class="line"><span class="cl"> <span class="n">lib</span> <span class="o">=</span> <span class="kn">import</span> <span class="s2">&#34;</span><span class="si">${</span><span class="n">inputs</span><span class="o">.</span><span class="n">home-manager</span><span class="si">}</span><span class="s2">/modules/lib/stdlib-extended.nix&#34;</span> <span class="n">pkgs</span><span class="o">.</span><span class="n">lib</span><span class="p">;</span>
2312 </span></span><span class="line"><span class="cl"> <span class="k">in</span>
2313 </span></span><span class="line"><span class="cl"> <span class="p">(</span><span class="n">lib</span><span class="o">.</span><span class="n">evalModules</span> <span class="p">{</span>
2314 </span></span><span class="line"><span class="cl"> <span class="n">modules</span> <span class="o">=</span> <span class="p">(</span><span class="kn">import</span> <span class="s2">&#34;</span><span class="si">${</span><span class="n">inputs</span><span class="o">.</span><span class="n">home-manager</span><span class="si">}</span><span class="s2">/modules/modules.nix&#34;</span><span class="p">)</span> <span class="p">{</span>
2315 </span></span><span class="line"><span class="cl"> <span class="k">inherit</span> <span class="n">lib</span> <span class="n">pkgs</span><span class="p">;</span>
2316 </span></span><span class="line"><span class="cl"> <span class="n">check</span> <span class="o">=</span> <span class="no">false</span><span class="p">;</span>
2317 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
2318 </span></span><span class="line"><span class="cl"> <span class="p">})</span>
2319 </span></span><span class="line"><span class="cl"><span class="p">)</span><span class="o">.</span><span class="n">options</span>
2320 </span></span></code></pre><p>The interesting part is: Home-Manager has its own extension of the module
2321 system (including <code>evalModules</code>). This includes e.g.: extra types used in
2322 Home-Manager only. Also, we need to disable <code>checks</code>, otherwise we will hit
2323 some validations (e.g.: missing <code>stateVersion</code>). I am not sure if this causes
2324 any issue for module completion yet, I may set it in the future.</p>
2325 <p>And for the final result:</p>
2326 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
2327 </span></span><span class="line"><span class="cl"> <span class="n">nixpkgs</span> <span class="o">=</span> <span class="p">{</span>
2328 </span></span><span class="line"><span class="cl"> <span class="n">expr</span> <span class="o">=</span> <span class="s1">&#39;import &#34;${flake.inputs.nixpkgs}&#34; { }&#39;</span><span class="p">,</span>
2329 </span></span><span class="line"><span class="cl"> <span class="p">},</span>
2330 </span></span><span class="line"><span class="cl"> <span class="n">options</span> <span class="o">=</span> <span class="p">{</span>
2331 </span></span><span class="line"><span class="cl"> <span class="n">nixos</span> <span class="o">=</span> <span class="p">{</span>
2332 </span></span><span class="line"><span class="cl"> <span class="n">expr</span> <span class="o">=</span> <span class="s1">&#39;(let pkgs = import &#34;${inputs.nixpkgs}&#34; { }; in (pkgs.lib.evalModules { modules = (import &#34;${inputs.nixpkgs}/nixos/modules/module-list.nix&#34;) ++ [ ({...}: { nixpkgs.hostPlatform = builtins.currentSystem;} ) ] ; })).options&#39;</span><span class="p">,</span>
2333 </span></span><span class="line"><span class="cl"> <span class="p">},</span>
2334 </span></span><span class="line"><span class="cl"> <span class="n">home_manager</span> <span class="o">=</span> <span class="p">{</span>
2335 </span></span><span class="line"><span class="cl"> <span class="n">expr</span> <span class="o">=</span> <span class="s1">&#39;(let pkgs = import &#34;${inputs.nixpkgs}&#34; { }; lib = import &#34;${inputs.home-manager}/modules/lib/stdlib-extended.nix&#34; pkgs.lib; in (lib.evalModules { modules = (import &#34;${inputs.home-manager}/modules/modules.nix&#34;) { inherit lib pkgs; check = false; }; })).options&#39;</span><span class="p">,</span>
2336 </span></span><span class="line"><span class="cl"> <span class="p">},</span>
2337 </span></span><span class="line"><span class="cl"> <span class="p">},</span>
2338 </span></span><span class="line"><span class="cl"><span class="p">}</span>
2339 </span></span></code></pre><p>Yes, it is quite a mouthful, but it makes module completion work in any
2340 repository, as long as you're using Flakes. And it is fast, since it doesn't
2341 need any network access. Since we are already here, let's define <code>nixpkgs</code> to
2342 not depend in the <code>NIX_PATH</code> being set too.</p></summary></entry><entry><title>Troubleshooting: ZSH lag and solutions with Nix</title><link href="https://kokada.dev/blog/troubleshoting-zsh-lag-and-solutions-with-nix/" rel="alternate"/><published>2024-08-01T00:00:00+00:00</published><id>https://kokada.dev/blog/troubleshoting-zsh-lag-and-solutions-with-nix/</id><summary type="html"><p>Inspired by this <a href="https://lock.cmpxchg8b.com/slowterm.html">blog post from
2343 Tavis</a>, I decided to document my own
2344 recent journey of reducing terminal (ZSH) lag startup. This post is way less
2345 interesting than the one from Tavis that uses a debugger to patch applications
2346 on the fly, but should still be interesting for some. And it also shows how
2347 powerful Nix can be for some things.</p>
2348 <p>For context, I have basically 3 systems where I interact with terminal
2349 frequently:</p>
2350 <ul>
2351 <li><a href="https://www.notebookcheck.net/Lenovo-ThinkPad-P14s-Gen-1-Laptop-Review-AMD-workstation-sans-dedicated-GPU.528746.0.html">Thinkpad P14s Gen 1</a>
2352 running NixOS, with a reasonable fast CPU (<a href="https://www.notebookcheck.net/AMD-Ryzen-7-PRO-4750U-Processor-Benchmarks-and-Specs.464396.0.html">AMD Ryzen 7 PRO
2353 4750U</a>)
2354 and disk (NVMe)</li>
2355 <li><a href="https://everymac.com/systems/apple/macbook_pro/specs/macbook-pro-m1-pro-10-core-cpu-16-core-gpu-16-2021-specs.html">MacBook
2356 Pro</a>
2357 with a really fast CPU (M1 Pro) and disk, but of course running macOS, being
2358 phased-out since this is a job owned machine and I am changing jobs right
2359 now, but should be replaced with another one soon™</li>
2360 <li><a href="https://chromeunboxed.com/lenovo-chromebook-duet-3-review-perfect-sequel">Chromebook Duet
2361 3</a>
2362 running ChromeOS, with slow CPU (<a href="https://www.notebookcheck.net/Qualcomm-Snapdragon-7c-Gen-2-Processor-Benchmarks-and-Specs.554426.0.html">Snapdragon 7c Gen
2363 2</a>)
2364 and disk (eMMC, really?)</li>
2365 </ul>
2366 <p>My experience is similar to Tavis, at around 300ms of startup time I don't care
2367 too much, but around 500ms+ is where I start to notice. I never had any issues
2368 with startup time in NixOS itself (I had issues with macOS before, but it was
2369 not actually the fault of macOS), but in the Chromebook it was awful: 600ms+
2370 with <a href="https://www.instabug.com/blog/understanding-cold-hot-and-warm-app-launch-time">hot
2371 start</a>,
2372 while cold start it could take multiple seconds.</p>
2373 <p>We can check how long ZSH takes to start by using:</p>
2374 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> <span class="nb">time</span> zsh -ci <span class="nb">exit</span>
2375 </span></span><span class="line"><span class="cl"><span class="go">zsh -ic exit 0.04s user 0.10s system 100% cpu 0.143 total
2376 </span></span></span></code></pre><p>The <code>-i</code> flag here is important, because we are interested in the interactive
2377 use of ZSH. Without this flag ZSH will ignore your <code>~/.zshrc</code> file, and the
2378 results will be meaningless.</p>
2379 <p>To do a more interesting benchmark, we can use
2380 <a href="https://github.com/sharkdp/hyperfine"><code>hyperfine</code></a>:</p>
2381 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> hyperfine <span class="s2">&#34;zsh -ic exit&#34;</span>
2382 </span></span><span class="line"><span class="cl"><span class="go">Benchmark 1: zsh -ic exit
2383 </span></span></span><span class="line"><span class="cl"><span class="go"> Time (mean ± σ): 145.4 ms ± 4.2 ms [User: 49.8 ms, System: 97.3 ms]
2384 </span></span></span><span class="line"><span class="cl"><span class="go"> Range (min … max): 138.6 ms … 155.3 ms 19 runs
2385 </span></span></span></code></pre><p>Hyperfine will run the command multiple times and take care of things like
2386 shell startup time. A really great tool to have in your toolbox by the way, but
2387 I digress.</p>
2388 <p>So let's do a little time travelling. Going back to commit
2389 <a href="https://github.com/thiagokokada/nix-configs/tree/b12757f90889653e359a1ab0a8cfd2f90cfabf31"><code>b12757f</code></a>
2390 from <a href="https://github.com/thiagokokada/nix-configs/">nix-configs</a>. Running
2391 <code>hyperfine</code> like above from my NixOS laptop, we have:</p>
2392 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> hyperfine <span class="s2">&#34;zsh -ic exit&#34;</span>
2393 </span></span><span class="line"><span class="cl"><span class="go">Benchmark 1: zsh -ic exit
2394 </span></span></span><span class="line"><span class="cl"><span class="go"> Time (mean ± σ): 218.6 ms ± 5.1 ms [User: 70.6 ms, System: 151.5 ms]
2395 </span></span></span><span class="line"><span class="cl"><span class="go"> Range (min … max): 210.3 ms … 227.0 ms 13 runs
2396 </span></span></span></code></pre><p>This doesn't look that bad, but let's see the same commit in my Chromebook:</p>
2397 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> hyperfine <span class="s2">&#34;zsh -ic exit&#34;</span>
2398 </span></span><span class="line"><span class="cl"><span class="go">Benchmark 1: zsh -ic exit
2399 </span></span></span><span class="line"><span class="cl"><span class="go"> Time (mean ± σ): 679.7 ms ± 40.2 ms [User: 230.8 ms, System: 448.5 ms]
2400 </span></span></span><span class="line"><span class="cl"><span class="go"> Range (min … max): 607.3 ms … 737.0 ms 10 runs
2401 </span></span></span></code></pre><p>Yikes, this is much worse. And those are the results after I retried the
2402 benchmark (so it is a hot start). The cold start times were above 3s. So let's
2403 investigate what is happening here. We can profile what is taking time during
2404 the startup of ZSH using <a href="https://www.bigbinary.com/blog/zsh-profiling">zprof</a>.
2405 You can add the following in your <code>~/.zshrc</code>:</p>
2406 <pre class="chroma"><code><span class="line"><span class="cl"><span class="c1"># At the top of your ~/.zshrc file</span>
2407 </span></span><span class="line"><span class="cl">zmodload zsh/zprof
2408 </span></span><span class="line"><span class="cl">
2409 </span></span><span class="line"><span class="cl"><span class="c1"># ...</span>
2410 </span></span><span class="line"><span class="cl">
2411 </span></span><span class="line"><span class="cl"><span class="c1"># At the end of your ~/.zshrc file</span>
2412 </span></span><span class="line"><span class="cl">zprof
2413 </span></span></code></pre><p>Or if using Home-Manager, use the
2414 <a href="https://nix-community.github.io/home-manager/options.xhtml#opt-programs.zsh.zprof.enable"><code>programs.zsh.zprof.enable</code></a>
2415 option. Once we restart ZSH, we will have something like:</p>
2416 <pre class="chroma"><code><span class="line"><span class="cl"><span class="go">num calls time self name
2417 </span></span></span><span class="line"><span class="cl"><span class="go">-----------------------------------------------------------------------------------
2418 </span></span></span><span class="line"><span class="cl"><span class="go"> 1) 1 36.91 36.91 34.29% 30.47 30.47 28.31% (anon) [/home/thiagoko/.zsh/plugins/zim-completion/init.zsh:13]
2419 </span></span></span><span class="line"><span class="cl"><span class="go"> 2) 1 25.43 25.43 23.63% 25.43 25.43 23.63% (anon) [/home/thiagoko/.zsh/plugins/zim-ssh/init.zsh:6]
2420 </span></span></span><span class="line"><span class="cl"><span class="go"> 3) 1 22.00 22.00 20.45% 21.92 21.92 20.36% _zsh_highlight_load_highlighters
2421 </span></span></span><span class="line"><span class="cl"><span class="go"> 4) 1 12.32 12.32 11.45% 12.32 12.32 11.45% autopair-init
2422 </span></span></span><span class="line"><span class="cl"><span class="go"> 5) 1 6.44 6.44 5.98% 6.44 6.44 5.98% compinit
2423 </span></span></span><span class="line"><span class="cl"><span class="go"> 6) 1 3.56 3.56 3.31% 3.48 3.48 3.23% prompt_pure_state_setup
2424 </span></span></span><span class="line"><span class="cl"><span class="go"> 7) 2 3.79 1.89 3.52% 2.85 1.43 2.65% async
2425 </span></span></span><span class="line"><span class="cl"><span class="go"> 8) 1 0.93 0.93 0.87% 0.93 0.93 0.87% async_init
2426 </span></span></span><span class="line"><span class="cl"><span class="go"> 9) 6 0.93 0.15 0.86% 0.93 0.15 0.86% is-at-least
2427 </span></span></span><span class="line"><span class="cl"><span class="go">10) 6 0.67 0.11 0.63% 0.67 0.11 0.63% add-zle-hook-widget
2428 </span></span></span><span class="line"><span class="cl"><span class="go">11) 1 8.25 8.25 7.66% 0.61 0.61 0.57% prompt_pure_setup
2429 </span></span></span><span class="line"><span class="cl"><span class="go">12) 1 0.40 0.40 0.37% 0.40 0.40 0.37% (anon) [/nix/store/p1zqypy7600fvfyl1v571bljx2l8zhay-zsh-autosuggestions-0.7.0/share/zsh-autosuggestions/zsh-autosuggestions.zsh:458]
2430 </span></span></span><span class="line"><span class="cl"><span class="go">13) 5 0.31 0.06 0.29% 0.31 0.06 0.29% add-zsh-hook
2431 </span></span></span><span class="line"><span class="cl"><span class="go">14) 1 0.60 0.60 0.56% 0.29 0.29 0.27% (anon) [/home/thiagoko/.zsh/plugins/zim-input/init.zsh:5]
2432 </span></span></span><span class="line"><span class="cl"><span class="go">15) 1 0.21 0.21 0.20% 0.21 0.21 0.20% compdef
2433 </span></span></span><span class="line"><span class="cl"><span class="go">16) 1 0.10 0.10 0.09% 0.10 0.10 0.09% _zsh_highlight__function_is_autoload_stub_p
2434 </span></span></span><span class="line"><span class="cl"><span class="go">17) 1 0.26 0.26 0.24% 0.08 0.08 0.08% _zsh_highlight__function_callable_p
2435 </span></span></span><span class="line"><span class="cl"><span class="go">18) 1 0.08 0.08 0.08% 0.08 0.08 0.08% prompt_pure_is_inside_container
2436 </span></span></span><span class="line"><span class="cl"><span class="go">19) 1 0.07 0.07 0.07% 0.07 0.07 0.07% _zsh_highlight__is_function_p
2437 </span></span></span><span class="line"><span class="cl"><span class="go">20) 1 0.01 0.01 0.01% 0.01 0.01 0.01% __wezterm_install_bash_prexec
2438 </span></span></span><span class="line"><span class="cl"><span class="go">21) 1 0.00 0.00 0.00% 0.00 0.00 0.00% _zsh_highlight_bind_widgets
2439 </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="gp">#</span> ...
2440 </span></span></code></pre><p>I ommited some output for brevit. The first 2 things that shows are from the
2441 <a href="https://github.com/zimfw/zimfw">zimfw</a>, the framework that I use to configure
2442 my ZSH (similar to Oh-My-Zsh). I actually don't use <code>zimfw</code> directly, instead I
2443 just load some modules that I find useful, like the <code>zim-completion</code> and
2444 <code>zim-ssh</code> that we can see above. By the way, Zim is generally really well
2445 optimised for startup time, but those 2 modules are kind slow.</p>
2446 <p>For <a href="https://github.com/zimfw/completion"><code>zim-completion</code></a>, after taking a
2447 look at it, there isn't much I could do. It seems that the reason
2448 <code>zim-completion</code> takes so long during startup is because it is trying to decide
2449 if it needs to recompile the completions (and replacing it with just a naive
2450 <code>autoload -U compinit &amp;&amp; compinit</code> is even worse for startup performance). I
2451 may eventually replace it for something else, but I really like what Zim brings
2452 here, so I decided to not touch it for now.</p>
2453 <p>However <a href="https://github.com/zimfw/ssh"><code>zim-ssh</code></a> is another history. The only
2454 reason I used it is to start a <code>ssh-agent</code> and keep it between multiple ZSH
2455 sessions. It shouldn't have that much influence in startup time. So I took a
2456 look the code (since it is small, I am reproducing it here):</p>
2457 <pre class="chroma"><code><span class="line"><span class="cl"><span class="c1">#</span>
2458 </span></span><span class="line"><span class="cl"><span class="c1"># Set up ssh-agent</span>
2459 </span></span><span class="line"><span class="cl"><span class="c1">#</span>
2460 </span></span><span class="line"><span class="cl">
2461 </span></span><span class="line"><span class="cl"><span class="c1"># Don&#39;t do anything unless we can actually use ssh-agent</span>
2462 </span></span><span class="line"><span class="cl"><span class="o">((</span> <span class="si">${</span><span class="p">+commands[ssh-agent]</span><span class="si">}</span> <span class="o">))</span> <span class="o">&amp;&amp;</span> <span class="o">()</span> <span class="o">{</span>
2463 </span></span><span class="line"><span class="cl"> ssh-add -l <span class="p">&amp;</span>&gt;/dev/null
2464 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="o">((</span> ? <span class="o">==</span> <span class="m">2</span> <span class="o">))</span><span class="p">;</span> <span class="k">then</span>
2465 </span></span><span class="line"><span class="cl"> <span class="c1"># Unable to contact the authentication agent</span>
2466 </span></span><span class="line"><span class="cl">
2467 </span></span><span class="line"><span class="cl"> <span class="c1"># Load stored agent connection info</span>
2468 </span></span><span class="line"><span class="cl"> <span class="nb">local</span> -r <span class="nv">ssh_env</span><span class="o">=</span><span class="si">${</span><span class="nv">HOME</span><span class="si">}</span>/.ssh-agent
2469 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="o">[[</span> -r <span class="si">${</span><span class="nv">ssh_env</span><span class="si">}</span> <span class="o">]]</span> <span class="nb">source</span> <span class="si">${</span><span class="nv">ssh_env</span><span class="si">}</span> &gt;/dev/null
2470 </span></span><span class="line"><span class="cl">
2471 </span></span><span class="line"><span class="cl"> ssh-add -l <span class="p">&amp;</span>&gt;/dev/null
2472 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="o">((</span> ? <span class="o">==</span> <span class="m">2</span> <span class="o">))</span><span class="p">;</span> <span class="k">then</span>
2473 </span></span><span class="line"><span class="cl"> <span class="c1"># Start agent and store agent connection info</span>
2474 </span></span><span class="line"><span class="cl"> <span class="o">(</span><span class="nb">umask</span> 066<span class="p">;</span> ssh-agent &gt;! <span class="si">${</span><span class="nv">ssh_env</span><span class="si">}</span><span class="o">)</span>
2475 </span></span><span class="line"><span class="cl"> <span class="nb">source</span> <span class="si">${</span><span class="nv">ssh_env</span><span class="si">}</span> &gt;/dev/null
2476 </span></span><span class="line"><span class="cl"> <span class="k">fi</span>
2477 </span></span><span class="line"><span class="cl"> <span class="k">fi</span>
2478 </span></span><span class="line"><span class="cl">
2479 </span></span><span class="line"><span class="cl"> <span class="c1"># Load identities</span>
2480 </span></span><span class="line"><span class="cl"> ssh-add -l <span class="p">&amp;</span>&gt;/dev/null
2481 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="o">((</span> ? <span class="o">==</span> <span class="m">1</span> <span class="o">))</span><span class="p">;</span> <span class="k">then</span>
2482 </span></span><span class="line"><span class="cl"> <span class="nb">local</span> -a zssh_ids
2483 </span></span><span class="line"><span class="cl"> zstyle -a <span class="s1">&#39;:zim:ssh&#39;</span> ids <span class="s1">&#39;zssh_ids&#39;</span>
2484 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="o">((</span> <span class="si">${#</span><span class="nv">zssh_ids</span><span class="si">}</span> <span class="o">))</span><span class="p">;</span> <span class="k">then</span>
2485 </span></span><span class="line"><span class="cl"> ssh-add <span class="si">${</span><span class="nv">HOME</span><span class="si">}</span>/.ssh/<span class="si">${</span><span class="p">^zssh_ids</span><span class="si">}</span> 2&gt;/dev/null
2486 </span></span><span class="line"><span class="cl"> <span class="k">else</span>
2487 </span></span><span class="line"><span class="cl"> ssh-add 2&gt;/dev/null
2488 </span></span><span class="line"><span class="cl"> <span class="k">fi</span>
2489 </span></span><span class="line"><span class="cl"> <span class="k">fi</span>
2490 </span></span><span class="line"><span class="cl"><span class="o">}</span>
2491 </span></span></code></pre><p>Well, this is bad. Let's assume the common path, where the <code>ssh-agent</code> is
2492 already running but you open a new shell instance (that doesn't have the
2493 connection info yet so it will need to load). This will run <code>ssh-add</code> at 4
2494 times. How long does <code>ssh-add</code> takes to run?</p>
2495 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> hyperfine -Ni <span class="s2">&#34;ssh-add -l&#34;</span>
2496 </span></span><span class="line"><span class="cl"><span class="go">Benchmark 1: ssh-add -l
2497 </span></span></span><span class="line"><span class="cl"><span class="go"> Time (mean ± σ): 4.6 ms ± 1.1 ms [User: 2.0 ms, System: 2.0 ms]
2498 </span></span></span><span class="line"><span class="cl"><span class="go"> Range (min … max): 3.4 ms … 8.7 ms 619 runs
2499 </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="err">
2500 </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="go"> Warning: Ignoring non-zero exit code.
2501 </span></span></span></code></pre><p>For those curious, <code>-N</code> disables the Shell usage, that works better when the
2502 command being tested is too fast.</p>
2503 <p>In average we have 4x4ms=16ms of startup time. But keep in mind the worst case
2504 can be much worse. The question is, how can we improve the situation here?</p>
2505 <p>After taking a look, I decided to write my own code, based in some ideas stolen
2506 from <a href="https://github.com/ohmyzsh/ohmyzsh/blob/67581c53c6458566e174620361e84b364b9034d2/plugins/ssh-agent/ssh-agent.plugin.zsh">Oh-My-Zsh ssh-agent
2507 plugin</a>.
2508 Here is final version of my
2509 <a href="https://github.com/thiagokokada/nix-configs/blob/e45a888f2bf3ce5644c3966f0b6371414d0291e2/home-manager/cli/ssh/ssh-agent.zsh">code</a>:</p>
2510 <pre class="chroma"><code><span class="line"><span class="cl">zmodload zsh/net/socket
2511 </span></span><span class="line"><span class="cl">
2512 </span></span><span class="line"><span class="cl">_check_agent<span class="o">(){</span>
2513 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="o">[[</span> -S <span class="s2">&#34;</span><span class="nv">$SSH_AUTH_SOCK</span><span class="s2">&#34;</span> <span class="o">]]</span> <span class="o">&amp;&amp;</span> zsocket <span class="s2">&#34;</span><span class="nv">$SSH_AUTH_SOCK</span><span class="s2">&#34;</span> 2&gt;/dev/null<span class="p">;</span> <span class="k">then</span>
2514 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="m">0</span>
2515 </span></span><span class="line"><span class="cl"> <span class="k">fi</span>
2516 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="m">1</span>
2517 </span></span><span class="line"><span class="cl"><span class="o">}</span>
2518 </span></span><span class="line"><span class="cl">
2519 </span></span><span class="line"><span class="cl">_start_agent<span class="o">()</span> <span class="o">{</span>
2520 </span></span><span class="line"><span class="cl"> <span class="c1"># Test if $SSH_AUTH_SOCK is visible, in case we start e.g.: ssh-agent via</span>
2521 </span></span><span class="line"><span class="cl"> <span class="c1"># systemd service</span>
2522 </span></span><span class="line"><span class="cl"> <span class="k">if</span> _check_agent<span class="p">;</span> <span class="k">then</span>
2523 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="m">0</span>
2524 </span></span><span class="line"><span class="cl"> <span class="k">fi</span>
2525 </span></span><span class="line"><span class="cl">
2526 </span></span><span class="line"><span class="cl"> <span class="c1"># Get the filename to store/lookup the environment from</span>
2527 </span></span><span class="line"><span class="cl"> <span class="nb">local</span> -r <span class="nv">ssh_env_cache</span><span class="o">=</span><span class="s2">&#34;</span><span class="nv">$HOME</span><span class="s2">/.ssh-agent&#34;</span>
2528 </span></span><span class="line"><span class="cl">
2529 </span></span><span class="line"><span class="cl"> <span class="c1"># Check if ssh-agent is already running</span>
2530 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="o">[[</span> -f <span class="s2">&#34;</span><span class="nv">$ssh_env_cache</span><span class="s2">&#34;</span> <span class="o">]]</span><span class="p">;</span> <span class="k">then</span>
2531 </span></span><span class="line"><span class="cl"> <span class="nb">source</span> <span class="s2">&#34;</span><span class="nv">$ssh_env_cache</span><span class="s2">&#34;</span> &gt; /dev/null
2532 </span></span><span class="line"><span class="cl">
2533 </span></span><span class="line"><span class="cl"> <span class="c1"># Test if $SSH_AUTH_SOCK is visible, e.g.: the ssh-agent is still alive</span>
2534 </span></span><span class="line"><span class="cl"> <span class="k">if</span> _check_agent<span class="p">;</span> <span class="k">then</span>
2535 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="m">0</span>
2536 </span></span><span class="line"><span class="cl"> <span class="k">fi</span>
2537 </span></span><span class="line"><span class="cl"> <span class="k">fi</span>
2538 </span></span><span class="line"><span class="cl">
2539 </span></span><span class="line"><span class="cl"> <span class="c1"># start ssh-agent and setup environment</span>
2540 </span></span><span class="line"><span class="cl"> <span class="o">(</span>
2541 </span></span><span class="line"><span class="cl"> <span class="nb">umask</span> <span class="m">066</span>
2542 </span></span><span class="line"><span class="cl"> ssh-agent -s &gt;! <span class="s2">&#34;</span><span class="nv">$ssh_env_cache</span><span class="s2">&#34;</span>
2543 </span></span><span class="line"><span class="cl"> <span class="o">)</span>
2544 </span></span><span class="line"><span class="cl"> <span class="nb">source</span> <span class="s2">&#34;</span><span class="nv">$ssh_env_cache</span><span class="s2">&#34;</span> &gt; /dev/null
2545 </span></span><span class="line"><span class="cl"><span class="o">}</span>
2546 </span></span><span class="line"><span class="cl">
2547 </span></span><span class="line"><span class="cl">_start_agent
2548 </span></span><span class="line"><span class="cl">unfunction _check_agent _start_agent
2549 </span></span></code></pre><p>The idea here is simple: using
2550 <a href="https://zsh.sourceforge.io/Doc/Release/Zsh-Modules.html#The-zsh_002fnet_002fsocket-Module"><code>zsocket</code></a>
2551 module from ZSH itself to check if the <code>ssh-agent</code> is working instead of
2552 executing <code>ssh-add -l</code>. The only case we run any program now is to start the
2553 agent itself if needed. Let's run <code>hyperfine</code> again:</p>
2554 <pre><code>$ hyperfine &quot;zsh -ic exit&quot;
2555 Benchmark 1: zsh -ic exit
2556 Time (mean ± σ): 188.3 ms ± 8.2 ms [User: 61.1 ms, System: 130.0 ms]
2557 Range (min … max): 170.9 ms … 198.4 ms 16 runs
2558 </code></pre>
2559 <p>Got a good improvement here already. Let's see <code>zprof</code> again:</p>
2560 <pre class="chroma"><code><span class="line"><span class="cl"><span class="go">num calls time self name
2561 </span></span></span><span class="line"><span class="cl"><span class="go">-----------------------------------------------------------------------------------
2562 </span></span></span><span class="line"><span class="cl"><span class="go"> 1) 1 41.23 41.23 48.66% 33.52 33.52 39.56% (anon) [/home/thiagoko/.zsh/plugins/zim-completion/init.zsh:13]
2563 </span></span></span><span class="line"><span class="cl"><span class="go"> 2) 1 22.23 22.23 26.24% 22.12 22.12 26.10% _zsh_highlight_load_highlighters
2564 </span></span></span><span class="line"><span class="cl"><span class="go"> 3) 1 8.90 8.90 10.51% 8.90 8.90 10.51% Gautopair-init
2565 </span></span></span><span class="line"><span class="cl"><span class="go"> 4) 1 7.71 7.71 9.10% 7.71 7.71 9.10% compinit
2566 </span></span></span><span class="line"><span class="cl"><span class="go"> 5) 1 5.74 5.74 6.77% 5.60 5.60 6.60% prompt_pure_state_setup
2567 </span></span></span><span class="line"><span class="cl"><span class="go"> 6) 6 1.19 0.20 1.41% 1.19 0.20 1.41% add-zle-hook-widget
2568 </span></span></span><span class="line"><span class="cl"><span class="go"> 7) 2 1.97 0.99 2.33% 1.14 0.57 1.34% async
2569 </span></span></span><span class="line"><span class="cl"><span class="go"> 8) 6 0.87 0.15 1.03% 0.87 0.15 1.03% is-at-least
2570 </span></span></span><span class="line"><span class="cl"><span class="go"> 9) 1 0.84 0.84 0.99% 0.84 0.84 0.99% async_init
2571 </span></span></span><span class="line"><span class="cl"><span class="go">10) 1 9.30 9.30 10.97% 0.72 0.72 0.84% prompt_pure_setup
2572 </span></span></span><span class="line"><span class="cl"><span class="go">11) 5 0.63 0.13 0.75% 0.63 0.13 0.75% add-zsh-hook
2573 </span></span></span><span class="line"><span class="cl"><span class="go">12) 1 0.41 0.41 0.48% 0.41 0.41 0.48% _start_agent
2574 </span></span></span><span class="line"><span class="cl"><span class="go">13) 1 0.31 0.31 0.37% 0.31 0.31 0.37% (anon) [/nix/store/p1zqypy7600fvfyl1v571bljx2l8zhay-zsh-autosuggestions-0.7.0/share/zsh-autosuggestions/zsh-autosuggestions.zsh:458]
2575 </span></span></span><span class="line"><span class="cl"><span class="go">14) 1 0.55 0.55 0.64% 0.24 0.24 0.28% (anon) [/home/thiagoko/.zsh/plugins/zim-input/init.zsh:5]
2576 </span></span></span><span class="line"><span class="cl"><span class="go">15) 1 0.14 0.14 0.16% 0.14 0.14 0.16% prompt_pure_is_inside_container
2577 </span></span></span><span class="line"><span class="cl"><span class="go">16) 1 0.14 0.14 0.16% 0.14 0.14 0.16% compdef
2578 </span></span></span><span class="line"><span class="cl"><span class="go">17) 1 0.09 0.09 0.11% 0.09 0.09 0.11% _zsh_highlight__function_is_autoload_stub_p
2579 </span></span></span><span class="line"><span class="cl"><span class="go">18) 1 0.25 0.25 0.29% 0.08 0.08 0.09% _zsh_highlight__function_callable_p
2580 </span></span></span><span class="line"><span class="cl"><span class="go">19) 1 0.07 0.07 0.09% 0.07 0.07 0.09% _zsh_highlight__is_function_p
2581 </span></span></span><span class="line"><span class="cl"><span class="go">20) 1 0.01 0.01 0.01% 0.01 0.01 0.01% __wezterm_install_bash_prexec
2582 </span></span></span><span class="line"><span class="cl"><span class="go">21) 1 0.01 0.01 0.01% 0.01 0.01 0.01% _zsh_highlight_bind_widgets
2583 </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="gp">#</span> ...
2584 </span></span></code></pre><p>Well, there is nothing interesting here anymore. I mean, <code>zim-completion</code> is
2585 still the main culprit, but nothing to do for now. Instead of looking at
2586 <code>zproof</code>, let's take a look at my <code>~/.zshrc</code> instead:</p>
2587 <pre class="chroma"><code><span class="line"><span class="cl"><span class="c1"># ...</span>
2588 </span></span><span class="line"><span class="cl"><span class="k">if</span> <span class="o">[[</span> <span class="nv">$options</span><span class="o">[</span>zle<span class="o">]</span> <span class="o">=</span> on <span class="o">]]</span><span class="p">;</span> <span class="k">then</span>
2589 </span></span><span class="line"><span class="cl"> <span class="nb">eval</span> <span class="s2">&#34;</span><span class="k">$(</span>/nix/store/sk6wsgp4h477baxypksz9rl8ldwwh9yg-fzf-0.54.0/bin/fzf --zsh<span class="k">)</span><span class="s2">&#34;</span>
2590 </span></span><span class="line"><span class="cl"><span class="k">fi</span>
2591 </span></span><span class="line"><span class="cl">
2592 </span></span><span class="line"><span class="cl"><span class="c1"># ...</span>
2593 </span></span><span class="line"><span class="cl">/nix/store/x3yblr73r5x76dmaanjk3333mvzxc49r-any-nix-shell-1.2.1/bin/any-nix-shell zsh <span class="p">|</span> <span class="nb">source</span> /dev/stdin
2594 </span></span><span class="line"><span class="cl">
2595 </span></span><span class="line"><span class="cl"><span class="c1"># ...</span>
2596 </span></span><span class="line"><span class="cl"><span class="nb">eval</span> <span class="s2">&#34;</span><span class="k">$(</span>/nix/store/330d6k81flfs6w46b44afmncxk57qggv-zoxide-0.9.4/bin/zoxide init zsh <span class="k">)</span><span class="s2">&#34;</span>
2597 </span></span><span class="line"><span class="cl">
2598 </span></span><span class="line"><span class="cl"><span class="c1"># ...</span>
2599 </span></span><span class="line"><span class="cl"><span class="nb">eval</span> <span class="s2">&#34;</span><span class="k">$(</span>/nix/store/8l9j9kdv9m0z0s30lp4yvrc9s5bcbgmx-direnv-2.34.0/bin/direnv hook zsh<span class="k">)</span><span class="s2">&#34;</span>
2600 </span></span></code></pre><p>So you see, starting all those programs during ZSH startup can hurt the shell
2601 startup considerable. Not necessary for commands fast like <code>fzf</code> (that is
2602 written in Go), but let's see
2603 <a href="https://github.com/haslersn/any-nix-shell"><code>any-nix-shell</code></a>, that is written
2604 in shell script:</p>
2605 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> hyperfine <span class="s2">&#34;any-nix-shell zsh&#34;</span>
2606 </span></span><span class="line"><span class="cl"><span class="go">Benchmark 1: any-nix-shell zsh
2607 </span></span></span><span class="line"><span class="cl"><span class="go"> Time (mean ± σ): 16.0 ms ± 1.8 ms [User: 5.6 ms, System: 10.5 ms]
2608 </span></span></span><span class="line"><span class="cl"><span class="go"> Range (min … max): 11.3 ms … 20.3 ms 143 runs
2609 </span></span></span></code></pre><p>This is bad, consistently bad actually. Even for commands that are fast, keep
2610 in mind that there is a difference between the cold and hot start again. For
2611 example, <code>fzf</code>:</p>
2612 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> hyperfine -N <span class="s2">&#34;fzf --zsh&#34;</span>
2613 </span></span><span class="line"><span class="cl"><span class="go">Benchmark 1: fzf --zsh
2614 </span></span></span><span class="line"><span class="cl"><span class="go"> Time (mean ± σ): 2.9 ms ± 0.9 ms [User: 0.6 ms, System: 2.3 ms]
2615 </span></span></span><span class="line"><span class="cl"><span class="go"> Range (min … max): 1.7 ms … 6.8 ms 1113 runs
2616 </span></span></span></code></pre><p>See the range? While 1.7ms is something that is probably difficult to notice,
2617 6.8ms can be noticiable, especially if this accumulates with other slow
2618 starting apps.</p>
2619 <p>And the thing is, all those commands are just generating in the end a fixed
2620 output, at least for the current version of the program. Can we pre-generate
2621 them instead? If using Nix, of course we can:</p>
2622 <pre class="chroma"><code><span class="line"><span class="cl"><span class="c1"># You need to disable the default integration</span>
2623 </span></span><span class="line"><span class="cl"><span class="n">programs</span><span class="o">.</span><span class="n">direnv</span><span class="o">.</span><span class="n">enableZshIntegration</span> <span class="err">=</span> <span class="no">false</span><span class="p">;</span>
2624 </span></span><span class="line"><span class="cl"><span class="n">programs</span><span class="o">.</span><span class="n">fzf</span><span class="o">.</span><span class="n">enableZshIntegration</span> <span class="err">=</span> <span class="no">false</span><span class="p">;</span>
2625 </span></span><span class="line"><span class="cl"><span class="n">programs</span><span class="o">.</span><span class="n">zoxide</span><span class="o">.</span><span class="n">enableZshIntegration</span> <span class="err">=</span> <span class="no">false</span><span class="p">;</span>
2626 </span></span><span class="line"><span class="cl">
2627 </span></span><span class="line"><span class="cl"><span class="n">programs</span><span class="o">.</span><span class="n">zsh</span><span class="o">.</span><span class="n">initExtra</span> <span class="err">=</span>
2628 </span></span><span class="line"><span class="cl"> <span class="c1"># bash</span>
2629 </span></span><span class="line"><span class="cl"> <span class="s1">&#39;&#39;
2630 </span></span></span><span class="line"><span class="cl"><span class="s1"> # any-nix-shell
2631 </span></span></span><span class="line"><span class="cl"><span class="s1"> source </span><span class="si">${</span>
2632 </span></span><span class="line"><span class="cl"> <span class="n">pkgs</span><span class="o">.</span><span class="n">runCommand</span> <span class="s2">&#34;any-nix-shell-zsh&#34;</span> <span class="p">{</span> <span class="p">}</span> <span class="s1">&#39;&#39;
2633 </span></span></span><span class="line"><span class="cl"><span class="s1"> </span><span class="si">${</span><span class="n">lib</span><span class="o">.</span><span class="n">getExe</span> <span class="n">pkgs</span><span class="o">.</span><span class="n">any-nix-shell</span><span class="si">}</span><span class="s1"> zsh &gt; $out
2634 </span></span></span><span class="line"><span class="cl"><span class="s1"> &#39;&#39;</span>
2635 </span></span><span class="line"><span class="cl"> <span class="si">}</span><span class="s1">
2636 </span></span></span><span class="line"><span class="cl"><span class="s1">
2637 </span></span></span><span class="line"><span class="cl"><span class="s1"> # fzf
2638 </span></span></span><span class="line"><span class="cl"><span class="s1"> source </span><span class="si">${</span><span class="n">config</span><span class="o">.</span><span class="n">programs</span><span class="o">.</span><span class="n">fzf</span><span class="o">.</span><span class="n">package</span><span class="si">}</span><span class="s1">/share/fzf/completion.zsh
2639 </span></span></span><span class="line"><span class="cl"><span class="s1"> source </span><span class="si">${</span><span class="n">config</span><span class="o">.</span><span class="n">programs</span><span class="o">.</span><span class="n">fzf</span><span class="o">.</span><span class="n">package</span><span class="si">}</span><span class="s1">/share/fzf/key-bindings.zsh
2640 </span></span></span><span class="line"><span class="cl"><span class="s1">
2641 </span></span></span><span class="line"><span class="cl"><span class="s1"> # zoxide
2642 </span></span></span><span class="line"><span class="cl"><span class="s1"> source </span><span class="si">${</span>
2643 </span></span><span class="line"><span class="cl"> <span class="n">pkgs</span><span class="o">.</span><span class="n">runCommand</span> <span class="s2">&#34;zoxide-init-zsh&#34;</span> <span class="p">{</span> <span class="p">}</span> <span class="s1">&#39;&#39;
2644 </span></span></span><span class="line"><span class="cl"><span class="s1"> </span><span class="si">${</span><span class="n">lib</span><span class="o">.</span><span class="n">getExe</span> <span class="n">config</span><span class="o">.</span><span class="n">programs</span><span class="o">.</span><span class="n">zoxide</span><span class="o">.</span><span class="n">package</span><span class="si">}</span><span class="s1"> init zsh &gt; $out
2645 </span></span></span><span class="line"><span class="cl"><span class="s1"> &#39;&#39;</span>
2646 </span></span><span class="line"><span class="cl"> <span class="si">}</span><span class="s1">
2647 </span></span></span><span class="line"><span class="cl"><span class="s1">
2648 </span></span></span><span class="line"><span class="cl"><span class="s1"> # direnv
2649 </span></span></span><span class="line"><span class="cl"><span class="s1"> source </span><span class="si">${</span>
2650 </span></span><span class="line"><span class="cl"> <span class="n">pkgs</span><span class="o">.</span><span class="n">runCommand</span> <span class="s2">&#34;direnv-hook-zsh&#34;</span> <span class="p">{</span> <span class="p">}</span> <span class="s1">&#39;&#39;
2651 </span></span></span><span class="line"><span class="cl"><span class="s1"> </span><span class="si">${</span><span class="n">lib</span><span class="o">.</span><span class="n">getExe</span> <span class="n">config</span><span class="o">.</span><span class="n">programs</span><span class="o">.</span><span class="n">direnv</span><span class="o">.</span><span class="n">package</span><span class="si">}</span><span class="s1"> hook zsh &gt; $out
2652 </span></span></span><span class="line"><span class="cl"><span class="s1"> &#39;&#39;</span>
2653 </span></span><span class="line"><span class="cl"> <span class="si">}</span><span class="s1">
2654 </span></span></span><span class="line"><span class="cl"><span class="s1"> &#39;&#39;</span><span class="p">;</span>
2655 </span></span></code></pre><p>So we can use <code>pkgs.runCommand</code> to run those commands during build time and
2656 <code>source</code> the result. <code>fzf</code> actually doesn't need this since we have the files
2657 already generated in the package. I think this is one of those things that
2658 really shows the power of Nix: I wouldn't do something similar if I didn't use
2659 Nix because the risk of breaking something later is big (e.g.: forgetting to
2660 update the generated files), but Nix makes those things trivial.</p>
2661 <p>Let's run <code>hyperfine</code> again:</p>
2662 <pre><code>$ hyperfine &quot;zsh -ic exit&quot;
2663 Benchmark 1: zsh -ic exit
2664 Time (mean ± σ): 162.3 ms ± 4.9 ms [User: 52.7 ms, System: 111.1 ms]
2665 Range (min … max): 153.0 ms … 173.4 ms 19 runs
2666 </code></pre>
2667 <p>Another good improvement. The last change I did is switching between
2668 <a href="https://github.com/zsh-users/zsh-syntax-highlighting"><code>zsh-syntax-highlighting</code></a>
2669 to
2670 <a href="https://github.com/zdharma-continuum/fast-syntax-highlighting"><code>zsh-fast-syntax-highlighting</code></a>,
2671 that is supposed to be faster and have better highlighting too. I got that from
2672 <code>_zsh_highlight_load_highlighters</code> using 26% of the time from my <code>zprof</code> above.
2673 And for the final <code>hyperfine</code> in my laptop:</p>
2674 <pre><code>$ hyperfine &quot;zsh -ic exit&quot;
2675 Benchmark 1: zsh -ic exit
2676 Time (mean ± σ): 138.3 ms ± 7.1 ms [User: 47.5 ms, System: 91.9 ms]
2677 Range (min … max): 123.8 ms … 157.9 ms 21 runs
2678 </code></pre>
2679 <p>A ~36% improvement, not bad. Let's see how it fares in my Chromebook:</p>
2680 <pre><code>$ hyperfine &quot;zsh -ic exit&quot;
2681 Benchmark 1: zsh -ic exit
2682 Time (mean ± σ): 278.2 ms ± 46.9 ms [User: 88.0 ms, System: 184.8 ms]
2683 Range (min … max): 204.7 ms … 368.5 ms 11 runs
2684 </code></pre>
2685 <p>An even more impressive ~59% improvement. And yes, the shell startup now feels
2686 much better.</p></summary></entry><entry><title>Quick bits: realise Nix symlinks</title><link href="https://kokada.dev/blog/quick-bits-realise-nix-symlinks/" rel="alternate"/><published>2024-08-01T00:00:00+00:00</published><id>https://kokada.dev/blog/quick-bits-realise-nix-symlinks/</id><summary type="html"><p>When you are using Nix, especially with
2687 <a href="https://github.com/nix-community/home-manager/">Home-Manager</a>, there are times
2688 when you want to test something or maybe debug some issue in your
2689 configuration. Those times it would be really convenient if you could avoid a
2690 rebuild of your Home-Manager configuration, since this takes time until
2691 evaluation and activation.</p>
2692 <p>For those times I have this small script in my Nix configuration called
2693 <code>realise-symlinks</code>, that is defined as:</p>
2694 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span> <span class="n">pkgs</span><span class="o">,</span> <span class="o">...</span> <span class="p">}:</span>
2695 </span></span><span class="line"><span class="cl"><span class="k">let</span>
2696 </span></span><span class="line"><span class="cl"> <span class="n">realise-symlink</span> <span class="o">=</span> <span class="n">pkgs</span><span class="o">.</span><span class="n">writeShellApplication</span> <span class="p">{</span>
2697 </span></span><span class="line"><span class="cl"> <span class="n">name</span> <span class="o">=</span> <span class="s2">&#34;realise-symlink&#34;</span><span class="p">;</span>
2698 </span></span><span class="line"><span class="cl"> <span class="n">runtimeInputs</span> <span class="o">=</span> <span class="k">with</span> <span class="n">pkgs</span><span class="p">;</span> <span class="p">[</span> <span class="n">coreutils</span> <span class="p">];</span>
2699 </span></span><span class="line"><span class="cl"> <span class="n">text</span> <span class="o">=</span> <span class="s1">&#39;&#39;
2700 </span></span></span><span class="line"><span class="cl"><span class="s1"> for file in &#34;$@&#34;; do
2701 </span></span></span><span class="line"><span class="cl"><span class="s1"> if [[ -L &#34;$file&#34; ]]; then
2702 </span></span></span><span class="line"><span class="cl"><span class="s1"> if [[ -d &#34;$file&#34; ]]; then
2703 </span></span></span><span class="line"><span class="cl"><span class="s1"> tmpdir=&#34;</span><span class="se">&#39;&#39;$</span><span class="s1">{file}.tmp&#34;
2704 </span></span></span><span class="line"><span class="cl"><span class="s1"> mkdir -p &#34;$tmpdir&#34;
2705 </span></span></span><span class="line"><span class="cl"><span class="s1"> cp --verbose --recursive &#34;$file&#34;/* &#34;$tmpdir&#34;
2706 </span></span></span><span class="line"><span class="cl"><span class="s1"> unlink &#34;$file&#34;
2707 </span></span></span><span class="line"><span class="cl"><span class="s1"> mv &#34;$tmpdir&#34; &#34;$file&#34;
2708 </span></span></span><span class="line"><span class="cl"><span class="s1"> chmod --changes --recursive +w &#34;$file&#34;
2709 </span></span></span><span class="line"><span class="cl"><span class="s1"> else
2710 </span></span></span><span class="line"><span class="cl"><span class="s1"> cp --verbose --remove-destination &#34;$(readlink &#34;$file&#34;)&#34; &#34;$file&#34;
2711 </span></span></span><span class="line"><span class="cl"><span class="s1"> chmod --changes +w &#34;$file&#34;
2712 </span></span></span><span class="line"><span class="cl"><span class="s1"> fi
2713 </span></span></span><span class="line"><span class="cl"><span class="s1"> else
2714 </span></span></span><span class="line"><span class="cl"><span class="s1"> &gt;&amp;2 echo &#34;Not a symlink: $file&#34;
2715 </span></span></span><span class="line"><span class="cl"><span class="s1"> exit 1
2716 </span></span></span><span class="line"><span class="cl"><span class="s1"> fi
2717 </span></span></span><span class="line"><span class="cl"><span class="s1"> done
2718 </span></span></span><span class="line"><span class="cl"><span class="s1"> &#39;&#39;</span><span class="p">;</span>
2719 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
2720 </span></span><span class="line"><span class="cl"><span class="k">in</span>
2721 </span></span><span class="line"><span class="cl"><span class="p">{</span>
2722 </span></span><span class="line"><span class="cl"> <span class="n">home</span><span class="o">.</span><span class="n">packages</span> <span class="o">=</span> <span class="p">[</span> <span class="n">realise-symlink</span> <span class="p">];</span>
2723 </span></span><span class="line"><span class="cl"><span class="p">}</span>
2724 </span></span></code></pre><p>The idea of this script is that you can call it against a symlink against Nix
2725 store and it will realise, e.g.: convert to an &quot;actual&quot; file, e.g.:</p>
2726 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> ls -lah .zshrc
2727 </span></span><span class="line"><span class="cl"><span class="go">lrwxrwxrwx 1 thiagoko users 69 Aug 1 00:10 .zshrc -&gt; /nix/store/glz018yyh0qfqc9lywx1yhr7c3l96lv7-home-manager-files/.zshrc
2728 </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="err">
2729 </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="gp">$</span> realise-symlink .zshrc
2730 </span></span><span class="line"><span class="cl"><span class="go">removed &#39;.zshrc&#39;
2731 </span></span></span><span class="line"><span class="cl"><span class="go">&#39;/nix/store/glz018yyh0qfqc9lywx1yhr7c3l96lv7-home-manager-files/.zshrc&#39; -&gt; &#39;.zshrc&#39;
2732 </span></span></span><span class="line"><span class="cl"><span class="go">mode of &#39;.zshrc&#39; changed from 0444 (r--r--r--) to 0644 (rw-r--r--)
2733 </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="err">
2734 </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="gp">$</span> ls -lah .zshrc
2735 </span></span><span class="line"><span class="cl"><span class="go">-rw-r--r-- 1 thiagoko users 5.8K Aug 1 00:16 .zshrc
2736 </span></span></span></code></pre><p>It also add write permissions to the resulting file, to make it easier to edit.
2737 By the way, it also works with directories:</p>
2738 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> ls -lah zim-completion
2739 </span></span><span class="line"><span class="cl"><span class="go">lrwxrwxrwx 1 thiagoko users 90 Aug 1 00:10 zim-completion -&gt; /nix/store/glz018yyh0qfqc9lywx1yhr7c3l96lv7-home-manager-files/.zsh/plugins/zim-completion
2740 </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="err">
2741 </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="gp">$</span> realise-symlink zim-completion
2742 </span></span><span class="line"><span class="cl"><span class="go">&#39;zim-completion/init.zsh&#39; -&gt; &#39;zim-completion.tmp/init.zsh&#39;
2743 </span></span></span><span class="line"><span class="cl"><span class="go">&#39;zim-completion/init.zsh.zwc&#39; -&gt; &#39;zim-completion.tmp/init.zsh.zwc&#39;
2744 </span></span></span><span class="line"><span class="cl"><span class="go">&#39;zim-completion/LICENSE&#39; -&gt; &#39;zim-completion.tmp/LICENSE&#39;
2745 </span></span></span><span class="line"><span class="cl"><span class="go">&#39;zim-completion/README.md&#39; -&gt; &#39;zim-completion.tmp/README.md&#39;
2746 </span></span></span><span class="line"><span class="cl"><span class="go">mode of &#39;zim-completion/init.zsh&#39; changed from 0444 (r--r--r--) to 0644 (rw-r--r--)
2747 </span></span></span><span class="line"><span class="cl"><span class="go">mode of &#39;zim-completion/init.zsh.zwc&#39; changed from 0444 (r--r--r--) to 0644 (rw-r--r--)
2748 </span></span></span><span class="line"><span class="cl"><span class="go">mode of &#39;zim-completion/LICENSE&#39; changed from 0444 (r--r--r--) to 0644 (rw-r--r--)
2749 </span></span></span><span class="line"><span class="cl"><span class="go">mode of &#39;zim-completion/README.md&#39; changed from 0444 (r--r--r--) to 0644 (rw-r--r--)
2750 </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="err">
2751 </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="gp">$</span> ls -lah zim-completion
2752 </span></span><span class="line"><span class="cl"><span class="go">total 28K
2753 </span></span></span><span class="line"><span class="cl"><span class="go">drwxr-xr-x 1 thiagoko users 72 Aug 1 00:18 .
2754 </span></span></span><span class="line"><span class="cl"><span class="go">drwxr-xr-x 1 thiagoko users 130 Aug 1 00:18 ..
2755 </span></span></span><span class="line"><span class="cl"><span class="go">-rw-r--r-- 1 thiagoko users 5.3K Aug 1 00:18 init.zsh
2756 </span></span></span><span class="line"><span class="cl"><span class="go">-rw-r--r-- 1 thiagoko users 12K Aug 1 00:18 init.zsh.zwc
2757 </span></span></span><span class="line"><span class="cl"><span class="go">-rw-r--r-- 1 thiagoko users 1.3K Aug 1 00:18 LICENSE
2758 </span></span></span><span class="line"><span class="cl"><span class="go">-rw-r--r-- 1 thiagoko users 2.2K Aug 1 00:18 README.md
2759 </span></span></span></code></pre><p>After you finish whatever you are testing, to return to your configuration you
2760 can just delete those files and re-run your Home-Manager activation:</p>
2761 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> rm -rf .zshrc
2762 </span></span><span class="line"><span class="cl"><span class="err">
2763 </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="gp">$</span> sudo systemctl restart home-manager-&lt;user&gt;.service <span class="c1"># or `home-manager switch`</span>
2764 </span></span><span class="line"><span class="cl"><span class="err">
2765 </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="gp">$</span> ls -lah .zshrc
2766 </span></span><span class="line"><span class="cl"><span class="go">lrwxrwxrwx 1 thiagoko users 69 Aug 1 00:20 .zshrc -&gt; /nix/store/glz018yyh0qfqc9lywx1yhr7c3l96lv7-home-manager-files/.zshrc
2767 </span></span></span></code></pre><p>It even works with system files (i.e.: NixOS):</p>
2768 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> sudo realise-symlink /etc/nix/nix.conf
2769 </span></span><span class="line"><span class="cl"><span class="go">[sudo] password for thiagoko:
2770 </span></span></span><span class="line"><span class="cl"><span class="go">removed &#39;nix.conf&#39;
2771 </span></span></span><span class="line"><span class="cl"><span class="go">&#39;/etc/static/nix/nix.conf&#39; -&gt; &#39;nix.conf&#39;
2772 </span></span></span><span class="line"><span class="cl"><span class="go">mode of &#39;nix.conf&#39; changed from 0444 (r--r--r--) to 0644 (rw-r--r--)
2773 </span></span></span></code></pre><p>But I never needed for this case since it is more rare to me to experiment with
2774 OS level configuration.</p></summary></entry><entry><title>Generating YAML files with Nix</title><link href="https://kokada.dev/blog/generating-yaml-files-with-nix/" rel="alternate"/><published>2024-07-31T00:00:00+00:00</published><id>https://kokada.dev/blog/generating-yaml-files-with-nix/</id><summary type="html"><p>I hate YAML. Instead of writing an essay on why I hate YAML, I can just link to
2775 <a href="https://noyaml.com/">noyaml.com</a>. In my personal projects I will never use it,
2776 preferring either JSON, <a href="https://toml.io/en/">TOML</a> or even plain old
2777 <a href="https://en.wikipedia.org/wiki/INI_file">INI</a> files depending on the use case.
2778 However the ship has sailed already, there are tons of projects everywhere that
2779 uses YAML: from most CI systems (<a href="https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions">GitHub
2780 Actions</a>,
2781 <a href="https://circleci.com/docs/introduction-to-yaml-configurations/">CircleCI</a>,
2782 <a href="https://docs.travis-ci.com/user/build-config-yaml">Travis</a>, <em>et tu</em>
2783 <a href="https://man.sr.ht/builds.sr.ht/">builds.sr.ht</a>), to
2784 <a href="https://kubernetes.io/docs/concepts/overview/working-with-objects/">Kubernetes</a>,
2785 or in almost every
2786 <a href="https://guides.rubyonrails.org/configuring.html#configuring-a-database">Rails</a>
2787 application.</p>
2788 <p>One way to avoid at least some issues with the language is to write YAML in
2789 another language. I will show my solution in one of my <a href="https://github.com/thiagokokada/nix-configs/">personal
2790 repositories</a>, writing Nix to
2791 generate GitHub Actions configuration files. Bonus points for validating the
2792 result against the schema of GitHub Actions, so the famous &quot;this is supposed to
2793 be string instead of a list of strings&quot; is gone.</p>
2794 <p>Let's start with the basics: YAML is supposed to be a <a href="https://stackoverflow.com/a/1729545">superset of
2795 JSON</a>. What that means is that a JSON file
2796 <a href="https://yaml.org/spec/1.2-old/spec.html#id2759572">can be parsed</a> by a YAML
2797 parser. And Nix itself generates JSON natively, after all, Nix can be imagined
2798 as <a href="https://nix.dev/tutorials/nix-language.html">&quot;JSON with functions&quot;</a>.</p>
2799 <p>To make things easier, I will assume that you have the <code>nix-commands</code> and
2800 <code>flakes</code> enabled as <code>experimental-features</code> in your Nix configuration. If not,
2801 go <a href="https://wiki.nixos.org/wiki/Flakes">here</a>.</p>
2802 <p>Using the <code>nix eval</code> command, we can generate a JSON expression from Nix by:</p>
2803 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> nix <span class="nb">eval</span> --expr <span class="s1">&#39;{ foo = &#34;bar&#34;; }&#39;</span> --json
2804 </span></span><span class="line"><span class="cl"><span class="go">{&#34;foo&#34;:&#34;bar&#34;}
2805 </span></span></span></code></pre><p>However, typing long excerpts of Nix code inside the console would be
2806 impractical. We can write the following code inside a <code>foo.nix</code> file instead:</p>
2807 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
2808 </span></span><span class="line"><span class="cl"> <span class="n">foo</span> <span class="o">=</span> <span class="s2">&#34;bar&#34;</span><span class="p">;</span>
2809 </span></span><span class="line"><span class="cl"><span class="p">}</span>
2810 </span></span></code></pre><p>And:</p>
2811 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> nix <span class="nb">eval</span> --file foo.nix --json
2812 </span></span><span class="line"><span class="cl"><span class="go">{&#34;foo&#34;:&#34;bar&#34;}
2813 </span></span></span></code></pre><p>While you can use a JSON output as an input for YAML parsers, it is probably
2814 not the <a href="https://metacpan.org/pod/JSON::XS#JSON-and-YAML">best idea</a>. Sadly (or
2815 maybe not), Nix has no native functionality to export data to YAML. However,
2816 since we are using Nix, it is trivial to use <code>nixpkgs</code> to use some program to
2817 convert from JSON to YAML.</p>
2818 <p>To start, let's create a new directory, move our <code>foo.nix</code> file to it, create a
2819 new <code>flake.nix</code> file and put the following contents:</p>
2820 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
2821 </span></span><span class="line"><span class="cl"> <span class="n">description</span> <span class="o">=</span> <span class="s2">&#34;Generate YAML files with Nix&#34;</span><span class="p">;</span>
2822 </span></span><span class="line"><span class="cl">
2823 </span></span><span class="line"><span class="cl"> <span class="n">inputs</span> <span class="o">=</span> <span class="p">{</span>
2824 </span></span><span class="line"><span class="cl"> <span class="n">nixpkgs</span><span class="o">.</span><span class="n">url</span> <span class="o">=</span> <span class="s2">&#34;github:NixOS/nixpkgs/nixos-unstable&#34;</span><span class="p">;</span>
2825 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
2826 </span></span><span class="line"><span class="cl">
2827 </span></span><span class="line"><span class="cl"> <span class="n">outputs</span> <span class="o">=</span> <span class="p">{</span> <span class="n">nixpkgs</span><span class="o">,</span> <span class="o">...</span> <span class="p">}:</span>
2828 </span></span><span class="line"><span class="cl"> <span class="p">{</span>
2829 </span></span><span class="line"><span class="cl"> <span class="n">packages</span><span class="o">.</span><span class="n">x86_64-linux</span> <span class="o">=</span>
2830 </span></span><span class="line"><span class="cl"> <span class="k">let</span>
2831 </span></span><span class="line"><span class="cl"> <span class="k">inherit</span> <span class="p">(</span><span class="n">nixpkgs</span><span class="p">)</span> <span class="n">lib</span><span class="p">;</span>
2832 </span></span><span class="line"><span class="cl"> <span class="n">pkgs</span> <span class="o">=</span> <span class="kn">import</span> <span class="n">nixpkgs</span> <span class="p">{</span> <span class="n">system</span> <span class="o">=</span> <span class="s2">&#34;x86_64-linux&#34;</span><span class="p">;</span> <span class="p">};</span>
2833 </span></span><span class="line"><span class="cl"> <span class="k">in</span>
2834 </span></span><span class="line"><span class="cl"> <span class="p">{</span>
2835 </span></span><span class="line"><span class="cl"> <span class="n">toYAML</span> <span class="o">=</span> <span class="n">pkgs</span><span class="o">.</span><span class="n">runCommand</span> <span class="s2">&#34;toYAML&#34;</span> <span class="p">{</span>
2836 </span></span><span class="line"><span class="cl"> <span class="n">buildInputs</span> <span class="o">=</span> <span class="k">with</span> <span class="n">pkgs</span><span class="p">;</span> <span class="p">[</span> <span class="n">yj</span> <span class="p">];</span>
2837 </span></span><span class="line"><span class="cl"> <span class="n">json</span> <span class="o">=</span> <span class="nb">builtins</span><span class="o">.</span><span class="n">toJSON</span> <span class="p">(</span><span class="kn">import</span> <span class="sr">./go.nix</span><span class="p">);</span>
2838 </span></span><span class="line"><span class="cl"> <span class="n">passAsFile</span> <span class="o">=</span> <span class="p">[</span> <span class="s2">&#34;json&#34;</span> <span class="p">];</span> <span class="c1"># will be available as `$jsonPath`</span>
2839 </span></span><span class="line"><span class="cl"> <span class="p">}</span> <span class="s1">&#39;&#39;
2840 </span></span></span><span class="line"><span class="cl"><span class="s1"> mkdir -p $out
2841 </span></span></span><span class="line"><span class="cl"><span class="s1"> yj -jy &lt; &#34;$jsonPath&#34; &gt; $out/go.yaml
2842 </span></span></span><span class="line"><span class="cl"><span class="s1"> &#39;&#39;</span><span class="p">;</span>
2843 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
2844 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
2845 </span></span><span class="line"><span class="cl"><span class="p">}</span>
2846 </span></span></code></pre><p>We are loading the <code>./foo.nix</code> as a Nix file, converting it to JSON with
2847 <code>builtins.toJSON</code> function, and finally, using <code>pkgs.runCommand</code> and its
2848 <code>passAsFile</code> option to load the contents of the JSON file into
2849 <a href="https://github.com/sclevine/yj">yj</a>, that converts between serialisation
2850 formats (<code>-jy</code> flag means &quot;JSON to YAML&quot;). The reason I choose <code>yj</code> is mostly
2851 because it is a single binary Go program, but you can use whatever you prefer.</p>
2852 <p>By the way, there is a
2853 <a href="https://github.com/NixOS/nixpkgs/blob/9f918d616c5321ad374ae6cb5ea89c9e04bf3e58/lib/generators.nix#L805"><code>lib.generators.toYAML</code></a>
2854 inside <code>nixpkgs.lib</code>, but as of the day of this post it only calls
2855 <code>lib.strings.toJSON</code> (that in turn, calls <code>builtins.toJSON</code>). So it doesn't
2856 really help here. Another option would be <code>pkgs.formats.yaml.generate</code>, that
2857 converts between formats, but it calls
2858 <a href="https://github.com/NixOS/nixpkgs/blob/008ceae1a2b47a84d7aa01e55f8468272c70b9ee/pkgs/pkgs-lib/formats.nix#L77-L84"><code>remarshal</code></a>
2859 (in Python), so not my favorite choice.</p>
2860 <p>If we run the following commands, we can see the result:</p>
2861 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> nix build .#packages.x86_64-linux.toYAML
2862 </span></span><span class="line"><span class="cl"><span class="gp">$</span> cat result/foo.yaml
2863 </span></span><span class="line"><span class="cl"><span class="go">foo: bar
2864 </span></span></span></code></pre><p>That is the basic idea. To have a more realistic example, let's convert the
2865 <a href="https://github.com/thiagokokada/blog/blob/4e3f25485c6682f3e066b219df2290934bc0d256/.github/workflows/go.yml"><code>go.yml</code></a>,
2866 that builds this blog, to Nix:</p>
2867 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
2868 </span></span><span class="line"><span class="cl"> <span class="n">name</span> <span class="o">=</span> <span class="s2">&#34;Go&#34;</span><span class="p">;</span>
2869 </span></span><span class="line"><span class="cl"> <span class="n">on</span><span class="o">.</span><span class="n">push</span><span class="o">.</span><span class="n">branches</span> <span class="o">=</span> <span class="p">[</span> <span class="s2">&#34;main&#34;</span> <span class="p">];</span>
2870 </span></span><span class="line"><span class="cl">
2871 </span></span><span class="line"><span class="cl"> <span class="n">jobs</span> <span class="o">=</span> <span class="p">{</span>
2872 </span></span><span class="line"><span class="cl"> <span class="n">build</span> <span class="o">=</span> <span class="p">{</span>
2873 </span></span><span class="line"><span class="cl"> <span class="n">runs-on</span> <span class="o">=</span> <span class="s2">&#34;ubuntu-latest&#34;</span><span class="p">;</span>
2874 </span></span><span class="line"><span class="cl"> <span class="n">permissions</span><span class="o">.</span><span class="n">contents</span> <span class="o">=</span> <span class="s2">&#34;write&#34;</span><span class="p">;</span>
2875 </span></span><span class="line"><span class="cl"> <span class="n">steps</span> <span class="o">=</span> <span class="p">[</span>
2876 </span></span><span class="line"><span class="cl"> <span class="p">{</span> <span class="n">uses</span> <span class="o">=</span> <span class="s2">&#34;actions/checkout@v4&#34;</span><span class="p">;</span> <span class="p">}</span>
2877 </span></span><span class="line"><span class="cl"> <span class="p">{</span>
2878 </span></span><span class="line"><span class="cl"> <span class="n">name</span> <span class="o">=</span> <span class="s2">&#34;Set up Go&#34;</span><span class="p">;</span>
2879 </span></span><span class="line"><span class="cl"> <span class="n">uses</span> <span class="o">=</span> <span class="s2">&#34;actions/checkout@v4&#34;</span><span class="p">;</span>
2880 </span></span><span class="line"><span class="cl"> <span class="s2">&#34;with&#34;</span><span class="o">.</span><span class="n">go-version</span> <span class="o">=</span> <span class="s2">&#34;1.21&#34;</span><span class="p">;</span>
2881 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
2882 </span></span><span class="line"><span class="cl"> <span class="p">{</span>
2883 </span></span><span class="line"><span class="cl"> <span class="n">name</span> <span class="o">=</span> <span class="s2">&#34;Update&#34;</span><span class="p">;</span>
2884 </span></span><span class="line"><span class="cl"> <span class="n">run</span> <span class="o">=</span> <span class="s2">&#34;make&#34;</span><span class="p">;</span>
2885 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
2886 </span></span><span class="line"><span class="cl"> <span class="p">{</span>
2887 </span></span><span class="line"><span class="cl"> <span class="n">name</span> <span class="o">=</span> <span class="s2">&#34;Publish&#34;</span><span class="p">;</span>
2888 </span></span><span class="line"><span class="cl"> <span class="n">run</span> <span class="o">=</span> <span class="s2">&#34;make publish&#34;</span><span class="p">;</span>
2889 </span></span><span class="line"><span class="cl"> <span class="n">env</span><span class="o">.</span><span class="n">MATAROA_TOKEN</span> <span class="o">=</span> <span class="s1">&#39;&#39;</span><span class="se">&#39;&#39;$</span><span class="s1">{{ secrets.MATAROA_TOKEN }}&#39;&#39;</span><span class="p">;</span>
2890 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
2891 </span></span><span class="line"><span class="cl"> <span class="p">{</span>
2892 </span></span><span class="line"><span class="cl"> <span class="n">name</span> <span class="o">=</span> <span class="s2">&#34;Commit&#34;</span><span class="p">;</span>
2893 </span></span><span class="line"><span class="cl"> <span class="n">uses</span> <span class="o">=</span> <span class="s2">&#34;stefanzweifel/git-auto-commit-action@v5&#34;</span><span class="p">;</span>
2894 </span></span><span class="line"><span class="cl"> <span class="s2">&#34;with&#34;</span><span class="o">.</span><span class="n">commit_message</span> <span class="o">=</span> <span class="s2">&#34;README/rss:update&#34;</span><span class="p">;</span>
2895 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
2896 </span></span><span class="line"><span class="cl"> <span class="p">];</span>
2897 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
2898 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
2899 </span></span><span class="line"><span class="cl"><span class="p">}</span>
2900 </span></span></code></pre><p>Some interesting things to highlight: <code>with</code> is a reserved word in Nix, so we
2901 need to quote it. Not a problem, but something to be aware. And the template
2902 string in GitHub Actions uses the same <code>${}</code> that Nix uses, so we need to
2903 escape.</p>
2904 <p>And after running the following commands:</p>
2905 <pre><code>$ nix build .#packages.x86_64-linux.toYAML
2906 $ cat result/go.yaml
2907 jobs:
2908 build:
2909 permissions:
2910 contents: write
2911 runs-on: ubuntu-latest
2912 steps:
2913 - uses: actions/checkout@v4
2914 - name: Set up Go
2915 uses: actions/checkout@v4
2916 with:
2917 go-version: &quot;1.21&quot;
2918 - name: Update
2919 run: make
2920 - env:
2921 MATAROA_TOKEN: ${{ secrets.MATAROA_TOKEN }}
2922 name: Publish
2923 run: make publish
2924 - name: Commit
2925 uses: stefanzweifel/git-auto-commit-action@v5
2926 with:
2927 commit_message: README/rss:update
2928 name: Go
2929 &quot;on&quot;:
2930 push:
2931 branches:
2932 - main
2933 </code></pre>
2934 <p>Yes, the keys are not in the same order as we defined, since Nix, like most
2935 programming languages (with the exception of
2936 <a href="https://mail.python.org/pipermail/python-dev/2017-December/151283.html">Python</a>),
2937 do not guarantee the insertion order in maps/dicts/attrsets/whatever. But I
2938 really hope whatever is consuming your YAML is not relying in the order the
2939 keys are defined (this would be more cursed than YAML already is).</p>
2940 <p>So that is basically it. For the bonus points that I talked at the start of the
2941 post, we can modify <code>pkgs.runCommand</code> to run some kind of validator. I use
2942 <a href="https://github.com/mpalmer/action-validator"><code>action-validator</code></a>, one that I
2943 particularly packaged in
2944 <a href="https://github.com/NixOS/nixpkgs/pull/260217">nixpkgs</a> to use in those cases.
2945 But you could use e.g.: a validator of Kubernetes YAML. Or a generic YAML lint
2946 like this <a href="https://github.com/adrienverge/yamllint">one</a>. The possibilities are
2947 endless.</p>
2948 <p>Let's modify our <code>flake.nix</code> to add the validation:</p>
2949 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
2950 </span></span><span class="line"><span class="cl"> <span class="c1"># ...</span>
2951 </span></span><span class="line"><span class="cl"> <span class="n">outputs</span> <span class="o">=</span> <span class="p">{</span> <span class="n">nixpkgs</span><span class="o">,</span> <span class="o">...</span> <span class="p">}:</span>
2952 </span></span><span class="line"><span class="cl"> <span class="p">{</span>
2953 </span></span><span class="line"><span class="cl"> <span class="n">packages</span><span class="o">.</span><span class="n">x86_64-linux</span> <span class="o">=</span>
2954 </span></span><span class="line"><span class="cl"> <span class="k">let</span>
2955 </span></span><span class="line"><span class="cl"> <span class="k">inherit</span> <span class="p">(</span><span class="n">nixpkgs</span><span class="p">)</span> <span class="n">lib</span><span class="p">;</span>
2956 </span></span><span class="line"><span class="cl"> <span class="n">pkgs</span> <span class="o">=</span> <span class="kn">import</span> <span class="n">nixpkgs</span> <span class="p">{</span> <span class="n">system</span> <span class="o">=</span> <span class="s2">&#34;x86_64-linux&#34;</span><span class="p">;</span> <span class="p">};</span>
2957 </span></span><span class="line"><span class="cl"> <span class="k">in</span>
2958 </span></span><span class="line"><span class="cl"> <span class="p">{</span>
2959 </span></span><span class="line"><span class="cl"> <span class="n">toYAML</span> <span class="o">=</span> <span class="n">pkgs</span><span class="o">.</span><span class="n">runCommand</span> <span class="s2">&#34;toYAML&#34;</span> <span class="p">{</span>
2960 </span></span><span class="line"><span class="cl"> <span class="n">buildInputs</span> <span class="o">=</span> <span class="k">with</span> <span class="n">pkgs</span><span class="p">;</span> <span class="p">[</span> <span class="n">action-validator</span> <span class="n">yj</span> <span class="p">];</span>
2961 </span></span><span class="line"><span class="cl"> <span class="n">json</span> <span class="o">=</span> <span class="nb">builtins</span><span class="o">.</span><span class="n">toJSON</span> <span class="p">(</span><span class="kn">import</span> <span class="sr">./go.nix</span><span class="p">);</span>
2962 </span></span><span class="line"><span class="cl"> <span class="n">passAsFile</span> <span class="o">=</span> <span class="p">[</span> <span class="s2">&#34;json&#34;</span> <span class="p">];</span>
2963 </span></span><span class="line"><span class="cl"> <span class="p">}</span> <span class="s1">&#39;&#39;
2964 </span></span></span><span class="line"><span class="cl"><span class="s1"> mkdir -p $out
2965 </span></span></span><span class="line"><span class="cl"><span class="s1"> yj -jy &lt; &#34;$jsonPath&#34; &gt; $out/go.yaml
2966 </span></span></span><span class="line"><span class="cl"><span class="s1"> action-validator -v $out/go.yaml
2967 </span></span></span><span class="line"><span class="cl"><span class="s1"> &#39;&#39;</span><span class="p">;</span>
2968 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
2969 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
2970 </span></span><span class="line"><span class="cl"><span class="p">}</span>
2971 </span></span></code></pre><p>And let's add an error in our <code>go.nix</code> file:</p>
2972 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gh">diff --git a/go.nix b/go.nix
2973 </span></span></span><span class="line"><span class="cl"><span class="gh">index 25e0596..8c00033 100644
2974 </span></span></span><span class="line"><span class="cl"><span class="gh"></span><span class="gd">--- a/go.nix
2975 </span></span></span><span class="line"><span class="cl"><span class="gd"></span><span class="gi">+++ b/go.nix
2976 </span></span></span><span class="line"><span class="cl"><span class="gi"></span><span class="gu">@@ -5,7 +5,7 @@
2977 </span></span></span><span class="line"><span class="cl"><span class="gu"></span> jobs = {
2978 </span></span><span class="line"><span class="cl"> build = {
2979 </span></span><span class="line"><span class="cl"> runs-on = &#34;ubuntu-latest&#34;;
2980 </span></span><span class="line"><span class="cl"><span class="gd">- permissions.contents = &#34;write&#34;;
2981 </span></span></span><span class="line"><span class="cl"><span class="gd"></span><span class="gi">+ permissions.contents = [ &#34;write&#34; ];
2982 </span></span></span><span class="line"><span class="cl"><span class="gi"></span> steps = [
2983 </span></span><span class="line"><span class="cl"> { uses = &#34;actions/checkout@v4&#34;; }
2984 </span></span><span class="line"><span class="cl"> {
2985 </span></span></code></pre><p>Finally, let's try to build our YAML file again:</p>
2986 <pre><code>$ nix build .#packages.x86_64-linux.toYAML
2987 error: builder for '/nix/store/j8wr6j1pvyf986sf74hqw8k31lvlzac5-toYAML.drv' failed with exit code 1;
2988 last 25 log lines:
2989 &gt; &quot;Additional property 'runs-on' is not allowed&quot;,
2990 &gt; ),
2991 &gt; path: &quot;/jobs/build&quot;,
2992 &gt; title: &quot;Property conditions are not met&quot;,
2993 &gt; },
2994 &gt; Properties {
2995 &gt; code: &quot;properties&quot;,
2996 &gt; detail: Some(
2997 &gt; &quot;Additional property 'steps' is not allowed&quot;,
2998 &gt; ),
2999 &gt; path: &quot;/jobs/build&quot;,
3000 &gt; title: &quot;Property conditions are not met&quot;,
3001 &gt; },
3002 &gt; Required {
3003 &gt; code: &quot;required&quot;,
3004 &gt; detail: None,
3005 &gt; path: &quot;/jobs/build/uses&quot;,
3006 &gt; title: &quot;This property is required&quot;,
3007 &gt; },
3008 &gt; ],
3009 &gt; },
3010 &gt; ],
3011 &gt; },
3012 &gt; ],
3013 &gt; }
3014 For full logs, run 'nix log /nix/store/j8wr6j1pvyf986sf74hqw8k31lvlzac5-toYAML.drv'.
3015 </code></pre>
3016 <p>Yes, the output of <code>action-validator</code> is awfully verbose, but it is still
3017 better than making <a href="https://x.com/eric_sink/status/1430954572848287744">&quot;8 commits/push in one
3018 hour&quot;</a>.</p>
3019 <p>If you are interested in how a more advantage usage of this technique is,
3020 including usage of functions and constants to share common steps between
3021 different actions, please take a look at the
3022 <a href="https://github.com/thiagokokada/nix-configs/tree/master/actions">actions</a>
3023 (<a href="https://github.com/thiagokokada/nix-configs/tree/fbbc8f935cc7f774f691196e1612bbf7dd2054b5/actions">permalink</a>)
3024 in my <code>nix-config</code> repository.</p></summary></entry><entry><title>First impressions: FPGBC</title><link href="https://kokada.dev/blog/first-impressions-fpgbc/" rel="alternate"/><published>2024-07-30T00:00:00+00:00</published><id>https://kokada.dev/blog/first-impressions-fpgbc/</id><summary type="html"><p>Here is something for nostalgia: I just put together a <a href="https://en.wikipedia.org/wiki/Game_Boy_Color">Game Boy
3025 Color</a> made of completely new
3026 parts for a friend: here is the
3027 <a href="https://funnyplaying.com/products/fpgbc-kit">FPGBC</a>.</p>
3028 <p>The <em>FP</em> part of the name comes from
3029 <a href="https://en.wikipedia.org/wiki/Field-programmable_gate_array">FPGA</a>, because
3030 instead of software emulation this device use FPGA to reproduce the device.
3031 While I am not convinced that FPGA is necessary more accurate than a good
3032 software emulator, one advantage of FPGA is the (possible) lower input latency
3033 thanks to the avoidance of complexity to handle the user input (e.g.: the
3034 Operational System). A quick playthrough against <a href="https://en.wikipedia.org/wiki/Motocross_Maniacs">Motocross
3035 Maniacs</a> seems to be fine, but
3036 I can't see much difference from my <a href="https://retrogamecorps.com/2022/05/15/miyoo-mini-v2-guide/">Miyoo
3037 Mini+</a> (I will do
3038 more comparisons between the two devices later), that is a software emulation
3039 device.</p>
3040 <p>But I think focusing in accuracy is wrong, the main reason of getting a device
3041 like this one is for nostalgia, and this definitely hit the mark. The quality
3042 of the case is as good as I remember the original, and most of the details are
3043 replicate perfectly, including reproduction stickers in the back of the device.
3044 The only differences that I can find is the usage of USB-C port for charging in
3045 place of the barrel jack power adapter (thanks!), and the fact that the screen
3046 bezels are smaller compared to the original (because the screen is bigger) and
3047 doesn't include the Game Boy Color logo (that is fine in my opinion, since it
3048 would look weird in the fine bezels). It even has a supposedly working <a href="https://en.wikipedia.org/wiki/Game_Link_Cable">Link
3049 Cable</a> (I don't have another
3050 Game Boy to test). Sadly it is missing the infrared sensor, but the usage of
3051 that was pretty limited anyway.</p>
3052 <p><a href="https://github.com/thiagokokada/blog/raw/main/posts/2024-07-30/PXL_20240729_175245569.jpg"><img src="https://github.com/thiagokokada/blog/raw/main/posts/2024-07-30/PXL_20240729_175245569.jpg" alt="FPGBC running Tetris."></a></p>
3053 <p><a href="https://github.com/thiagokokada/blog/raw/main/posts/2024-07-30/PXL_20240729_175131157.jpg"><img src="https://github.com/thiagokokada/blog/raw/main/posts/2024-07-30/PXL_20240729_175131157.jpg" alt="Back of FPGBC. It includes even reproduction stickers of the original."></a></p>
3054 <p>So how well does it work? I can't say for sure. I don't have any original games
3055 with me, so I am relying in backups and a
3056 <a href="https://en.wikipedia.org/wiki/Flashcard">flashcard</a> for now. Many games that I
3057 tested works fine, a few of them have graphical issues that can be fixed in the
3058 menu (more about it later), and some of them doesn't boot. But I don't know if
3059 the issue with the games not booting are because of the roms, the flashcard or
3060 the device itself.</p>
3061 <p>By the way, the flashcard I am using is a cheap knockoff of an <a href="https://gbatemp.net/review/everdrive-gb.141/">Everdrive
3062 GB</a>. This FPGBC came with
3063 firmware v1.09, while there is an update available for v1.10 in the
3064 <a href="https://funnyplaying.com/products/fpgbc-kit">website</a>. I had an weird issue in
3065 the new firmware where no games would boot with this knockoff Everdrive so I
3066 had to go back to v1.09, but again, I am not sure if the issue was fact that I
3067 am using a knockoff device or this would happen with an original Everdrive GB.
3068 If you are going to buy a proper Everdrive, you probably wouldn't get a
3069 Everdrive GB anyway since it is discontinued, and it seems the <a href="https://www.reddit.com/r/Gameboy/comments/1atwjh3/fpgbc_everdrive_compatibility/">newer
3070 versions</a>
3071 have better compatibility with FPGBC.</p>
3072 <p>Sadly that the update didn't work, since there is this
3073 <a href="https://github.com/makhowastaken/GWGBC_FW">repository</a> that patches the
3074 firmware to boot the original logo instead of the ugly FPGBC one. And yes, for
3075 some reason the v1.09 firmware from this repository still doesn't work with my
3076 knockoff Everdrive.</p>
3077 <p>By the way, it seems the device is not easy to brick: I borked the firmware
3078 update process once while trying to downgrade back to v1.09, resulting in a
3079 black screen when I turned on the console. But just connecting the device to
3080 the computer and powering on, I could flash the firmware again and the device
3081 came back to life.</p>
3082 <p>About the features of the device: if you press the volume button (yes, you can
3083 press it now), it opens the following menu:</p>
3084 <p><a href="https://github.com/thiagokokada/blog/raw/main/posts/2024-07-30/PXL_20240729_210604830.jpg"><img src="https://github.com/thiagokokada/blog/raw/main/posts/2024-07-30/PXL_20240729_210604830.jpg" alt="FPGBC menu."></a></p>
3085 <p>The first 2 options are the LCD backlight (<code>BKLT</code>) and volume (<code>VOL</code>). I didn't talk about
3086 those, but the LCD screen seems to be IPS, and the quality is really good, and
3087 also looks bright enough to play even under bad lightining conditions. And the
3088 speaker has good quality, the sound is better than I remember, but sadly the
3089 maximum volume is kind low. Still should be enough for playing in a quiet room.</p>
3090 <p><code>DISPMOD</code> is probably the most controversial option: it allow you to set which
3091 scale you want. Anything with <code>EMU</code> at the end means emulating the original
3092 colors, and as far I remember it gets really close. You can also chose betwen
3093 <code>X4</code>, <code>X4P</code> and <code>FUL</code>, the last one is the one shown in the photos where the
3094 image fills the whole screen at the cost of non-integer scaling. <code>X4</code> is
3095 integer scaling, however the image doesn't fill the whole screen. The <code>X4P</code>
3096 also includes a pixel effect that makes the image closer than the original
3097 screen. It actually looks good, but the fact that I chose a white border for
3098 this FPGBC makes the border really distracting. Maybe the black one is a better
3099 choice if you want integer scale.</p>
3100 <p><code>CORE</code> is simple: you can choose between <code>GB</code> (Game Boy) or <code>GBC</code> (Game Boy
3101 Color). For those who don't know, you can run Game Boy games in Game Boy Color
3102 and they will be automatically colorised. Some people don't like this and
3103 prefer the colors of <code>GB</code>, so you have this option. The <code>GB_PALETTE</code> allows you
3104 to chose the color in GB mode, for example, the green-ish colors from the
3105 original Game Boy or the blue-ish colors from <a href="https://nintendo.fandom.com/wiki/Game_Boy_Light">Game Boy
3106 Light</a>. And yes, you can
3107 choose the color palette for Game Boy games running in <code>GBC</code> mode by pressing a
3108 <a href="https://gbstudiocentral.com/tips/game-boy-color-modes/">button combination</a> at
3109 the boot screen, but it seems not working in my unit and again, not sure if the
3110 fault is my knockoff Everdrive.</p>
3111 <p><code>FRAME_MIX</code> basically is an option that makes some effects, like transparency
3112 in <a href="https://en.wikipedia.org/wiki/Wave_Race">Wave Race</a>, to work at the cost of
3113 introducing blurriness. The reason for this is that those effects depends in
3114 the fact that the Game Boy screen was slow refresh, so you could rely on it by
3115 rapidly changing pixels to create some interesting effects, but sadly those
3116 effects doesn't work well in modern displays.</p>
3117 <p><code>GB_CLRFIX</code> is the option I mentioned before, where some Game Boy games just
3118 get completely wrong colors for some reason, e.g.: <a href="https://en.wikipedia.org/wiki/The_Addams_Family_(video_game)">The Addams
3119 Family</a>. Turning
3120 on fixes those games, but I am not sure if this option breaks other games.</p>
3121 <p>Finally, <code>SPD</code> allows you to increase or decrease the CPU clock, slowing or
3122 speeding up the games (including the sound). The result can be hilarious, so I
3123 think this is a nice addition to the features. Sadly you can't know what the
3124 default speed is, so you need to rely on sound to adjust back to the default.</p>
3125 <p>So in the end, can I recommend a FPGBC? I am not sure. If you want a device to
3126 play games, I still think something like a Miyoo Mini+ is a better choice. Not
3127 only you will have access to more games from different platforms, you also
3128 don't need to rely on flashcards or cartridges. Also it has way more features
3129 than FPGBC, like wireless multiplayer,
3130 <a href="https://retroachievements.org/">RetroArchivements</a> and save states.</p>
3131 <p>But the actual reason to get a FPGBC is nostalgia, and for that I think the
3132 FPGBC is difficult to beat. The price of the <a href="https://funnyplaying.com/products/fpgbc-kit">kit to
3133 assemble</a> ($69.90) is cheaper than
3134 most Game Boy's in good condition you can find in eBay, and you get for that
3135 price a rechargable battery, an amazing quality screen, the PCB and the
3136 speaker. You need to buy separately the case and the buttons, but in total you
3137 will still end up paying less, and allows you to fully customise your build.
3138 And the result device is not only in mint condition, it is really convenient
3139 too: recharging batteries (via USB-C even) is much more convenient than buying
3140 AA batteries, and the screen not only is better but it even has backlight. You
3141 can also buy a fully built console for
3142 <a href="https://funnyplaying.com/products/fpgbc-console">$99.00</a>, but you have less
3143 options of customisation.</p>
3144 <p>This is the classic case of do what I say, don't do what I do. This FPGBC is a
3145 gift, and I will buy another one soon. Can't wait to play <a href="https://en.wikipedia.org/wiki/Pok%C3%A9mon_Gold_and_Silver">Pokémon
3146 Gold</a> in (almost)
3147 original hardware again.</p>
3148 <p><a href="https://github.com/thiagokokada/blog/raw/main/posts/2024-07-30/PXL_20240729_123847458.jpg"><img src="https://github.com/thiagokokada/blog/raw/main/posts/2024-07-30/PXL_20240729_123847458.jpg" alt="The kit before assemble."></a></p></summary></entry><entry><title>Quick bits: why you should automate everything</title><link href="https://kokada.dev/blog/quick-bits-why-you-should-automate-everything/" rel="alternate"/><published>2024-07-29T00:00:00+00:00</published><id>https://kokada.dev/blog/quick-bits-why-you-should-automate-everything/</id><summary type="html"><p>If everything works as expected this blog post should appear in <a href="https://kokada.capivaras.dev/">in my
3149 blog</a> without I ever touching the
3150 <a href="https://capivaras.dev">capivaras.dev</a> website. I rewrote my <a href="https://kokada.dev/blog/using-github-as-a-bad-blog-platform">previous Python
3151 script</a> to Go
3152 (<a href="https://github.com/thiagokokada/blog/blob/3c39e0f7cd58b1af885f69871490b05bf6fc7d99/blog.go">permalink</a>)
3153 since my attempt to generate proper description to the RSS feed resulted in
3154 slow startup times (not because of Python, but because of my usage of
3155 <code>nix-shell</code> since I didn't want to deal with
3156 <a href="https://docs.python.org/3/library/venv.html">venv</a> or anything to manage my
3157 Python dependencies).</p>
3158 <p>My previous workflow of this blog already involved me writing the texts in
3159 <a href="https://neovim.io/">neovim</a>, copying and pasting the result in the
3160 <a href="https://capivaras.dev">capivaras.dev</a> website and publishing. This was not
3161 that bad, except that it seems I have a heavy tendency of editing my posts
3162 multiple times. Copying and pasting data between neovim and the website became
3163 tedious, so I decided to give up and automate the whole process.</p>
3164 <p><a href="https://mataroa.blog/">Mataroa</a> (the blog platform
3165 <a href="https://capivaras.dev">capivaras.dev</a> run) has a reasonable good
3166 <a href="https://mataroa.blog/api/docs/">API</a>, and it only took a few hours to get a
3167 version of publishing working (it would take less if
3168 <a href="https://www.djangoproject.com/">Django</a>, the framework Mataroa is written, did
3169 not have a weird behavior with URLs missing a trailing <code>/</code>). An additional few
3170 lines of
3171 <a href="https://github.com/thiagokokada/blog/blob/51b20612335c7f4312a51a0f436235b4b701ce8b/.github/workflows/go.yml">YAML</a>
3172 to make GitHub Actions trigger a pipeline and now I should never have to
3173 manually update my blog again.</p>
3174 <p>I could have not done this. I mean, I probably wasted more time writing an
3175 automation than I actually wasted publishing manually. But the manual process
3176 is easy to miss, and I already did multiple mistakes publishing in the manual
3177 method. For example, when writing the Markdown files, each post is written in a
3178 particular format, where the first header is considered the title, so I need to
3179 remove it from the contents during publication. But of course, this is easy to
3180 miss, and I had to fix this multiple times already.</p>
3181 <p>So yes, I think this is a good lesson on why you should automate everything. It
3182 is more than just about <a href="https://xkcd.com/1205/">time savings</a>, it is about
3183 reducing mistakes and even documenting (even if poorly) a process. I mean, the
3184 code I wrote is not that great, but I can definitely rely on it in the future
3185 to remember what I need to do. It will be much faster than trying to learn from
3186 scratch again.</p></summary></entry><entry><title>Go, a reasonable good language</title><link href="https://kokada.dev/blog/go-a-reasonable-good-language/" rel="alternate"/><published>2024-07-29T00:00:00+00:00</published><id>https://kokada.dev/blog/go-a-reasonable-good-language/</id><summary type="html"><p>Go was one of the languages that I always was interested to learn, but never
3187 got the hang of it. I first got interested in the language when I was in my
3188 first job, between 2016-2018. At the time the language was a completely
3189 different beast: no modules, no generics, no easy way to error wrap yet, etc.</p>
3190 <p>Go forward 2023 (no pun indented), I wrote my <a href="https://github.com/thiagokokada/twenty-twenty-twenty/">first project in
3191 Go</a>, wrote some scripts
3192 at <code>$CURRENT_JOB</code> in the language, and now wrote <a href="https://github.com/thiagokokada/hyprland-go/">my first
3193 library</a> (with an <a href="https://kokada.dev/blog/writing-nixos-tests-for-fun-and-profit">interesting
3194 way to run
3195 CI</a>). I am also
3196 writing more
3197 <a href="https://github.com/thiagokokada/nix-configs/blob/8c559527ed12e1d4f57a3fc5c72630b956f4c290/home-manager/desktop/wayland/hyprland/hyprtabs/hyprtabs.go">scripts</a>
3198 in the language, where I would prefer to use Bash or Python before. Heck, even
3199 this blog is automatically published with a <a href="https://kokada.dev/blog/quick-bits-why-you-should-automate-everything">Go
3200 script</a>,
3201 that used to be a <a href="https://kokada.dev/blog/using-github-as-a-bad-blog-platform">Python
3202 one</a> before. I can
3203 say that nowadays it is another language in my toolbox, and while it is still a
3204 love and hate relationship, recently it is more about love and less about hate.</p>
3205 <p>The points that I love about Go is probably obvious for some, but still
3206 interesting to talk about anyway. The fact that the language generates static
3207 binaries by default and have fast compilation times is something that I
3208 apreciate since I first heard about the language, and now that I am using the
3209 language frequently, are points thatI appreciate even more. Something about
3210 getting almost instant feedback after changing a line of code and running <code>go run</code> (even with its quirks) are great for the developer experience. This is the
3211 main reason why I am using the language more frequently for scripts.</p>
3212 <p>Then we have the fast startup times. I am kind of sensitive to latency,
3213 especially of command line utilities that need to answer fast when I expect
3214 them to be fast (e.g.: <code>foo --help</code>). This is one part where I could have
3215 issues in Python, especially for more complex programs, but in Go it is rarely
3216 an issue.</p>
3217 <p>Modules are also fantastic. It is not without its weirdness (like everything in
3218 Go ecossystem), but the fact that it is so easy to add and manage dependencies
3219 in a project using only the <code>go</code> CLI is great. I also like that it generates a
3220 hash of every dependency, making it reproducible (well, probably not at Nix
3221 level, but still reproducible).</p>
3222 <p>Since I started to talk about <code>go</code> CLI, what a great tool! The fact that you
3223 can manage dependencies, generate documentation, format code, lint, run
3224 tests/<a href="https://blog.logrocket.com/benchmarking-golang-improve-function-performance/">benchmarks</a>/<a href="https://go.dev/doc/security/fuzz/">fuzzing</a>,
3225 check code for <a href="https://go.dev/doc/articles/race_detector">races</a> etc., all
3226 with just the &quot;compiler&quot; for the language is excelent. Still probably one of
3227 the best developer experiences I know in any programming language.</p>
3228 <p>I will not even talk about the things that everyone talks about Go, like
3229 <a href="https://go.dev/doc/effective_go#goroutines">goroutines</a>, because I just don't
3230 think I can add anything interesting to the topic.</p>
3231 <p>Now for the parts that I like less, the test part still quirks me that it is
3232 not based in assertions, but thankfully it is easy to write assertions with
3233 generics nowadays:</p>
3234 <pre class="chroma"><code><span class="line"><span class="cl"><span class="kd">func</span> <span class="nx">Equal</span><span class="p">[</span><span class="nx">T</span> <span class="nx">comparable</span><span class="p">](</span><span class="nx">t</span> <span class="o">*</span><span class="nx">testing</span><span class="p">.</span><span class="nx">T</span><span class="p">,</span> <span class="nx">got</span><span class="p">,</span> <span class="nx">want</span> <span class="nx">T</span><span class="p">)</span> <span class="p">{</span>
3235 </span></span><span class="line"><span class="cl"> <span class="nx">t</span><span class="p">.</span><span class="nf">Helper</span><span class="p">()</span>
3236 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="nx">got</span> <span class="o">!=</span> <span class="nx">want</span> <span class="p">{</span>
3237 </span></span><span class="line"><span class="cl"> <span class="nx">t</span><span class="p">.</span><span class="nf">Errorf</span><span class="p">(</span><span class="s">&#34;got: %#v, want: %#v&#34;</span><span class="p">,</span> <span class="nx">got</span><span class="p">,</span> <span class="nx">want</span><span class="p">)</span>
3238 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
3239 </span></span><span class="line"><span class="cl"><span class="p">}</span>
3240 </span></span><span class="line"><span class="cl">
3241 </span></span><span class="line"><span class="cl"><span class="kd">func</span> <span class="nx">GreaterOrEqual</span><span class="p">[</span><span class="nx">T</span> <span class="nx">cmp</span><span class="p">.</span><span class="nx">Ordered</span><span class="p">](</span><span class="nx">t</span> <span class="o">*</span><span class="nx">testing</span><span class="p">.</span><span class="nx">T</span><span class="p">,</span> <span class="nx">actual</span><span class="p">,</span> <span class="nx">expected</span> <span class="nx">T</span><span class="p">)</span> <span class="p">{</span>
3242 </span></span><span class="line"><span class="cl"> <span class="nx">t</span><span class="p">.</span><span class="nf">Helper</span><span class="p">()</span>
3243 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="nx">actual</span> <span class="p">&lt;</span> <span class="nx">expected</span> <span class="p">{</span>
3244 </span></span><span class="line"><span class="cl"> <span class="nx">t</span><span class="p">.</span><span class="nf">Errorf</span><span class="p">(</span><span class="s">&#34;got: %v; want: &gt;=%v&#34;</span><span class="p">,</span> <span class="nx">actual</span><span class="p">,</span> <span class="nx">expected</span><span class="p">)</span>
3245 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
3246 </span></span><span class="line"><span class="cl"><span class="p">}</span>
3247 </span></span><span class="line"><span class="cl">
3248 </span></span><span class="line"><span class="cl"><span class="c1">// etc...
3249 </span></span></span></code></pre><p>Just one of those things that I end up re-writing in every project. Yes, I know
3250 about <a href="https://github.com/stretchr/testify">testify</a> and other assertion
3251 libraries, but quoting <a href="https://www.youtube.com/watch?v=PAAkCSZUG1c&amp;t=568s">Rob
3252 Pike</a> here, &quot;a little
3253 copying is better than a little dependency&quot;. As long the code you write is
3254 trivial, it is better to duplicate the code than try to import a dependency.</p>
3255 <p>About another piece of code that generics allows me to write and I always end
3256 up re-writing in every project is the <code>must*</code> family of functions:</p>
3257 <pre class="chroma"><code><span class="line"><span class="cl"><span class="kd">func</span> <span class="nf">must</span><span class="p">(</span><span class="nx">err</span> <span class="kt">error</span><span class="p">)</span> <span class="p">{</span>
3258 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="nx">err</span> <span class="o">!=</span> <span class="kc">nil</span> <span class="p">{</span>
3259 </span></span><span class="line"><span class="cl"> <span class="nb">panic</span><span class="p">(</span><span class="nx">err</span><span class="p">)</span>
3260 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
3261 </span></span><span class="line"><span class="cl"><span class="p">}</span>
3262 </span></span><span class="line"><span class="cl">
3263 </span></span><span class="line"><span class="cl"><span class="kd">func</span> <span class="nx">must1</span><span class="p">[</span><span class="nx">T</span> <span class="nx">any</span><span class="p">](</span><span class="nx">v</span> <span class="nx">T</span><span class="p">,</span> <span class="nx">err</span> <span class="kt">error</span><span class="p">)</span> <span class="nx">T</span> <span class="p">{</span>
3264 </span></span><span class="line"><span class="cl"> <span class="nf">must</span><span class="p">(</span><span class="nx">err</span><span class="p">)</span>
3265 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="nx">v</span>
3266 </span></span><span class="line"><span class="cl"><span class="p">}</span>
3267 </span></span><span class="line"><span class="cl">
3268 </span></span><span class="line"><span class="cl"><span class="kd">func</span> <span class="nx">must2</span><span class="p">[</span><span class="nx">T1</span> <span class="nx">any</span><span class="p">,</span> <span class="nx">T2</span> <span class="nx">any</span><span class="p">](</span><span class="nx">v1</span> <span class="nx">T1</span><span class="p">,</span> <span class="nx">v2</span> <span class="nx">T2</span><span class="p">,</span> <span class="nx">err</span> <span class="kt">error</span><span class="p">)</span> <span class="nx">T</span> <span class="p">{</span>
3269 </span></span><span class="line"><span class="cl"> <span class="nf">must</span><span class="p">(</span><span class="nx">err</span><span class="p">)</span>
3270 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="nx">v1</span><span class="p">,</span> <span class="nx">v2</span>
3271 </span></span><span class="line"><span class="cl"><span class="p">}</span>
3272 </span></span><span class="line"><span class="cl">
3273 </span></span><span class="line"><span class="cl"><span class="c1">// must3, must4, etc...
3274 </span></span></span></code></pre><p>Those functions are so useful, especially for scripts where I generally don't
3275 want to handle each error: if I have an error, I want the program to halt and
3276 print a stack trace (exactly as I would have with a language with exceptions).
3277 It basically allow me to convert code from:</p>
3278 <pre class="chroma"><code><span class="line"><span class="cl"><span class="nx">contents</span><span class="p">,</span> <span class="nx">err</span> <span class="o">:=</span> <span class="nx">os</span><span class="p">.</span><span class="nf">ReadFile</span><span class="p">(</span><span class="s">&#34;file&#34;</span><span class="p">)</span>
3279 </span></span><span class="line"><span class="cl"><span class="k">if</span> <span class="nx">err</span> <span class="o">!=</span> <span class="kc">nil</span> <span class="p">{</span>
3280 </span></span><span class="line"><span class="cl"> <span class="nb">panic</span><span class="p">(</span><span class="nx">err</span><span class="p">)</span>
3281 </span></span><span class="line"><span class="cl"><span class="p">}</span>
3282 </span></span></code></pre><p>To:</p>
3283 <pre class="chroma"><code><span class="line"><span class="cl"><span class="nx">contents</span> <span class="o">:=</span> <span class="nf">must1</span><span class="p">(</span><span class="nx">os</span><span class="p">.</span><span class="nf">ReadFile</span><span class="p">(</span><span class="s">&#34;file&#34;</span><span class="p">))</span>
3284 </span></span></code></pre><p>This brings Go closer to Python to me, and I think for scripts this is
3285 something great.</p>
3286 <p>Finally, for the things that I hate, well the biggest one currently is the lack
3287 of nullability (or in Go terms,
3288 <a href="https://github.com/golang/go/issues/49202">nillability</a>). After using
3289 languages that has it, like Kotlin, or even something like
3290 <a href="https://www.mypy-lang.org/">mypy</a>, this is one of those things that completely
3291 changes the developer experience. I also still don't like the error handling
3292 (but <code>must*</code> goes far by improving the situation, when it is possible to use
3293 it), especially because it is easy to lose context on it:</p>
3294 <pre class="chroma"><code><span class="line"><span class="cl"><span class="c1">// bad
3295 </span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="kd">func</span> <span class="nf">readFileContents</span><span class="p">(</span><span class="nx">file</span><span class="p">)</span> <span class="p">([]</span><span class="kt">byte</span><span class="p">,</span> <span class="kt">error</span><span class="p">)</span> <span class="p">{</span>
3296 </span></span><span class="line"><span class="cl"> <span class="nx">contents</span><span class="p">,</span> <span class="nx">err</span> <span class="o">:=</span> <span class="nx">os</span><span class="p">.</span><span class="nf">ReadFile</span><span class="p">(</span><span class="nx">file</span><span class="p">)</span>
3297 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="nx">err</span> <span class="o">!=</span> <span class="kc">nil</span> <span class="p">{</span>
3298 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="kc">nil</span><span class="p">,</span> <span class="nx">err</span>
3299 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
3300 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="nx">contents</span><span class="p">,</span> <span class="kc">nil</span>
3301 </span></span><span class="line"><span class="cl"><span class="p">}</span>
3302 </span></span><span class="line"><span class="cl">
3303 </span></span><span class="line"><span class="cl"><span class="c1">// good
3304 </span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="kd">func</span> <span class="nf">readFileContents</span><span class="p">(</span><span class="nx">file</span><span class="p">)</span> <span class="p">([]</span><span class="kt">byte</span><span class="p">,</span> <span class="kt">error</span><span class="p">)</span> <span class="p">{</span>
3305 </span></span><span class="line"><span class="cl"> <span class="nx">contents</span><span class="p">,</span> <span class="nx">err</span> <span class="o">:=</span> <span class="nx">os</span><span class="p">.</span><span class="nf">ReadFile</span><span class="p">(</span><span class="nx">file</span><span class="p">)</span>
3306 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="nx">err</span> <span class="o">!=</span> <span class="kc">nil</span> <span class="p">{</span>
3307 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="kc">nil</span><span class="p">,</span> <span class="nx">fmt</span><span class="p">.</span><span class="nf">Errorf</span><span class="p">(</span><span class="s">&#34;readFileContents: error while reading a file: %w&#34;</span><span class="p">,</span> <span class="nx">err</span><span class="p">)</span>
3308 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
3309 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="nx">contents</span><span class="p">,</span> <span class="kc">nil</span>
3310 </span></span><span class="line"><span class="cl"><span class="p">}</span>
3311 </span></span></code></pre><p>I also have some grips about the mutate everything approach of the language. I
3312 prefer immutability by default, but I find that in general as long as you split
3313 your functions at a reasonable size it is generally fine.</p>
3314 <p>And for a language that is supposed to be straightforward, it is strange on how
3315 much magic the language relies on, in the form of things like <a href="https://go.dev/doc/modules/layout"><code>internal</code> and
3316 <code>main</code></a> packages, name capitalisation to
3317 indicate visibility (<code>private</code> vs <code>Public</code>), conditional compiling by filenames
3318 (e.g.: <code>foo_amd64.go</code>, <code>bar_linux.go</code>), magic comments (e.g.: <code>//go:build</code>),
3319 etc.</p>
3320 <p>I expect to write more Go code going forward. Not because it is the perfect
3321 language or whatever, but just because it a is language that has some really
3322 good qualities that makes the language attractive even with the issues that I
3323 have. That makes it a reasonable good language, and at least for me this is
3324 good enough.</p></summary></entry><entry><title>Quick bits: nix-shell is cursed</title><link href="https://kokada.dev/blog/quick-bits-nix-shell-is-cursed/" rel="alternate"/><published>2024-07-27T00:00:00+00:00</published><id>https://kokada.dev/blog/quick-bits-nix-shell-is-cursed/</id><summary type="html"><p>The other day I had to run a PHP project in my machine. I have no idea how PHP
3325 ecosystem work, I just wanted to get it to run.</p>
3326 <p>The easiest way to get a script to run if you use Nix is to use <code>nix-shell</code>. As
3327 many of you probably know, you can add <code>nix-shell</code> as a shebang in your scripts
3328 to run them as <code>./script</code>. This was a PHP script so I wanted to do the same.
3329 Easy right?</p>
3330 <pre class="chroma"><code><span class="line"><span class="cl"><span class="c1">#!/use/bin/env nix-shell
3331 </span></span></span><span class="line"><span class="cl"><span class="c1">#!nix-shell -i php -p php83
3332 </span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="o">&lt;?</span><span class="nx">php</span>
3333 </span></span><span class="line"><span class="cl"><span class="k">declare</span><span class="p">(</span><span class="nx">strict_types</span><span class="o">=</span><span class="mi">1</span><span class="p">);</span>
3334 </span></span></code></pre><p>And:</p>
3335 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> ./index.php
3336 </span></span><span class="line"><span class="cl"><span class="go">Fatal error: strict_types declaration must be the very first statement in the script in index.php on line 4
3337 </span></span></span></code></pre><p>So it seems that <code>declare(strict_types=1)</code> needs to be the first line in a PHP
3338 script if used. I removed <code>declare(strict_types=1)</code> and while the script works,
3339 I don't have enough expertise in PHP to know if this would be safe or not.</p>
3340 <p>I decided to try something that initially looked really dumb:</p>
3341 <pre class="chroma"><code><span class="line"><span class="cl"><span class="c1">#!/use/bin/env nix-shell
3342 </span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="o">&lt;?</span><span class="nx">php</span>
3343 </span></span><span class="line"><span class="cl"><span class="k">declare</span><span class="p">(</span><span class="nx">strict_types</span><span class="o">=</span><span class="mi">1</span><span class="p">);</span>
3344 </span></span><span class="line"><span class="cl"><span class="c1">#!nix-shell -i php -p php83
3345 </span></span></span></code></pre><p>And:</p>
3346 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> ./index.php
3347 </span></span><span class="line"><span class="cl"><span class="go">Works
3348 </span></span></span></code></pre><p>Wat? I mean, it is not dumb if it works, but this at least looks cursed.</p>
3349 <p>Eventually I found this
3350 <a href="https://github.com/NixOS/nix/issues/2570#issuecomment-446220517">comment</a> in a
3351 Nix issue talking about cases where <code>nix-shell</code> shebang doesn't work. It looks
3352 like the classic case of a <a href="https://github.com/NixOS/nix/issues/2570#issuecomment-446222206">bug that becomes a
3353 feature</a>.</p>
3354 <p><em>Update:</em> after posting this in
3355 <a href="https://lobste.rs/s/gkcgza/quick_bits_nix_shell_is_cursed">Lobte.rs</a>, it seems
3356 someone decided to open a <a href="https://github.com/NixOS/nix/pull/11202">Pull
3357 Request</a> to document this behavior.
3358 Also the equivalent for the new <a href="https://nix.dev/manual/nix/2.23/command-ref/new-cli/nix#shebang-interpreter">nix
3359 CLI</a>
3360 explicitly documents this behavior:</p>
3361 <blockquote>
3362 <p>Note that the <code>#! nix</code> lines don't need to follow after the first line, to
3363 accomodate other interpreters.</p>
3364 </blockquote></summary></entry><entry><title>Using GitHub as a (bad) blog platform</title><link href="https://kokada.dev/blog/using-github-as-a-bad-blog-platform/" rel="alternate"/><published>2024-07-26T00:00:00+00:00</published><id>https://kokada.dev/blog/using-github-as-a-bad-blog-platform/</id><summary type="html"><p>I finally started a new blog, thanks to the offer of
3365 <a href="https://gluer.org/">@ratsclub</a> to give me free access to
3366 <a href="https://capivaras.dev/">capivaras.dev</a>. But considering how small this blog
3367 platform is supposed to be, I want to have at least somewhere to have a backup
3368 of my posts. I know <a href="https://mataroa.blog/">Mataroa</a>, the blog platform that
3369 <a href="https://capivaras.dev/">capivaras.dev</a> runs, has automatic e-mail backups, but
3370 I want something more reliable.</p>
3371 <p>I am writing all my posts in Markdown (the format that <a href="https://1.mataroa.blog/guides/markdown/">Mataroa
3372 supports</a>) files inside
3373 <a href="https://neovim.io/">neovim</a> anyway, so why not store all my Markdown files in
3374 Git? So this is what I did, I now have an unofficial mirror in
3375 <a href="https://github.com/thiagokokada/blog">GitHub</a>.</p>
3376 <p>While I am here, why not overcomplicate? Can I make an usable blog platform
3377 from GitHub? And by that I don't mean GitHub pages, the repository itself. I
3378 mean, it already renders Markdown files by default, so no need to do anything
3379 in that space. To reach feature parity with
3380 <a href="https://capivaras.dev/">capivaras.dev</a>, I only need to have an index and RSS
3381 (since comments are not supported anyway). No need for newsletter since GitHub
3382 has a <a href="https://docs.github.com/en/account-and-profile/managing-subscriptions-and-notifications-on-github/managing-subscriptions-for-activity-on-github/viewing-your-subscriptions">watch
3383 feature</a>
3384 already.</p>
3385 <p>After a couple of hours hacking a Python script, you can see the result of this
3386 monstrosity <a href="https://github.com/thiagokokada/blog">here</a>. The script, called
3387 <code>gen_blog.py</code>, is available at the same repository (here is a
3388 <a href="https://github.com/thiagokokada/blog/blob/c8986d1ab1b94c0986fd814629bb8eb4034fb6e7/gen_blog.py">permalink</a>).
3389 It automatically generates an index at
3390 <a href="https://github.com/thiagokokada/blog/blob/main/README.md"><code>README.md</code></a> with
3391 each blog post and a
3392 <a href="https://raw.githubusercontent.com/thiagokokada/blog/main/rss.xml"><code>rss.xml</code></a>
3393 file at the root of the repository.</p>
3394 <p>Instead of trying to explain the code, I am going to explain the general idea,
3395 because I think that if you want to replicate this idea it is better to rewrite
3396 it in a way that you understand. It shouldn't take more than 2 hours in any
3397 decent programming language. But if you really want, the script itself is
3398 licensed in <a href="https://en.wikipedia.org/wiki/WTFPL">WTFPL</a> license. The code only
3399 uses Python 3's standard library and should work in any relatively recent
3400 version (anything newer than 3.9 should work).</p>
3401 <p>So the idea is basically to organise the repository and the Markdown files in a
3402 easy way that makes it trivial to parse in a deterministic way. For example, my
3403 repository is organised in the following way:</p>
3404 <pre><code>.
3405 ├── 2024-07-26
3406 │ ├── 01-writing-nixos-tests-for-fun-and-profit.md
3407 │ └── 02-using-github-as-a-bad-blog-platform.md &lt;- this file
3408 ├── gen_blog.py
3409 ├── README.md
3410 └── rss.xml
3411 </code></pre>
3412 <p>Each day that you write a new blog post will be on its own directory. This is
3413 nice because Markdown files may include extra files than the posts themselves,
3414 e.g.: images, and this organisation make it trivial to organise everything.</p>
3415 <p>Each post has its own Markdown file. I put a two digit number before each post,
3416 to ensure that when publishing multiple posts at the same day I keep them in
3417 the same order of publishing. But if you don't care about it, you can just name
3418 the files whatever you want.</p>
3419 <p>Also, I am assuming that each Markdown file has a header starting with <code># </code>,
3420 and that is the title of the blog post.</p>
3421 <p>Using the above organisation, I have this function that scrapes the repository
3422 and collect the necessary information to generate the index and RSS files:</p>
3423 <pre class="chroma"><code><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">grab_posts</span><span class="p">(</span><span class="n">pwd</span><span class="p">:</span> <span class="n">Path</span><span class="p">):</span>
3424 </span></span><span class="line"><span class="cl"> <span class="n">posts</span> <span class="o">=</span> <span class="n">defaultdict</span><span class="p">(</span><span class="nb">list</span><span class="p">)</span>
3425 </span></span><span class="line"><span class="cl">
3426 </span></span><span class="line"><span class="cl"> <span class="k">for</span> <span class="nb">dir</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">pwd</span><span class="o">.</span><span class="n">iterdir</span><span class="p">(),</span> <span class="n">reverse</span><span class="o">=</span><span class="kc">True</span><span class="p">):</span>
3427 </span></span><span class="line"><span class="cl"> <span class="c1"># Ignore non-directories or hidden files</span>
3428 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="ow">not</span> <span class="nb">dir</span><span class="o">.</span><span class="n">is_dir</span><span class="p">()</span> <span class="ow">or</span> <span class="nb">dir</span><span class="o">.</span><span class="n">name</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="s2">&#34;.&#34;</span><span class="p">:</span>
3429 </span></span><span class="line"><span class="cl"> <span class="k">continue</span>
3430 </span></span><span class="line"><span class="cl">
3431 </span></span><span class="line"><span class="cl"> <span class="c1"># Try to parse date from directory name</span>
3432 </span></span><span class="line"><span class="cl"> <span class="k">try</span><span class="p">:</span>
3433 </span></span><span class="line"><span class="cl"> <span class="n">date</span> <span class="o">=</span> <span class="n">datetime</span><span class="o">.</span><span class="n">strptime</span><span class="p">(</span><span class="nb">dir</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="s2">&#34;%Y-%m-</span><span class="si">%d</span><span class="s2">&#34;</span><span class="p">)</span>
3434 </span></span><span class="line"><span class="cl"> <span class="k">except</span> <span class="ne">ValueError</span><span class="p">:</span>
3435 </span></span><span class="line"><span class="cl"> <span class="nb">print</span><span class="p">(</span><span class="sa">f</span><span class="s2">&#34;WARN: ignoring non-date directory: </span><span class="si">{</span><span class="nb">dir</span><span class="si">}</span><span class="s2">&#34;</span><span class="p">,</span> <span class="n">file</span><span class="o">=</span><span class="n">sys</span><span class="o">.</span><span class="n">stderr</span><span class="p">)</span>
3436 </span></span><span class="line"><span class="cl"> <span class="k">continue</span>
3437 </span></span><span class="line"><span class="cl">
3438 </span></span><span class="line"><span class="cl"> <span class="c1"># Iterate between the files in the date directory</span>
3439 </span></span><span class="line"><span class="cl"> <span class="k">for</span> <span class="n">post</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="nb">dir</span><span class="o">.</span><span class="n">iterdir</span><span class="p">(),</span> <span class="n">reverse</span><span class="o">=</span><span class="kc">True</span><span class="p">):</span>
3440 </span></span><span class="line"><span class="cl"> <span class="c1"># Ignore non-markdown files or hidden files (draft)</span>
3441 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="ow">not</span> <span class="n">post</span><span class="o">.</span><span class="n">suffix</span> <span class="o">==</span> <span class="s2">&#34;.md&#34;</span> <span class="ow">or</span> <span class="n">post</span><span class="o">.</span><span class="n">name</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="s2">&#34;.&#34;</span><span class="p">:</span>
3442 </span></span><span class="line"><span class="cl"> <span class="k">continue</span>
3443 </span></span><span class="line"><span class="cl">
3444 </span></span><span class="line"><span class="cl"> <span class="c1"># Grab the first H1 section to parse as title</span>
3445 </span></span><span class="line"><span class="cl"> <span class="n">text</span> <span class="o">=</span> <span class="n">post</span><span class="o">.</span><span class="n">read_text</span><span class="p">()</span>
3446 </span></span><span class="line"><span class="cl"> <span class="n">mTitle</span> <span class="o">=</span> <span class="n">re</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="sa">r</span><span class="s2">&#34;# (?P&lt;title&gt;.*)\r?\n&#34;</span><span class="p">,</span> <span class="n">text</span><span class="p">)</span>
3447 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="n">mTitle</span> <span class="ow">and</span> <span class="p">(</span><span class="n">title</span> <span class="o">:=</span> <span class="n">mTitle</span><span class="o">.</span><span class="n">groupdict</span><span class="p">()</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&#34;title&#34;</span><span class="p">)):</span>
3448 </span></span><span class="line"><span class="cl"> <span class="n">posts</span><span class="p">[</span><span class="n">date</span><span class="p">]</span><span class="o">.</span><span class="n">append</span><span class="p">({</span><span class="s2">&#34;title&#34;</span><span class="p">:</span> <span class="n">title</span><span class="p">,</span> <span class="s2">&#34;file&#34;</span><span class="p">:</span> <span class="nb">str</span><span class="p">(</span><span class="n">post</span><span class="p">)})</span>
3449 </span></span><span class="line"><span class="cl"> <span class="k">else</span><span class="p">:</span>
3450 </span></span><span class="line"><span class="cl"> <span class="nb">print</span><span class="p">(</span><span class="sa">f</span><span class="s2">&#34;WARN: did not find title for file: </span><span class="si">{</span><span class="n">post</span><span class="si">}</span><span class="s2">&#34;</span><span class="p">,</span> <span class="n">file</span><span class="o">=</span><span class="n">sys</span><span class="o">.</span><span class="n">stderr</span><span class="p">)</span>
3451 </span></span><span class="line"><span class="cl">
3452 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="n">posts</span>
3453 </span></span></code></pre><p>Some interesting tidbits: if a Markdown file has a <code>.</code> at the start I assume it
3454 is a draft post, and ignore it from my scraper. I added a bunch of <code>WARN</code>
3455 prints to make sure that the me in the future doesn't do anything dumb. Also,
3456 sorting in reverse since reverse chronological order is the one most people
3457 expect in blogs (i.e.: more recent blog posts at top).</p>
3458 <p>After running the function above, I have a resulting dictionary that I can use
3459 to generate either a <code>README.md</code> file or Markdown:</p>
3460 <pre class="chroma"><code><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">gen_readme</span><span class="p">(</span><span class="n">posts</span><span class="p">):</span>
3461 </span></span><span class="line"><span class="cl"> <span class="n">titles</span> <span class="o">=</span> <span class="p">[]</span>
3462 </span></span><span class="line"><span class="cl">
3463 </span></span><span class="line"><span class="cl"> <span class="k">for</span> <span class="n">date</span><span class="p">,</span> <span class="n">dayPosts</span> <span class="ow">in</span> <span class="n">posts</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
3464 </span></span><span class="line"><span class="cl"> <span class="k">for</span> <span class="n">post</span> <span class="ow">in</span> <span class="n">dayPosts</span><span class="p">:</span>
3465 </span></span><span class="line"><span class="cl"> <span class="c1"># This creates a relative link to the Markdown file, .e.g.:</span>
3466 </span></span><span class="line"><span class="cl"> <span class="c1"># ./02-using-github-as-a-bad-blog-platform.md</span>
3467 </span></span><span class="line"><span class="cl"> <span class="n">link</span> <span class="o">=</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="s2">&#34;.&#34;</span><span class="p">,</span> <span class="n">post</span><span class="p">[</span><span class="s2">&#34;file&#34;</span><span class="p">])</span>
3468 </span></span><span class="line"><span class="cl"> <span class="c1"># This formats the title, e.g.:</span>
3469 </span></span><span class="line"><span class="cl"> <span class="c1"># - [Using GitHub as a (bad) blog platform](./2024-07-26/02-using-github-as-a-bad-blog-platform.md) - 2024-07-26</span>
3470 </span></span><span class="line"><span class="cl"> <span class="n">title</span> <span class="o">=</span> <span class="n">date</span><span class="o">.</span><span class="n">strftime</span><span class="p">(</span><span class="sa">f</span><span class="s2">&#34;- [</span><span class="si">{</span><span class="n">post</span><span class="p">[</span><span class="s1">&#39;title&#39;</span><span class="p">]</span><span class="si">}</span><span class="s2">](</span><span class="si">{</span><span class="n">link</span><span class="si">}</span><span class="s2">) - %Y-%m-%d&#34;</span><span class="p">)</span>
3471 </span></span><span class="line"><span class="cl"> <span class="c1"># This appends to the list to generate the content later</span>
3472 </span></span><span class="line"><span class="cl"> <span class="n">titles</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">title</span><span class="p">)</span>
3473 </span></span><span class="line"><span class="cl">
3474 </span></span><span class="line"><span class="cl"> <span class="c1"># README_TEMPLATE is a string with the static part of the README</span>
3475 </span></span><span class="line"><span class="cl"> <span class="nb">print</span><span class="p">(</span><span class="n">README_TEMPLATE</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">posts</span><span class="o">=</span><span class="s2">&#34;</span><span class="se">\n</span><span class="s2">&#34;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">titles</span><span class="p">)))</span>
3476 </span></span><span class="line"><span class="cl">
3477 </span></span><span class="line"><span class="cl">
3478 </span></span><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">gen_rss</span><span class="p">(</span><span class="n">posts</span><span class="p">):</span>
3479 </span></span><span class="line"><span class="cl"> <span class="c1"># Got most of the specification from here:</span>
3480 </span></span><span class="line"><span class="cl"> <span class="c1"># https://www.w3schools.com/XML/xml_rss.asp</span>
3481 </span></span><span class="line"><span class="cl"> <span class="n">rss</span> <span class="o">=</span> <span class="n">ET</span><span class="o">.</span><span class="n">Element</span><span class="p">(</span><span class="s2">&#34;rss&#34;</span><span class="p">,</span> <span class="n">version</span><span class="o">=</span><span class="s2">&#34;2.0&#34;</span><span class="p">)</span>
3482 </span></span><span class="line"><span class="cl">
3483 </span></span><span class="line"><span class="cl"> <span class="c1"># Here are the RSS metadata for the blog itself</span>
3484 </span></span><span class="line"><span class="cl"> <span class="n">channel</span> <span class="o">=</span> <span class="n">ET</span><span class="o">.</span><span class="n">SubElement</span><span class="p">(</span><span class="n">rss</span><span class="p">,</span> <span class="s2">&#34;channel&#34;</span><span class="p">)</span>
3485 </span></span><span class="line"><span class="cl"> <span class="n">ET</span><span class="o">.</span><span class="n">SubElement</span><span class="p">(</span><span class="n">channel</span><span class="p">,</span> <span class="s2">&#34;title&#34;</span><span class="p">)</span><span class="o">.</span><span class="n">text</span> <span class="o">=</span> <span class="s2">&#34;kokada&#39;s blog&#34;</span>
3486 </span></span><span class="line"><span class="cl"> <span class="n">ET</span><span class="o">.</span><span class="n">SubElement</span><span class="p">(</span><span class="n">channel</span><span class="p">,</span> <span class="s2">&#34;link&#34;</span><span class="p">)</span><span class="o">.</span><span class="n">text</span> <span class="o">=</span> <span class="s2">&#34;https://github.com/thiagokokada/blog&#34;</span>
3487 </span></span><span class="line"><span class="cl"> <span class="n">ET</span><span class="o">.</span><span class="n">SubElement</span><span class="p">(</span><span class="n">channel</span><span class="p">,</span> <span class="s2">&#34;description&#34;</span><span class="p">)</span><span class="o">.</span><span class="n">text</span> <span class="o">=</span> <span class="s2">&#34;dd if=/dev/urandom of=/dev/brain0&#34;</span>
3488 </span></span><span class="line"><span class="cl">
3489 </span></span><span class="line"><span class="cl"> <span class="c1"># You create one item for each blog post</span>
3490 </span></span><span class="line"><span class="cl"> <span class="k">for</span> <span class="n">date</span><span class="p">,</span> <span class="n">dayPost</span> <span class="ow">in</span> <span class="n">posts</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
3491 </span></span><span class="line"><span class="cl"> <span class="k">for</span> <span class="n">post</span> <span class="ow">in</span> <span class="n">dayPost</span><span class="p">:</span>
3492 </span></span><span class="line"><span class="cl"> <span class="n">item</span> <span class="o">=</span> <span class="n">ET</span><span class="o">.</span><span class="n">SubElement</span><span class="p">(</span><span class="n">channel</span><span class="p">,</span> <span class="s2">&#34;item&#34;</span><span class="p">)</span>
3493 </span></span><span class="line"><span class="cl"> <span class="n">link</span> <span class="o">=</span> <span class="n">urljoin</span><span class="p">(</span><span class="n">RSS_POST_LINK_PREFIX</span><span class="p">,</span> <span class="n">post</span><span class="p">[</span><span class="s2">&#34;file&#34;</span><span class="p">])</span>
3494 </span></span><span class="line"><span class="cl"> <span class="n">ET</span><span class="o">.</span><span class="n">SubElement</span><span class="p">(</span><span class="n">item</span><span class="p">,</span> <span class="s2">&#34;title&#34;</span><span class="p">)</span><span class="o">.</span><span class="n">text</span> <span class="o">=</span> <span class="n">post</span><span class="p">[</span><span class="s2">&#34;title&#34;</span><span class="p">]</span>
3495 </span></span><span class="line"><span class="cl"> <span class="n">ET</span><span class="o">.</span><span class="n">SubElement</span><span class="p">(</span><span class="n">item</span><span class="p">,</span> <span class="s2">&#34;guid&#34;</span><span class="p">)</span><span class="o">.</span><span class="n">text</span> <span class="o">=</span> <span class="n">link</span>
3496 </span></span><span class="line"><span class="cl"> <span class="n">ET</span><span class="o">.</span><span class="n">SubElement</span><span class="p">(</span><span class="n">item</span><span class="p">,</span> <span class="s2">&#34;link&#34;</span><span class="p">)</span><span class="o">.</span><span class="n">text</span> <span class="o">=</span> <span class="n">link</span>
3497 </span></span><span class="line"><span class="cl"> <span class="n">ET</span><span class="o">.</span><span class="n">SubElement</span><span class="p">(</span><span class="n">item</span><span class="p">,</span> <span class="s2">&#34;pubDate&#34;</span><span class="p">)</span><span class="o">.</span><span class="n">text</span> <span class="o">=</span> <span class="n">date</span><span class="o">.</span><span class="n">strftime</span><span class="p">(</span><span class="s1">&#39;</span><span class="si">%a</span><span class="s1">, </span><span class="si">%d</span><span class="s1"> %b %Y %H:%M:%S GMT&#39;</span><span class="p">)</span>
3498 </span></span><span class="line"><span class="cl">
3499 </span></span><span class="line"><span class="cl"> <span class="c1"># Generate the XML and indent</span>
3500 </span></span><span class="line"><span class="cl"> <span class="n">tree</span> <span class="o">=</span> <span class="n">ET</span><span class="o">.</span><span class="n">ElementTree</span><span class="p">(</span><span class="n">rss</span><span class="p">)</span>
3501 </span></span><span class="line"><span class="cl"> <span class="n">ET</span><span class="o">.</span><span class="n">indent</span><span class="p">(</span><span class="n">tree</span><span class="p">,</span> <span class="n">space</span><span class="o">=</span><span class="s2">&#34;</span><span class="se">\t</span><span class="s2">&#34;</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>
3502 </span></span><span class="line"><span class="cl"> <span class="n">tree</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="s2">&#34;rss.xml&#34;</span><span class="p">,</span> <span class="n">xml_declaration</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">encoding</span><span class="o">=</span><span class="s2">&#34;UTF-8&#34;</span><span class="p">)</span>
3503 </span></span></code></pre><p>To publish a new Post, a basically write a Markdown file, run `./gen_readme.py</p>
3504 <blockquote>
3505 <p>README.md` at the root of the repository, and see the magic happen.</p>
3506 </blockquote>
3507 <p>It works much better than I initially anticipated. The <code>README.md</code> is properly
3508 populated with the titles and links. The RSS is kind of empty since it has no
3509 description, but it seems to work fine (at least in
3510 <a href="https://www.inoreader.com/">Inoreader</a>, my RSS reader of choice). I can
3511 probably fill the post description with more information if I really want, but
3512 it is enough for now (update: it is working now, you just need to render the
3513 Markdown as HTML and escape the tags;
3514 <a href="https://github.com/thiagokokada/blog/blob/9506051cc3d49f203304174b335ff62de9d17a05/gen_blog.py">permalink</a>
3515 for the updated script). Not sure who is that interested in my writing that
3516 will want to use this RSS feed instead the one available in
3517 <a href="https://kokada.capivaras.dev/rss/">capivaras.dev</a> anyway.</p>
3518 <p>Also, while I am using GitHub here, the same idea would work in GitLab, Gitea,
3519 sr.ht or whatever. As long as your source hub supports Markdown files it should
3520 work.</p>
3521 <p>So that is it. I am not saying this is a good idea for your primary blog
3522 platform or whatever, and I still prefer to publish to a platform that doesn't
3523 track users or have tons of JavaScript or whatever. But if you want a backup of
3524 your posts and you are already writing Markdown anyway, well, there are worse
3525 ways to do it I think.</p>
3526 <p><em>Update:</em> I rewrote the script again using Go
3527 (<a href="https://github.com/thiagokokada/blog/blob/3ece3853e7dd32977e7ec5238d2955e530ca4beb/blog.go">permalink</a>).
3528 The reason for it is because when I started rendering Markdown (for
3529 descriptions) the Python version got quite slow (not the fault of Python
3530 itself, mostly because of the usage of <code>nix-shell</code> to manage dependencies;
3531 something that Go doesn't need). Took about half an hour, showing how easy it
3532 is to do the same.</p></summary></entry><entry><title>Writing NixOS tests for fun and profit</title><link href="https://kokada.dev/blog/writing-nixos-tests-for-fun-and-profit/" rel="alternate"/><published>2024-07-26T00:00:00+00:00</published><id>https://kokada.dev/blog/writing-nixos-tests-for-fun-and-profit/</id><summary type="html"><p>I recently started a <a href="https://github.com/thiagokokada/hyprland-go">new side
3533 project</a> writing an IPC library in
3534 Go for <a href="https://hyprland.org/">Hyprland</a>, a Window Manager for Wayland.</p>
3535 <p>Once I got past the Work-in-Progress phase, I realise I had an issue: I wrote
3536 some tests, but I was running then inside my system running Hyprland. And the
3537 tests themselves were annoying: since they send commands to the current running
3538 Hyprland instance, I was having programs being opened and settings being
3539 changed, because this was the only way to have a reasonable good confidence
3540 that what I was doing was correct. So I need to do like any good developer and
3541 implement a CI, but how?</p>
3542 <p>One approach would be to create something like a mock client and test against
3543 my mock. Since this mock wouldn't need a running Hyprland instance the tests
3544 could run everywhere (even in non-Linux systems!), but they wouldn't be much
3545 useful. Mocks are great for testing business logic, but not really for making
3546 sure everything is working correctly.</p>
3547 <p>I need something more akin to an integration test, but this is tricky. It is
3548 not like I am doing integration with e.g.: PostgreSQL that has thousands of
3549 libraries available to make integration tests easier, I am doing integration
3550 with a Window Manager that is a moving target with multiple breaking changes in
3551 each release. And this is where NixOS tests enter, a way to run tests inside
3552 Virtual Machines configured in Nix.</p>
3553 <p>I am a long time NixOS user and commiter, but I never wrote a NixOS test
3554 outside of <a href="https://github.com/NixOS/nixpkgs">nixpkgs</a> itself. However I knew
3555 it was possible, and after doing a quick reading of the <a href="https://wiki.nixos.org/wiki/NixOS_VM_tests">Wiki
3556 entry</a> about it, I was ready to
3557 start.</p>
3558 <p>The first part is to call <code>pkgs.testers.runNixOSTest</code> and configure the machine
3559 as any other NixOS system, e.g.:</p>
3560 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span> <span class="n">pkgs</span><span class="o">,</span> <span class="o">...</span> <span class="p">}:</span>
3561 </span></span><span class="line"><span class="cl"><span class="n">pkgs</span><span class="o">.</span><span class="n">testers</span><span class="o">.</span><span class="n">runNixOSTest</span> <span class="p">{</span>
3562 </span></span><span class="line"><span class="cl"> <span class="n">name</span> <span class="o">=</span> <span class="s2">&#34;hyprland-go&#34;</span><span class="p">;</span>
3563 </span></span><span class="line"><span class="cl">
3564 </span></span><span class="line"><span class="cl"> <span class="n">nodes</span><span class="o">.</span><span class="n">machine</span> <span class="o">=</span>
3565 </span></span><span class="line"><span class="cl"> <span class="p">{</span> <span class="n">config</span><span class="o">,</span> <span class="n">pkgs</span><span class="o">,</span> <span class="n">lib</span><span class="o">,</span> <span class="o">...</span> <span class="p">}:</span>
3566 </span></span><span class="line"><span class="cl"> <span class="p">{</span>
3567 </span></span><span class="line"><span class="cl"> <span class="c1"># bootloader related configuration</span>
3568 </span></span><span class="line"><span class="cl"> <span class="n">boot</span><span class="o">.</span><span class="n">loader</span><span class="o">.</span><span class="n">systemd-boot</span><span class="o">.</span><span class="n">enable</span> <span class="o">=</span> <span class="no">true</span><span class="p">;</span>
3569 </span></span><span class="line"><span class="cl"> <span class="n">boot</span><span class="o">.</span><span class="n">loader</span><span class="o">.</span><span class="n">efi</span><span class="o">.</span><span class="n">canTouchEfiVariables</span> <span class="o">=</span> <span class="no">true</span><span class="p">;</span>
3570 </span></span><span class="line"><span class="cl">
3571 </span></span><span class="line"><span class="cl"> <span class="c1"># enable hyprland</span>
3572 </span></span><span class="line"><span class="cl"> <span class="n">programs</span><span class="o">.</span><span class="n">hyprland</span><span class="o">.</span><span class="n">enable</span> <span class="o">=</span> <span class="no">true</span><span class="p">;</span>
3573 </span></span><span class="line"><span class="cl">
3574 </span></span><span class="line"><span class="cl"> <span class="c1"># create a user called alice</span>
3575 </span></span><span class="line"><span class="cl"> <span class="n">users</span><span class="o">.</span><span class="n">users</span><span class="o">.</span><span class="n">alice</span> <span class="o">=</span> <span class="p">{</span>
3576 </span></span><span class="line"><span class="cl"> <span class="n">isNormalUser</span> <span class="o">=</span> <span class="no">true</span><span class="p">;</span>
3577 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
3578 </span></span><span class="line"><span class="cl">
3579 </span></span><span class="line"><span class="cl"> <span class="c1"># add some extra packages that we need during tests</span>
3580 </span></span><span class="line"><span class="cl"> <span class="n">environment</span><span class="o">.</span><span class="n">systemPackages</span> <span class="o">=</span> <span class="k">with</span> <span class="n">pkgs</span><span class="p">;</span> <span class="p">[</span> <span class="n">go</span> <span class="n">kitty</span> <span class="p">];</span>
3581 </span></span><span class="line"><span class="cl">
3582 </span></span><span class="line"><span class="cl"> <span class="c1"># auto login as alice</span>
3583 </span></span><span class="line"><span class="cl"> <span class="n">services</span><span class="o">.</span><span class="n">getty</span><span class="o">.</span><span class="n">autologinUser</span> <span class="o">=</span> <span class="s2">&#34;alice&#34;</span><span class="p">;</span>
3584 </span></span><span class="line"><span class="cl">
3585 </span></span><span class="line"><span class="cl"> <span class="c1"># configure VM, increase memory and CPU and enable OpenGL via LLVMPipe</span>
3586 </span></span><span class="line"><span class="cl"> <span class="n">virtualisation</span><span class="o">.</span><span class="n">qemu</span> <span class="o">=</span> <span class="p">{</span>
3587 </span></span><span class="line"><span class="cl"> <span class="n">options</span> <span class="o">=</span> <span class="p">[</span>
3588 </span></span><span class="line"><span class="cl"> <span class="s2">&#34;-smp 2&#34;</span>
3589 </span></span><span class="line"><span class="cl"> <span class="s2">&#34;-m 4G&#34;</span>
3590 </span></span><span class="line"><span class="cl"> <span class="s2">&#34;-vga none&#34;</span>
3591 </span></span><span class="line"><span class="cl"> <span class="s2">&#34;-device virtio-gpu-pci&#34;</span>
3592 </span></span><span class="line"><span class="cl"> <span class="p">];</span>
3593 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
3594 </span></span><span class="line"><span class="cl">
3595 </span></span><span class="line"><span class="cl"> <span class="c1"># Start hyprland at login</span>
3596 </span></span><span class="line"><span class="cl"> <span class="n">programs</span><span class="o">.</span><span class="n">bash</span><span class="o">.</span><span class="n">loginShellInit</span> <span class="o">=</span> <span class="s2">&#34;Hyprland&#34;</span><span class="p">;</span>
3597 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
3598 </span></span><span class="line"><span class="cl">
3599 </span></span><span class="line"><span class="cl"> <span class="n">testScript</span> <span class="o">=</span> <span class="s2">&#34;start_all()&#34;</span><span class="p">;</span>
3600 </span></span><span class="line"><span class="cl"><span class="p">}</span>
3601 </span></span></code></pre><p>A few details that I want to bring to attention. The first one is how easy it
3602 is to setup things like a normal user account, add some extra packages we need
3603 for testing, add Hyprland itself and configure auto-login. I have no idea how
3604 painful it would be to automatise all those steps in e.g.: Ansible, but here we
3605 are in a few lines of Nix code. This is, of course, thanks to all the
3606 contributors to nixpkgs that implement something that help their own use case,
3607 but once combined make it greater than the sum of the parts.</p>
3608 <p>Second is something that I took a while to figure out: how to enable GPU
3609 acceleration inside the VM. You see, Hyprland, different from other Window
3610 Managers, requires OpenGL support. This is basically why the flag <code>-device virtio-gpu-pci</code> is in <code>virtualisation.qemu.options</code>, this enables OpenGL
3611 rendering via LLVMPipe, that while being slow since it is rendered in CPU, is
3612 sufficient for this case.</p>
3613 <p>Putting the above code inside a
3614 <a href="https://wiki.nixos.org/wiki/Flakes"><code>flake.nix</code></a> for reproducibility, I had
3615 something similar to:</p>
3616 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
3617 </span></span><span class="line"><span class="cl"> <span class="n">description</span> <span class="o">=</span> <span class="s2">&#34;Hyprland&#39;s IPC bindings for Go&#34;</span><span class="p">;</span>
3618 </span></span><span class="line"><span class="cl">
3619 </span></span><span class="line"><span class="cl"> <span class="n">inputs</span> <span class="o">=</span> <span class="p">{</span>
3620 </span></span><span class="line"><span class="cl"> <span class="n">nixpkgs</span><span class="o">.</span><span class="n">url</span> <span class="o">=</span> <span class="s2">&#34;github:NixOS/nixpkgs/nixos-unstable&#34;</span><span class="p">;</span>
3621 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
3622 </span></span><span class="line"><span class="cl">
3623 </span></span><span class="line"><span class="cl"> <span class="n">outputs</span> <span class="o">=</span> <span class="p">{</span> <span class="n">nixpkgs</span><span class="o">,</span> <span class="o">...</span> <span class="p">}:</span>
3624 </span></span><span class="line"><span class="cl"> <span class="p">{</span>
3625 </span></span><span class="line"><span class="cl"> <span class="n">checks</span><span class="o">.</span><span class="n">x86_64-linux</span> <span class="o">=</span>
3626 </span></span><span class="line"><span class="cl"> <span class="k">let</span>
3627 </span></span><span class="line"><span class="cl"> <span class="n">pkgs</span> <span class="o">=</span> <span class="kn">import</span> <span class="n">nixpkgs</span> <span class="p">{</span> <span class="n">system</span> <span class="o">=</span> <span class="s2">&#34;x86_64-linux&#34;</span><span class="p">;</span> <span class="p">};</span>
3628 </span></span><span class="line"><span class="cl"> <span class="k">in</span>
3629 </span></span><span class="line"><span class="cl"> <span class="p">{</span>
3630 </span></span><span class="line"><span class="cl"> <span class="n">testVm</span> <span class="o">=</span> <span class="n">pkgs</span><span class="o">.</span><span class="n">testers</span><span class="o">.</span><span class="n">runNixOSTest</span> <span class="p">{</span>
3631 </span></span><span class="line"><span class="cl"> <span class="c1"># the code above</span>
3632 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
3633 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
3634 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
3635 </span></span><span class="line"><span class="cl"><span class="p">}</span>
3636 </span></span></code></pre><p>I can now run <code>nix build .#checks.x86_64-linux.testVm -L</code> to build and run the
3637 VM. However it is not really useful right now, since we didn't add any useful
3638 code in <code>testScript</code>, the core of the NixOS test framework. We can also run
3639 <code>nix build .#checks.x86_64-linux.testVm.driverInteractive</code> and
3640 <code>./result/bin/nixos-test-driver</code>: this will start a Python console where we can
3641 manually play with the VM (try typing <code>start_all()</code> for example).</p>
3642 <p>The <code>testScript</code> is a sequence of Python statements that perform various
3643 actions, such as starting VMs, executing commands in the VMs, and so on. More
3644 about it in the official
3645 <a href="https://nixos.org/manual/nixos/stable/index.html#sec-nixos-tests">documentation</a>.
3646 For our case we can start with something like this:</p>
3647 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
3648 </span></span><span class="line"><span class="cl"> <span class="n">testScript</span> <span class="o">=</span> <span class="cm">/* python */</span> <span class="s1">&#39;&#39;
3649 </span></span></span><span class="line"><span class="cl"><span class="s1"> start_all()
3650 </span></span></span><span class="line"><span class="cl"><span class="s1">
3651 </span></span></span><span class="line"><span class="cl"><span class="s1"> machine.wait_for_unit(&#34;multi-user.target&#34;)
3652 </span></span></span><span class="line"><span class="cl"><span class="s1"> machine.wait_for_file(&#34;/home/alice/test-finished&#34;)
3653 </span></span></span><span class="line"><span class="cl"><span class="s1"> &#39;&#39;</span><span class="p">;</span>
3654 </span></span><span class="line"><span class="cl"><span class="p">}</span>
3655 </span></span></code></pre><p>The first statement, <code>start_all()</code>, starts all VMs, in this case we have only
3656 one, called <code>machine</code>. We send two further commands to <code>machine</code>:
3657 <code>wait_for_unit(&quot;multi-user.target&quot;)</code> and
3658 <code>wait_for_file(&quot;/home/alice/test-finished&quot;)</code>.</p>
3659 <p>The first command waits until systemd's <code>multi-user.target</code> is ready, a good
3660 way to ensure that the system is ready for further commands. The second one we
3661 wait for a file called <code>test-finished</code> to appear in Alice's <code>$HOME</code> (basically,
3662 a canary), but how can we generate this file?</p>
3663 <p>Remember that we added <code>programs.bash.loginShellInit = &quot;Hyprland&quot;</code>, that
3664 automatically starts Hyprland when Alice logs in. We need to modify that
3665 command to run the Go tests from our library. The good thing is that Hyprland
3666 configuration file supports a
3667 <a href="https://wiki.hyprland.org/Configuring/Keywords/#executing"><code>exec-once</code></a>
3668 command that runs a command during Hyprland launch. We can abuse this to launch
3669 a terminal emulator and run our tests:</p>
3670 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
3671 </span></span><span class="line"><span class="cl"> <span class="n">programs</span><span class="o">.</span><span class="n">bash</span><span class="o">.</span><span class="n">loginShellInit</span> <span class="o">=</span>
3672 </span></span><span class="line"><span class="cl"> <span class="k">let</span>
3673 </span></span><span class="line"><span class="cl"> <span class="n">testScript</span> <span class="o">=</span> <span class="n">pkgs</span><span class="o">.</span><span class="n">writeShellScript</span> <span class="s2">&#34;hyprland-go-test&#34;</span> <span class="s1">&#39;&#39;
3674 </span></span></span><span class="line"><span class="cl"><span class="s1"> set -euo pipefail
3675 </span></span></span><span class="line"><span class="cl"><span class="s1">
3676 </span></span></span><span class="line"><span class="cl"><span class="s1"> trap &#39;echo $? &gt; $HOME/test-finished&#39; EXIT # creates the canary when the script finishes
3677 </span></span></span><span class="line"><span class="cl"><span class="s1">
3678 </span></span></span><span class="line"><span class="cl"><span class="s1"> cd </span><span class="si">${</span><span class="sr">./.</span><span class="si">}</span><span class="s1"> # go to the library directory
3679 </span></span></span><span class="line"><span class="cl"><span class="s1"> go test -v ./... &gt; $HOME/test.log 2&gt;&amp;1 # run Go tests
3680 </span></span></span><span class="line"><span class="cl"><span class="s1"> &#39;&#39;</span><span class="p">;</span>
3681 </span></span><span class="line"><span class="cl"> <span class="n">hyprlandConf</span> <span class="o">=</span> <span class="n">pkgs</span><span class="o">.</span><span class="n">writeText</span> <span class="s2">&#34;hyprland.conf&#34;</span> <span class="s1">&#39;&#39;
3682 </span></span></span><span class="line"><span class="cl"><span class="s1"> exec-once = kitty sh -c </span><span class="si">${</span><span class="n">testScript</span><span class="si">}</span><span class="s1">
3683 </span></span></span><span class="line"><span class="cl"><span class="s1"> &#39;&#39;</span><span class="p">;</span>
3684 </span></span><span class="line"><span class="cl"> <span class="k">in</span> <span class="s1">&#39;&#39;
3685 </span></span></span><span class="line"><span class="cl"><span class="s1"> Hyprland --config </span><span class="si">${</span><span class="n">hyprlandConf</span><span class="si">}</span><span class="s1">
3686 </span></span></span><span class="line"><span class="cl"><span class="s1"> &#39;&#39;</span><span class="p">;</span>
3687 </span></span><span class="line"><span class="cl"><span class="p">}</span>
3688 </span></span></code></pre><p>So we are basically creating a custom Hyprland config that starts a
3689 <a href="https://sw.kovidgoyal.net/kitty/">Kitty</a> terminal emulator, that then launches
3690 a shell script that runs the test. Since we have no way to get the results of
3691 the test, we pipe the output to a file that we can collect later (e.g.:
3692 <code>machine.succeded(&quot;cat /home/alice/test.log&quot;)</code>). And once the script exit, we
3693 create the canary file <code>$HOME/test-finished</code>, that allows the <code>testScript</code>
3694 knows that the test finished and it can destroy the VM safely.</p>
3695 <p>If you want to take a look at the final result, it is
3696 <a href="https://github.com/thiagokokada/hyprland-go/blob/v0.0.1/flake.nix">here</a>. This
3697 tests run in any Linux machine that supports KVM, and also works in <a href="https://github.com/thiagokokada/hyprland-go/actions/workflows/nix.yaml">GitHub
3698 Actions</a>
3699 thanks to the the
3700 <a href="https://github.com/DeterminateSystems/nix-installer-action">nix-installer-action</a>.</p>
3701 <p>And now I have a proper CI pipeline in a way that I never imagined would be
3702