kokada.dev.rss.xml - sfeed_tests - sfeed tests and RSS and Atom files
(HTM) git clone git://git.codemadness.org/sfeed_tests
(DIR) Log
(DIR) Files
(DIR) Refs
(DIR) README
(DIR) LICENSE
---
kokada.dev.rss.xml (494372B)
---
1 <?xml version="1.0" encoding="utf-8"?>
2 <rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>kokada</title><link>https://kokada.dev</link><description>
3 # dd if=/dev/urandom of=/dev/brain0
4 dd: error writing &#39;/dev/brain0&#39;: No space left on device
5
6 </description><atom:link href="https://kokada.dev/rss/" rel="self"/><language>en-us</language><lastBuildDate>Wed, 04 Dec 2024 00:00:00 +0000</lastBuildDate><item><title>Praise to scdoc to generate man pages</title><link>https://kokada.dev/blog/praise-to-scdoc-to-generate-man-pages/</link><description><p>Hey, its been a long time since my <a href="posts/2024-10-07/01-enabling-le-audio-lc3-in-wf-1000xm5.md">last blog
7 post</a>. It is mostly
8 because I ran out of things to write, but I expected this. This is probably
9 more likely how I am actually going to post from now. At least, it shows that
10 my plan to have a <a href="https://kokada.dev/blog/making-a-blog-for-the-next-10-years">blog for a long
11 time</a>, that is
12 easy to go back when I wanted is working fine, but I digress.</p>
13 <p>Going back to the theme of the today blog post, I needed to write a <a href="https://en.wikipedia.org/wiki/Man_page">man
14 page</a> for the first time in years. I
15 hate <a href="https://en.wikipedia.org/wiki/Troff">troff</a>, the typesetting system used
16 for man pages (similar to <a href="https://en.wikipedia.org/wiki/LaTeX">LaTeX</a> for
17 documents). It is one of the weirdest languages that I ever saw, and even the
18 example in Wikipedia shows that:</p>
19 <pre><code class="language-troff">.ND &quot;January 10, 1993&quot;
20 .AU &quot;Ms. Jane Smith&quot;
21 .AT &quot;Upcoming appointment&quot;
22 .MT 5
23 .DS
24 Reference #A12345
25 .sp 4
26 Mr. Samuel Jones
27 Field director, Bureau of Inspections
28 1010 Government Plaza
29 Capitoltown, ST
30 .sp 3
31 Dear Mr. Jones,
32 .sp 2
33 .P
34 Making reference to the noted obligation to submit for state inspection our newly created production process, we request that you consider the possible inappropriateness of subjecting the innovative technologies of tomorrow to the largely antiquated requirements of yesterday. If our great state is to prosper in the twenty-first century, we must take steps
35 .B now ,
36 in
37 .I this
38 year of
39 .I this
40 decade, to prepare our industrial base for the interstate and international competition that is sure to appear. Our new process does precisely that. Please do not let it be undone by a regulatory environment that is no longer apt.
41 .P
42 Thank you for your consideration of our position.
43 .FC Sincerely
44 .SG
45 </code></pre>
46 <p>Keep in mind that the break lines are necessary every time you introduce a
47 macro, like <code>.I this</code> (that I <em>think</em> it is for italics). Yes, this format is
48 as illegible as hell, and it is worse that the format lacks good tooling (or at
49 least I didn't find any good ones).</p>
50 <p>Most people when they need to write a man page nowadays ends up using some
51 other format that generates a man page. For example, in the past I used
52 <a href="https://pandoc.org/">Pandoc</a> to convert Markdown to a man page, but even if
53 Pandoc is a great project the result is sub-optimal at best: Markdowns are, at
54 the end, designed for generating HTML (and a subset of it), and not man pages,
55 so you basically ends up fighting the format for it to do what you want.
56 Also, Pandoc is a big project, with a ~200MB binary (at least it is the default
57 Pandoc binary in Nix).</p>
58 <p>For this specific project I needed something small. I am trying to replace one
59 of the most essential pieces inside NixOS, <code>nixos-rebuild</code>, written in Bash,
60 with a <a href="https://discourse.nixos.org/t/nixos-rebuild-ng-a-nixos-rebuild-rewrite/55606/">full rewritten in
61 Python</a>
62 (sorry Rust zealots!), called <code>nixos-rebuild-ng</code>.</p>
63 <p>Since this project will eventually (if successful) be in the critical path for
64 NixOS, I want to reduce the number of dependencies as much as possible, so
65 something as big as Pandoc is out. I could use
66 <a href="https://asciidoc.org/">AsciiDoc</a>, but it is a big complicated Python project
67 (this may seem ironic, but <code>nixos-rebuild-ng</code> has only one runtime dependency,
68 that is optional). And I also hated the last time I tried to use it to generate
69 man pages: it more flexible than Markdown, but still far from optimal.</p>
70 <p>Thanks to Drew DeVault (creator of <a href="https://swaywm.org/">SwayWM</a>) that seems it
71 had the same issues in the past and created
72 <a href="https://drewdevault.com/2018/05/13/scdoc.html"><code>scdoc</code></a>, a very simple man
73 page generator using a DSL inspired in Markdown, but specific to generate man
74 pages. The binary is written in C (and advantage in this case since it means it
75 is easier to bootstrap), is small (~1 Kloc) and has no dependencies, so it
76 fits the requirement.</p>
77 <p>While the language suffers from being a niche project for a niche segment, the
78 <a href="https://man.archlinux.org/man/scdoc.5.en">man page</a> for it is actually really
79 nice. It is terse though and lacks examples, and this is what this blog post
80 will try to accomplish.</p>
81 <p>To start, let's have a quick summary of the syntax, written in <code>scdoc</code> as
82 comments:</p>
83 <pre><code class="language-scdoc">; quick summary:
84 ; # new section
85 ; comments starts with ;
86 ; - this is a list
87 ; - sub-list
88 ; - *bold*: _underline_, force a line break++
89 ; - [tables], \[ can be used to force an actual [
90 ; . numbered list
91 ; please configure your editor to use hard tabs
92 ; see `man 5 scdoc` for more information about syntax
93 ; or https://man.archlinux.org/man/scdoc.5.en
94 </code></pre>
95 <p>I actually added this summary in the <code>.scd</code> (the <code>scdoc</code> extension) files that
96 I wrote, so it is easy for someone that never saw the format to start
97 collaborating.</p>
98 <p>And here an example of a (summarised) man page in <code>.scd</code> format:</p>
99 <pre class="chroma"><code><span class="line"><span class="cl">nixos-rebuild-ng(8)
100 </span></span><span class="line"><span class="cl">
101 </span></span><span class="line"><span class="cl"><span class="gh"># NAME
102 </span></span></span><span class="line"><span class="cl"><span class="gh"></span>
103 </span></span><span class="line"><span class="cl">nixos-rebuild - reconfigure a NixOS machine
104 </span></span><span class="line"><span class="cl">
105 </span></span><span class="line"><span class="cl"><span class="gh"># SYNOPSIS
106 </span></span></span><span class="line"><span class="cl"><span class="gh"></span>
107 </span></span><span class="line"><span class="cl"><span class="ge">_nixos-rebuild_</span> \[--upgrade] [--upgrade-all]++
108 </span></span><span class="line"><span class="cl"> \[{switch,boot}]
109 </span></span><span class="line"><span class="cl">
110 </span></span><span class="line"><span class="cl"><span class="gh"># DESCRIPTION
111 </span></span></span><span class="line"><span class="cl"><span class="gh"></span>
112 </span></span><span class="line"><span class="cl">This command has one required argument, which specifies the desired operation.
113 </span></span><span class="line"><span class="cl">It must be one of the following:
114 </span></span><span class="line"><span class="cl">
115 </span></span><span class="line"><span class="cl"><span class="ge">*switch*</span>
116 </span></span><span class="line"><span class="cl"> Build and activate the new configuration, and make it the boot default.
117 </span></span><span class="line"><span class="cl"> That is, the configuration is added to the GRUB boot menu as the
118 </span></span><span class="line"><span class="cl"> default menu entry, so that subsequent reboots will boot the system
119 </span></span><span class="line"><span class="cl"> into the new configuration. Previous configurations activated with
120 </span></span><span class="line"><span class="cl"> nixos-rebuild switch or nixos-rebuild boot remain available in the GRUB
121 </span></span><span class="line"><span class="cl"> menu.
122 </span></span><span class="line"><span class="cl">
123 </span></span><span class="line"><span class="cl"><span class="ge">*boot*</span>
124 </span></span><span class="line"><span class="cl"> Build the new configuration and make it the boot default (as with
125 </span></span><span class="line"><span class="cl"> <span class="ge">*nixos-rebuild switch*</span>), but do not activate it. That is, the system
126 </span></span><span class="line"><span class="cl"> continues to run the previous configuration until the next reboot.
127 </span></span><span class="line"><span class="cl">
128 </span></span><span class="line"><span class="cl"><span class="gh"># OPTIONS
129 </span></span></span><span class="line"><span class="cl"><span class="gh"></span>
130 </span></span><span class="line"><span class="cl"><span class="ge">*--upgrade, --upgrade-all*</span>
131 </span></span><span class="line"><span class="cl"> Update the root user&#39;s channel named &#39;nixos&#39; before rebuilding the
132 </span></span><span class="line"><span class="cl"> system.
133 </span></span><span class="line"><span class="cl">
134 </span></span><span class="line"><span class="cl"> In addition to the &#39;nixos&#39; channel, the root user&#39;s channels which have
135 </span></span><span class="line"><span class="cl"> a file named &#39;.update-on-nixos-rebuild&#39; in their base directory will
136 </span></span><span class="line"><span class="cl"> also be updated.
137 </span></span><span class="line"><span class="cl">
138 </span></span><span class="line"><span class="cl"> Passing <span class="ge">*--upgrade-all*</span> updates all of the root user&#39;s channels.
139 </span></span><span class="line"><span class="cl">
140 </span></span><span class="line"><span class="cl">See the Nix manual, <span class="ge">*nix flake lock --help*</span> or <span class="ge">*nix-build --help*</span> for details.
141 </span></span><span class="line"><span class="cl">
142 </span></span><span class="line"><span class="cl"><span class="gh"># ENVIRONMENT
143 </span></span></span><span class="line"><span class="cl"><span class="gh"></span>
144 </span></span><span class="line"><span class="cl">NIXOS_CONFIG
145 </span></span><span class="line"><span class="cl"> Path to the main NixOS configuration module. Defaults to
146 </span></span><span class="line"><span class="cl"> <span class="ge">_/etc/nixos/configuration.nix_</span>.
147 </span></span><span class="line"><span class="cl">
148 </span></span><span class="line"><span class="cl"><span class="gh"># FILES
149 </span></span></span><span class="line"><span class="cl"><span class="gh"></span>
150 </span></span><span class="line"><span class="cl">/etc/nixos/flake.nix
151 </span></span><span class="line"><span class="cl"> If this file exists, then <span class="ge">*nixos-rebuild*</span> will use it as if the
152 </span></span><span class="line"><span class="cl"> <span class="ge">*--flake*</span> option was given. This file may be a symlink to a
153 </span></span><span class="line"><span class="cl"> flake.nix in an actual flake; thus <span class="ge">_/etc/nixos_</span> need not be a
154 </span></span><span class="line"><span class="cl"> flake.
155 </span></span><span class="line"><span class="cl">
156 </span></span><span class="line"><span class="cl"><span class="gh"># AUTHORS
157 </span></span></span><span class="line"><span class="cl"><span class="gh"></span>
158 </span></span><span class="line"><span class="cl">Nixpkgs/NixOS contributors
159 </span></span></code></pre><p>And here is a screenshot of the result:</p>
160 <p><a href="https://github.com/thiagokokada/blog/raw/main/posts/2024-12-04/2024-12-04-230955_hyprshot.png"><img src="https://github.com/thiagokokada/blog/raw/main/posts/2024-12-04/2024-12-04-230955_hyprshot.png" alt="Man page rendered from scd
161 file"></a></p>
162 <p>One of nice things that I found is how looking at the plain text looks kind
163 like the man page result already. And if you know Markdown, you can basically
164 understand most things that is happening. There are a few differences, like
165 <code>*bold*</code> instead of <code>**bold**</code>, and while they're unfortunate they're not the
166 end of the world.</p>
167 <p>Now, the format has its quirks. The first line being the name of the program
168 and section in parenthesis is required, but this makes sense, since you need
169 this information for the corners. But for one, it requires the usage of hard
170 tabs to create indentation, and the error messages are awful, in a situation
171 that kind remembers me of <code>Makefile</code>. Also the choice of <code>[</code> to start a table
172 means that the traditional <code>app [command]</code> needs in many cases to be escaped as
173 <code>app \[command]</code>. I found this a strange choice since this is supposed to be a
174 format that is only used for man pages, and using <code>[command]</code> to indicate an
175 optional is common, but at least it is easy to escape.</p>
176 <p>In the end, I think all that matters is the result. And for the first time for
177 all those years trying to write a man page, I am satisfied with the result. The
178 man page looks exactly as I wanted once rendered, and the <code>.scd</code> file looks
179 reasonable good that it can work as a documentation for someone that for one
180 reason or another can't use the man page (can't say the same for the troff
181 version). Also, it is really easy for someone to just go there and update the
182 man page, even without experience in the format (except for maybe the
183 requirement of tabs). So all in all, I really liked the format, and will use it
184 again if I need to write another man page in the future.</p></description><pubDate>Wed, 04 Dec 2024 00:00:00 +0000</pubDate><guid>https://kokada.dev/blog/praise-to-scdoc-to-generate-man-pages/</guid></item><item><title>Enabling LE Audio/LC3 in WF-1000XM5</title><link>https://kokada.dev/blog/enabling-le-audio-lc3-in-wf-1000xm5/</link><description><p>One of things that I hate the most about the fact that we are all using
185 wireless earbuds instead of wired earphones is the latency: it is bad, getting
186 up to seconds(!) depending on your particular combination of OS/earbuds/device.</p>
187 <p>There is a solution though: Bluetooth LE Audio, that is supposed to fix
188 multiple issues with the original design for Bluetooth Classic Audio, including
189 a much lower latency, improved efficiency (e.g.: less battery power) and even
190 multiple streams of audio. LE Audio also includes a new default codec for
191 improved audio quality, <a href="https://en.wikipedia.org/wiki/LC3_(codec)">LC3</a>, that
192 replaces the venerable <a href="https://en.wikipedia.org/wiki/SBC_(codec)">SBC</a> codec
193 for audio.</p>
194 <p>However, the standard is a mess right now: a few wireless headphones already
195 support it, but they're generally disabled by default and it is pretty messy to
196 enable. And even after enabling it, getting it to work can be a pain.</p>
197 <p>I have pretty much the best setup to use LE Audio right now: a recently
198 released Pixel 9 Pro with Sony's
199 <a href="https://www.sony.ie/headphones/products/wf-1000xm5">WF-1000XM5</a> earbuds, and
200 after lots of tries I got it to work. You can see below the versions of
201 everything I am using:</p>
202 <ul>
203 <li>Android: 14</li>
204 <li><a href="https://play.google.com/store/apps/details?id=com.sony.songpal.mdr">Sound
205 Connect</a>:
206 11.0.1</li>
207 <li>WM-1000XM5: 4.0.2</li>
208 </ul>
209 <p>The first thing you need to do is enable in &quot;Sound Connect&quot; app &quot;LE Audio
210 Priority&quot; in &quot;Device Settings -&gt; System&quot;:</p>
211 <p><a href="https://github.com/thiagokokada/blog/raw/main/posts/2024-10-07/photo_4909454744305642922_y.jpg"><img src="https://github.com/thiagokokada/blog/raw/main/posts/2024-10-07/photo_4909454744305642922_y.jpg" alt="LE Audio option inside Sound
212 Connect"></a></p>
213 <p>After this, you will need to pair your headset with the device again. You can
214 do this as same as always: press and hold the button in case for a few seconds
215 until a blue light starts to blink. However, this is where things starts to get
216 janky: I couldn't get the headset to pair with Android again.</p>
217 <p>A few of the things that I needed to do (in no specific order):</p>
218 <ul>
219 <li>Remove the previous paired headset</li>
220 <li>Restart the Android</li>
221 <li>Clean-up &quot;Sound Connect&quot; storage (Long press the app icon -&gt; &quot;App info&quot; -&gt;
222 &quot;Storage and Cache&quot; -&gt; &quot;Clear storage&quot;)</li>
223 </ul>
224 <p>If you can get the headset to connect, go to the &quot;Bluetooth&quot; settings in
225 Android, click in the gear icon for the headset and enable &quot;LE Audio&quot; option:</p>
226 <p><a href="https://github.com/thiagokokada/blog/raw/main/posts/2024-10-07/photo_4909454744305642937_y.jpg"><img src="https://github.com/thiagokokada/blog/raw/main/posts/2024-10-07/photo_4909454744305642937_y.jpg" alt="LE Audio option Bluetooth
227 Settings"></a></p>
228 <p>If you can't, you may want to <a href="https://helpguide.sony.net/mdr/2963/v1/en/contents/TP1000783925.html">restore the headset to factory
229 settings</a>
230 and try again from the start (that means pairing your device with &quot;Sound
231 Connect&quot; again, and you may want to try to clear the storage before doing so).</p>
232 <p>Yes, the process is extremely janky, but I think this is why both &quot;Sound
233 Connect&quot; and Android marks this feature as beta/experimental. And I still need
234 to test the latency, but from my initial testing there are some glitches when
235 the audio is only used for a short period of time (e.g.: Duolingo only enables
236 the audio when the character is speaking). So I only recommend this if you want
237 to test how LE Audio will behave, since it is clear that this needs more
238 polish.</p></description><pubDate>Mon, 07 Oct 2024 00:00:00 +0000</pubDate><guid>https://kokada.dev/blog/enabling-le-audio-lc3-in-wf-1000xm5/</guid></item><item><title>Quick bits: where does Nix store Flake's trusted settings?</title><link>https://kokada.dev/blog/quick-bits-where-does-nix-store-flakes-trusted-settings/</link><description><p>Have you ever run a <code>nix build</code> command and had this prompt?</p>
239 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> nix run .#darwinActivations/Sekai-MacBook-Pro
240 </span></span><span class="line"><span class="cl"><span class="go">do you want to allow configuration setting &#39;extra-substituters&#39; to be set to &#39;https://nix-community.cachix.org https://thiagokokada-nix-configs.cachix.org&#39; (y/N)? y
241 </span></span></span><span class="line"><span class="cl"><span class="go">do you want to permanently mark this value as trusted (y/N)? y
242 </span></span></span></code></pre><p>And realise that you did/didn't want to mark this value as trusted? But where
243 is this stored? Well, I had to look at the Nix source code to find the answer,
244 since I can't find this anywhere in the web or in ChatGPT (but I am sure that
245 now that I posted this it will eventually be searchable), and the answer can be
246 found
247 <a href="https://github.com/NixOS/nix/blob/c116030605bf7fecd232d0ff3b6fe066f23e4620/src/libflake/flake/config.cc#L13-L16">here</a>:</p>
248 <pre class="chroma"><code><span class="line"><span class="cl"><span class="n">Path</span> <span class="nf">trustedListPath</span><span class="p">()</span>
249 </span></span><span class="line"><span class="cl"><span class="p">{</span>
250 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="n">getDataDir</span><span class="p">()</span> <span class="o">+</span> <span class="s">&#34;/trusted-settings.json&#34;</span><span class="p">;</span>
251 </span></span><span class="line"><span class="cl"><span class="p">}</span>
252 </span></span></code></pre><p>Where is <code>getDataDir()</code> though? I found the answer
253 <a href="https://github.com/NixOS/nix/blob/c116030605bf7fecd232d0ff3b6fe066f23e4620/src/libutil/users.cc#L52-L65">here</a>:</p>
254 <pre class="chroma"><code><span class="line"><span class="cl"><span class="n">Path</span> <span class="nf">getDataDir</span><span class="p">()</span>
255 </span></span><span class="line"><span class="cl"><span class="p">{</span>
256 </span></span><span class="line"><span class="cl"> <span class="k">auto</span> <span class="n">dir</span> <span class="o">=</span> <span class="n">getEnv</span><span class="p">(</span><span class="s">&#34;NIX_DATA_HOME&#34;</span><span class="p">);</span>
257 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="p">(</span><span class="n">dir</span><span class="p">)</span> <span class="p">{</span>
258 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="o">*</span><span class="n">dir</span><span class="p">;</span>
259 </span></span><span class="line"><span class="cl"> <span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
260 </span></span><span class="line"><span class="cl"> <span class="k">auto</span> <span class="n">xdgDir</span> <span class="o">=</span> <span class="n">getEnv</span><span class="p">(</span><span class="s">&#34;XDG_DATA_HOME&#34;</span><span class="p">);</span>
261 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="p">(</span><span class="n">xdgDir</span><span class="p">)</span> <span class="p">{</span>
262 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="o">*</span><span class="n">xdgDir</span> <span class="o">+</span> <span class="s">&#34;/nix&#34;</span><span class="p">;</span>
263 </span></span><span class="line"><span class="cl"> <span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
264 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="n">getHome</span><span class="p">()</span> <span class="o">+</span> <span class="s">&#34;/.local/share/nix&#34;</span><span class="p">;</span>
265 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
266 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
267 </span></span><span class="line"><span class="cl"><span class="p">}</span>
268 </span></span></code></pre><p>So we solved the mystery:</p>
269 <ul>
270 <li>If <code>NIX_DATA_HOME</code> is set, the file will be in
271 <code>$NIX_DATA_HOME/trusted-settings.json</code></li>
272 <li>If <code>XDG_DATA_HOME</code> is set, the file will be in
273 <code>$XDG_DATA_HOME/nix/trusted-settings.json</code></li>
274 <li>Otherwise Nix will fallback to <code>$HOME/.local/share/nix/trusted-settings.json</code></li>
275 </ul>
276 <p>By the way, if you don't know why you got this prompt, if <code>flake.nix</code> has a
277 <code>nixConfig</code> attribute inside <code>outputs</code> <strong>and</strong> this <code>nixConfig</code> is an unsafe
278 attribute (like <code>extra-substituters</code>) you will get this prompt, unless you set
279 <a href="https://nix.dev/manual/nix/2.23/command-ref/conf-file#conf-accept-flake-config"><code>accept-flake-config = true</code></a>
280 in your Nix configuration (please <strong>do not do this</strong>, it is dangerous because
281 it may allow running possible unsafe options without asking you first).</p>
282 <p>You can inspect the JSON file or delete it and Nix will prompt you again the
283 next time you run a <code>nix</code> command. And yes, saving this preference is
284 controversial considering this is Nix, but having the power to set <code>nixConfig</code>
285 attributes is really powerful, and with great powers comes great
286 responsibilities.</p></description><pubDate>Mon, 30 Sep 2024 00:00:00 +0000</pubDate><guid>https://kokada.dev/blog/quick-bits-where-does-nix-store-flakes-trusted-settings/</guid></item><item><title>My coding experience with LLM</title><link>https://kokada.dev/blog/my-coding-experience-with-llm/</link><description><p>I don't want to talk about AI. AI fatigue is real, with (garbage) AI content
287 everywhere you go. However, I like to talk about tools, and AIs, or more
288 specifically, <a href="https://en.wikipedia.org/wiki/Large_language_model">Large Language
289 Models</a> (LLM), can be
290 really useful tools. Maybe someday we will have our AI overlords dominating
291 everything (including my job), but right now I don't feel particularly
292 threatened given my experience with AIs.</p>
293 <p>Given that <a href="https://mashable.com/article/google-search-low-quality-research">Google Search is getting
294 worse</a>, I am
295 slowly relying in LLMs more and more. This is the focus of this post, basically
296 <a href="https://www.rottentomatoes.com/m/the_good_the_bad_and_the_ugly">the good, the bad and the
297 ugly</a> (by the
298 way, never saw this film) of LLMs, especially focused for solving coding
299 problems. I will take a look at the my chat history in
300 <a href="https://chatgpt.com/">ChatGPT</a> and see where it was a good tool, where it
301 shined, and where it just completely wasted my time.</p>
302 <p>A disclaimer, I never bothered to buy a ChatGPT subscription because I don't
303 feel that the tool is useful enough (at least yet), so most of the examples
304 here are either with <a href="https://openai.com/index/chatgpt/">GPT-3.5</a> or <a href="https://openai.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/">GPT-4o
305 mini</a>,
306 since those where the models offered by free in ChatGPT.</p>
307 <p>An addendum, I have some experience with <a href="https://github.com/features/copilot">GitHub
308 Copilot</a> (another LLM, but this one runs
309 inside your IDE or favorite text editor) in <code>$PREVIOUS_JOB</code>. But my experience
310 with it is even more underwhelming than what I am going to describe in this
311 post: it was basically really good for generating fixture for tests and between
312 ok-ish to plain bad for everything else. Yes, that's would fit in a tweet (or
313 xeet?).</p>
314 <h2>The good</h2>
315 <p><a href="https://chatgpt.com/share/c58a1432-2761-4954-acac-cade29942c34">Chat history</a>.</p>
316 <blockquote>
317 <p>Is there any way that I can pass a context for a net.conn to cancel its
318 Read()?</p>
319 </blockquote>
320 <p>This is one of the good cases where ChatGPT felt to me that I was talking with
321 an expert in Go. To start, I didn't even mention in my initial prompt that I
322 was talking about Go, but it correctly inferred so. Also, the solution overview
323 was great:</p>
324 <blockquote>
325 <ol>
326 <li><strong>Use a separate goroutine</strong> to handle the context cancellation.</li>
327 <li><strong>Set a deadline</strong> on the connection using <code>SetReadDeadline()</code> to unblock
328 the <code>Read()</code> when the context is canceled.</li>
329 </ol>
330 </blockquote>
331 <p>Before asking ChatGPT about this issue I had already coded my own naive
332 solution that was something similar to this:</p>
333 <pre class="chroma"><code><span class="line"><span class="cl"><span class="kd">func</span> <span class="nf">foo</span><span class="p">(</span><span class="nx">ctx</span> <span class="nx">context</span><span class="p">.</span><span class="nx">Context</span><span class="p">)</span> <span class="p">{</span>
334 </span></span><span class="line"><span class="cl"> <span class="c1">// ...
335 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="nx">done</span> <span class="o">:=</span> <span class="nb">make</span><span class="p">(</span><span class="kd">chan</span> <span class="kd">struct</span><span class="p">{})</span>
336 </span></span><span class="line"><span class="cl"> <span class="kd">var</span> <span class="nx">n</span> <span class="kt">int</span>
337 </span></span><span class="line"><span class="cl"> <span class="kd">var</span> <span class="nx">err</span> <span class="kt">error</span>
338 </span></span><span class="line"><span class="cl">
339 </span></span><span class="line"><span class="cl"> <span class="k">go</span> <span class="kd">func</span><span class="p">()</span> <span class="p">{</span>
340 </span></span><span class="line"><span class="cl"> <span class="nx">n</span><span class="p">,</span> <span class="nx">err</span> <span class="p">=</span> <span class="nx">conn</span><span class="p">.</span><span class="nf">Read</span><span class="p">(</span><span class="nx">buf</span><span class="p">)</span>
341 </span></span><span class="line"><span class="cl"> <span class="nx">done</span> <span class="o">&lt;-</span><span class="kd">struct</span><span class="p">{}{}</span>
342 </span></span><span class="line"><span class="cl"> <span class="p">}()</span>
343 </span></span><span class="line"><span class="cl">
344 </span></span><span class="line"><span class="cl"> <span class="k">select</span> <span class="p">{</span>
345 </span></span><span class="line"><span class="cl"> <span class="k">case</span> <span class="o">&lt;-</span><span class="nx">ctx</span><span class="p">.</span><span class="nf">Done</span><span class="p">():</span>
346 </span></span><span class="line"><span class="cl"> <span class="nx">conn</span><span class="p">.</span><span class="nf">Close</span><span class="p">()</span>
347 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">ctx</span><span class="p">.</span><span class="nf">Err</span><span class="p">()</span>
348 </span></span><span class="line"><span class="cl"> <span class="k">case</span> <span class="o">&lt;-</span><span class="nx">done</span><span class="p">:</span>
349 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="nx">n</span><span class="p">,</span> <span class="nx">err</span>
350 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
351 </span></span><span class="line"><span class="cl"> <span class="c1">// ...
352 </span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="p">}</span>
353 </span></span></code></pre><p>Compared this solution with the one from ChatGPT:</p>
354 <pre class="chroma"><code><span class="line"><span class="cl"><span class="kd">func</span> <span class="nf">readWithContext</span><span class="p">(</span><span class="nx">ctx</span> <span class="nx">context</span><span class="p">.</span><span class="nx">Context</span><span class="p">,</span> <span class="nx">conn</span> <span class="nx">net</span><span class="p">.</span><span class="nx">Conn</span><span class="p">,</span> <span class="nx">buf</span> <span class="p">[]</span><span class="kt">byte</span><span class="p">)</span> <span class="p">(</span><span class="kt">int</span><span class="p">,</span> <span class="kt">error</span><span class="p">)</span> <span class="p">{</span>
355 </span></span><span class="line"><span class="cl"> <span class="nx">done</span> <span class="o">:=</span> <span class="nb">make</span><span class="p">(</span><span class="kd">chan</span> <span class="kd">struct</span><span class="p">{})</span>
356 </span></span><span class="line"><span class="cl"> <span class="kd">var</span> <span class="nx">n</span> <span class="kt">int</span>
357 </span></span><span class="line"><span class="cl"> <span class="kd">var</span> <span class="nx">err</span> <span class="kt">error</span>
358 </span></span><span class="line"><span class="cl">
359 </span></span><span class="line"><span class="cl"> <span class="c1">// Start a goroutine to perform the read
360 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="k">go</span> <span class="kd">func</span><span class="p">()</span> <span class="p">{</span>
361 </span></span><span class="line"><span class="cl"> <span class="nx">n</span><span class="p">,</span> <span class="nx">err</span> <span class="p">=</span> <span class="nx">conn</span><span class="p">.</span><span class="nf">Read</span><span class="p">(</span><span class="nx">buf</span><span class="p">)</span>
362 </span></span><span class="line"><span class="cl"> <span class="nb">close</span><span class="p">(</span><span class="nx">done</span><span class="p">)</span>
363 </span></span><span class="line"><span class="cl"> <span class="p">}()</span>
364 </span></span><span class="line"><span class="cl">
365 </span></span><span class="line"><span class="cl"> <span class="k">select</span> <span class="p">{</span>
366 </span></span><span class="line"><span class="cl"> <span class="k">case</span> <span class="o">&lt;-</span><span class="nx">ctx</span><span class="p">.</span><span class="nf">Done</span><span class="p">():</span> <span class="c1">// Context was canceled or timed out
367 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="c1">// Set a short deadline to unblock the Read()
368 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="nx">conn</span><span class="p">.</span><span class="nf">SetReadDeadline</span><span class="p">(</span><span class="nx">time</span><span class="p">.</span><span class="nf">Now</span><span class="p">())</span>
369 </span></span><span class="line"><span class="cl"> <span class="o">&lt;-</span><span class="nx">done</span> <span class="c1">// Wait for the read to finish
370 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="k">return</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">ctx</span><span class="p">.</span><span class="nf">Err</span><span class="p">()</span>
371 </span></span><span class="line"><span class="cl"> <span class="k">case</span> <span class="o">&lt;-</span><span class="nx">done</span><span class="p">:</span> <span class="c1">// Read finished successfully
372 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="k">return</span> <span class="nx">n</span><span class="p">,</span> <span class="nx">err</span>
373 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
374 </span></span><span class="line"><span class="cl"><span class="p">}</span>
375 </span></span></code></pre><p>The ChatGPT version is much better because:</p>
376 <ol>
377 <li>It doesn't close the connection socket allowing it to be reused</li>
378 <li>It waits until the goroutine is finished to avoid leaks</li>
379 <li>It closes the channel after it finishes</li>
380 </ol>
381 <p>Now, I don't claim to be a Go expert (far from it), but this was one of those
382 cases where I felt that I was being guided by someone that is coding in Go for
383 years.</p>
384 <p>But it didn't end there, I asked:</p>
385 <blockquote>
386 <p>If I want to reuse the connection, do I need to reset the SetReadDeadline?</p>
387 </blockquote>
388 <p>Since in this particular case I wanted the underlying connection to be reused,
389 and ChatGPT correctly amended the code:</p>
390 <pre class="chroma"><code><span class="line"><span class="cl"><span class="kd">func</span> <span class="nf">readWithContext</span><span class="p">(</span><span class="nx">ctx</span> <span class="nx">context</span><span class="p">.</span><span class="nx">Context</span><span class="p">,</span> <span class="nx">conn</span> <span class="nx">net</span><span class="p">.</span><span class="nx">Conn</span><span class="p">,</span> <span class="nx">buf</span> <span class="p">[]</span><span class="kt">byte</span><span class="p">)</span> <span class="p">(</span><span class="kt">int</span><span class="p">,</span> <span class="kt">error</span><span class="p">)</span> <span class="p">{</span>
391 </span></span><span class="line"><span class="cl"> <span class="c1">// Same as before
392 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="k">select</span> <span class="p">{</span>
393 </span></span><span class="line"><span class="cl"> <span class="k">case</span> <span class="o">&lt;-</span><span class="nx">ctx</span><span class="p">.</span><span class="nf">Done</span><span class="p">():</span> <span class="c1">// Context was canceled or timed out
394 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="c1">// Set a short deadline to unblock the Read()
395 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="nx">conn</span><span class="p">.</span><span class="nf">SetReadDeadline</span><span class="p">(</span><span class="nx">time</span><span class="p">.</span><span class="nf">Now</span><span class="p">())</span>
396 </span></span><span class="line"><span class="cl"> <span class="o">&lt;-</span><span class="nx">done</span> <span class="c1">// Wait for the read to finish
397 </span></span></span><span class="line"><span class="cl"><span class="c1"></span>
398 </span></span><span class="line"><span class="cl"> <span class="c1">// Reset the deadline so the connection can be reused
399 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="nx">conn</span><span class="p">.</span><span class="nf">SetReadDeadline</span><span class="p">(</span><span class="nx">time</span><span class="p">.</span><span class="nx">Time</span><span class="p">{})</span>
400 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">ctx</span><span class="p">.</span><span class="nf">Err</span><span class="p">()</span>
401 </span></span><span class="line"><span class="cl"> <span class="k">case</span> <span class="o">&lt;-</span><span class="nx">done</span><span class="p">:</span> <span class="c1">// Read finished successfully
402 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="k">return</span> <span class="nx">n</span><span class="p">,</span> <span class="nx">err</span>
403 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
404 </span></span><span class="line"><span class="cl"><span class="p">}</span>
405 </span></span></code></pre><p>Also the explanation on why this happens was good:</p>
406 <blockquote>
407 <p>When you set a read deadline with <code>conn.SetReadDeadline(time.Now())</code> to unblock
408 the <code>Read()</code> call, this deadline remains in effect for future <code>Read()</code>
409 operations. This means that if you attempt another read operation after this,
410 it could immediately fail with a timeout error if the deadline is still in
411 effect.
412 To reuse the connection, you'll need to remove the deadline or set a new one.
413 You can do this by calling <code>conn.SetReadDeadline(time.Time{})</code>, which clears
414 any existing deadline.</p>
415 </blockquote>
416 <p>I had one last question that I kind already knew the answer:</p>
417 <blockquote>
418 <p>Do we need to defer the clearing?</p>
419 </blockquote>
420 <p>And ChatGPT correctly returned:</p>
421 <blockquote>
422 <p>Whether you need to defer the clearing of the read deadline depends on how
423 your code is structured. If you want to ensure that the read deadline is
424 always cleared after a context cancellation, even if the code panics or exits
425 early, then deferring the <code>conn.SetReadDeadline(time.Time{})</code> call is a good
426 idea.</p>
427 </blockquote>
428 <p>This whole interaction was one of the best ones I ever had in ChatGPT, one that
429 it gave me an actual functional code with good explanation on what was
430 happening. But still, it didn't talked at all about the gotcha of setting the
431 read deadline and reusing the underlying connection, needing another prompt to
432 give me this information.</p>
433 <p>Also, there is one issue in ChatGPT solution though: we are discarding the
434 values of the buffer once the context is done by returning <code>n = 0</code> instead of
435 the number of bytes read. In this particular case this is what I wanted, but
436 this is a possible issue that ChatGPT didn't talk.</p>
437 <p>For comparison,
438 <a href="https://stackoverflow.com/questions/61637359/how-to-use-context-context-with-tcp-connection-read">this</a>
439 is the best that I got searching in both DuckDuckGo and Google, that is where I
440 kind based my first solution on. I still not sure where ChatGPT could give me
441 such a good answer for this question, because I just couldn't find anything
442 remotely similar during my searches.</p>
443 <h2>The amazing</h2>
444 <p><a href="https://chatgpt.com/share/0c9a148f-4cfd-4b4e-aa84-7486e185125b">Chat history</a>.</p>
445 <blockquote>
446 <p>I want to write an extension to <a href="https://github.com/yuin/goldmark/">https://github.com/yuin/goldmark/</a> that allows
447 me to rewrite <code>[link](../2024-07-30/post.md)</code> to
448 <code>[link](https://kokada.capivaras.dev/slug)</code></p>
449 </blockquote>
450 <p>Here is the generated code:</p>
451 <pre class="chroma"><code><span class="line"><span class="cl"><span class="kn">package</span> <span class="nx">linkrewriter</span>
452 </span></span><span class="line"><span class="cl">
453 </span></span><span class="line"><span class="cl"><span class="kn">import</span> <span class="p">(</span>
454 </span></span><span class="line"><span class="cl"> <span class="s">&#34;fmt&#34;</span>
455 </span></span><span class="line"><span class="cl"> <span class="s">&#34;path&#34;</span>
456 </span></span><span class="line"><span class="cl"> <span class="s">&#34;strings&#34;</span>
457 </span></span><span class="line"><span class="cl">
458 </span></span><span class="line"><span class="cl"> <span class="s">&#34;github.com/yuin/goldmark&#34;</span>
459 </span></span><span class="line"><span class="cl"> <span class="s">&#34;github.com/yuin/goldmark/ast&#34;</span>
460 </span></span><span class="line"><span class="cl"> <span class="s">&#34;github.com/yuin/goldmark/parser&#34;</span>
461 </span></span><span class="line"><span class="cl"> <span class="s">&#34;github.com/yuin/goldmark/renderer&#34;</span>
462 </span></span><span class="line"><span class="cl"> <span class="s">&#34;github.com/yuin/goldmark/util&#34;</span>
463 </span></span><span class="line"><span class="cl"><span class="p">)</span>
464 </span></span><span class="line"><span class="cl">
465 </span></span><span class="line"><span class="cl"><span class="c1">// LinkRewriter is the main struct for your extension
466 </span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="kd">type</span> <span class="nx">LinkRewriter</span> <span class="kd">struct</span><span class="p">{}</span>
467 </span></span><span class="line"><span class="cl">
468 </span></span><span class="line"><span class="cl"><span class="c1">// NewLinkRewriter returns a new instance of LinkRewriter
469 </span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="kd">func</span> <span class="nf">NewLinkRewriter</span><span class="p">()</span> <span class="o">*</span><span class="nx">LinkRewriter</span> <span class="p">{</span>
470 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="o">&amp;</span><span class="nx">LinkRewriter</span><span class="p">{}</span>
471 </span></span><span class="line"><span class="cl"><span class="p">}</span>
472 </span></span><span class="line"><span class="cl">
473 </span></span><span class="line"><span class="cl"><span class="c1">// Extend will be called by Goldmark to add your extension
474 </span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="kd">func</span> <span class="p">(</span><span class="nx">e</span> <span class="o">*</span><span class="nx">LinkRewriter</span><span class="p">)</span> <span class="nf">Extend</span><span class="p">(</span><span class="nx">m</span> <span class="nx">goldmark</span><span class="p">.</span><span class="nx">Markdown</span><span class="p">)</span> <span class="p">{</span>
475 </span></span><span class="line"><span class="cl"> <span class="nx">m</span><span class="p">.</span><span class="nf">Parser</span><span class="p">().</span><span class="nf">AddOptions</span><span class="p">(</span><span class="nx">parser</span><span class="p">.</span><span class="nf">WithASTTransformers</span><span class="p">(</span><span class="nx">util</span><span class="p">.</span><span class="nf">Prioritized</span><span class="p">(</span><span class="nx">e</span><span class="p">,</span> <span class="mi">0</span><span class="p">)))</span>
476 </span></span><span class="line"><span class="cl"><span class="p">}</span>
477 </span></span><span class="line"><span class="cl">
478 </span></span><span class="line"><span class="cl"><span class="c1">// Transform is the method that modifies the AST
479 </span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="kd">func</span> <span class="p">(</span><span class="nx">e</span> <span class="o">*</span><span class="nx">LinkRewriter</span><span class="p">)</span> <span class="nf">Transform</span><span class="p">(</span><span class="nx">node</span> <span class="o">*</span><span class="nx">ast</span><span class="p">.</span><span class="nx">Document</span><span class="p">,</span> <span class="nx">reader</span> <span class="nx">text</span><span class="p">.</span><span class="nx">Reader</span><span class="p">,</span> <span class="nx">pc</span> <span class="nx">parser</span><span class="p">.</span><span class="nx">Context</span><span class="p">)</span> <span class="p">{</span>
480 </span></span><span class="line"><span class="cl"> <span class="nx">ast</span><span class="p">.</span><span class="nf">Walk</span><span class="p">(</span><span class="nx">node</span><span class="p">,</span> <span class="kd">func</span><span class="p">(</span><span class="nx">n</span> <span class="nx">ast</span><span class="p">.</span><span class="nx">Node</span><span class="p">,</span> <span class="nx">entering</span> <span class="kt">bool</span><span class="p">)</span> <span class="p">(</span><span class="nx">ast</span><span class="p">.</span><span class="nx">WalkStatus</span><span class="p">,</span> <span class="kt">error</span><span class="p">)</span> <span class="p">{</span>
481 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="p">!</span><span class="nx">entering</span> <span class="p">{</span>
482 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="nx">ast</span><span class="p">.</span><span class="nx">WalkContinue</span><span class="p">,</span> <span class="kc">nil</span>
483 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
484 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="nx">link</span><span class="p">,</span> <span class="nx">ok</span> <span class="o">:=</span> <span class="nx">n</span><span class="p">.(</span><span class="o">*</span><span class="nx">ast</span><span class="p">.</span><span class="nx">Link</span><span class="p">);</span> <span class="nx">ok</span> <span class="p">{</span>
485 </span></span><span class="line"><span class="cl"> <span class="nf">rewriteLink</span><span class="p">(</span><span class="nx">link</span><span class="p">)</span>
486 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
487 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="nx">ast</span><span class="p">.</span><span class="nx">WalkContinue</span><span class="p">,</span> <span class="kc">nil</span>
488 </span></span><span class="line"><span class="cl"> <span class="p">})</span>
489 </span></span><span class="line"><span class="cl"><span class="p">}</span>
490 </span></span><span class="line"><span class="cl">
491 </span></span><span class="line"><span class="cl"><span class="c1">// rewriteLink modifies the link URL
492 </span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="kd">func</span> <span class="nf">rewriteLink</span><span class="p">(</span><span class="nx">link</span> <span class="o">*</span><span class="nx">ast</span><span class="p">.</span><span class="nx">Link</span><span class="p">)</span> <span class="p">{</span>
493 </span></span><span class="line"><span class="cl"> <span class="nx">url</span> <span class="o">:=</span> <span class="nb">string</span><span class="p">(</span><span class="nx">link</span><span class="p">.</span><span class="nx">Destination</span><span class="p">)</span>
494 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="nx">strings</span><span class="p">.</span><span class="nf">HasPrefix</span><span class="p">(</span><span class="nx">url</span><span class="p">,</span> <span class="s">&#34;../&#34;</span><span class="p">)</span> <span class="p">{</span>
495 </span></span><span class="line"><span class="cl"> <span class="c1">// Modify this part according to your slug generation logic
496 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="nx">slug</span> <span class="o">:=</span> <span class="nf">generateSlug</span><span class="p">(</span><span class="nx">url</span><span class="p">)</span>
497 </span></span><span class="line"><span class="cl"> <span class="nx">link</span><span class="p">.</span><span class="nx">Destination</span> <span class="p">=</span> <span class="p">[]</span><span class="nb">byte</span><span class="p">(</span><span class="nx">fmt</span><span class="p">.</span><span class="nf">Sprintf</span><span class="p">(</span><span class="s">&#34;https://kokada.capivaras.dev/%s&#34;</span><span class="p">,</span> <span class="nx">slug</span><span class="p">))</span>
498 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
499 </span></span><span class="line"><span class="cl"><span class="p">}</span>
500 </span></span><span class="line"><span class="cl">
501 </span></span><span class="line"><span class="cl"><span class="c1">// generateSlug generates a slug from the given URL
502 </span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="kd">func</span> <span class="nf">generateSlug</span><span class="p">(</span><span class="nx">url</span> <span class="kt">string</span><span class="p">)</span> <span class="kt">string</span> <span class="p">{</span>
503 </span></span><span class="line"><span class="cl"> <span class="c1">// Assuming the slug is the last part of the URL without the extension
504 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="k">return</span> <span class="nx">strings</span><span class="p">.</span><span class="nf">TrimSuffix</span><span class="p">(</span><span class="nx">path</span><span class="p">.</span><span class="nf">Base</span><span class="p">(</span><span class="nx">url</span><span class="p">),</span> <span class="nx">path</span><span class="p">.</span><span class="nf">Ext</span><span class="p">(</span><span class="nx">url</span><span class="p">))</span>
505 </span></span><span class="line"><span class="cl"><span class="p">}</span>
506 </span></span></code></pre><pre class="chroma"><code><span class="line"><span class="cl"><span class="kn">package</span> <span class="nx">main</span>
507 </span></span><span class="line"><span class="cl">
508 </span></span><span class="line"><span class="cl"><span class="kn">import</span> <span class="p">(</span>
509 </span></span><span class="line"><span class="cl"> <span class="s">&#34;github.com/yuin/goldmark&#34;</span>
510 </span></span><span class="line"><span class="cl"> <span class="s">&#34;github.com/yuin/goldmark/renderer/html&#34;</span>
511 </span></span><span class="line"><span class="cl"> <span class="nx">linkrewriter</span> <span class="s">&#34;path/to/your/linkrewriter&#34;</span>
512 </span></span><span class="line"><span class="cl"><span class="p">)</span>
513 </span></span><span class="line"><span class="cl">
514 </span></span><span class="line"><span class="cl"><span class="kd">func</span> <span class="nf">main</span><span class="p">()</span> <span class="p">{</span>
515 </span></span><span class="line"><span class="cl"> <span class="nx">md</span> <span class="o">:=</span> <span class="nx">goldmark</span><span class="p">.</span><span class="nf">New</span><span class="p">(</span>
516 </span></span><span class="line"><span class="cl"> <span class="nx">goldmark</span><span class="p">.</span><span class="nf">WithExtensions</span><span class="p">(</span>
517 </span></span><span class="line"><span class="cl"> <span class="nx">linkrewriter</span><span class="p">.</span><span class="nf">NewLinkRewriter</span><span class="p">(),</span>
518 </span></span><span class="line"><span class="cl"> <span class="p">),</span>
519 </span></span><span class="line"><span class="cl"> <span class="nx">goldmark</span><span class="p">.</span><span class="nf">WithRendererOptions</span><span class="p">(</span>
520 </span></span><span class="line"><span class="cl"> <span class="nx">html</span><span class="p">.</span><span class="nf">WithHardWraps</span><span class="p">(),</span>
521 </span></span><span class="line"><span class="cl"> <span class="p">),</span>
522 </span></span><span class="line"><span class="cl"> <span class="p">)</span>
523 </span></span><span class="line"><span class="cl">
524 </span></span><span class="line"><span class="cl"> <span class="c1">// Use `md.Convert` to convert your markdown to HTML as usual
525 </span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="p">}</span>
526 </span></span></code></pre><p>This one will need some context. <a href="https://github.com/yuin/goldmark/">Goldmark</a>,
527 the Markdown renderer that <a href="https://kokada.dev/blog/making-a-blog-for-the-next-10-years">powers this
528 blog</a>, does not
529 have good examples or documentation (at least I didn't found neither).</p>
530 <p>So I was not really hopeful I would get any answers in ChatGPT, but not only it
531 put me in the correct path, the code was almost correct. It did had some unused
532 and missing imports, and there were some compilation errors that were easily
533 fixed. And when it registered the extension, it also added a
534 <code>html.WithHardWraps()</code> renderer for no reason at all.</p>
535 <p>But this is one of the experiences in LLM that most impressed me. I had no idea
536 at all on how to solve this problem, and ChatGPT basically gave me the
537 solution. A <a href="https://github.com/thiagokokada/blog/blob/3a2ea58a0638d1005617d25eabeae0626933db46/link_rewriter.go">modified version of the
538 code</a>
539 from this chat is still used to power this blog to this day, that also
540 demonstrates that the code is not garbage (since now I understand what it does
541 and I can even extend it).</p>
542 <p>I am still not sure how the license situation of this code is though. Does
543 licenses even make sense in AI generated code?</p>
544 <h2>The bad</h2>
545 <p><a href="https://chatgpt.com/share/95fcd777-b03c-4cfe-8d1d-12e53f5a4dfe">Chat history</a>.</p>
546 <blockquote>
547 <p>I want to configure VM options inside IntelliJ by project</p>
548 </blockquote>
549 <blockquote>
550 <p>My project uses sbt, and I want to set this in a way that I can share with my
551 colleagues</p>
552 </blockquote>
553 <blockquote>
554 <p>I am using a build.sbt file, but IntelliJ is not loading it's JVM options and
555 I need to set this up manually. I want to automate this</p>
556 </blockquote>
557 <p>This is an example of my most common interaction with ChatGPT: it generally
558 gives some options that could work, but not in my specific case. So I start to
559 ask for more information until it eventually starts to
560 <a href="https://en.wikipedia.org/wiki/Hallucination_(artificial_intelligence)">hallucinate</a>.</p>
561 <p>In this particular case, the hallucination was in:</p>
562 <blockquote>
563 <p>The sbt-idea-plugin can generate IntelliJ IDEA project files based on your
564 SBT configuration, including JVM options. This allows you to ensure that
565 IntelliJ loads the correct settings from the SBT configuration.</p>
566 <p>Steps:</p>
567 <ol>
568 <li>Add the <code>sbt-idea-plugin</code> to your <code>project/plugins.sbt</code> file:</li>
569 </ol>
570 <pre class="chroma"><code><span class="line"><span class="cl"><span class="n">addSbtPlugin</span><span class="o">(</span><span class="s">&#34;com.github.mwz&#34;</span> <span class="o">%</span> <span class="s">&#34;sbt-idea&#34;</span> <span class="o">%</span> <span class="s">&#34;2.4.0&#34;</span><span class="o">)</span>
571 </span></span></code></pre><ol start="2">
572 <li>Run the following command in your terminal to regenerate IntelliJ IDEA
573 project files:</li>
574 </ol>
575 <pre class="chroma"><code><span class="line"><span class="cl"><span class="go">sbt gen-idea
576 </span></span></span></code></pre></blockquote>
577 <p>Yes, there is a
578 <a href="https://github.com/JetBrains/sbt-idea-plugin"><code>sbt-idea-plugin</code></a>, but no, it
579 doesn't generate project files based in my SBT configuration. It is not even
580 from the repository that it is supposed to be pointing too.</p>
581 <p>I will just not say that this interaction was a waste of time because some of
582 the other ideas can be applied, but this was the kind of case where I ended up
583 just being disappointed because the thing that seemed to be a possible solution
584 for my problem ended up not existing at all.</p>
585 <h2>The ugly</h2>
586 <p>There are lots of examples where ChatGPT just plain disappointed me. Here is a
587 few:</p>
588 <blockquote>
589 <p>What is the language that is a fork of Go that has extra features like map?</p>
590 </blockquote>
591 <p><a href="https://chatgpt.com/share/2b854bbd-d376-4049-97fe-b47321c1ef07">Chat history</a>.</p>
592 <p>This is one of those cases where I saw something once and was trying to
593 remember. ChatGPT just gave me lots of answers that were not a fork of Go
594 (<a href="https://vlang.io/">V</a>, <a href="https://github.com/gluon-lang/gluon">Gluon</a>,
595 <a href="https://www.ponylang.io">Pony</a>), hallucinated a programming language (Gomini),
596 and said that <a href="https://ziglang.org/">Zig</a> is focused in STEM.</p>
597 <p>By the way, I eventually remember what the language was:
598 <a href="https://goplus.org/">Go+</a>. To be fair with ChatGPT, I misremembered that the
599 Go+ had map (as the <a href="https://en.wikipedia.org/wiki/Map_(higher-order_function)">high-order
600 function</a>, not a
601 hashmap that both Go and Go+ have). But still, Go+ explicit announces that it
602 is a language for STEM everywhere in their website, and even them ChatGPT
603 couldn't answer.</p>
604 <blockquote>
605 <p>In IntelliJ, I need to pass some VM options to the run configuration for my
606 application to run tests, e.g.:</p>
607 <pre><code>--add-exports=java.base/sun.security.x509=ALL-UNNAMED
608 --add-opens=java.base/sun.security.ssl=ALL-UNNAMED
609 -Dconfig.resource=api.test.conf
610 </code></pre>
611 <p>However, the <code>api.test.conf</code> may be <code>jobs.api.conf</code> or something else
612 depending on the subproject I am running inside Idea. Can I parametrize this
613 in some way?</p>
614 </blockquote>
615 <p><a href="https://chatgpt.com/share/aa9bd051-65de-49d9-8d56-0e28de94722c">Chat history</a></p>
616 <p>So quick answer: you can't. IntelliJ macros are not powerful enough for this.
617 ChatGPT itself said so when I asked about the IntelliJ macros
618 <a href="https://chatgpt.com/share/7a057bf5-bcec-458b-9432-98e3f551ad28">directly</a>.</p>
619 <p>But in this particular case, ChatGPT happily said &quot;yes, you can parameterize VM
620 options in IntelliJ IDEA to make them more flexible depending on the subproject
621 or other context&quot;. Thanks for wasting my time, I guess.</p>
622 <h2>Conclusion</h2>
623 <p>I do think that LLMs, in particular ChatGPT since this is the one I have more
624 experience with, are improving. But still, if there is anything that I
625 concluded after putting all those experiences in a post is that you still need
626 someone that knows at least as much about whatever you're asking ChatGPT to
627 actually validate the answer.</p>
628 <p>Not only because LLMs hallucinates a lot, but also because even in the &quot;good&quot;
629 example the LLM needed extra input to figure out about the possible drawback of
630 the solution. The only case where I felt that ChatGPT outsmarted me was the
631 &quot;amazing&quot; case, but this was literally the only example that I could find in
632 all my chat history with ChatGPT that was like this.</p>
633 <p>So yes, at least for now I still don't think AI will be our overlords. Maybe
634 someday.</p></description><pubDate>Mon, 02 Sep 2024 00:00:00 +0000</pubDate><guid>https://kokada.dev/blog/my-coding-experience-with-llm/</guid></item><item><title>Making a blog for the next 10 years</title><link>https://kokada.dev/blog/making-a-blog-for-the-next-10-years/</link><description><p>So one thing that I realise after starting writing this blog is that I care
635 more about it than some of my other projects. For some reason or another I want
636 to make sure that this blog will continue with me for a long time. This is one
637 of the reasons why I use <a href="https://kokada.dev/blog/using-github-as-a-bad-blog-platform">GitHub as mirror
638 blog</a> and why I
639 created a <a href="https://kokada.dev/blog/quick-bits-why-you-should-automate-everything">bunch of
640 automation</a>
641 to make sure I never forget how to maintain this blog.</p>
642 <p>Still, there are a bunch of dependencies that I need to make sure they're
643 working so I can publish blog posts:</p>
644 <ul>
645 <li>Markdown</li>
646 <li>A text editor</li>
647 <li>Hosting and blog platform</li>
648 <li>Shell script and
649 <a href="https://kokada.dev/blog/abusing-makefiles-for-fun-and-profit">Makefile</a></li>
650 <li>Go toolchain and modules</li>
651 </ul>
652 <p>Let's start with the most important one: the texts themselves, they're all
653 written in <a href="https://www.markdownguide.org/">Markdown</a>. The nice thing about
654 Markdown is that it is just plain text files with some special notation that
655 allow you to format text, but the text itself is still legible even if you have
656 no support to render Markdowns. So it means as long as I can read a plain text
657 file, I can still read the posts, no issue there. Closely related issue here is
658 the text editor, but the nice thing about Markdown is that if tomorrow I decide
659 to change from <code>neovim</code> to, say, <a href="https://zed.dev/">Zed</a>, I can still write it
660 Markdown files without issue. I also use a standardised Markdown implementation
661 called <a href="https://commonmark.org/">CommonMark</a>, that fixes a bunch of ambiguities
662 of the original Markdown spec.</p>
663 <p>The next point is the host (<a href="capivaras.dev">capivaras.dev</a>) and the blog
664 platform (<a href="https://github.com/mataroa-blog/mataroa">Mataroa</a>). One of the nice
665 things about Mataroa is that it avoids platform lock-in by having multiple ways
666 to <a href="https://mataroa.blog/export/">export your posts</a>. I could export everything
667 to <a href="https://gohugo.io/">Hugo</a>, for example, and serve the blog from <a href="https://pages.github.com/">GitHub
668 Pages</a>.</p>
669 <p>This is especially nice consider how small <a href="capivaras.dev">capivaras.dev</a> is,
670 so it is good to know that if I eventually have issues I could move to
671 somewhere else. I also have an e-mail backup every month with all posts made
672 (another <a href="https://hey.mataroa.blog/blog/monthly-auto-exports-via-email/">Mataroa
673 feature</a>), and
674 of course I have a <a href="https://github.com/thiagokokada/blog">Git repo</a> that also
675 acts as a <a href="https://kokada.dev/blog/using-github-as-a-bad-blog-platform">mirror of this
676 blog</a>. So I would
677 say the chance of losing access to the content is slim.</p>
678 <p>One other issue is the URL of the posts that are indexed in many different
679 places, like <a href="https://news.ycombinator.com/">Hacker News</a>,
680 <a href="https://lobste.rs/">Lobte.rs</a>, etc. This is why I finally decide to bite the
681 bullet and purchase a proper domain, and this website should now be available
682 in <a href="kokada.dev">kokada.dev</a>. This means that in my eventual exit from
683 <a href="capivaras.dev">capivaras.dev</a>, I can just point my new blog location to my own
684 domain if needed (it is not as easy since I also need to preserve the post
685 URLs, but shouldn't be difficult to fix this if I ever need to do so).</p>
686 <p>Now for the tools that I use to publish from the original Markdown files to
687 everything else. Let's start with shell script(s) and Makefile: I decided that
688 they're less of an issue if they eventually stop working: they're only used to
689 make my life easier, but I can still publish files manually if needed. Still, I
690 tried to rewrite both the
691 <a href="https://github.com/thiagokokada/blog/commit/a0d421ca90f3da059998295c5e3c6c7a6a3f0688">shell</a>
692 and
693 <a href="https://github.com/thiagokokada/blog/commit/074580065b21fbdaf930aa51968e69f015d49505">Makefile</a>
694 to avoid GNUisms, so in the eventual case that I decide to e.g.: stop using a
695 GNU/Linux system like NixOS and use a *BSD system instead, I am covered.</p>
696 <p>In other projects I would probably add a <code>shell.nix</code> file and add the GNU
697 versions of the tools that I am using, but I am trying to avoid Nix while
698 building this blog as much as I can, mainly because it is not available
699 everywhere (e.g.: *BSD systems).</p>
700 <p>Go is the more important part: the tooling used to <a href="https://kokada.dev/blog/quick-bits-why-you-should-automate-everything">publish this blog is
701 written in
702 Go</a>. Go
703 is a good language when you want to ensure that things will work for a long
704 time because of its <a href="https://go.dev/blog/compat">backwards compatibility
705 guarantee</a>. Also I don't expect Google dropping Go
706 development soon, but even if this happen (<a href="https://killedbygoogle.com/">&quot;killed by
707 Google&quot;</a> is a thing after all), it is very likely
708 some other group or company would adopt its development quickly, considering
709 <a href="https://www.tiobe.com/tiobe-index/go/">how popular the language</a> is.</p>
710 <p>However, the <a href="https://github.com/thiagokokada/blog/blob/main/go.mod">Go
711 modules</a> that I depend
712 are another story:</p>
713 <ul>
714 <li><a href="https://github.com/elliotchance/orderedmap/">elliotchance/orderedmap</a>: an
715 ordered map implementation that I use until Go adds it in the <a href="https://kokada.dev/blog/an-unordered-list-of-things-i-miss-in-go">standard
716 library</a></li>
717 <li><a href="https://github.com/gorilla/feeds">gorilla/feeds</a>: a RSS generator library</li>
718 <li><a href="https://github.com/gosimple/slug">gosimple/slug</a>: a
719 <a href="https://developer.mozilla.org/en-US/docs/Glossary/Slug">slug</a> generator
720 library</li>
721 <li><a href="https://github.com/yuin/goldmark">yuin/goldmark</a>: a CommonMark parser and
722 renderer</li>
723 <li><a href="https://github.com/teekennedy/goldmark-markdown">teekennedy/goldmark-markdown</a>:
724 a renderer for Goldmark to render back to Markdown (since Goldmark itself
725 doesn't have this capacity)</li>
726 </ul>
727 <p>In common for all those modules are that they're clearly small projects
728 maintained mostly by one developer. They're all very good, don't get me wrong,
729 but they're still an reliability issue in the future. There is no guarantee
730 those repositories will not be deleted tomorrow, for example.</p>
731 <p>Yes, <a href="https://proxy.golang.org/">Go Proxy</a> exist, but from what I understood
732 reading its page is that while it caches modules contents, this is not
733 guarantee:</p>
734 <blockquote>
735 <p>proxy.golang.org does not save all modules forever. There are a number of
736 reasons for this, but one reason is if proxy.golang.org is not able to detect
737 a suitable license. In this case, only a temporarily cached copy of the
738 module will be made available, and may become unavailable if it is removed
739 from the original source and becomes outdated. The checksums will still
740 remain in the checksum database regardless of whether or not they have become
741 unavailable in the mirror.</p>
742 </blockquote>
743 <p>This is why this is the first project that made sense to me to use <a href="https://go.dev/ref/mod#go-mod-vendor"><code>go mod vendor</code></a>. Now I have a copy of the source
744 code of all modules inside the
745 <a href="https://github.com/thiagokokada/blog/tree/0b97630d6b30551ffe05b5d8124305b1065f729d/vendor">vendor</a>
746 directory in the repository, avoiding the risks I commented above. This allows
747 me to ensure that this blog will still be publishable in the future, as long as
748 I have a working Go toolchain (and Go toolchain makes this
749 <a href="https://kokada.dev/blog/quick-bits-go-automatically-downloads-a-newer-toolchain-if-needed">easy</a>).</p>
750 <p>There are a few other things that can bitrot this blog, for example links going
751 nowhere. I always try to use
752 <a href="https://en.wikipedia.org/wiki/Permalink">permalinks</a> when it makes sense, but
753 the only actual way to ensure those links would work in the future would be to
754 point them to <a href="https://archive.org/">archive.org</a> (but even archive.org may not
755 exist forever). Maybe something to fix in the future, hope not in the far
756 future, before things start to break.</p></description><pubDate>Sat, 24 Aug 2024 00:00:00 +0000</pubDate><guid>https://kokada.dev/blog/making-a-blog-for-the-next-10-years/</guid></item><item><title>An unordered list of hidden gems inside NixOS</title><link>https://kokada.dev/blog/an-unordered-list-of-hidden-gems-inside-nixos/</link><description><p>After using <a href="https://nixos.org/">NixOS</a> for the last 5+ years as my main OS, I
757 end up with a <a href="https://github.com/thiagokokada/nix-configs/">configuration</a>
758 with many things that are interesting for one reason or another, but it is not
759 listed anywhere (well, except if you are the kind of person that reads <code>man 5 configuration.nix</code> or the release notes in every release).</p>
760 <p>So kind in the same spirit as my <a href="https://kokada.dev/blog/an-unordered-list-of-things-i-miss-in-go">list of things that I miss in
761 Go</a>, here is
762 a list of modules that I find neat in NixOS and are not default already. Again,
763 the list is unordered since this makes it easier to update in the future if I
764 find something else, but also I don't want to think too hard about an order
765 here.</p>
766 <p>With all above, let's start.</p>
767 <h2><a href="https://github.com/NixOS/nixpkgs/blob/6afb255d976f85f3359e4929abd6f5149c323a02/nixos/modules/services/networking/nftables.nix"><code>networking.nftables</code></a></h2>
768 <p><a href="https://www.nftables.org/">nftables</a> is, accordingly to Wikipedia:</p>
769 <blockquote>
770 <p>nftables is a subsystem of the Linux kernel providing filtering and
771 classification of network packets/datagrams/frames.</p>
772 </blockquote>
773 <p>It is basically a replacement of the venerable
774 <a href="https://en.wikipedia.org/wiki/Iptables">iptables</a>, that still exist and is the
775 default program to configure the famous <code>networking.firewall</code>, the declarative
776 <a href="https://wiki.nixos.org/wiki/Firewall">Firewall</a> that NixOS enable by default.</p>
777 <p>To enable, it is simple, just add to your configuration:</p>
778 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
779 </span></span><span class="line"><span class="cl"> <span class="n">networking</span><span class="o">.</span><span class="n">nftables</span><span class="o">.</span><span class="n">enable</span> <span class="o">=</span> <span class="no">true</span><span class="p">;</span>
780 </span></span><span class="line"><span class="cl"><span class="p">}</span>
781 </span></span></code></pre><p>And thanks to the fact that NixOS's Firewall is declarative, everything should
782 still work as expect: any service that you set <code>openFirewall = true</code> will still
783 have its ports open, if you set <code>networking.firewall.allowPing = false</code> it will
784 still disable pings like before, etc.</p>
785 <p>If you look at the documentation of the above option, you will find the
786 following warning:</p>
787 <blockquote>
788 <p>Note that if you have Docker enabled you will not be able to use nftables
789 without intervention. Docker uses iptables internally to setup NAT for
790 containers. This module disables the ip_tables kernel module, however Docker
791 automatically loads the module. Please see
792 <a href="https://github.com/NixOS/nixpkgs/issues/24318#issuecomment-289216273">https://github.com/NixOS/nixpkgs/issues/24318#issuecomment-289216273</a> for
793 more information.</p>
794 </blockquote>
795 <p>I don't use Docker (switched to Podman instead for quite a long time), so I
796 don't know how bad the situation is. Also keep in mind that <code>nftables</code> does
797 offer <code>iptables-compat</code> for compatibility with old iptables scripts, so it is
798 most likely Docker doing something weird here.</p>
799 <p>Now, the actual advantage from the user here is not clear: the main advantage
800 from my point of view (and the reason I used to use in other distros like Arch)
801 is the improved syntax, however if you are using the declarative NixOS's
802 Firewall you are not interacting with either <code>iptables</code> or <code>nftables</code> directly
803 anyway. <code>nftables</code> is supposed to be more efficient, but not sure most users
804 will care about this.</p>
805 <p>However if you are the kind of person that needs custom rules, switching to
806 <code>nftables</code> does bring a few benefits, including
807 <code>networking.nftables.checkRuleset</code> (enabled by default), that checks if your
808 ruleset has syntax errors during build time. Really valuable to avoid issues
809 only after switch.</p>
810 <p>Anyway, this is one of those options that I think it should be the default for
811 a long time, since most of the new development in NixOS firewall seems to be
812 focusing <code>nftables</code> for a while.</p>
813 <h2><a href="https://github.com/NixOS/nixpkgs/blob/877d19523edcac81b167e8fd716ad2658da2adca/nixos/modules/system/activation/switchable-system.nix#L30-L38"><code>system.switch.enableNg</code></a></h2>
814 <p><a href="https://github.com/NixOS/nixpkgs/pull/308801">This one</a> I just discovered
815 today, but it has been available for a while (~2 months if you're using
816 <code>nixos-unstable</code>). Finally someone is rewriting
817 <a href="https://github.com/NixOS/nixpkgs/blob/b1eff03c35aa7c90ab3a4d9f6ef297dae5fba37b/nixos/modules/system/activation/switch-to-configuration.pl"><code>switch-to-configuration.pl</code></a>,
818 the Perl script that is called everytime you run <code>nixos-rebuild switch</code>.</p>
819 <p>Now, I am not one of those &quot;rewrite in Rust&quot; zealots, but in this case this is
820 definitely worth it: <code>switch-to-configuration.pl</code> is one of those pieces of
821 code in NixOS that most people avoid touching at the fear of breaking
822 something. There is a reason why
823 <a href="https://github.com/NixOS/nixpkgs/commit/eeb2588a59c938042b74183ce1da7052a6ef7e59"><code>nixos-rebuild</code></a>
824 is as convoluted as it is, because even if it is a messy shell script, most
825 people preferred to workaround issues from the <code>switch-to-configuration.pl</code>
826 inside it than trying to understand the mess that <code>switch-to-configuration.pl</code>
827 is.</p>
828 <p>Trying this one is easy:</p>
829 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
830 </span></span><span class="line"><span class="cl"> <span class="n">system</span><span class="o">.</span><span class="n">switch</span> <span class="o">=</span> <span class="p">{</span>
831 </span></span><span class="line"><span class="cl"> <span class="n">enable</span> <span class="o">=</span> <span class="no">false</span><span class="p">;</span>
832 </span></span><span class="line"><span class="cl"> <span class="n">enableNg</span> <span class="o">=</span> <span class="no">true</span><span class="p">;</span>
833 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
834 </span></span><span class="line"><span class="cl"><span class="p">}</span>
835 </span></span></code></pre><p>Yes, you need to explicit set <code>system.switch.enable = false</code>, since the default
836 is <code>true</code>.</p>
837 <p>By the way, what is the reason you would want to set <code>system.switch.enable = false</code> before the <code>enableNg</code> appeared you ask? For systems that are immutable
838 and updated by e.g.: image upgrades instead of modifying root.</p>
839 <p>Enabling <code>switch-to-configuration-ng</code> right now is mostly for testing purposes,
840 but one of the advantages that I saw is that system switches are (slightly)
841 faster:</p>
842 <pre><code>$ hyperfine &quot;sudo nixos-rebuild switch&quot; # switch-to-configuration.pl
843 Benchmark 1: sudo nixos-rebuild switch
844 Time (mean ± σ): 3.576 s ± 0.035 s [User: 0.004 s, System: 0.014 s]
845 Range (min … max): 3.522 s … 3.645 s 10 runs
846 </code><p><code>$ hyperfine &quot;sudo nixos-rebuild switch&quot; # switch-to-configuration-ng
847 Benchmark 1: sudo nixos-rebuild switch
848 Time (mean ± σ): 3.394 s ± 0.080 s [User: 0.004 s, System: 0.013 s]
849 Range (min … max): 3.325 s … 3.608 s 10 runs
850 </code></p></pre><p></p>
851 <p>But yes, the difference is not enough to make a significant impact, and it is
852 not the objective anyway. The real reason for the rewrite is to make it easier
853 to collaborate. I hope one day we also have someone brave enough to rewrite the
854 <code>nixos-rebuild</code> script in something saner.</p>
855 <h2><a href="https://github.com/NixOS/nixpkgs/blob/cce9aef6fd8f010d288d685b9d2a38f3b6ac47e9/nixos/modules/system/boot/systemd/initrd.nix"><code>boot.initrd.systemd</code></a></h2>
856 <p>A quick recap on how a modern Linux distro generally boots: the first thing
857 that the bootloader (say <a href="https://www.gnu.org/software/grub/">GRUB</a> or
858 <a href="https://systemd.io/BOOT/">systemd-boot</a>) loads is <code>initrd</code> (<em>initial
859 ramdisk</em>), a small image that runs from RAM and includes the Linux kernel and
860 some utilities that are responsible for setting up the main system. For
861 example, one of the responsibilities of the <code>initrd</code> is to mount the disks and
862 start init system (<code>systemd</code>).</p>
863 <p>It may surprising that this <code>initrd</code> image does <strong>not</strong> generally include
864 <code>systemd</code>. Traditionally <code>initrd</code> is composed by a bunch of shell scripts and a
865 minimal runtime (e.g.: <a href="https://www.busybox.net/">busybox</a>), however <code>systemd</code>
866 can also do this job since a long time ago. It is just the role of the distros
867 to integrate <code>systemd</code> inside the <code>initrd</code>.</p>
868 <p>This is what <code>boot.initrd.systemd</code> does: enable <code>systemd</code> inside the <code>initrd</code>.
869 It make a few subtle changes:</p>
870 <ul>
871 <li>If you are using <a href="https://wiki.nixos.org/wiki/Full_Disk_Encryption">Full Disk Encryption via
872 LUKS</a>, you will get a
873 different password prompt at login</li>
874 <li>You will get <code>initrd</code> time information if using <code>systemd-analyze</code> to measure
875 boot time
876 <ul>
877 <li>You can get even more information (bootloader) if you also use
878 <code>systemd-boot</code></li>
879 </ul>
880 </li>
881 <li>You will also get <code>systemd</code> style status about services during <code>initrd</code> (not
882 only afterwards)</li>
883 </ul>
884 <p>But I think the main reason is that since <code>systemd</code> is event-driven, it should
885 make boot more reliable, especially in challenging situations (like booting
886 from network). I can't say that I have any system like this to test if it is
887 actually more reliable or not, but I don't remember having any issues since I
888 set <code>boot.initrd.systemd.enable = true</code>, so there is that.</p>
889 <h2><a href="https://github.com/NixOS/nixpkgs/blob/b4a09f1f9d1599478afadffa782a02690550447c/pkgs/development/libraries/pipewire/default.nix"><code>services.pipewire</code></a></h2>
890 <p>If there is something in that list that has a good chance that you're using
891 already, it is this one, especially if you're using
892 <a href="https://wayland.freedesktop.org/">Wayland</a>. Still, I think it is interesting
893 to include in this list since <a href="https://www.pipewire.org/">PipeWire</a> is great.</p>
894 <p>The experience with PipeWire until now for me was seamless: I never had any
895 issues with it, all my applications still work exactly as it always worked. I
896 also didn't had any issues with
897 <a href="https://www.freedesktop.org/wiki/Software/PulseAudio/">PulseAudio</a> for a
898 while, but I still remember when I first tried PulseAudio during the 0.x in
899 Fedora and having tons of issues. So bonus points for PipeWire developers for
900 polishing the experience of enough that most people will feel no difference.</p>
901 <p>To enable PipeWire, I would recommend:</p>
902 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
903 </span></span><span class="line"><span class="cl"> <span class="n">services</span><span class="o">.</span><span class="n">pipewire</span> <span class="o">=</span> <span class="p">{</span>
904 </span></span><span class="line"><span class="cl"> <span class="n">enable</span> <span class="o">=</span> <span class="no">true</span><span class="p">;</span>
905 </span></span><span class="line"><span class="cl"> <span class="n">alsa</span><span class="o">.</span><span class="n">enable</span> <span class="o">=</span> <span class="no">true</span><span class="p">;</span>
906 </span></span><span class="line"><span class="cl"> <span class="n">pulse</span><span class="o">.</span><span class="n">enable</span> <span class="o">=</span> <span class="no">true</span><span class="p">;</span>
907 </span></span><span class="line"><span class="cl"> <span class="c1"># jack.enable = true;</span>
908 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
909 </span></span><span class="line"><span class="cl"> <span class="n">security</span><span class="o">.</span><span class="n">rtkit</span><span class="o">.</span><span class="n">enable</span> <span class="o">=</span> <span class="no">true</span><span class="p">;</span>
910 </span></span><span class="line"><span class="cl"><span class="p">}</span>
911 </span></span></code></pre><p>This enables both ALSA and PulseAudio emulation support in PipeWire for maximum
912 compatibility with desktop applications (you can also enable
913 <a href="https://jackaudio.org/"><code>jack</code></a> if you use professional audio applications).
914 It also enables <a href="https://github.com/heftig/rtkit"><code>rtkit</code></a>, allowing PipeWire
915 to get (soft) realtime, helping avoiding cracks during high CPU load.</p>
916 <p>I also recommend taking a look at the <a href="https://wiki.nixos.org/wiki/PipeWire">Wiki
917 article</a>, that has multiple interesting
918 configurations that can be added for low-latency setups or improved codecs for
919 Bluetooth devices.</p>
920 <h2><a href="https://github.com/NixOS/nixpkgs/blob/c9ec8289781a3c4ac4dd5c42c8d50dd65360e79c/nixos/modules/services/networking/networkmanager.nix#L264-L271"><code>networking.networkmanager.wifi.backend = &quot;iwd&quot;</code></a></h2>
921 <p>There is a good change that you're using
922 <a href="https://www.networkmanager.dev/"><code>NetworkManager</code></a> to manage network,
923 especially for Wi-Fi. And if that is the case, I can't recommend enough
924 changing the backend from the default <code>wpa_supplicant</code> to
925 <a href="https://iwd.wiki.kernel.org/"><code>iwd</code></a>.</p>
926 <p>If you think that your Wi-Fi takes a long time to connect/re-connect, it may be
927 because <code>wpa_supplicant</code>. <code>iwd</code> seems much more optimised in this regard, and
928 since switching to it I never felt that my Wi-Fi was worse than other OSes (and
929 generally slightly better than Windows, but keep in mind that this is a
930 non-scientific comparison).</p>
931 <p>Not saying that I never had Wi-Fi issues since switching to <code>iwd</code>, however
932 switching back to <code>wpa_supplicant</code> in those cases never fixed the issue (it was
933 the same or worse), so I assume either bad hardware or drivers in those cases.</p>
934 <h2><a href="https://github.com/NixOS/nixpkgs/blob/d5badef1e0416160298aad6dd8604eaf8b6e7e52/nixos/modules/system/boot/tmp.nix#L25-L32"><code>boot.tmp.useTmpfs</code></a></h2>
935 <p><em>Added in 2024-08-22</em></p>
936 <p>Most Linux distro nowadays mount <code>/tmp</code> inside
937 <a href="https://en.wikipedia.org/wiki/Tmpfs">tmpfs</a>, storing the files inside RAM (and
938 making them actually temporary, e.g.: does not persist between reboots). There
939 is a <a href="https://lwn.net/Articles/499410/">long discussion</a> whether this makes
940 sense or not, but it is a fact that using <code>/tmp</code> as an storage for small files
941 generally makes sense.</p>
942 <p>However, NixOS still stores <code>/tmp</code> inside <code>/</code> by default. The main reason for
943 this is because Nix, by default, still builds everything in <code>TMPDIR</code> (that
944 defaults to <code>/tmp</code> if not set). To fix the situation you can do:</p>
945 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
946 </span></span><span class="line"><span class="cl"> <span class="n">boot</span><span class="o">.</span><span class="n">tmp</span><span class="o">.</span><span class="n">useTmpfs</span> <span class="o">=</span> <span class="no">true</span><span class="p">;</span>
947 </span></span><span class="line"><span class="cl"> <span class="n">systemd</span><span class="o">.</span><span class="n">services</span><span class="o">.</span><span class="n">nix-daemon</span> <span class="o">=</span> <span class="p">{</span>
948 </span></span><span class="line"><span class="cl"> <span class="n">environment</span><span class="o">.</span><span class="n">TMPDIR</span> <span class="o">=</span> <span class="s2">&#34;/var/tmp&#34;</span><span class="p">;</span>
949 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
950 </span></span><span class="line"><span class="cl"><span class="p">}</span>
951 </span></span></code></pre><p>This ensures that Nix will build in <code>/var/tmp</code>, that is the temporary directory
952 for large files.</p>
953 <p>If you don't want to use <code>/tmp</code> inside tmpfs for some reason, I recommend at
954 least setting <code>boot.tmp.cleanOnBoot = true</code>. This is to avoid issues with e.g.:
955 mistakenly writing a program that depends in <code>/tmp</code> being persistent, because
956 this is the behavior that most other distros expects.</p>
957 <h2><a href="https://github.com/NixOS/nixpkgs/blob/68142254d20256e617f596a5d50f9950f98fa36a/nixos/modules/config/zram.nix"><code>zramSwap</code></a></h2>
958 <p><em>Added in 2024-08-22</em></p>
959 <p>Have a system with small amounts of RAM? <del>You can download more RAM for free,
960 just click this <a href="https://downloadmoreram.com/">link</a>.</del> Well, just kidding, but
961 you can actually improve RAM usage by compressing it. This is what
962 <a href="https://en.wikipedia.org/wiki/Zram"><code>zram</code></a> does: it will create a block
963 device using part of your RAM that will be compressed.</p>
964 <p>While this means that whatever is in the compressed part can't be used as-is
965 (it needs to be decompressed first), it means you can store a lot more
966 information. And since RAM is much faster than disk, this generally is a good
967 trade-off. It is still recommended that you have some swap space though, e.g.:
968 to swap pages that are unused.</p>
969 <p>To enable it, you can do:</p>
970 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
971 </span></span><span class="line"><span class="cl"> <span class="n">zramSwap</span> <span class="o">=</span> <span class="p">{</span>
972 </span></span><span class="line"><span class="cl"> <span class="n">enable</span> <span class="o">=</span> <span class="no">true</span><span class="p">;</span>
973 </span></span><span class="line"><span class="cl"> <span class="n">algorithm</span> <span class="o">=</span> <span class="s2">&#34;zstd&#34;</span><span class="p">;</span>
974 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
975 </span></span><span class="line"><span class="cl"><span class="p">}</span>
976 </span></span></code></pre><p>You can play a little with the <code>zramSwap.algorithm</code> parameter, but I recommend
977 either <code>lz4</code> or <code>zstd</code>. It is important for the compression algorithm to be
978 fast, since you probably want to minimise CPU usage. Well, unless you have a
979 very good CPU and small amounts of RAM, them it may make sense to use an
980 algorithm that compress better but uses more CPU.</p>
981 <h2><a href="https://github.com/NixOS/nixpkgs/blob/5356420466c4d7901b63acc5e337c5bf30573f8a/nixos/modules/services/misc/fstrim.nix"><code>services.fstrim</code></a></h2>
982 <p><em>Added in 2024-08-22</em></p>
983 <p>Do you have a SSD? If so, you probably want to
984 <a href="https://en.wikipedia.org/wiki/Trim_(computing)">trim</a> it regularly to keep a
985 good performance. This should be as easy as:</p>
986 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
987 </span></span><span class="line"><span class="cl"> <span class="n">services</span><span class="o">.</span><span class="n">fstrim</span><span class="o">.</span><span class="n">enable</span> <span class="o">=</span> <span class="no">true</span><span class="p">;</span>
988 </span></span><span class="line"><span class="cl"><span class="p">}</span>
989 </span></span></code></pre><p>One caveat though: if you are using Full Disk Encryption with LUKS, you also
990 need to enable <code>boot.initrd.luks.devices.&lt;name&gt;.allowDiscards</code>, otherwise
991 <code>fstrim</code> will not work. The reason that this is not enabled by default is
992 because there are some <a href="https://asalor.blogspot.com/2011/08/trim-dm-crypt-problems.html">security
993 implications</a>.
994 It shouldn't be an issue for most people, but if you are paranoid please assess
995 your risks first.</p>
996 <h2><a href="https://github.com/NixOS/nixpkgs/blob/e83fc891118e5e5660ccca000040e33b0a0a3080/nixos/modules/system/boot/binfmt.nix"><code>boot.binfmt.emulatedSystems</code></a></h2>
997 <p><em>Added in 2024-08-22</em></p>
998 <p>This is one of my favorites. Do you want to have the power to run binaries from
999 other architectures like
1000 <a href="https://en.wikipedia.org/wiki/ARM_architecture_family">ARM</a> or
1001 <a href="https://en.wikipedia.org/wiki/RISC-V">RISC-V</a>? In NixOS it is easy:</p>
1002 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
1003 </span></span><span class="line"><span class="cl"> <span class="n">boot</span><span class="o">.</span><span class="n">binfmt</span><span class="o">.</span><span class="n">emulatedSystems</span> <span class="o">=</span> <span class="p">[</span> <span class="s2">&#34;aarch64-linux&#34;</span> <span class="s2">&#34;riscv64-linux&#34;</span> <span class="p">];</span>
1004 </span></span><span class="line"><span class="cl"><span class="p">}</span>
1005 </span></span></code></pre><p>This will install a QEMU emulator for the target architectures, and configure
1006 <a href="https://en.wikipedia.org/wiki/Binfmt_misc"><code>binfmt</code></a> so that the kernel will
1007 automatically detect when a binary from another architecture is run,
1008 running it with the QEMU emulator instead.</p>
1009 <p>Now, keep in mind that you still need e.g.: libraries for the target
1010 architecture. However this is where the Nix magic comes in, you can easily
1011 compile something to another architecture as:</p>
1012 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> uname -a
1013 </span></span><span class="line"><span class="cl"><span class="go">Linux sankyuu-nixos 6.10.3 #1-NixOS SMP PREEMPT_DYNAMIC Sat Aug 3 07:01:09 UTC 2024 x86_64 GNU/Linux
1014 </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="gp">$</span> <span class="nb">cd</span> nixpkgs
1015 </span></span><span class="line"><span class="cl"><span class="gp">$</span> nix build .#hello --system aarch64-linux
1016 </span></span><span class="line"><span class="cl"><span class="gp">$</span> file ./result/bin/hello
1017 </span></span><span class="line"><span class="cl"><span class="go">./result/bin/hello: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /nix/store/kz7xglxzhad64v667wwpn8vrxhjwcbna-glibc-2.39-52/lib/ld-linux-aarch64.so.1, for GNU/Linux 3.10.0, not stripped
1018 </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="gp">$</span> ./result/bin/hello
1019 </span></span><span class="line"><span class="cl"><span class="go">Hello, world!
1020 </span></span></span></code></pre><p>And since the binary will be linked with the libraries from the target
1021 architecture, everything works as expect.</p>
1022 <p>You also need to temper your expectations: QEMU emulation is slow. It is
1023 sufficient fast to build small programs, but if you need to build something
1024 bigger, I would recommend you to do it via <a href="https://kokada.dev/blog/building-static-binaries-in-nix">cross-compilation
1025 instead</a>.</p>
1026 <h2><a href="https://github.com/NixOS/nixpkgs/blob/d9cf6ea6f809b862b73766705bdadb1236eeeff8/nixos/modules/services/system/dbus.nix#L40-L50"><code>services.dbus.implementation = &quot;broker&quot;</code></a></h2>
1027 <p><em>Added in 2024-08-22</em></p>
1028 <p>Someone in
1029 <a href="https://lobste.rs/s/ymmale/unordered_list_hidden_gems_inside_nixos#c_dz1hck">Lobste.rs</a>
1030 remembered me of this gem: <a href="https://github.com/bus1/dbus-broker">dbus-broker</a>,
1031 a high performance implementation of
1032 <a href="https://en.wikipedia.org/wiki/D-Bus">D-Bus</a>.</p>
1033 <p>For those who doesn't know, D-Bus is the <a href="https://en.wikipedia.org/wiki/Inter-process_communication">Inter-Process
1034 Communication</a>
1035 protocol used mostly in Linux desktop. And distros like
1036 <a href="https://archlinux.org/news/making-dbus-broker-our-default-d-bus-daemon/">Arch</a>
1037 and
1038 <a href="https://fedoraproject.org/wiki/Changes/DbusBrokerAsTheDefaultDbusImplementation">Fedora</a>
1039 already use it to default, so there aren't lots of reason to not use it.</p>
1040 <h2><a href="https://github.com/NixOS/nixpkgs/blob/6afb255d976f85f3359e4929abd6f5149c323a02/nixos/modules/services/hardware/irqbalance.nix"><code>services.irqbalance</code></a></h2>
1041 <p><em>Added in 2024-08-23</em></p>
1042 <p>Here is one that I have no experience but got curious after this comment in
1043 <a href="https://lobste.rs/s/ymmale/unordered_list_hidden_gems_inside_nixos#c_aghpwy">Lobte.rs</a>:</p>
1044 <blockquote>
1045 <p>One more option to add is maybe <code>services.irqbalance.enable</code>. For my old
1046 laptop it’s a life savior: no more freezes of the DE or video playback during
1047 heavy operations like e.g. compilation.</p>
1048 </blockquote>
1049 <p>Never tried it before but will give a try!</p></description><pubDate>Wed, 21 Aug 2024 00:00:00 +0000</pubDate><guid>https://kokada.dev/blog/an-unordered-list-of-hidden-gems-inside-nixos/</guid></item><item><title>Quick bits: using forks with Go modules</title><link>https://kokada.dev/blog/quick-bits-using-forks-with-go-modules/</link><description><p>There are 2 types of blog posts: the ones you write for others, and the ones
1050 that you write for yourself. This blog post is the latter kind. What I am going
1051 to talk here is probably something most people know, but I didn't, and the
1052 documentation is all scattered in the internet. So I am writing mostly to
1053 remember myself, in future, if I have the same kind of issue what I need to do.</p>
1054 <p>The context: <a href="https://mataroa.blog/">Mataroa</a>, the blog platform
1055 <a href="https://capivaras.dev">capivaras.dev</a> is hosted on, relies in
1056 <a href="https://python-markdown.github.io/">python-markdown</a>, predates
1057 <a href="https://commonmark.org/">CommonMark</a> specification and as such, has some
1058 incompatibilities. One of those incompatibilities with CommonMark is the way
1059 sub lists are handled. From the documentation:</p>
1060 <blockquote>
1061 <p>The syntax rules clearly state that when a list item consists of multiple
1062 paragraphs, “each subsequent paragraph in a list item must be indented by
1063 either 4 spaces or one tab” (emphasis added). However, many implementations
1064 do not enforce this rule and allow less than 4 spaces of indentation. The
1065 implementers of Python-Markdown consider it a bug to not enforce this rule.</p>
1066 </blockquote>
1067 <p>CommonMark <a href="https://spec.commonmark.org/0.31.2/#lists">relax those
1068 restrictions</a>, allowing a sublist to
1069 be defined with just 2 spaces of indentation.</p>
1070 <p>So I have automated all posts from this blog <a href="https://kokada.dev/blog/quick-bits-why-you-should-automate-everything">using
1071 Go</a> and
1072 a CommonMark renderer called <a href="https://github.com/yuin/goldmark/">Goldmark</a>. I
1073 them re-render the Markdown to Mataroa using a <a href="https://github.com/teekennedy/goldmark-markdown">Markdown
1074 renderer</a> before publising to
1075 <a href="https://capivaras.dev">capivaras.dev</a>, because this allow me to do some
1076 transformations in the original Markdown. It mostly works fine except for sub
1077 lists, thanks to the fact that the Markdown renderer I am using renders sub
1078 lists with 2 spaces.</p>
1079 <p>The only reason sub lists are working right now is because
1080 <a href="https://gluer.org/">@ratsclub</a> fixed this issue in the fork that
1081 <a href="https://capivaras.dev">capivaras.dev</a> runs. But I want to be compatible with
1082 the official instance if I ever need to migrate.</p>
1083 <p>The solution? Let's fix this in a
1084 <a href="https://github.com/teekennedy/goldmark-markdown/pull/21">PR</a>. However now that
1085 I have code to fix the issue, how can I use it without waiting upstream to
1086 merge my code?</p>
1087 <p>If you are using Go modules it is easy, you just need to use the <a href="https://go.dev/ref/mod#go-mod-file-replace"><code>replace</code>
1088 directive</a>:</p>
1089 <pre class="chroma"><code><span class="line"><span class="cl"><span class="nx">module</span> <span class="nx">github</span><span class="p">.</span><span class="nx">com</span><span class="o">/</span><span class="nx">thiagokokada</span><span class="o">/</span><span class="nx">blog</span>
1090 </span></span><span class="line"><span class="cl">
1091 </span></span><span class="line"><span class="cl"><span class="k">go</span> <span class="mf">1.23</span>
1092 </span></span><span class="line"><span class="cl">
1093 </span></span><span class="line"><span class="cl"><span class="nf">require</span> <span class="p">(</span>
1094 </span></span><span class="line"><span class="cl"> <span class="nx">github</span><span class="p">.</span><span class="nx">com</span><span class="o">/</span><span class="nx">elliotchance</span><span class="o">/</span><span class="nx">orderedmap</span><span class="o">/</span><span class="nx">v2</span> <span class="nx">v2</span><span class="mf">.4.0</span>
1095 </span></span><span class="line"><span class="cl"> <span class="nx">github</span><span class="p">.</span><span class="nx">com</span><span class="o">/</span><span class="nx">gorilla</span><span class="o">/</span><span class="nx">feeds</span> <span class="nx">v1</span><span class="mf">.2.0</span>
1096 </span></span><span class="line"><span class="cl"> <span class="nx">github</span><span class="p">.</span><span class="nx">com</span><span class="o">/</span><span class="nx">gosimple</span><span class="o">/</span><span class="nx">slug</span> <span class="nx">v1</span><span class="mf">.14.0</span>
1097 </span></span><span class="line"><span class="cl"> <span class="nx">github</span><span class="p">.</span><span class="nx">com</span><span class="o">/</span><span class="nx">teekennedy</span><span class="o">/</span><span class="nx">goldmark</span><span class="o">-</span><span class="nx">markdown</span> <span class="nx">v0</span><span class="mf">.3.0</span>
1098 </span></span><span class="line"><span class="cl"> <span class="nx">github</span><span class="p">.</span><span class="nx">com</span><span class="o">/</span><span class="nx">yuin</span><span class="o">/</span><span class="nx">goldmark</span> <span class="nx">v1</span><span class="mf">.7.4</span>
1099 </span></span><span class="line"><span class="cl"> <span class="nx">github</span><span class="p">.</span><span class="nx">com</span><span class="o">/</span><span class="nx">yuin</span><span class="o">/</span><span class="nx">goldmark</span><span class="o">-</span><span class="nx">highlighting</span> <span class="nx">v0</span><span class="mf">.0.0</span><span class="o">-</span><span class="mi">20220208100518</span><span class="o">-</span><span class="mi">594</span><span class="nx">be1970594</span>
1100 </span></span><span class="line"><span class="cl"><span class="p">)</span>
1101 </span></span><span class="line"><span class="cl">
1102 </span></span><span class="line"><span class="cl"><span class="nf">require</span> <span class="p">(</span>
1103 </span></span><span class="line"><span class="cl"> <span class="nx">github</span><span class="p">.</span><span class="nx">com</span><span class="o">/</span><span class="nx">alecthomas</span><span class="o">/</span><span class="nx">chroma</span> <span class="nx">v0</span><span class="mf">.10.0</span> <span class="c1">// indirect
1104 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="nx">github</span><span class="p">.</span><span class="nx">com</span><span class="o">/</span><span class="nx">dlclark</span><span class="o">/</span><span class="nx">regexp2</span> <span class="nx">v1</span><span class="mf">.11.4</span> <span class="c1">// indirect
1105 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="nx">github</span><span class="p">.</span><span class="nx">com</span><span class="o">/</span><span class="nx">gosimple</span><span class="o">/</span><span class="nx">unidecode</span> <span class="nx">v1</span><span class="mf">.0.1</span> <span class="c1">// indirect
1106 </span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="p">)</span>
1107 </span></span><span class="line"><span class="cl">
1108 </span></span><span class="line"><span class="cl"><span class="nx">replace</span> <span class="nx">github</span><span class="p">.</span><span class="nx">com</span><span class="o">/</span><span class="nx">teekennedy</span><span class="o">/</span><span class="nx">goldmark</span><span class="o">-</span><span class="nx">markdown</span> <span class="p">=&gt;</span> <span class="nx">github</span><span class="p">.</span><span class="nx">com</span><span class="o">/</span><span class="nx">thiagokokada</span><span class="o">/</span><span class="nx">goldmark</span><span class="o">-</span><span class="nx">markdown</span> <span class="nx">v0</span><span class="mf">.0.0</span><span class="o">-</span><span class="mi">20240820111219</span><span class="o">-</span><span class="nx">f30775d8ed15</span>
1109 </span></span></code></pre><p>This will replace all usages of <code>github.com/teekennedy/goldmark-markdown</code> to my
1110 fork in <code>github.com/thiagokokada/goldmark-markdown</code>. You even get all the
1111 reproducibility of modules since Go automatically pins the commit.</p>
1112 <p>Since the Go format for versions is quite unique, you can just set to the
1113 desired branch (e.g.: instead of <code>v0.0.0-20240820111219-f30775d8ed15</code>, you can
1114 use <code>add-sublist-length-opt</code> that is the branch name) and run <code>go mod tidy</code> to
1115 fix the format.</p></description><pubDate>Tue, 20 Aug 2024 00:00:00 +0000</pubDate><guid>https://kokada.dev/blog/quick-bits-using-forks-with-go-modules/</guid></item><item><title>Quick bits: Go automatically downloads a newer toolchain if needed</title><link>https://kokada.dev/blog/quick-bits-go-automatically-downloads-a-newer-toolchain-if-needed/</link><description><p>I am using
1116 <a href="https://github.com/elliotchance/orderedmap/">elliotchance/orderedmap</a> as my
1117 choice of ordered maps (since Go <a href="https://kokada.dev/blog/an-unordered-list-of-things-i-miss-in-go">doesn't have
1118 one</a> in
1119 standard library yet). I recently did a
1120 <a href="https://github.com/elliotchance/orderedmap/pull/41">PR</a> to implement Go 1.23
1121 iterators, because I find them neat, however I was avoiding to use it in the
1122 code that generates this <a href="https://github.com/thiagokokada/blog">blog</a> since Go
1123 1.23 was just released and is still not the default Go in
1124 <a href="https://github.com/NixOS/nixpkgs">nixpkgs</a>.</p>
1125 <p>I decided that I would create a
1126 <a href="https://github.com/thiagokokada/blog/pull/2">branch</a> and leave there for a few
1127 months, until I decided to try to run the code locally and got this:</p>
1128 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> go build
1129 </span></span><span class="line"><span class="cl"><span class="go">go: downloading go1.23.0 (darwin/arm64)
1130 </span></span></span></code></pre><p>Nice. And before you ask, yes, the compiled binary works perfectly:</p>
1131 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> make
1132 </span></span><span class="line"><span class="cl"><span class="go">./blog &gt; README.md
1133 </span></span></span><span class="line"><span class="cl"><span class="go">./blog -rss &gt; rss.xml
1134 </span></span></span></code></pre><p>So how does this work? Take a look at the documentation in the official <a href="https://tip.golang.org/doc/toolchain">Golang
1135 page</a>:</p>
1136 <blockquote>
1137 <p>Starting in Go 1.21, the Go distribution consists of a go command and a
1138 bundled Go toolchain, which is the standard library as well as the compiler,
1139 assembler, and other tools. The go command can use its bundled Go toolchain
1140 as well as other versions that it finds in the local PATH or downloads as
1141 needed.</p>
1142 </blockquote>
1143 <p>There are a bunch of rules here that I am not going to enter in detail (I
1144 recommend you to read the official documentation), but a quick summary:</p>
1145 <ul>
1146 <li>
1147 <p>Go will download a toolchain when either <code>go</code> or <code>toolchain</code> lines <code>go.mod</code>
1148 is set to a Go version higher than your current <code>go</code> binary</p>
1149 <ul>
1150 <li>But only if your <code>go</code> binary is at least version 1.21, since this is the
1151 version that introduces this behavior</li>
1152 </ul>
1153 </li>
1154 <li>
1155 <p>You can force a specific toolchain with <code>GOTOOLCHAIN</code> environment setting,
1156 e.g.: <code>GOTOOLCHAIN=1.23</code></p>
1157 <ul>
1158 <li>The default value for <code>GOTOOLCHAIN</code> is <code>auto</code>, that basically has the
1159 behavior described in this post</li>
1160 <li>You can also set to <code>local</code> to always use the current <code>go</code> binary, or the
1161 previous behaviour pre-1.21 Go</li>
1162 <li>There is also <code>&lt;name&gt;+auto</code> and <code>path</code> options, that can be seen in the
1163 docs</li>
1164 </ul>
1165 </li>
1166 <li>
1167 <p>The downloaded toolchains go to whatever your <code>GOPATH</code> is, inside
1168 <code>golang.org/toolchain</code> module, and version <code>v0.0.1-goVERSION.GOOS-GOARCH</code>, for
1169 example:</p>
1170 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> ls -lah <span class="nv">$GOPATH</span>/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.darwin-arm64
1171 </span></span><span class="line"><span class="cl"><span class="go">total 64
1172 </span></span></span><span class="line"><span class="cl"><span class="go">dr-xr-xr-x 14 user staff 448B 19 Aug 12:01 .
1173 </span></span></span><span class="line"><span class="cl"><span class="go">drwxr-xr-x 4 user staff 128B 19 Aug 12:01 ..
1174 </span></span></span><span class="line"><span class="cl"><span class="go">-r--r--r-- 1 user staff 1.3K 19 Aug 12:01 CONTRIBUTING.md
1175 </span></span></span><span class="line"><span class="cl"><span class="go">-r--r--r-- 1 user staff 1.4K 19 Aug 12:01 LICENSE
1176 </span></span></span><span class="line"><span class="cl"><span class="go">-r--r--r-- 1 user staff 1.3K 19 Aug 12:01 PATENTS
1177 </span></span></span><span class="line"><span class="cl"><span class="go">-r--r--r-- 1 user staff 1.4K 19 Aug 12:01 README.md
1178 </span></span></span><span class="line"><span class="cl"><span class="go">-r--r--r-- 1 user staff 426B 19 Aug 12:01 SECURITY.md
1179 </span></span></span><span class="line"><span class="cl"><span class="go">-r--r--r-- 1 user staff 35B 19 Aug 12:01 VERSION
1180 </span></span></span><span class="line"><span class="cl"><span class="go">dr-xr-xr-x 4 user staff 128B 19 Aug 12:01 bin
1181 </span></span></span><span class="line"><span class="cl"><span class="go">-r--r--r-- 1 user staff 52B 19 Aug 12:01 codereview.cfg
1182 </span></span></span><span class="line"><span class="cl"><span class="go">-r--r--r-- 1 user staff 505B 19 Aug 12:01 go.env
1183 </span></span></span><span class="line"><span class="cl"><span class="go">dr-xr-xr-x 3 user staff 96B 19 Aug 12:01 lib
1184 </span></span></span><span class="line"><span class="cl"><span class="go">dr-xr-xr-x 4 user staff 128B 19 Aug 12:01 pkg
1185 </span></span></span><span class="line"><span class="cl"><span class="go">dr-xr-xr-x 77 user staff 2.4K 19 Aug 12:02 src
1186 </span></span></span></code></pre></li>
1187 </ul>
1188 <p>By the way, this only works well because Go toolchain binaries are static.</p>
1189 <p>While I don't like a program downloading random binaries from the internet, I
1190 like what Go is doing here. It makes the whole bootstrapping process for a Go
1191 project much easier: as long as you have a reasonable up-to-date <code>go</code> binary in
1192 your <code>PATH</code>, you should be ready to go (pun intended). And Go modules are
1193 already reasonable secure, ensuring that each module have a proper checksum. As
1194 long as nobody else can publish modules in <code>golang.org/toolchain</code> namespace I
1195 can't see much of a security issue here, but I am not a security expert.</p>
1196 <p>But if you don't like this behavior, you can always disable it by setting
1197 <code>GOTOOLCHAIN=local</code>. And just do not forget to set this in your
1198 <a href="https://brandur.org/fragments/go-version-matrix">CI</a>, unless you don't care
1199 about Go versions.</p></description><pubDate>Mon, 19 Aug 2024 00:00:00 +0000</pubDate><guid>https://kokada.dev/blog/quick-bits-go-automatically-downloads-a-newer-toolchain-if-needed/</guid></item><item><title>An unordered list of things I miss in Go</title><link>https://kokada.dev/blog/an-unordered-list-of-things-i-miss-in-go/</link><description><p>I like Go. I think it is a <a href="https://kokada.dev/blog/go-a-reasonable-good-language">reasonable good
1200 language</a>, and has some
1201 good qualities that makes up for its flaws. However, this doesn't mean I think
1202 the language couldn't be better, far from it.</p>
1203 <p>This blog post is a list of things that I miss from Go from other languages.
1204 Some of the things here could probably be implemented soon, some other would
1205 probably need a major revision of the language. The list is unordered, because
1206 this makes it easier for me to update in the future if I found something else,
1207 but also because I don't want to think too hard about giving each point here a
1208 rank.</p>
1209 <p>With all above, let's start.</p>
1210 <h2>Ordered maps in standard library</h2>
1211 <p>When I first learned about
1212 <a href="https://docs.python.org/3/library/stdtypes.html#typesmapping">dictionaries</a> in
1213 Python it quickly became one of my favorite data structures ever. They're
1214 extremely versatile, and most modern programming languages have something
1215 similar in its standard library. Go isn't different, it has
1216 <a href="https://go.dev/blog/maps"><code>map</code></a>, that is Go implementation of a <a href="https://en.wikipedia.org/wiki/Hash_table">hash
1217 table</a>. However <code>map</code> in Go are
1218 quirky, for example:</p>
1219 <pre class="chroma"><code><span class="line"><span class="cl"><span class="kn">package</span> <span class="nx">main</span>
1220 </span></span><span class="line"><span class="cl">
1221 </span></span><span class="line"><span class="cl"><span class="kd">func</span> <span class="nf">main</span><span class="p">()</span> <span class="p">{</span>
1222 </span></span><span class="line"><span class="cl"> <span class="nx">m</span> <span class="o">:=</span> <span class="kd">map</span><span class="p">[</span><span class="kt">string</span><span class="p">]</span><span class="kt">bool</span><span class="p">{</span><span class="s">&#34;foo&#34;</span><span class="p">:</span> <span class="kc">true</span><span class="p">,</span> <span class="s">&#34;bar&#34;</span><span class="p">:</span> <span class="kc">false</span><span class="p">,</span> <span class="s">&#34;baz&#34;</span><span class="p">:</span> <span class="kc">true</span><span class="p">,</span> <span class="s">&#34;qux&#34;</span><span class="p">:</span> <span class="kc">false</span><span class="p">,</span> <span class="s">&#34;quux&#34;</span><span class="p">:</span> <span class="kc">true</span><span class="p">}</span>
1223 </span></span><span class="line"><span class="cl">
1224 </span></span><span class="line"><span class="cl"> <span class="k">for</span> <span class="nx">k</span> <span class="o">:=</span> <span class="k">range</span> <span class="nx">m</span> <span class="p">{</span>
1225 </span></span><span class="line"><span class="cl"> <span class="nb">println</span><span class="p">(</span><span class="nx">k</span><span class="p">)</span>
1226 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
1227 </span></span><span class="line"><span class="cl"><span class="p">}</span>
1228 </span></span></code></pre><pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> go run ./test.go
1229 </span></span><span class="line"><span class="cl"><span class="go">bar
1230 </span></span></span><span class="line"><span class="cl"><span class="go">baz
1231 </span></span></span><span class="line"><span class="cl"><span class="go">qux
1232 </span></span></span><span class="line"><span class="cl"><span class="go">quux
1233 </span></span></span><span class="line"><span class="cl"><span class="go">foo
1234 </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="err">
1235 </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="gp">$</span> go run ./test.go
1236 </span></span><span class="line"><span class="cl"><span class="go">foo
1237 </span></span></span><span class="line"><span class="cl"><span class="go">bar
1238 </span></span></span><span class="line"><span class="cl"><span class="go">baz
1239 </span></span></span><span class="line"><span class="cl"><span class="go">qux
1240 </span></span></span><span class="line"><span class="cl"><span class="go">quux
1241 </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="err">
1242 </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="gp">$</span> go run ./test.go
1243 </span></span><span class="line"><span class="cl"><span class="go">qux
1244 </span></span></span><span class="line"><span class="cl"><span class="go">quux
1245 </span></span></span><span class="line"><span class="cl"><span class="go">foo
1246 </span></span></span><span class="line"><span class="cl"><span class="go">bar
1247 </span></span></span><span class="line"><span class="cl"><span class="go">baz
1248 </span></span></span></code></pre><p>Now, I don't expect any hash table implementation to keep the order of the
1249 elements, but Go actually <a href="https://victoriametrics.com/blog/go-map/">randomise each map
1250 instance</a>:</p>
1251 <blockquote>
1252 <p>But here’s the deal, while the hash function used for maps in Go is
1253 consistent across all maps with <strong>the same key type</strong>, the <code>seed</code> used by
1254 that hash function is different for each map instance. So, when you create a
1255 new map, Go generates a random seed just for that map.</p>
1256 </blockquote>
1257 <p>While I understand the reason for this (i.e.: to avoid developers relying in a
1258 specific iteration order), I still find it weird, and I think this is something
1259 unique for Go. This decision means that even if you don't care about a specific
1260 order, you will still need to sort the map before doing something else if you
1261 want reproducibility, something that I care a lot.</p>
1262 <p>The fix for this? Go could offer an ordered map implementation inside the
1263 standard library. An ordered map ensure that the iteration order of the map is
1264 the same as the insertion order (that is, by the way, a powerful property that
1265 allow maps to be used in other contexts, not just my pet peeve above).</p>
1266 <p>Python actually does this for any dictionaries since <a href="https://stackoverflow.com/a/39980744">Python
1267 3.6</a>, but it offered an
1268 <a href="https://docs.python.org/3/library/collections.html#collections.OrderedDict">OrderedDict</a>
1269 before it (and <code>OrderedDict</code> still has some methods that normal <code>dict</code> doesn't,
1270 that maybe useful in specific cases).</p>
1271 <p>Before generics it would be impossible to have a type-safe API for such data
1272 structure without introducing a new data type in the language (like <code>slices</code>),
1273 but now Go has generics so it is not an issue anymore. The other issue is that
1274 you would be forced to iterate manually in this new data structure, but thanks
1275 to the new <a href="https://tip.golang.org/doc/go1.23#language"><code>range-over-func</code></a> in
1276 Go 1.23, it means we can iterate in an ordered map as a library almost exactly
1277 like we can do as a <code>map</code>:</p>
1278 <pre class="chroma"><code><span class="line"><span class="cl"><span class="kn">import</span> <span class="s">&#34;orderedmap&#34;</span>
1279 </span></span><span class="line"><span class="cl">
1280 </span></span><span class="line"><span class="cl"><span class="kd">func</span> <span class="nf">main</span><span class="p">()</span> <span class="p">{</span>
1281 </span></span><span class="line"><span class="cl"> <span class="nx">m</span> <span class="o">:=</span> <span class="nx">orderedmap</span><span class="p">.</span><span class="nx">New</span><span class="p">[</span><span class="kt">string</span><span class="p">,</span> <span class="kt">bool</span><span class="p">]()</span>
1282 </span></span><span class="line"><span class="cl"> <span class="nx">m</span><span class="p">.</span><span class="nf">Set</span><span class="p">(</span><span class="s">&#34;foo&#34;</span><span class="p">,</span> <span class="kc">true</span><span class="p">)</span>
1283 </span></span><span class="line"><span class="cl"> <span class="nx">m</span><span class="p">.</span><span class="nf">Set</span><span class="p">(</span><span class="s">&#34;bar&#34;</span><span class="p">,</span> <span class="kc">false</span><span class="p">)</span>
1284 </span></span><span class="line"><span class="cl"> <span class="nx">m</span><span class="p">.</span><span class="nf">Set</span><span class="p">(</span><span class="s">&#34;baz&#34;</span><span class="p">,</span> <span class="kc">true</span><span class="p">)</span>
1285 </span></span><span class="line"><span class="cl">
1286 </span></span><span class="line"><span class="cl"> <span class="k">for</span> <span class="nx">k</span> <span class="o">:=</span> <span class="k">range</span> <span class="nx">m</span><span class="p">.</span><span class="nf">Iterator</span><span class="p">()</span> <span class="p">{</span>
1287 </span></span><span class="line"><span class="cl"> <span class="nb">println</span><span class="p">(</span><span class="nx">k</span><span class="p">)</span> <span class="c1">// Order always will be: foo, bar, baz
1288 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="p">}</span>
1289 </span></span><span class="line"><span class="cl"><span class="p">}</span>
1290 </span></span></code></pre><p>Now, of course the lack of Ordered Map in the standard library can be filled
1291 with third party implementations, e.g.: I am using this
1292 <a href="https://github.com/elliotchance/orderedmap">one</a> in one of my projects. But
1293 being in standard library reduces the friction: if there was some
1294 implementation in standard library, I would generally prefer it unless I have
1295 some specific needs. However when the standard library doesn't offer what I
1296 need, I need to find it myself a suitable library, and this ends up taking time
1297 since generally there are lots of alternatives.</p>
1298 <h2>Keyword and default arguments for functions</h2>
1299 <p>Something that comes straight from Python that I miss sometimes in Go is that
1300 you can do things like this when declaring a function:</p>
1301 <pre class="chroma"><code><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">hello</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s2">&#34;World&#34;</span><span class="p">):</span>
1302 </span></span><span class="line"><span class="cl"> <span class="nb">print</span><span class="p">(</span><span class="sa">f</span><span class="s2">&#34;Hello, </span><span class="si">{</span><span class="n">name</span><span class="si">}</span><span class="s2">&#34;</span><span class="p">)</span>
1303 </span></span><span class="line"><span class="cl">
1304 </span></span><span class="line"><span class="cl"><span class="n">hello</span><span class="p">(</span><span class="s2">&#34;Foo&#34;</span><span class="p">)</span> <span class="c1"># &#34;normal&#34; function call</span>
1305 </span></span><span class="line"><span class="cl"><span class="n">hello</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s2">&#34;Bar&#34;</span><span class="p">)</span> <span class="c1"># calling with keyword arguments</span>
1306 </span></span><span class="line"><span class="cl"><span class="n">hello</span><span class="p">()</span> <span class="c1"># calling with default arguments</span>
1307 </span></span></code></pre><pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> python hello.py
1308 </span></span><span class="line"><span class="cl"><span class="go">Hello, Foo
1309 </span></span></span><span class="line"><span class="cl"><span class="go">Hello, Bar
1310 </span></span></span><span class="line"><span class="cl"><span class="go">Hello, World
1311 </span></span></span></code></pre><p>The lack of default arguments especially affects even some of the API decisions
1312 for Go standard library, for example, <code>string.Replace</code>:</p>
1313 <blockquote>
1314 <p><code>func Replace(s, old, new string, n int) string</code></p>
1315 <p>Replace returns a copy of the string s with the first n non-overlapping
1316 instances of old replaced by new. If old is empty, it matches at the
1317 beginning of the string and after each UTF-8 sequence, yielding up to k+1
1318 replacements for a k-rune string. If n &lt; 0, there is no limit on the number
1319 of replacements.</p>
1320 </blockquote>
1321 <p>If Go had default arguments, <code>Replace</code> could have e.g.: <code>func Replace(s, old, new string, n int = -1)</code> signature, that would mean we don't need
1322 <code>strings.ReplaceAll</code> (since this is basically what this function does, call
1323 <code>strings.Replace(s, old, new, -1)</code>).</p>
1324 <h2>Nullability (or nillability)</h2>
1325 <p>I talked I little about this in <a href="https://kokada.dev/blog/go-a-reasonable-good-language">my previous post about
1326 Go</a>, but I want to
1327 expand here.</p>
1328 <p>First, I don't think the language needs to support the generic solution for
1329 nullability, that would be either having proper Union or Sum types. Kotlin
1330 AFAIK doesn't support neither, but my 2 years experience with Kotlin showed
1331 that just having nullable types already helped a lot in ensuring type safety.</p>
1332 <p>Second, I do feel that Go has less issues with <code>nil</code> values, than say, Java,
1333 because its decision of using zero values instead of <code>nil</code> in many cases. So
1334 for example, a string can never be <code>nil</code>, however a string pointer can be. This
1335 means that this is fine:</p>
1336 <pre class="chroma"><code><span class="line"><span class="cl"><span class="kd">func</span><span class="p">(</span><span class="nx">s</span> <span class="kt">string</span><span class="p">)</span> <span class="p">{</span>
1337 </span></span><span class="line"><span class="cl"> <span class="c1">// do something with s
1338 </span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="p">}</span>
1339 </span></span></code></pre><p>However:</p>
1340 <pre class="chroma"><code><span class="line"><span class="cl"><span class="kd">func</span><span class="p">(</span><span class="nx">s</span> <span class="o">*</span><span class="kt">string</span><span class="p">)</span> <span class="p">{</span>
1341 </span></span><span class="line"><span class="cl"> <span class="c1">// s maybe nil here, better check first
1342 </span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="p">}</span>
1343 </span></span></code></pre><p>Still, I get more <code>panic</code> for <code>nil</code> pointer deference than I get in other
1344 languages that offer nullables (heck, even Python with
1345 <a href="https://www.mypy-lang.org/"><code>mypy</code></a> is safer).</p>
1346 <p>Sadly this is the change in this post that is more likely to need a completely
1347 new revision of the language.
1348 <a href="https://github.com/golang/go/issues/49202">nillability</a> was proposed before,
1349 but it is really unlikely it can be done without breaking backwards
1350 compatibility.</p>
1351 <p>It could be done the Java way by adding a <code>nullable</code> type to the standard
1352 library (<a href="https://jcp.org/en/jsr/detail?id=305">JSR305</a>), but the fact that
1353 <a href="https://stackoverflow.com/questions/2289694/what-is-the-status-of-jsr-305">JSR305 is considerd
1354 dead</a>
1355 by many shows how difficult it is to do something like this without a major
1356 change in the language. Dart is the only language that I know that <a href="https://dart.dev/null-safety/understanding-null-safety">did this
1357 successfully</a>, but
1358 definitely it was not without its pains. And the fact that most people that
1359 program in Dart probably does because of Flutter (that eventually required
1360 newer versions with null-safety) is not a good sign.</p>
1361 <h2>Lambdas</h2>
1362 <p><em>Added in 2024-08-18</em></p>
1363 <p>Go is a surprising good language for some functional code, thanks to having
1364 first class functions and closures. Sadly the syntax doesn't help, since the
1365 only way you can use anonymous functions in Go is using <code>func</code>. Especially if
1366 the types are complex, this can result in some convoluted code. Take the
1367 example from the <a href="https://go.dev/wiki/RangefuncExperiment"><code>range-over-func</code>
1368 experiment</a>:</p>
1369 <pre class="chroma"><code><span class="line"><span class="cl"><span class="kn">package</span> <span class="nx">slices</span>
1370 </span></span><span class="line"><span class="cl">
1371 </span></span><span class="line"><span class="cl"><span class="kd">func</span> <span class="nx">Backward</span><span class="p">[</span><span class="nx">E</span> <span class="nx">any</span><span class="p">](</span><span class="nx">s</span> <span class="p">[]</span><span class="nx">E</span><span class="p">)</span> <span class="kd">func</span><span class="p">(</span><span class="kd">func</span><span class="p">(</span><span class="kt">int</span><span class="p">,</span> <span class="nx">E</span><span class="p">)</span> <span class="kt">bool</span><span class="p">)</span> <span class="p">{</span>
1372 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="kd">func</span><span class="p">(</span><span class="nx">yield</span> <span class="kd">func</span><span class="p">(</span><span class="kt">int</span><span class="p">,</span> <span class="nx">E</span><span class="p">)</span> <span class="kt">bool</span><span class="p">)</span> <span class="p">{</span>
1373 </span></span><span class="line"><span class="cl"> <span class="k">for</span> <span class="nx">i</span> <span class="o">:=</span> <span class="nb">len</span><span class="p">(</span><span class="nx">s</span><span class="p">)</span><span class="o">-</span><span class="mi">1</span><span class="p">;</span> <span class="nx">i</span> <span class="o">&gt;=</span> <span class="mi">0</span><span class="p">;</span> <span class="nx">i</span><span class="o">--</span> <span class="p">{</span>
1374 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="p">!</span><span class="nf">yield</span><span class="p">(</span><span class="nx">i</span><span class="p">,</span> <span class="nx">s</span><span class="p">[</span><span class="nx">i</span><span class="p">])</span> <span class="p">{</span>
1375 </span></span><span class="line"><span class="cl"> <span class="k">return</span>
1376 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
1377 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
1378 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
1379 </span></span><span class="line"><span class="cl"><span class="p">}</span>
1380 </span></span></code></pre><p>If Go had a syntax for lambdas, especially if we could elide the types, this
1381 could be simplified a lot:</p>
1382 <pre class="chroma"><code><span class="line"><span class="cl"><span class="kn">package</span> <span class="nx">slices</span>
1383 </span></span><span class="line"><span class="cl">
1384 </span></span><span class="line"><span class="cl"><span class="kd">func</span> <span class="nx">Backward</span><span class="p">[</span><span class="nx">E</span> <span class="nx">any</span><span class="p">](</span><span class="nx">s</span> <span class="p">[]</span><span class="nx">E</span><span class="p">)</span> <span class="kd">func</span><span class="p">(</span><span class="kd">func</span><span class="p">(</span><span class="kt">int</span><span class="p">,</span> <span class="nx">E</span><span class="p">)</span> <span class="kt">bool</span><span class="p">)</span> <span class="p">{</span>
1385 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="p">(</span><span class="nx">yield</span><span class="p">)</span> <span class="p">=&gt;</span> <span class="p">{</span>
1386 </span></span><span class="line"><span class="cl"> <span class="k">for</span> <span class="nx">i</span> <span class="o">:=</span> <span class="nb">len</span><span class="p">(</span><span class="nx">s</span><span class="p">)</span><span class="o">-</span><span class="mi">1</span><span class="p">;</span> <span class="nx">i</span> <span class="o">&gt;=</span> <span class="mi">0</span><span class="p">;</span> <span class="nx">i</span><span class="o">--</span> <span class="p">{</span>
1387 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="p">!</span><span class="nf">yield</span><span class="p">(</span><span class="nx">i</span><span class="p">,</span> <span class="nx">s</span><span class="p">[</span><span class="nx">i</span><span class="p">])</span> <span class="p">{</span>
1388 </span></span><span class="line"><span class="cl"> <span class="k">return</span>
1389 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
1390 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
1391 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
1392 </span></span><span class="line"><span class="cl"><span class="p">}</span>
1393 </span></span></code></pre><p>Or even something like this would already help, no special syntax but allowing
1394 the types to be elided in an unnamed function:</p>
1395 <pre class="chroma"><code><span class="line"><span class="cl"><span class="kn">package</span> <span class="nx">slices</span>
1396 </span></span><span class="line"><span class="cl">
1397 </span></span><span class="line"><span class="cl"><span class="kd">func</span> <span class="nx">Backward</span><span class="p">[</span><span class="nx">E</span> <span class="nx">any</span><span class="p">](</span><span class="nx">s</span> <span class="p">[]</span><span class="nx">E</span><span class="p">)</span> <span class="kd">func</span><span class="p">(</span><span class="kd">func</span><span class="p">(</span><span class="kt">int</span><span class="p">,</span> <span class="nx">E</span><span class="p">)</span> <span class="kt">bool</span><span class="p">)</span> <span class="p">{</span>
1398 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="kd">func</span><span class="p">(</span><span class="nx">yield</span><span class="p">)</span> <span class="p">{</span>
1399 </span></span><span class="line"><span class="cl"> <span class="k">for</span> <span class="nx">i</span> <span class="o">:=</span> <span class="nb">len</span><span class="p">(</span><span class="nx">s</span><span class="p">)</span><span class="o">-</span><span class="mi">1</span><span class="p">;</span> <span class="nx">i</span> <span class="o">&gt;=</span> <span class="mi">0</span><span class="p">;</span> <span class="nx">i</span><span class="o">--</span> <span class="p">{</span>
1400 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="p">!</span><span class="nf">yield</span><span class="p">(</span><span class="nx">i</span><span class="p">,</span> <span class="nx">s</span><span class="p">[</span><span class="nx">i</span><span class="p">])</span> <span class="p">{</span>
1401 </span></span><span class="line"><span class="cl"> <span class="k">return</span>
1402 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
1403 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
1404 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
1405 </span></span><span class="line"><span class="cl"><span class="p">}</span>
1406 </span></span></code></pre><p>This feature I am still somewhat hopeful that may become a reality in some
1407 future version of the language, since they didn't close the
1408 <a href="https://github.com/golang/go/issues/21498">issue</a> yet, and the discussion
1409 about the possibility of this feature is still ongoing.</p>
1410 <h2>Error on unused return values</h2>
1411 <p><em>Added in 2024-09-12</em></p>
1412 <p>In a <a href="https://kokada.dev/blog/my-coding-experience-with-llm">recent post about
1413 LLMs</a>, I talked about
1414 this function that I asked ChatGPT to generate to create a socket that accepted
1415 a <a href="https://pkg.go.dev/context">context</a>:</p>
1416 <pre class="chroma"><code><span class="line"><span class="cl"><span class="kd">func</span> <span class="nf">readWithContext</span><span class="p">(</span><span class="nx">ctx</span> <span class="nx">context</span><span class="p">.</span><span class="nx">Context</span><span class="p">,</span> <span class="nx">conn</span> <span class="nx">net</span><span class="p">.</span><span class="nx">Conn</span><span class="p">,</span> <span class="nx">buf</span> <span class="p">[]</span><span class="kt">byte</span><span class="p">)</span> <span class="p">(</span><span class="kt">int</span><span class="p">,</span> <span class="kt">error</span><span class="p">)</span> <span class="p">{</span>
1417 </span></span><span class="line"><span class="cl"> <span class="nx">done</span> <span class="o">:=</span> <span class="nb">make</span><span class="p">(</span><span class="kd">chan</span> <span class="kd">struct</span><span class="p">{})</span>
1418 </span></span><span class="line"><span class="cl"> <span class="kd">var</span> <span class="nx">n</span> <span class="kt">int</span>
1419 </span></span><span class="line"><span class="cl"> <span class="kd">var</span> <span class="nx">err</span> <span class="kt">error</span>
1420 </span></span><span class="line"><span class="cl">
1421 </span></span><span class="line"><span class="cl"> <span class="c1">// Start a goroutine to perform the read
1422 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="k">go</span> <span class="kd">func</span><span class="p">()</span> <span class="p">{</span>
1423 </span></span><span class="line"><span class="cl"> <span class="nx">n</span><span class="p">,</span> <span class="nx">err</span> <span class="p">=</span> <span class="nx">conn</span><span class="p">.</span><span class="nf">Read</span><span class="p">(</span><span class="nx">buf</span><span class="p">)</span>
1424 </span></span><span class="line"><span class="cl"> <span class="nb">close</span><span class="p">(</span><span class="nx">done</span><span class="p">)</span>
1425 </span></span><span class="line"><span class="cl"> <span class="p">}()</span>
1426 </span></span><span class="line"><span class="cl">
1427 </span></span><span class="line"><span class="cl"> <span class="k">select</span> <span class="p">{</span>
1428 </span></span><span class="line"><span class="cl"> <span class="k">case</span> <span class="o">&lt;-</span><span class="nx">ctx</span><span class="p">.</span><span class="nf">Done</span><span class="p">():</span> <span class="c1">// Context was canceled or timed out
1429 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="c1">// Set a short deadline to unblock the Read()
1430 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="nx">conn</span><span class="p">.</span><span class="nf">SetReadDeadline</span><span class="p">(</span><span class="nx">time</span><span class="p">.</span><span class="nf">Now</span><span class="p">())</span>
1431 </span></span><span class="line"><span class="cl"> <span class="o">&lt;-</span><span class="nx">done</span> <span class="c1">// Wait for the read to finish
1432 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="k">return</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">ctx</span><span class="p">.</span><span class="nf">Err</span><span class="p">()</span>
1433 </span></span><span class="line"><span class="cl"> <span class="k">case</span> <span class="o">&lt;-</span><span class="nx">done</span><span class="p">:</span> <span class="c1">// Read finished successfully
1434 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="k">return</span> <span class="nx">n</span><span class="p">,</span> <span class="nx">err</span>
1435 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
1436 </span></span><span class="line"><span class="cl"><span class="p">}</span>
1437 </span></span></code></pre><p>However while reading <a href="https://joeduffyblog.com/2016/02/07/the-error-model/">this blog
1438 post</a> about &quot;The Error
1439 Model&quot;, I realised that this function is lacking error check during
1440 <code>conn.SetReadDeadline()</code> calls:</p>
1441 <pre class="chroma"><code><span class="line"><span class="cl"><span class="kd">func</span> <span class="nf">readWithContext</span><span class="p">(</span><span class="nx">ctx</span> <span class="nx">context</span><span class="p">.</span><span class="nx">Context</span><span class="p">,</span> <span class="nx">conn</span> <span class="nx">net</span><span class="p">.</span><span class="nx">Conn</span><span class="p">,</span> <span class="nx">buf</span> <span class="p">[]</span><span class="kt">byte</span><span class="p">)</span> <span class="p">(</span><span class="nx">n</span> <span class="kt">int</span><span class="p">,</span> <span class="nx">err</span> <span class="kt">error</span><span class="p">)</span> <span class="p">{</span>
1442 </span></span><span class="line"><span class="cl"> <span class="nx">done</span> <span class="o">:=</span> <span class="nb">make</span><span class="p">(</span><span class="kd">chan</span> <span class="kd">struct</span><span class="p">{})</span>
1443 </span></span><span class="line"><span class="cl">
1444 </span></span><span class="line"><span class="cl"> <span class="c1">// Start a goroutine to perform the read
1445 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="k">go</span> <span class="kd">func</span><span class="p">()</span> <span class="p">{</span>
1446 </span></span><span class="line"><span class="cl"> <span class="nx">n</span><span class="p">,</span> <span class="nx">err</span> <span class="p">=</span> <span class="nx">conn</span><span class="p">.</span><span class="nf">Read</span><span class="p">(</span><span class="nx">buf</span><span class="p">)</span>
1447 </span></span><span class="line"><span class="cl"> <span class="nb">close</span><span class="p">(</span><span class="nx">done</span><span class="p">)</span>
1448 </span></span><span class="line"><span class="cl"> <span class="p">}()</span>
1449 </span></span><span class="line"><span class="cl">
1450 </span></span><span class="line"><span class="cl"> <span class="k">select</span> <span class="p">{</span>
1451 </span></span><span class="line"><span class="cl"> <span class="k">case</span> <span class="o">&lt;-</span><span class="nx">done</span><span class="p">:</span>
1452 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="nx">n</span><span class="p">,</span> <span class="nx">err</span>
1453 </span></span><span class="line"><span class="cl"> <span class="k">case</span> <span class="o">&lt;-</span><span class="nx">ctx</span><span class="p">.</span><span class="nf">Done</span><span class="p">():</span>
1454 </span></span><span class="line"><span class="cl"> <span class="c1">// Set a short deadline to unblock the Read()
1455 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="nx">err</span> <span class="p">=</span> <span class="nx">conn</span><span class="p">.</span><span class="nf">SetReadDeadline</span><span class="p">(</span><span class="nx">time</span><span class="p">.</span><span class="nf">Now</span><span class="p">())</span>
1456 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="nx">err</span> <span class="o">!=</span> <span class="kc">nil</span> <span class="p">{</span>
1457 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">err</span>
1458 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
1459 </span></span><span class="line"><span class="cl"> <span class="c1">// Reset read deadline
1460 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="k">defer</span> <span class="kd">func</span><span class="p">()</span> <span class="p">{</span>
1461 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="nx">e</span> <span class="o">:=</span> <span class="nx">conn</span><span class="p">.</span><span class="nf">SetReadDeadline</span><span class="p">(</span><span class="nx">time</span><span class="p">.</span><span class="nx">Time</span><span class="p">{});</span> <span class="nx">e</span> <span class="o">!=</span> <span class="kc">nil</span> <span class="p">{</span>
1462 </span></span><span class="line"><span class="cl"> <span class="nx">err</span> <span class="p">=</span> <span class="nx">errors</span><span class="p">.</span><span class="nf">Join</span><span class="p">(</span><span class="nx">err</span><span class="p">,</span> <span class="nx">e</span><span class="p">)</span>
1463 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
1464 </span></span><span class="line"><span class="cl"> <span class="p">}()</span>
1465 </span></span><span class="line"><span class="cl"> <span class="c1">// Make sure that the goroutine is done to avoid leaks
1466 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="o">&lt;-</span><span class="nx">done</span>
1467 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">errors</span><span class="p">.</span><span class="nf">Join</span><span class="p">(</span><span class="nx">err</span><span class="p">,</span> <span class="nx">ctx</span><span class="p">.</span><span class="nf">Err</span><span class="p">())</span>
1468 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
1469 </span></span><span class="line"><span class="cl"><span class="p">}</span>
1470 </span></span></code></pre><p>I could blame the LLM on this, but this is the type of mistake that I could see
1471 happening even in a Pull Request done by an actual human.</p>
1472 <p>Ignoring errors is bad, especially because once the error is ignored it is
1473 gone, forever. You may have strange issues in the code that are impossible to
1474 debug because you can't know about something that doesn't exist. This is one of
1475 the reasons I think exceptions are better, since this is the kind of error that
1476 would be impossible to ignore if the language had exceptions, and the exception
1477 would eventually propagate in your stack until it is captured or you get a
1478 crash (with a stack trace to debug the issue).</p>
1479 <p>Now, I don't think this is a fault of using error as values instead of
1480 exceptions. Go has the tools to encapsulate the errors and propagate them
1481 properly, like it is shown in the fixed code. But the fact that someone can do
1482 this mistake without any warning or error from the compiler is bad.</p>
1483 <p>If Go had a warning or error for unused return values, this would be different:</p>
1484 <pre class="chroma"><code><span class="line"><span class="cl"><span class="kd">func</span> <span class="nf">foo</span><span class="p">(</span><span class="nx">conn</span> <span class="o">*</span><span class="nx">net</span><span class="p">.</span><span class="nx">Conn</span><span class="p">)</span> <span class="p">{</span>
1485 </span></span><span class="line"><span class="cl"> <span class="c1">// ...
1486 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="nx">_</span> <span class="p">=</span> <span class="nx">conn</span><span class="p">.</span><span class="nf">SetReadDeadline</span><span class="p">(</span><span class="nx">time</span><span class="p">.</span><span class="nf">Now</span><span class="p">())</span>
1487 </span></span><span class="line"><span class="cl"><span class="p">}</span>
1488 </span></span></code></pre><p>Now in this case it is clear: I am ignoring the error explicitly, probably for
1489 a good reason. I can ask during a Pull Request why the committer is ignoring if
1490 it is lacking appropriate context, and maybe even ask for some comments to be
1491 added why this would be safe. What can't happen is this being ignored by
1492 mistake.</p>
1493 <p>I am not completely sure if I want this only for errors or for any unused value
1494 though. There are lots of cases where ignoring non-error values is ok, but I
1495 also don't see the issue of having something like:</p>
1496 <pre class="chroma"><code><span class="line"><span class="cl"><span class="kd">func</span> <span class="nf">foo</span><span class="p">()</span> <span class="p">{</span>
1497 </span></span><span class="line"><span class="cl"> <span class="c1">// ...
1498 </span></span></span><span class="line"><span class="cl"><span class="c1"></span> <span class="nx">_</span><span class="p">,</span> <span class="nx">_</span> <span class="p">=</span> <span class="nf">FuncThatReturnIntAndError</span><span class="p">()</span>
1499 </span></span><span class="line"><span class="cl"><span class="p">}</span>
1500 </span></span></code></pre><p>If anything, it is clear for whoever is reading this code later that you're
1501 only calling this function only for its side-effects.</p>
1502 <p>By the way, yes, <a href="https://github.com/kisielk/errcheck">errcheck</a> linter exists,
1503 but if the language itself doesn't enforce this it means that there are lots of
1504 libraries that are probably mistakenly ignoring errors. And if the library is
1505 ignoring errors by mistake, you can't do much when you actually have them.</p></description><pubDate>Sat, 17 Aug 2024 00:00:00 +0000</pubDate><guid>https://kokada.dev/blog/an-unordered-list-of-things-i-miss-in-go/</guid></item><item><title>Quick bits: NixOS's boot.kernelPackages is a attrset</title><link>https://kokada.dev/blog/quick-bits-nixoss-boot-kernelpackages-is-a-attrset/</link><description><p>I don't know if this is just something that got me by surprise or not, but
1506 <a href="https://github.com/NixOS/nixpkgs/blob/bb16119a4a7639ebbc91ad0f516b324b0f7c9b68/nixos/modules/system/boot/kernel.nix#L40-L71"><code>boot.kernelPackages</code></a>
1507 option does not receive a derivation like most other packages, but instead
1508 receives a function and returns an attribute set with all packages. Here is the
1509 documentation:</p>
1510 <blockquote>
1511 <p>This option allows you to override the Linux kernel used by NixOS. Since
1512 things like external kernel module packages are tied to the kernel you’re
1513 using, it also overrides those. This option is a function that takes Nixpkgs
1514 as an argument (as a convenience), and returns an attribute set containing at
1515 the very least an attribute kernel. Additional attributes may be needed
1516 depending on your configuration. For instance, if you use the NVIDIA X
1517 driver, then it also needs to contain an attribute <code>nvidia_x11</code>.</p>
1518 </blockquote>
1519 <p>The kernel package itself is referenced by the
1520 <a href="https://github.com/NixOS/nixpkgs/blob/bb16119a4a7639ebbc91ad0f516b324b0f7c9b68/nixos/modules/system/boot/kernel.nix#L331-L332"><code>kernel</code></a>
1521 derivation inside this attribute set:</p>
1522 <pre class="chroma"><code><span class="line"><span class="cl"><span class="go">nix-repl&gt; nixosConfigurations.sankyuu-nixos.config.boot.kernelPackages.kernel
1523 </span></span></span><span class="line"><span class="cl"><span class="go">«derivation /nix/store/5zyjvf3qgfk52qmgxh36l4dkr9lf100x-linux-6.10.3.drv»
1524 </span></span></span></code></pre><p>The reason for this is because it ensure that things like modules are built
1525 with the same kernel version you are booting.</p>
1526 <p>However one less obvious consequence about this is that if you want packages
1527 that come from <code>linuxPackages</code>, say for example
1528 <a href="https://github.com/NixOS/nixpkgs/blob/nixos-unstable/pkgs/os-specific/linux/cpupower/default.nix"><code>cpupower</code></a>
1529 or
1530 <a href="https://github.com/NixOS/nixpkgs/blob/nixos-unstable/pkgs/os-specific/linux/cpupower/default.nix"><code>turbostat</code></a>,
1531 it is better to do:</p>
1532 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span> <span class="n">config</span><span class="o">,</span> <span class="o">...</span> <span class="p">}:</span>
1533 </span></span><span class="line"><span class="cl"><span class="p">{</span>
1534 </span></span><span class="line"><span class="cl"> <span class="n">environment</span><span class="o">.</span><span class="n">systemPackages</span> <span class="o">=</span> <span class="p">[</span>
1535 </span></span><span class="line"><span class="cl"> <span class="n">config</span><span class="o">.</span><span class="n">boot</span><span class="o">.</span><span class="n">kernelPackages</span><span class="o">.</span><span class="n">cpupower</span>
1536 </span></span><span class="line"><span class="cl"> <span class="n">config</span><span class="o">.</span><span class="n">boot</span><span class="o">.</span><span class="n">kernelPackages</span><span class="o">.</span><span class="n">turbostat</span>
1537 </span></span><span class="line"><span class="cl"> <span class="p">];</span>
1538 </span></span><span class="line"><span class="cl"><span class="p">}</span>
1539 </span></span></code></pre><p>Instead of:</p>
1540 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span> <span class="n">pkgs</span><span class="o">,</span> <span class="o">...</span> <span class="p">}:</span>
1541 </span></span><span class="line"><span class="cl"><span class="p">{</span>
1542 </span></span><span class="line"><span class="cl"> <span class="n">environment</span><span class="o">.</span><span class="n">systemPackages</span> <span class="o">=</span> <span class="k">with</span> <span class="n">pkgs</span><span class="p">;</span> <span class="p">[</span>
1543 </span></span><span class="line"><span class="cl"> <span class="n">linuxPackages</span><span class="o">.</span><span class="n">cpupower</span>
1544 </span></span><span class="line"><span class="cl"> <span class="n">linuxPackages</span><span class="o">.</span><span class="n">turbostat</span>
1545 </span></span><span class="line"><span class="cl"> <span class="p">];</span>
1546 </span></span><span class="line"><span class="cl"><span class="p">}</span>
1547 </span></span></code></pre><p>Now, I used the later for a long time and never had issues. But technically
1548 those packages depends in a specific kernel version for a reason, so getting
1549 them from <code>config.boot.kernelPackages</code> reduces the chance of you having some
1550 compatibility issue in the future.</p></description><pubDate>Thu, 15 Aug 2024 00:00:00 +0000</pubDate><guid>https://kokada.dev/blog/quick-bits-nixoss-boot-kernelpackages-is-a-attrset/</guid></item><item><title>Abusing Makefiles for fun and profit</title><link>https://kokada.dev/blog/abusing-makefiles-for-fun-and-profit/</link><description><p>If you are following this blog for a while, it should be no surprise that most
1551 of the workflow in this blog is <a href="https://kokada.dev/blog/quick-bits-why-you-should-automate-everything">automated using
1552 Go</a>. I
1553 basically write Markdown files with some special rules inside the
1554 <a href="https://github.com/thiagokokada/blog">repository</a>, commit and push it. In
1555 seconds, the CI (currently <a href="https://github.com/thiagokokada/blog/blob/4e3f25485c6682f3e066b219df2290934bc0d256/.github/workflows/go.yml">GitHub
1556 Actions</a>)
1557 will take the latest commit, generate some files (since I use the <a href="https://kokada.dev/blog/using-github-as-a-bad-blog-platform">repository
1558 itself</a> as a
1559 backup blog) and publish to the <a href="https://kokada.capivaras.dev/">capivaras.dev
1560 website</a>.</p>
1561 <p>Now, considering how much about <a href="https://nixos.org/">Nix</a> I talk in this blog,
1562 it should be a surprise that the workflow above has <strong>zero</strong> Nix code inside
1563 it. I am not saying this blog will never have it, but I am only going to add if
1564 this is necessary, for example if I start using a tool to build this blog that
1565 I generally don't expect it to be installed by the machine I am currently
1566 using. Go is an exception of this rule since it is relatively straightfoward to
1567 install (just download the <a href="https://go.dev/doc/install">binary</a>) and because
1568 its <a href="https://go.dev/doc/go1compat">stability guarantee</a> means (hopefully) no
1569 breakage. But most other things I consider moving targets, and I wouldn't be
1570 comfortable to use unless I have Nix to ensure reproducibility.</p>
1571 <p>This is why the other tool that this blog (ab)uses during its workflow is
1572 <a href="https://en.wikipedia.org/wiki/Make_(software)"><code>Make</code></a>, one of the oldest
1573 build automation tool that exist. It is basically available in any *nix (do not
1574 confuse with <a href="https://nixos.org/">Nix</a>) system, from most Linux distros to
1575 macOS, by default. So it is the tool I choose to automatise some tasks in this
1576 blog, even if I consider writing a <code>Makefile</code> (the domain-specific language
1577 that <code>Make</code> uses) kind of a lost, dark art.</p>
1578 <p>To be clear, the idea of this post is not to be a <code>Makefile</code> tutorial. I will
1579 explain some basic concepts, but if you want an actual tutorial a good one can
1580 be found <a href="https://makefiletutorial.com/">here</a>. Also, while I am using <code>Make</code>
1581 thanks to the reasons above, you can use many other tools for a similar
1582 objective, like <a href="https://github.com/casey/just">Justfiles</a>,
1583 <a href="https://taskfile.dev/">Taskfiles</a> (sadly it uses
1584 <a href="https://kokada.dev/blog/generating-yaml-files-with-nix">YAML</a>), or even a
1585 small script written in any language you want. The reason that I am writing
1586 this post is why you should do it, not how.</p>
1587 <p>A quick recap on how this blog works: inside the
1588 <a href="https://github.com/thiagokokada/blog">repository</a>, a post is basically a
1589 Markdown post following the directory structure below
1590 (<a href="https://github.com/thiagokokada/blog/tree/894a388c61ca3a38dfc9d4cbe88dc684fd964bb7">permalink</a>
1591 for the current version of this blog):</p>
1592 <pre class="chroma"><code><span class="line"><span class="cl"><span class="go">.
1593 </span></span></span><span class="line"><span class="cl"><span class="go">&lt;...&gt;
1594 </span></span></span><span class="line"><span class="cl"><span class="go">├── 2024-08-07
1595 </span></span></span><span class="line"><span class="cl"><span class="go">│ ├── 01-quick-bits-is-crostini-a-microvm.md
1596 </span></span></span><span class="line"><span class="cl"><span class="go">│ └── 02-meta-are-quick-bits-really-quick.md
1597 </span></span></span><span class="line"><span class="cl"><span class="go">├── 2024-08-11
1598 </span></span></span><span class="line"><span class="cl"><span class="go">│ └── 01-building-static-binaries-in-nix.md
1599 </span></span></span><span class="line"><span class="cl"><span class="go">├── 2024-08-12
1600 </span></span></span><span class="line"><span class="cl"><span class="go">│ ├── 01-things-i-dont-like-in-my-chromebook-duet-3.md
1601 </span></span></span><span class="line"><span class="cl"><span class="go">│ └── Screenshot_2024-08-12_20.50.42.png
1602 </span></span></span><span class="line"><span class="cl"><span class="go">├── 2024-08-13
1603 </span></span></span><span class="line"><span class="cl"><span class="go">│ ├── 01-abusing-makefiles-for-fun-and-profit.md &lt;-- this file
1604 </span></span></span><span class="line"><span class="cl"><span class="go">├── .github
1605 </span></span></span><span class="line"><span class="cl"><span class="go">│ └── workflows
1606 </span></span></span><span class="line"><span class="cl"><span class="go">│ └── go.yml
1607 </span></span></span><span class="line"><span class="cl"><span class="go">├── .gitignore
1608 </span></span></span><span class="line"><span class="cl"><span class="go">├── go.mod
1609 </span></span></span><span class="line"><span class="cl"><span class="go">├── go.sum
1610 </span></span></span><span class="line"><span class="cl"><span class="go">├── LICENSE
1611 </span></span></span><span class="line"><span class="cl"><span class="go">├── link_rewriter.go
1612 </span></span></span><span class="line"><span class="cl"><span class="go">├── Makefile
1613 </span></span></span><span class="line"><span class="cl"><span class="go">├── mataroa.go
1614 </span></span></span><span class="line"><span class="cl"><span class="go">├── README.md
1615 </span></span></span><span class="line"><span class="cl"><span class="go">├── rss.xml
1616 </span></span></span><span class="line"><span class="cl"><span class="go">└── .scripts
1617 </span></span></span><span class="line"><span class="cl"><span class="go"> └── gen-post.sh
1618 </span></span></span></code></pre><p>So I just create a new Markdown file following the
1619 <code>YYYY-MM-DD/XX-title-slug.md</code> format. It <strong>must</strong> start with a <code>h1</code> header,
1620 that will be automatically extract to be used as the post title, but otherwise
1621 there is no other formatting rules. It is a highly optionated structure, but
1622 the nice thing about being optionated is that we can extract lots of
1623 information just from how the files are organised in the filesystem.</p>
1624 <p>Most of the magic that converts those Markdown files to actual blog posts are
1625 in the Go files that you can see above: <code>blog.go</code> is the main logic that walks
1626 in the repository and extracts the necessary information, <code>mataroa.go</code> is
1627 responsible for the <a href="https://capivaras.dev/">capivaras.dev</a> integration (that
1628 uses <a href="https://mataroa.blog/">Mataroa</a> platform), while <code>link_rewriter.go</code> is
1629 responsible to do some transformations in the Markdown files before posting.</p>
1630 <p>While I could manage everything by just using <code>go</code> CLI and a few other *nix
1631 commands, to make it easier to manager everything I have the following
1632 <a href="https://github.com/thiagokokada/blog/blob/527466a2a7c8baae532281bff5db3f0695f018cb/Makefile"><code>Makefile</code></a>:</p>
1633 <pre class="chroma"><code><span class="line"><span class="cl"><span class="nv">MARKDOWN</span> <span class="o">:=</span> <span class="k">$(</span>wildcard ./**/*.md<span class="k">)</span>
1634 </span></span><span class="line"><span class="cl">
1635 </span></span><span class="line"><span class="cl"><span class="nf">.PHONY</span><span class="o">:</span> <span class="n">all</span>
1636 </span></span><span class="line"><span class="cl"><span class="nf">all</span><span class="o">:</span> <span class="n">README</span>.<span class="n">md</span> <span class="n">rss</span>.<span class="n">xml</span>
1637 </span></span><span class="line"><span class="cl">
1638 </span></span><span class="line"><span class="cl"><span class="nf">blog</span><span class="o">:</span> *.<span class="n">go</span> <span class="n">go</span>.*
1639 </span></span><span class="line"><span class="cl"> go build
1640 </span></span><span class="line"><span class="cl">
1641 </span></span><span class="line"><span class="cl"><span class="nf">README.md</span><span class="o">:</span> <span class="n">blog</span> <span class="k">$(</span><span class="nv">MARKDOWN</span><span class="k">)</span>
1642 </span></span><span class="line"><span class="cl"> ./blog &gt; README.md
1643 </span></span><span class="line"><span class="cl">
1644 </span></span><span class="line"><span class="cl"><span class="nf">rss.xml</span><span class="o">:</span> <span class="n">blog</span> <span class="k">$(</span><span class="nv">MARKDOWN</span><span class="k">)</span>
1645 </span></span><span class="line"><span class="cl"> ./blog -rss &gt; rss.xml
1646 </span></span><span class="line"><span class="cl">
1647 </span></span><span class="line"><span class="cl"><span class="nf">.PHONY</span><span class="o">:</span> <span class="n">publish</span>
1648 </span></span><span class="line"><span class="cl"><span class="nf">publish</span><span class="o">:</span> <span class="n">blog</span>
1649 </span></span><span class="line"><span class="cl"> ./blog -publish
1650 </span></span><span class="line"><span class="cl">
1651 </span></span><span class="line"><span class="cl"><span class="nv">DAY</span> <span class="o">:=</span> <span class="k">$(</span>shell date<span class="k">)</span>
1652 </span></span><span class="line"><span class="cl"><span class="nv">_PARSED_DAY</span> <span class="o">:=</span> <span class="k">$(</span>shell date <span class="s1">&#39;+%Y-%m-%d&#39;</span> -d <span class="s1">&#39;$(DAY)&#39;</span><span class="k">)</span>
1653 </span></span><span class="line"><span class="cl"><span class="nf">.PHONY</span><span class="o">:</span> <span class="n">day</span>
1654 </span></span><span class="line"><span class="cl"><span class="nf">day</span><span class="o">:</span>
1655 </span></span><span class="line"><span class="cl"> mkdir -p <span class="s1">&#39;$(_PARSED_DAY)&#39;</span>
1656 </span></span><span class="line"><span class="cl">
1657 </span></span><span class="line"><span class="cl"><span class="nv">TITLE</span> <span class="o">=</span> <span class="k">$(</span>error TITLE is not defined<span class="k">)</span>
1658 </span></span><span class="line"><span class="cl"><span class="nf">.PHONY</span><span class="o">:</span> <span class="n">post</span>
1659 </span></span><span class="line"><span class="cl"><span class="nf">post</span><span class="o">:</span> <span class="n">blog</span> <span class="n">day</span>
1660 </span></span><span class="line"><span class="cl"> ./.scripts/gen-post.sh <span class="s1">&#39;$(_PARSED_DAY)&#39;</span> <span class="s1">&#39;$(TITLE)&#39;</span>
1661 </span></span><span class="line"><span class="cl">
1662 </span></span><span class="line"><span class="cl"><span class="nv">FILE</span> <span class="o">=</span> <span class="k">$(</span>error FILE is not defined<span class="k">)</span>
1663 </span></span><span class="line"><span class="cl"><span class="nf">.PHONY</span><span class="o">:</span> <span class="n">draft</span>
1664 </span></span><span class="line"><span class="cl"><span class="nf">draft</span><span class="o">:</span>
1665 </span></span><span class="line"><span class="cl"> <span class="nb">cd</span> <span class="s1">&#39;$(dir $(FILE))&#39;</span> <span class="o">&amp;&amp;</span> mv <span class="s1">&#39;$(notdir $(FILE))&#39;</span> <span class="s1">&#39;.$(notdir $(FILE))&#39;</span>
1666 </span></span><span class="line"><span class="cl">
1667 </span></span><span class="line"><span class="cl"><span class="nf">.PHONY</span><span class="o">:</span> <span class="n">undraft</span>
1668 </span></span><span class="line"><span class="cl"><span class="nf">undraft</span><span class="o">:</span>
1669 </span></span><span class="line"><span class="cl"> <span class="nb">cd</span> <span class="s1">&#39;$(dir $(FILE))&#39;</span> <span class="o">&amp;&amp;</span> mv <span class="s1">&#39;$(notdir $(FILE))&#39;</span> <span class="s1">&#39;$(patsubst .%,%,$(notdir $(FILE)))&#39;</span>
1670 </span></span><span class="line"><span class="cl">
1671 </span></span><span class="line"><span class="cl"><span class="nf">.PHONY</span><span class="o">:</span> <span class="n">clean</span>
1672 </span></span><span class="line"><span class="cl"><span class="nf">clean</span><span class="o">:</span>
1673 </span></span><span class="line"><span class="cl"> rm -rf blog
1674 </span></span></code></pre><p>For those unfamiliar with <code>Makefile</code>, a quick explanation on how it works from
1675 <a href="https://en.wikipedia.org/wiki/Make_(software)#Makefile">Wikipedia</a>:</p>
1676 <blockquote>
1677 <p>Each rule begins with a <em>dependency line</em> which consists of the rule's target
1678 name followed by a colon (:) and optionally a list of targets on which the
1679 rule's target depends, its prerequisites.</p>
1680 </blockquote>
1681 <p>So if we look for example at the <code>blog</code> binary, the dependencies are all the
1682 <code>.go</code> files and Go module files like <code>go.mod</code> and <code>go.sum</code>. We can make the
1683 <code>blog</code> binary by running:</p>
1684 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> make blog
1685 </span></span><span class="line"><span class="cl"><span class="go">go build
1686 </span></span></span></code></pre><p>One nice thing about <code>Makefile</code> is that they track if any of the source files
1687 has a newer timestamp than the target file, and only trigger the build again if
1688 there are changes, for example:</p>
1689 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> make blog
1690 </span></span><span class="line"><span class="cl"><span class="go">make: &#39;blog&#39; is up to date.
1691 </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="err">
1692 </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="gp">$</span> touch blog.go
1693 </span></span><span class="line"><span class="cl"><span class="err">
1694 </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="gp">$</span> make blog
1695 </span></span><span class="line"><span class="cl"><span class="go">go build
1696 </span></span></span></code></pre><p>But sometimes this property is undesirable. In those cases we can declare a
1697 target as <code>.PHONY</code>, that basically instructs <code>Makefile</code> to always make the
1698 target. One classic example is <code>clean</code> target, that removes build artifacts:</p>
1699 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> make clean
1700 </span></span><span class="line"><span class="cl"><span class="go">rm -rf blog
1701 </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="err">
1702 </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="gp">$</span> make clean
1703 </span></span><span class="line"><span class="cl"><span class="go">rm -rf blog
1704 </span></span></span></code></pre><p>By the way, it is better to declare a target as <code>.PHONY</code> than declaring
1705 dependencies incorrectly, especially in languages that has fast build times
1706 like e.g.: Go. The worst thing that can happen is something not being rebuild
1707 when it needs to. So my recomendation if you are writing your first <code>Makefile</code>
1708 is to just declare everything as <code>.PHONY</code>. You can always improve it later.</p>
1709 <p>One last basic concept that I want to explain about <code>Makefile</code> is the default
1710 target: it is the target that is run if you just run <code>make</code> without arguments
1711 inside the directory that contains a <code>Makefile</code>. The default target is
1712 generally the first target in the <code>Makefile</code>. It is common to have an <code>all</code>
1713 target (that is also marked as <code>.PHONY</code>) that has as dependencies all the
1714 targets that you want to build by default. In this particular case I declare
1715 the <code>README.md</code> and <code>rss.xml</code> files to be build by default, and they themselves
1716 depends in <code>blog</code> binary being build. So once I run <code>make</code> you get as result:</p>
1717 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> make
1718 </span></span><span class="line"><span class="cl"><span class="go">go build
1719 </span></span></span><span class="line"><span class="cl"><span class="go">./blog &gt; README.md
1720 </span></span></span><span class="line"><span class="cl"><span class="go">./blog -rss &gt; rss.xml
1721 </span></span></span></code></pre><p>And this result above highlights the first reason I think you should have a
1722 <code>Makefile</code> or something similar in your projects: you don't need to remember
1723 the exactly steps that you need to get things working. If I see one project of
1724 mine having a <code>Makefile</code>, I can be reasonably confident that I can get it
1725 working by just running <code>make</code>.</p>
1726 <p>But now let's focus in the other targets that I have in the <code>Makefile</code> that are
1727 not related to the build process but are there to help me manage my blog posts.
1728 Remember the rules I explained above? Maybe not, but it should be no problem,
1729 because:</p>
1730 <pre><code>$ make post TITLE=&quot;My new blog post&quot;
1731 mkdir -p &quot;2024-08-13&quot;
1732 ./.scripts/gen-post.sh &quot;2024-08-13&quot; &quot;My new blog post&quot;
1733 Creating file: 2024-08-13/02-my-new-blog-post.md
1734 <p>$ cat 2024-08-13/02-my-new-blog-post.md</p>
1735 <h1>My new blog post</h1>
1736 </code><p><code></code></p></pre><p></p>
1737 <p>This command, <code>make post</code>, is responsible for:</p>
1738 <ol>
1739 <li>Create a new directory for today, if it doesn't exist</li>
1740 <li>Run the
1741 <a href="https://github.com/thiagokokada/blog/blob/6a3b06970729f7650e5bee5fb0e1f9f2541ffea8/.scripts/gen-post.sh"><code>gen-post.sh</code></a>
1742 script, that:
1743 <ol>
1744 <li>Enumerates all posts from the day, so we can number the new post correctly
1745 <ul>
1746 <li>We already had this post planned for 2024-08-13, so the new post is 02</li>
1747 </ul>
1748 </li>
1749 <li>Slugify the title, so we can create each Markdown file with the correct
1750 filename</li>
1751 <li>Creates a new Markdown file with the title as a <code>h1</code> header</li>
1752 </ol>
1753 </li>
1754 </ol>
1755 <p>The steps above may or may not seen trivial, and for a while I was doing them
1756 manually. But not having to think what is the current date or if I already
1757 posted that day or what is the slug is for the title make (pun intended) my
1758 like much easier.</p>
1759 <p>Yes, the code is ugly. The way variables works in <code>Make</code> is that you can
1760 declare then inside the <code>Makefile</code>, but they can be overwritten in the terminal
1761 if you pass them. I used this to allow <code>make post</code> to also work for future
1762 posts:</p>
1763 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> make post <span class="nv">TITLE</span><span class="o">=</span><span class="s2">&#34;Another new blog post&#34;</span> <span class="nv">DAY</span><span class="o">=</span>2024-12-12
1764 </span></span><span class="line"><span class="cl"><span class="go">mkdir -p &#34;2024-12-12&#34;
1765 </span></span></span><span class="line"><span class="cl"><span class="go">./.scripts/gen-post.sh &#34;2024-12-12&#34; &#34;Another new blog post&#34;
1766 </span></span></span><span class="line"><span class="cl"><span class="go">Creating file: 2024-12-12/01-another-new-blog-post.md
1767 </span></span></span></code></pre><p>So in the above case, <code>DAY</code> is filled with the value passed in the terminal
1768 instead of default (that would be the current day), and <code>_PARSED_DAY</code> is the
1769 day we use to actually create the directory. We can actually pass any date
1770 format recognised by
1771 <a href="https://www.gnu.org/software/coreutils/manual/html_node/Examples-of-date.html"><code>date</code></a>,
1772 not just <code>YYYY-MM-DD</code>.</p>
1773 <p>I have 2 other phony targets that I want to talk, <code>draft</code> and <code>undraft</code>. They
1774 expect a <code>FILE</code> to be passed, and I use them to either hide or unhide a file:</p>
1775 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> make draft <span class="nv">FILE</span><span class="o">=</span>2024-12-12/01-another-new-blog-post.md
1776 </span></span><span class="line"><span class="cl"><span class="go">mv &#34;2024-12-12/01-another-new-blog-post.md&#34; &#34;2024-12-12/.01-another-new-blog-post.md&#34;
1777 </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="err">
1778 </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="gp">$</span> make undraft <span class="nv">FILE</span><span class="o">=</span>2024-12-12/.01-another-new-blog-post.md
1779 </span></span><span class="line"><span class="cl"><span class="go">mv &#34;2024-12-12/.01-another-new-blog-post.md&#34; &#34;2024-12-12/01-another-new-blog-post.md&#34;
1780 </span></span></span></code></pre><p>Why? Because hidden files are <a href="https://github.com/thiagokokada/blog/blob/894a388c61ca3a38dfc9d4cbe88dc684fd964bb7/blog.go#L101-L104">explicit
1781 ignored</a>
1782 during my directory parser to mean they're a draft post and not ready to be
1783 published. And the reason I created those targets is because I was tired of
1784 trying to hide or unhide a file manually.</p>
1785 <p>So that's it, for the same reason you <a href="https://kokada.dev/blog/quick-bits-why-you-should-automate-everything">should probably automate
1786 everything</a>,
1787 you also need to have some way to automate your tasks. <code>Makefile</code> is one way to
1788 do it, maybe not the best way to do it, but it works and it is available
1789 anywhere.</p></description><pubDate>Tue, 13 Aug 2024 00:00:00 +0000</pubDate><guid>https://kokada.dev/blog/abusing-makefiles-for-fun-and-profit/</guid></item><item><title>Things I don't like in my Chromebook Duet 3</title><link>https://kokada.dev/blog/things-i-dont-like-in-my-chromebook-duet-3/</link><description><p>So this is kind of a continuation from my <a href="https://kokada.dev/blog/my-favorite-device-is-a-chromebook">previous
1790 post</a> talking why
1791 my favorite device is a Chromebook. In this post I want to talk about what
1792 makes me this device unhappy, and comment about things that if changed would
1793 make it a much better device.</p>
1794 <p>But before talking about the negative aspects, let me talk about a positive
1795 aspect that I just briefly talked in the previous post: the screen. It is a
1796 HiDPI screen (2000x1200 resolution in 10.95''), that is unexpected bright (400
1797 nits according to the
1798 <a href="https://www.lenovo.com/us/en/p/laptops/lenovo/lenovo-edu-chromebooks/ideapad-duet-3-chromebook-11-inch,-qlc/len101i0034">specs</a>).
1799 It is difficult to find laptops at the same price with a screen that good. At
1800 10.95'' in its default resolution I find it too small (like 1250x750), but I
1801 find the font size acceptable at 115% scale (1087x652). Yes, it result in a
1802 small workspace, but this is not a big issue for what I do in this device. It
1803 is also only 60Hz, but I thought I would miss high refresh rate more than I
1804 actually miss in this device.</p>
1805 <p>Update: I forgot to say one thing about the screen: it scratches really easy. I
1806 got my screen scratched after the first day of usage, and considering the price
1807 I don't think the screen has a hardened glass. I bought a cheap glass screen
1808 protector and this did the trick though, even hiding the previous scratch, and
1809 I have zero issues with the screen now.</p>
1810 <p>Now the first aspect that I don't like: the speakers. They sound tiny and even
1811 at maximum volume it is not really loud. The speakers is the only reason why I
1812 still keep my <a href="https://www.gsmarena.com/xiaomi_pad_5-11042.php">Xiaomi Pad 5</a>,
1813 because I like to watch animes/videos before sleep and having good speakers is
1814 a must.</p>
1815 <p>The keyboard has that issue that I mentioned in the previous post: sometimes
1816 the key get stuck, and I get duplicated characters. But it also has some minor
1817 issues that I didn't talked about: the first one is the UK layout that has some
1818 extra keys that I have no use for, but this also makes the keys that I use
1819 smaller. Very much a &quot;me&quot; problem, since if I had got a US version I wouldn't
1820 have those issues, but an issue nonetheless that gets worse considering how
1821 small the keyboard is. I am actually suprised how fast I can type considering
1822 how many issues this keyboard has, so maybe this is a testament that this
1823 keyboard is not actually that bad.</p>
1824 <p>The other keyboard issue is a problem that affects all Chromebooks: its custom
1825 layout. Google replaced a few keys like Fn keys with shortcuts and replaced the
1826 Caps Lock with a
1827 <a href="https://chromeunboxed.com/chromebook-launcher-now-everything-button">&quot;Everything&quot;</a>
1828 key (that is similar to the Windows Key), while removing Windows Key from its
1829 place. I actually have less issue with this than I initially though: I don't
1830 care too much about Fn keys (except when using IntelliJ, but that is something
1831 that I only use at <code>$CURRENT_JOB</code>), and ChromeOS is surprisingly powerful in
1832 its customisation, allowing you to swap key functionality. I remap Everything
1833 key with Esc, and Esc for the Everything key, and I can get productive in my
1834 <code>neovim</code> setup.</p>
1835 <p>And finally, let me talk more about the performance: yes, it is bad, but
1836 bearable once you get used to. The issue is both the CPU and IO. While the CPU,
1837 a <a href="https://www.qualcomm.com/products/mobile/snapdragon/laptops-and-tablets/snapdragon-mobile-compute-platforms/snapdragon-7c-gen-2-compute-platform">Snapdragon 7c Gen
1838 2</a>
1839 is octa-core, it has only 2 high performance CPU cores vs 6 low performance
1840 ones (2xARM Cortex A76 vs 6xARM Cortex A55). If it was something like 4x4, it
1841 would be much better. The fact that the cores are old doesn't help either.</p>
1842 <p>But the worst part is the IO. Not only it uses a eMMC module, it is slow:</p>
1843 <p><a href="https://github.com/thiagokokada/blog/raw/main/posts/2024-08-12/Screenshot_2024-08-12_20.50.42.png"><img src="https://github.com/thiagokokada/blog/raw/main/posts/2024-08-12/Screenshot_2024-08-12_20.50.42.png" alt="CPDT Benchmark results from Chromebook Duet 3."></a></p>
1844 <p>I don't know how much more expensive it would be to put a
1845 <a href="https://en.wikipedia.org/wiki/Universal_Flash_Storage">UFS</a> instead of eMMC in
1846 this device, but this is probably the choice that would most increase
1847 performance in this device, especially considering how aggressive Chromebooks
1848 use (z)swap.</p>
1849 <p>Update 2: I forgot to talk about the fact that the exterior of the device is
1850 covered in cloth. I thought I would hate this at first, but nowadays I kind
1851 like it. And it is also nice that it will never get scratched, I don't care too
1852 much about the exterior of this device and it is the only device that I have at
1853 home that doesn't have additional protection (except the screen protector
1854 mentioned above).</p></description><pubDate>Mon, 12 Aug 2024 00:00:00 +0000</pubDate><guid>https://kokada.dev/blog/things-i-dont-like-in-my-chromebook-duet-3/</guid></item><item><title>Building static binaries in Nix</title><link>https://kokada.dev/blog/building-static-binaries-in-nix/</link><description><p>I recently had laser eye surgery for my near-sightedness, and while evaluating
1855 if I could have the surgery I discovered that I was suffering from dry eyes.
1856 Thanks to this, my ophthalmologist recommended that every 20 minutes while
1857 using screens, I look somewhere 20 feet away for 20 seconds, a technique known
1858 as <a href="https://www.healthline.com/health/eye-health/20-20-20-rule">20-20-20 rule</a>.</p>
1859 <p>I had issues following this rule because I never remembered to do the pauses. I
1860 initially tried to setup an alarm, but this became annoying, so I decided to
1861 try to find a program. I found
1862 <a href="https://tonyh4156.github.io/20-20-20.github.io/">this</a> one for macOS that
1863 worked fine, but it bothered me that it was compiled for x86_64 while I was
1864 using a M1 MacBook Pro at the time, and also I needed something that worked in
1865 Linux.</p>
1866 <p>Since I never found a good multi-platform alternative, I decided to write my
1867 own. This became
1868 <a href="https://github.com/thiagokokada/twenty-twenty-twenty/">twenty-twenty-twenty</a>,
1869 the first <a href="https://kokada.dev/blog/go-a-reasonable-good-language">Go</a> program
1870 that I ever wrote. I wrote it in Go because I wanted to learn the language, but
1871 also because Go made it easy to build static binaries. And the first version I
1872 could build static binaries without issues because I was using
1873 <a href="https://github.com/gen2brain/beeep">beeep</a>, that uses pure Go code in all
1874 supported platforms. However, it also meant that the notifications in macOS
1875 looked ugly, since it used
1876 <a href="https://github.com/gen2brain/beeep/blob/master/beep_darwin.go#L20">osascript</a>.</p>
1877 <p>I wanted better integration with macOS, so this meant switching libraries.
1878 After searching for a while, the
1879 <a href="https://pkg.go.dev/gioui.org/x/notify">notify</a> library from
1880 <a href="https://gioui.org/">GioUI</a> is the one that seemed to work better. It
1881 implements notification in macOS using its native framework, so it works much
1882 better, but sadly it meant losing static binaries because it depends in CGO.</p>
1883 <p>Not a big loss initially, because I am only depending in Foundation inside
1884 macOS (that should always be available), and in Linux I could still statically
1885 compile. However I eventually added more features like sound
1886 (via <a href="https://github.com/gopxl/beep">beep</a>) and tray icon (via
1887 <a href="https://github.com/fyne-io/systray">systray</a>), that meant I needed CGO in both
1888 macOS and Linux.</p>
1889 <p>Losing static binaries in Linux is a much bigger deal, since Linux is a moving
1890 target. The general recommendation for building CGO binaries statically is
1891 using
1892 <a href="https://eli.thegreenplace.net/2024/building-static-binaries-with-go-on-linux/">musl</a>,
1893 but this also means building all dependencies that we need using musl (e.g.:
1894 <a href="https://github.com/ebitengine/oto?tab=readme-ov-file#linux"><code>ALSA</code></a> for
1895 [beep/oto]). This generally means pain, but Nix makes it easy.</p>
1896 <p>Let's start by creating a <a href="https://github.com/thiagokokada/twenty-twenty-twenty/blob/main/twenty-twenty-twenty.nix">Nix
1897 file</a>
1898 that builds our Go module (simplified below for brevity):</p>
1899 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span> <span class="n">lib</span>
1900 </span></span><span class="line"><span class="cl"><span class="o">,</span> <span class="n">stdenv</span>
1901 </span></span><span class="line"><span class="cl"><span class="o">,</span> <span class="n">alsa-lib</span>
1902 </span></span><span class="line"><span class="cl"><span class="o">,</span> <span class="n">buildGoModule</span>
1903 </span></span><span class="line"><span class="cl"><span class="o">,</span> <span class="n">pkg-config</span>
1904 </span></span><span class="line"><span class="cl"><span class="o">,</span> <span class="n">withStatic</span> <span class="o">?</span> <span class="no">false</span>
1905 </span></span><span class="line"><span class="cl"><span class="p">}:</span>
1906 </span></span><span class="line"><span class="cl">
1907 </span></span><span class="line"><span class="cl"><span class="n">buildGoModule</span> <span class="p">{</span>
1908 </span></span><span class="line"><span class="cl"> <span class="n">pname</span> <span class="o">=</span> <span class="s2">&#34;twenty-twenty-twenty&#34;</span><span class="p">;</span>
1909 </span></span><span class="line"><span class="cl"> <span class="n">version</span> <span class="o">=</span> <span class="s2">&#34;1.0.0&#34;</span><span class="p">;</span>
1910 </span></span><span class="line"><span class="cl"> <span class="n">src</span> <span class="o">=</span> <span class="n">lib</span><span class="o">.</span><span class="n">cleanSource</span> <span class="sr">./.</span><span class="p">;</span>
1911 </span></span><span class="line"><span class="cl"> <span class="n">vendorHash</span> <span class="o">=</span> <span class="s2">&#34;sha256-NzDhpJRogIfL2IYoqAUHoPh/ZdNnvnhEQ+kn8A+ZyBw=&#34;</span><span class="p">;</span>
1912 </span></span><span class="line"><span class="cl">
1913 </span></span><span class="line"><span class="cl"> <span class="n">CGO_ENABLED</span> <span class="o">=</span> <span class="s2">&#34;1&#34;</span><span class="p">;</span>
1914 </span></span><span class="line"><span class="cl">
1915 </span></span><span class="line"><span class="cl"> <span class="n">nativeBuildInputs</span> <span class="o">=</span> <span class="n">lib</span><span class="o">.</span><span class="n">optionals</span> <span class="p">(</span><span class="n">stdenv</span><span class="o">.</span><span class="n">hostPlatform</span><span class="o">.</span><span class="n">isLinux</span><span class="p">)</span> <span class="p">[</span>
1916 </span></span><span class="line"><span class="cl"> <span class="n">pkg-config</span>
1917 </span></span><span class="line"><span class="cl"> <span class="p">];</span>
1918 </span></span><span class="line"><span class="cl">
1919 </span></span><span class="line"><span class="cl"> <span class="n">buildInputs</span> <span class="o">=</span> <span class="n">lib</span><span class="o">.</span><span class="n">optionals</span> <span class="p">(</span><span class="n">stdenv</span><span class="o">.</span><span class="n">hostPlatform</span><span class="o">.</span><span class="n">isLinux</span><span class="p">)</span> <span class="p">[</span>
1920 </span></span><span class="line"><span class="cl"> <span class="n">alsa-lib</span>
1921 </span></span><span class="line"><span class="cl"> <span class="p">];</span>
1922 </span></span><span class="line"><span class="cl">
1923 </span></span><span class="line"><span class="cl"> <span class="n">ldflags</span> <span class="o">=</span> <span class="p">[</span> <span class="s2">&#34;-X=main.version=</span><span class="si">${</span><span class="n">version</span><span class="si">}</span><span class="s2">&#34;</span> <span class="s2">&#34;-s&#34;</span> <span class="s2">&#34;-w&#34;</span> <span class="p">]</span>
1924 </span></span><span class="line"><span class="cl"> <span class="o">++</span> <span class="n">lib</span><span class="o">.</span><span class="n">optionals</span> <span class="n">withStatic</span> <span class="p">[</span> <span class="s2">&#34;-linkmode external&#34;</span> <span class="s1">&#39;&#39;-extldflags &#34;-static&#34;&#39;&#39;</span> <span class="p">];</span>
1925 </span></span><span class="line"><span class="cl">
1926 </span></span><span class="line"><span class="cl"> <span class="n">meta</span> <span class="o">=</span> <span class="k">with</span> <span class="n">lib</span><span class="p">;</span> <span class="p">{</span>
1927 </span></span><span class="line"><span class="cl"> <span class="n">description</span> <span class="o">=</span> <span class="s2">&#34;Alerts every 20 minutes to look something at 20 feet away for 20 seconds&#34;</span><span class="p">;</span>
1928 </span></span><span class="line"><span class="cl"> <span class="n">homepage</span> <span class="o">=</span> <span class="s2">&#34;https://github.com/thiagokokada/twenty-twenty-twenty&#34;</span><span class="p">;</span>
1929 </span></span><span class="line"><span class="cl"> <span class="n">license</span> <span class="o">=</span> <span class="n">licenses</span><span class="o">.</span><span class="n">mit</span><span class="p">;</span>
1930 </span></span><span class="line"><span class="cl"> <span class="n">mainProgram</span> <span class="o">=</span> <span class="s2">&#34;twenty-twenty-twenty&#34;</span><span class="p">;</span>
1931 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
1932 </span></span><span class="line"><span class="cl"><span class="p">}</span>
1933 </span></span></code></pre><p>And we can build it with the following <code>flake.nix</code>:</p>
1934 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
1935 </span></span><span class="line"><span class="cl"> <span class="n">description</span> <span class="o">=</span> <span class="s2">&#34;twenty-twenty-twenty&#34;</span><span class="p">;</span>
1936 </span></span><span class="line"><span class="cl">
1937 </span></span><span class="line"><span class="cl"> <span class="n">inputs</span> <span class="o">=</span> <span class="p">{</span>
1938 </span></span><span class="line"><span class="cl"> <span class="n">nixpkgs</span><span class="o">.</span><span class="n">url</span> <span class="o">=</span> <span class="s2">&#34;nixpkgs/nixpkgs-unstable&#34;</span><span class="p">;</span>
1939 </span></span><span class="line"><span class="cl"> <span class="n">flake-compat</span><span class="o">.</span><span class="n">url</span> <span class="o">=</span> <span class="s2">&#34;github:edolstra/flake-compat&#34;</span><span class="p">;</span>
1940 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
1941 </span></span><span class="line"><span class="cl">
1942 </span></span><span class="line"><span class="cl"> <span class="n">outputs</span> <span class="o">=</span> <span class="p">{</span> <span class="n">self</span><span class="o">,</span> <span class="n">nixpkgs</span><span class="o">,</span> <span class="o">...</span> <span class="p">}:</span>
1943 </span></span><span class="line"><span class="cl"> <span class="k">let</span>
1944 </span></span><span class="line"><span class="cl"> <span class="n">supportedSystems</span> <span class="o">=</span> <span class="p">[</span> <span class="s2">&#34;x86_64-linux&#34;</span> <span class="s2">&#34;aarch64-linux&#34;</span> <span class="p">];</span>
1945 </span></span><span class="line"><span class="cl">
1946 </span></span><span class="line"><span class="cl"> <span class="c1"># Helper function to generate an attrset &#39;{ x86_64-linux = f &#34;x86_64-linux&#34;; ... }&#39;.</span>
1947 </span></span><span class="line"><span class="cl"> <span class="n">forAllSystems</span> <span class="o">=</span> <span class="n">nixpkgs</span><span class="o">.</span><span class="n">lib</span><span class="o">.</span><span class="n">genAttrs</span> <span class="n">supportedSystems</span><span class="p">;</span>
1948 </span></span><span class="line"><span class="cl">
1949 </span></span><span class="line"><span class="cl"> <span class="c1"># Nixpkgs instantiated for supported system types.</span>
1950 </span></span><span class="line"><span class="cl"> <span class="n">nixpkgsFor</span> <span class="o">=</span> <span class="n">forAllSystems</span> <span class="p">(</span><span class="n">system</span><span class="p">:</span> <span class="kn">import</span> <span class="n">nixpkgs</span> <span class="p">{</span> <span class="k">inherit</span> <span class="n">system</span><span class="p">;</span> <span class="p">});</span>
1951 </span></span><span class="line"><span class="cl"> <span class="k">in</span>
1952 </span></span><span class="line"><span class="cl"> <span class="p">{</span>
1953 </span></span><span class="line"><span class="cl"> <span class="n">packages</span> <span class="o">=</span> <span class="n">forAllSystems</span> <span class="p">(</span><span class="n">system</span><span class="p">:</span>
1954 </span></span><span class="line"><span class="cl"> <span class="k">let</span> <span class="n">pkgs</span> <span class="o">=</span> <span class="n">nixpkgsFor</span><span class="o">.</span><span class="si">${</span><span class="n">system</span><span class="si">}</span><span class="p">;</span>
1955 </span></span><span class="line"><span class="cl"> <span class="k">in</span> <span class="p">{</span>
1956 </span></span><span class="line"><span class="cl"> <span class="n">default</span> <span class="o">=</span> <span class="n">self</span><span class="o">.</span><span class="n">packages</span><span class="o">.</span><span class="si">${</span><span class="n">system</span><span class="si">}</span><span class="o">.</span><span class="n">twenty-twenty-twenty</span><span class="p">;</span>
1957 </span></span><span class="line"><span class="cl"> <span class="n">twenty-twenty-twenty</span> <span class="o">=</span> <span class="n">pkgs</span><span class="o">.</span><span class="n">callPackage</span> <span class="sr">./twenty-twenty-twenty.nix</span> <span class="p">{</span> <span class="p">};</span>
1958 </span></span><span class="line"><span class="cl"> <span class="n">twenty-twenty-twenty-static</span> <span class="o">=</span> <span class="n">pkgs</span><span class="o">.</span><span class="n">pkgsStatic</span><span class="o">.</span><span class="n">callPackage</span> <span class="sr">./twenty-twenty-twenty.nix</span> <span class="p">{</span>
1959 </span></span><span class="line"><span class="cl"> <span class="n">withStatic</span> <span class="o">=</span> <span class="no">true</span><span class="p">;</span>
1960 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
1961 </span></span><span class="line"><span class="cl"> <span class="p">});</span>
1962 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
1963 </span></span><span class="line"><span class="cl"><span class="p">}</span>
1964 </span></span></code></pre><p>I think this shows how powerful Nix is: the only difference between the normal
1965 build and a static build the usage of <code>pkgs.pkgsStatic</code> instead of <code>pkgs</code>. This
1966 automatically builds all packages statically with <code>musl</code>. Also we need pass
1967 some <a href="https://honnef.co/articles/statically-compiled-go-programs-always-even-with-cgo-using-musl/">extra
1968 flags</a>
1969 to the Go compiler (i.e.: <code>-linkmode external -extldflags &quot;-static&quot;</code>), but this
1970 is a requirement from Go.</p>
1971 <p>So, does it work? Let's test:</p>
1972 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> nix build .#twenty-twenty-twenty-static
1973 </span></span><span class="line"><span class="cl"><span class="err">
1974 </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="gp">$</span> file result/bin/twenty-twenty-twenty
1975 </span></span><span class="line"><span class="cl"><span class="go">result/bin/twenty-twenty-twenty: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, stripped
1976 </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="err">
1977 </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="gp">$</span> ./result/bin/twenty-twenty-twenty
1978 </span></span><span class="line"><span class="cl"><span class="go">ALSA lib pcm.c:2712:(snd_pcm_open_conf) Either /nix/store/dhn51w2km4fyf9ivi00rz03qs8q4mpng-pipewire-1.2.1/lib/alsa-lib/libasound_module_pcm_pipewire.so cannot be opened or _snd_pcm_pipewire_open was not defined inside
1979 </span></span></span><span class="line"><span class="cl"><span class="go">ALSA lib pcm.c:2712:(snd_pcm_open_conf) Either /nix/store/ly9d7llymzjyf6gi1455qzayqipk2kab-pipewire-1.2.1/lib/alsa-lib/libasound_module_pcm_pipewire.so cannot be opened or _snd_pcm_pipewire_open was not defined inside
1980 </span></span></span><span class="line"><span class="cl"><span class="go">ALSA lib pcm.c:2712:(snd_pcm_open_conf) Either /nix/store/dhn51w2km4fyf9ivi00rz03qs8q4mpng-pipewire-1.2.1/lib/alsa-lib/libasound_module_pcm_pipewire.so cannot be opened or _snd_pcm_pipewire_open was not defined inside
1981 </span></span></span><span class="line"><span class="cl"><span class="go">ALSA lib pcm.c:2712:(snd_pcm_open_conf) Either /nix/store/ly9d7llymzjyf6gi1455qzayqipk2kab-pipewire-1.2.1/lib/alsa-lib/libasound_module_pcm_pipewire.so cannot be opened or _snd_pcm_pipewire_open was not defined inside
1982 </span></span></span><span class="line"><span class="cl"><span class="go">ALSA lib pcm.c:2712:(snd_pcm_open_conf) Either /nix/store/dhn51w2km4fyf9ivi00rz03qs8q4mpng-pipewire-1.2.1/lib/alsa-lib/libasound_module_pcm_pipewire.so cannot be opened or _snd_pcm_pipewire_open was not defined inside
1983 </span></span></span><span class="line"><span class="cl"><span class="go">ALSA lib pcm.c:2712:(snd_pcm_open_conf) Either /nix/store/ly9d7llymzjyf6gi1455qzayqipk2kab-pipewire-1.2.1/lib/alsa-lib/libasound_module_pcm_pipewire.so cannot be opened or _snd_pcm_pipewire_open was not defined inside
1984 </span></span></span><span class="line"><span class="cl"><span class="go">2024-08-11T19:26:33+01:00 INFO Running twenty-twenty-twenty every 20.0 minute(s), with 20 second(s) duration and sound set to true
1985 </span></span></span></code></pre><p>There are some warns and sadly the sound doesn't work. I think the issue is
1986 related because of my usage of PipeWire and the binary may work in a pure ALSA
1987 system, but I don't have access to one. Maybe adding <code>pipewire</code> to
1988 <code>buildInputs</code> would fix this issue, but I can't get <code>pipewire</code> to be compiled
1989 statically (because of its dependencies). I think this is a good show how easy
1990 it is to statically compilation is in Nix, but also how complex static binaries
1991 are to get correctly.</p>
1992 <p>Bonus points for
1993 <a href="https://nix.dev/tutorials/cross-compilation.html">cross-compilation</a>. We can
1994 easily cross-compile by using <code>pkgsCross</code>:</p>
1995 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
1996 </span></span><span class="line"><span class="cl"> <span class="c1"># ...</span>
1997 </span></span><span class="line"><span class="cl"> <span class="n">outputs</span> <span class="o">=</span> <span class="p">{</span> <span class="n">self</span><span class="o">,</span> <span class="n">nixpkgs</span><span class="o">,</span> <span class="o">...</span> <span class="p">}:</span>
1998 </span></span><span class="line"><span class="cl"> <span class="k">let</span>
1999 </span></span><span class="line"><span class="cl"> <span class="c1"># ...</span>
2000 </span></span><span class="line"><span class="cl"> <span class="k">in</span>
2001 </span></span><span class="line"><span class="cl"> <span class="p">{</span>
2002 </span></span><span class="line"><span class="cl"> <span class="n">packages</span> <span class="o">=</span> <span class="n">forAllSystems</span> <span class="p">(</span><span class="n">system</span><span class="p">:</span>
2003 </span></span><span class="line"><span class="cl"> <span class="k">let</span> <span class="n">pkgs</span> <span class="o">=</span> <span class="n">nixpkgsFor</span><span class="o">.</span><span class="si">${</span><span class="n">system</span><span class="si">}</span><span class="p">;</span>
2004 </span></span><span class="line"><span class="cl"> <span class="k">in</span> <span class="p">{</span>
2005 </span></span><span class="line"><span class="cl"> <span class="n">twenty-twenty-twenty-static-aarch64</span> <span class="o">=</span> <span class="n">pkgs</span><span class="o">.</span><span class="n">pkgsCross</span><span class="o">.</span><span class="n">aarch64-multiplatform</span><span class="o">.</span><span class="n">pkgsStatic</span><span class="o">.</span><span class="n">callPackage</span> <span class="sr">./twenty-twenty-twenty.nix</span> <span class="p">{</span>
2006 </span></span><span class="line"><span class="cl"> <span class="n">withStatic</span> <span class="o">=</span> <span class="no">true</span><span class="p">;</span>
2007 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
2008 </span></span><span class="line"><span class="cl"> <span class="p">});</span>
2009 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
2010 </span></span><span class="line"><span class="cl"><span class="p">}</span>
2011 </span></span></code></pre><p>The idea of <code>pkgsCross</code> is to select a target platform (e.g.:
2012 <code>aarch64-multiplatform</code>) and use it as any other <code>pkgs</code>. We can even chain
2013 <code>pkgsStatic</code> to statically cross compile binaries:</p>
2014 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> nix build .#twenty-twenty-twenty-static-aarch64
2015 </span></span><span class="line"><span class="cl"><span class="err">
2016 </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="gp">$</span> file result/bin/twenty-twenty-twenty
2017 </span></span><span class="line"><span class="cl"><span class="go">result/bin/twenty-twenty-twenty: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, stripped
2018 </span></span></span></code></pre></description><pubDate>Sun, 11 Aug 2024 00:00:00 +0000</pubDate><guid>https://kokada.dev/blog/building-static-binaries-in-nix/</guid></item><item><title>Meta: are quick bits really quick?</title><link>https://kokada.dev/blog/meta-are-quick-bits-really-quick/</link><description><p>When I wrote my first <a href="https://kokada.dev/blog/quick-bits-nix-shell-is-cursed">quick
2019 bits</a> post in this
2020 blog, I gave that title prefix without much thought: I knew it was supposed to
2021 be a quicker post than my previous one, but I never thought too deeply about
2022 it. But after:</p>
2023 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> ls -lah **/*quick-bits*.md <span class="p">|</span> wc -l
2024 </span></span><span class="line"><span class="cl"><span class="go">4
2025 </span></span></span></code></pre><p>Well, 4 blog posts starting with the same prefix, I was curious: are those
2026 quick bits really quick, or at least quicker? Let's see:</p>
2027 <pre><code>$ wc -w **/*.md
2028 1107 2024-07-26/01-writing-nixos-tests-for-fun-and-profit.md
2029 1220 2024-07-26/02-using-github-as-a-bad-blog-platform.md
2030 286 2024-07-27/01-quick-bits-nix-shell-is-cursed.md
2031 387 2024-07-29/01-quick-bits-why-you-should-automate-everything.md
2032 1060 2024-07-29/02-go-a-reasonable-good-language.md
2033 1380 2024-07-30/01-first-impressions-fpgbc.md
2034 1238 2024-07-31/01-generating-yaml-files-with-nix.md
2035 2308 2024-08-01/01-troubleshooting-zsh-lag-and-solutions-with-nix.md
2036 504 2024-08-01/02-quick-bits-realise-nix-symlinks.md
2037 834 2024-08-04/01-make-nixd-modules-completion-work-anywhere-with-flakes.md
2038 1147 2024-08-05/01-my-favorite-device-is-a-chromebook.md
2039 394 2024-08-07/01-quick-bits-is-crostini-a-microvm.md
2040 120 README.md
2041 11985 total
2042 </code></pre>
2043 <p>While using <code>wc</code> is probably not the best way to measure word count (especially
2044 in this blog, since I tend to write lots of code snippets), I think this at
2045 least it gives me a good insight: yes, quick bits are quicker, and they're
2046 basically posts with a soft limit around 500 words. So expect in future this
2047 limit to be used.</p>
2048 <p>By the way, at:</p>
2049 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> wc -w 2024-08-07/02-meta-are-quick-bits-really-quick.md
2050 </span></span><span class="line"><span class="cl"><span class="go">220 2024-08-07/02-meta-are-quick-bits-really-quick.md
2051 </span></span></span></code></pre><p>This post is also technically a quick bits post, but &quot;quick bits meta&quot; would be
2052 too much. And yes, that last block of code is also meta ;).</p></description><pubDate>Wed, 07 Aug 2024 00:00:00 +0000</pubDate><guid>https://kokada.dev/blog/meta-are-quick-bits-really-quick/</guid></item><item><title>Quick bits: is Crostini a micro VM?</title><link>https://kokada.dev/blog/quick-bits-is-crostini-a-micro-vm/</link><description><p><strong>Disclaimer</strong>: I am not a Virtual Machine specialist, this post is mainly the
2053 conclusion I got after some searching.</p>
2054 <p>Someone asked me in my <a href="https://kokada.dev/blog/my-favorite-device-is-a-chromebook">previous
2055 post</a> about my
2056 Chromebook if Crostini could be considered a micro VM. This was a interesting
2057 question, so I decided to do another post.</p>
2058 <p>To start, it is really difficult to get a good definition of what a &quot;micro VM&quot;
2059 is. <a href="https://firecracker-microvm.github.io/">Firecracker</a> defines itself as a
2060 micro VM, and describes itself in its website as:</p>
2061 <blockquote>
2062 <p>Firecracker is a virtual machine monitor (VMM) that uses the Linux
2063 Kernel-based Virtual Machine (KVM) to create and manage microVMs. Firecracker
2064 has a minimalist design. It excludes unnecessary devices and guest
2065 functionality to reduce the memory footprint and attack surface area of each
2066 microVM. This improves security, decreases the startup time, and increases
2067 hardware utilization.</p>
2068 </blockquote>
2069 <p>Now looking at Crostini, its heart is a VMM called
2070 <a href="https://crosvm.dev/">crosvm</a>. It is described in its
2071 <a href="https://chromium.googlesource.com/chromiumos/platform/crosvm/+/HEAD/README.md">README</a>
2072 as:</p>
2073 <blockquote>
2074 <p>crosvm is a virtual machine monitor (VMM) based on Linux’s KVM hypervisor,
2075 with a focus on simplicity, security, and speed. crosvm is intended to run
2076 Linux guests, originally as a security boundary for running native
2077 applications on the ChromeOS platform. Compared to QEMU, crosvm doesn’t
2078 emulate architectures or real hardware, instead concentrating on
2079 paravirtualized devices, such as the virtio standard.</p>
2080 </blockquote>
2081 <p>Similar descriptions right? Actually Firecracker website says it &quot;started from
2082 Chromium OS's Virtual Machine Monitor, crosvm, an open source VMM written in
2083 Rust&quot;. So I would say it is safe to say crosvm itself is a micro VM.</p>
2084 <p>But
2085 <a href="https://www.chromium.org/chromium-os/developer-library/guides/containers/containers-and-vms/">Crostini</a>
2086 itself is a combination of virtualization AND containerization. Basically
2087 inside the VM it runs a Linux kernel and <a href="https://linuxcontainers.org/">LXC</a>,
2088 that can start arbitrary containers inside it. From the Crostini documentation
2089 this choice seems to be to keep startup times down, and also to increase
2090 security (e.g.: in case of a security issue inside the container).</p>
2091 <p>This is definitely an interesting choice, since containers allow the overhead
2092 of each distro that you run inside Crostini to be low, and the main VM itself
2093 (called
2094 <a href="https://chromium.googlesource.com/chromiumos/overlays/board-overlays/+/HEAD/project-termina/">Termina</a>)
2095 should have low overhead too thanks to crosvm.</p>
2096 <p>By the way, if you want to learn more how &quot;devices&quot; works inside a micro VM
2097 like crosvm, I recommend <a href="https://prilik.com/blog/post/crosvm-paravirt/">this blog
2098 post</a> talking about
2099 paravirtualized devices in crosvm.</p></description><pubDate>Wed, 07 Aug 2024 00:00:00 +0000</pubDate><guid>https://kokada.dev/blog/quick-bits-is-crostini-a-micro-vm/</guid></item><item><title>My favorite device is a Chromebook</title><link>https://kokada.dev/blog/my-favorite-device-is-a-chromebook/</link><description><p>Most of the posts in this blog (including this one) and most of I would call
2100 &quot;personal computing&quot; that I do nowadays is mostly done in one of the most
2101 unremarkable devices that I own: a cheap <a href="https://www.lenovo.com/us/en/p/laptops/lenovo/lenovo-edu-chromebooks/ideapad-duet-3-chromebook-11-inch,-qlc/len101i0034">Chromebook Duet
2102 3</a>,
2103 that I bought for around EUR300. I was thinking why, because it is woefully
2104 underpowered: a <a href="https://www.anandtech.com/show/16696/qualcomm-announces-snapdragon-7c-gen-2-entrylevel-pc-and-chromebook-refresh">Snapdragon 7c Gen
2105 2</a>,
2106 a CPU that was already considered slow 3 years ago, coupled with an eMMC for
2107 storage, that is not much faster than a HDD. At least I have the 8GB RAM
2108 version instead of the 4GB one.</p>
2109 <p>It is a hybrid device, one that can be used as either a tablet or laptop, but
2110 it is compromised experience in both cases: as a tablet, it lacks the better
2111 touch optimised interface from iOS or Android; as a laptop, you have to depend
2112 on the stand to adjust the screen, and the detachable keyboard is worse than
2113 any laptop I have ever owned: getting keys stucked and characters being
2114 duplicated as a result is a common occurence. It is not so bad that I can't get
2115 things done though. About the trackpad: its biggest quality is that I never
2116 feel the need to use the touchscreen in laptop mode, that is to say that it is
2117 acceptable. Just crank up the pointer speed in ChromeOS settings, otherwise you
2118 never get anywhere since the trackpad is so small. There is also an active
2119 stylus, that helped me sometimes when I needed to sign something but otherwise
2120 I can't comment too much.</p>
2121 <p>But I really love this device. It is generally the only device that I bring in
2122 trips nowadays, because while it is compromised it works well enough: I can use
2123 to consume media in tablet mode (the fact that ChromeOS supports Android apps
2124 is a plus in those cases), browse the web and even do Linux stuff (more about
2125 this later). The fact that it is small (the size remembers me of a
2126 <a href="https://en.wikipedia.org/wiki/Netbook">netbook</a>), lightweight (~1KG, including
2127 the keyboard), has a good screen (that is bright and HiDPI) and good battery
2128 life (I don't have numbers but I almost never worry about it) is what makes
2129 this device the perfect companion to trips.</p>
2130 <p>Also, it has 2 USB-C ports and supports DisplayPort alt-mode, so it means you
2131 can charge it, connect to a external display and peripherals, all at the same
2132 time. Sadly, the maximum output resolution I got was 1080p (2560x1080),
2133 although some people at Reddit <a href="https://www.reddit.com/r/chromeos/comments/zh27tg/comment/izku724/?utm_source=share&amp;utm_medium=web3x&amp;utm_name=web3xcss&amp;utm_term=1&amp;utm_content=share_button">seems to have
2134 success</a>
2135 at 1440p, and the specs suggests it supports 4k. It may be my Dell S3423DWC
2136 monitor being wonky, the fact that it is Ultrawide or the cable, who knows? I
2137 even tried to change the monitor to &quot;High Resolution&quot; mode in settings, but to
2138 no avail.</p>
2139 <p><em>Update:</em> looking at the <a href="https://www.qualcomm.com/content/dam/qcomm-martech/dm-assets/documents/prod_brief_qcom_sd7c_gen2.pdf">product
2140 brief</a>
2141 for Snapdragon 7c Gen 2, it seems it supports up to QHD@60Hz (1440p) for the
2142 external display, and not 4k. This explains why it doesn't work at maximum
2143 resolution in my Dell S3423DWC, since while it is 1440p it is Ultrawide, so the
2144 resolution is bigger than QHD (3440x1440 vs 2560x1440).</p>
2145 <p>ChromeOS is also really interesting nowadays. To start, it is designed from the
2146 ground up to be a <a href="https://support.google.com/chromebook/answer/3438631">secure computing
2147 environment</a>, probably
2148 the most secure OS for consumers right now. Being a Chrome-first OS makes it a
2149 compromised experience, for example, it is the only device that I use Chrome as
2150 my main browser (since I personally prefer Firefox). But having a OS that boots
2151 fast is great: I never worry about OS updates because I know the device will be
2152 ready in seconds after a reboot. And the whole desktop experience inside the
2153 ChromeOS desktop is good, having shortcuts for many operations so you can get
2154 things done fast, and support for virtual desktops (ChromeOS call it &quot;desks&quot;)
2155 means you can organise your windows as much as you want.</p>
2156 <p>And what I think makes ChromeOS really powerful is
2157 <a href="https://chromeos.dev/en/linux">Crostini</a>, a full Linux VM that you can run
2158 inside ChromeOS. It runs Debian (it seems you can <a href="https://www.reddit.com/r/Crostini/wiki/howto/run-other-distros/">run other
2159 distros</a>
2160 though) with a deep integration with ChromeOS, so you can run even graphical
2161 programs without issues (including OpenGL!):</p>
2162 <p><a href="https://github.com/thiagokokada/blog/raw/main/posts/2024-08-05/Screenshot_2024-08-05_21.22.29.png"><img src="https://github.com/thiagokokada/blog/raw/main/posts/2024-08-05/Screenshot_2024-08-05_21.22.29.png" alt="Fastfetch inside Crostini with gitk running side-by-side."></a></p>
2163 <p><a href="https://github.com/thiagokokada/blog/raw/main/posts/2024-08-05/Screenshot_2024-08-05_21.39.58.png"><img src="https://github.com/thiagokokada/blog/raw/main/posts/2024-08-05/Screenshot_2024-08-05_21.39.58.png" alt="Running glxgears inside Crostini."></a></p>
2164 <p>This is all thanks to
2165 <a href="https://chromium.googlesource.com/chromiumos/platform2/+/HEAD/vm_tools/sommelier/README.md">Sommelier</a>,
2166 a nested Wayland compositor that runs inside Crostini and allow both Wayland
2167 and X11 applications to be forwarded to ChromeOS. The integration is so good
2168 that I can run Firefox inside Crostini and works well enough, but sadly Firefox
2169 is too slow in this device (I am not sure if the issue is ChromeOS or Firefox,
2170 but I suspect the later since Google does some optimisation per device).</p>
2171 <p>One interesting tidbit about the OpenGL situation in this device: this seems to
2172 be the first Chromebook to ship with open source drivers, thanks to Freedreno.
2173 There is <a href="https://www.youtube.com/watch?v=8mnjSmN03VM">this</a> very interesting
2174 presentation done by Rob Clark in XDC 2021, that I recommended anyone
2175 interested in free drivers to watch (the reference design of Duet 3 is
2176 <a href="https://chromeunboxed.com/chromebook-tablet-snapdragon-7c-homestar-coachz-strongbad">Strongbad</a>).</p>
2177 <p>The Crostini integration is probably the best VM integration with Linux I ever
2178 saw in an OS: you can manage files inside the VM, share directories between the
2179 OS and VM, copy and paste works between the two, GUI applications installed
2180 inside the VM appear in the ChromeOS menu, memory allocation inside the VM is
2181 transparent, etc. Even the themes for Linux GUI applications are customised to
2182 match ChromeOS. It is unironically one of the best Linux desktop experiences I
2183 ever had.</p>
2184 <p>Of course I am using Nix, but since the Crostini integration depends on some
2185 services being configured and installed, I decided to run Nix inside Debian
2186 instead of NixOS and run <a href="https://nix-community.github.io/home-manager/index.xhtml#sec-install-standalone">Home-Manager
2187 standalone</a>.
2188 I recommend checking the official <a href="https://wiki.nixos.org/wiki/Installing_Nix_on_Crostini">NixOS Wiki article about
2189 Crostini</a>, that details
2190 how to register applications in ChromeOS (so desktop applications appear in
2191 menu) and use <a href="https://github.com/nix-community/nixGL">nixGL</a> to make OpenGL
2192 applications work.</p>
2193 <p>Like I said at the start of the article, the device is woefully slow thanks to
2194 its CPU and eMMC. It does mean that, for example, activating my Home-Manager
2195 configuration takes a while (around 1 minute, vs a few seconds in my laptop).
2196 But it is much faster than say,
2197 <a href="https://github.com/nix-community/nix-on-droid-app">nix-on-droid</a>, that the
2198 last time I tried in a much more powerful device (<a href="https://www.gsmarena.com/xiaomi_pad_5-11042.php">Xiaomi Pad
2199 5</a>), took 30 minutes until I
2200 just decided to cancel the operation. Having a proper VM instead of
2201 <a href="https://wiki.termux.com/wiki/PRoot">proot</a> makes all the difference here.</p>
2202 <p>I can even do some light programming here: using my
2203 <a href="https://kokada.dev/blog/troubleshoting-zsh-lag-and-solutions-with-nix">ZSH</a>
2204 and neovim configuration (including LSP for coding) is reasonable fast. For
2205 example, I did most of the code that <a href="https://kokada.dev/blog/quick-bits-why-you-should-automate-everything">powers this
2206 blog</a>
2207 using this Chromebook. If I need more power, I can use the <a href="https://tailscale.com/kb/1267/install-chromebook">Tailscale app for
2208 Android</a> to connect to any
2209 other of my hosts via SSH. Yes, the Tailscale app works in Crostini, sadly
2210 without MagicDNS, so you need to use the internal Tailscale IPs instead.</p>
2211 <p>Until Google decides to give us a proper VM or user namespaces in Android or
2212 release a hybrid Chromebook device with better specs, this small Chromebook
2213 will probably stay as my travel companion, and is one of my favorite devices.</p></description><pubDate>Mon, 05 Aug 2024 00:00:00 +0000</pubDate><guid>https://kokada.dev/blog/my-favorite-device-is-a-chromebook/</guid></item><item><title>Make nixd module completion to work anywhere (with Flakes)</title><link>https://kokada.dev/blog/make-nixd-module-completion-to-work-anywhere-with-flakes/</link><description><p>If you want the TL;DR, go to the bottom of the post (search for &quot;final
2214 result&quot;).</p>
2215 <p>I recently switched from <a href="https://github.com/oxalica/nil">nil</a> to
2216 <a href="https://github.com/nix-community/nixd">nixd</a> as my LSP of choice for Nix. I
2217 was curious in <code>nixd</code> for a long time since the fact that it can eval Nix code
2218 means it can offer much more powerful completion than the <code>nil</code>'s static
2219 analysis, however it used to be difficult to setup. Nowadays it is much easier,
2220 basically doing the right thing as long as you have <code>NIX_PATH</code> setup, and you
2221 get both package and NixOS modules completion.</p>
2222 <p>Getting Home-Manager modules though needs some setup. The recommended way to
2223 setup accordingly to the <a href="https://github.com/nix-community/nixd/blob/fe202307eaf7e89c4366ed927af761482a6065c8/nixd/docs/configuration.md">official
2224 documentation</a>
2225 is to use the following for Flake based configurations (using neovim
2226 configuration here, but it should be easy to adapt to other editors):</p>
2227 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
2228 </span></span><span class="line"><span class="cl"> <span class="n">nixpkgs</span> <span class="o">=</span> <span class="p">{</span>
2229 </span></span><span class="line"><span class="cl"> <span class="n">expr</span> <span class="o">=</span> <span class="s2">&#34;import &lt;nixpkgs&gt; { }&#34;</span><span class="p">,</span>
2230 </span></span><span class="line"><span class="cl"> <span class="p">},</span>
2231 </span></span><span class="line"><span class="cl"> <span class="n">options</span> <span class="o">=</span> <span class="p">{</span>
2232 </span></span><span class="line"><span class="cl"> <span class="n">nixos</span> <span class="o">=</span> <span class="p">{</span>
2233 </span></span><span class="line"><span class="cl"> <span class="n">expr</span> <span class="o">=</span> <span class="s1">&#39;(builtins.getFlake (&#34;git+file://&#34; + toString ./.)).nixosConfigurations.miku-nixos.options&#39;</span><span class="p">,</span>
2234 </span></span><span class="line"><span class="cl"> <span class="p">},</span>
2235 </span></span><span class="line"><span class="cl"> <span class="n">home_manager</span> <span class="o">=</span> <span class="p">{</span>
2236 </span></span><span class="line"><span class="cl"> <span class="n">expr</span> <span class="o">=</span> <span class="s1">&#39;(builtins.getFlake (&#34;git+file://&#34; + toString ./.)).homeConfigurations.home-linux.options&#39;</span><span class="p">,</span>
2237 </span></span><span class="line"><span class="cl"> <span class="p">},</span>
2238 </span></span><span class="line"><span class="cl"> <span class="p">},</span>
2239 </span></span><span class="line"><span class="cl"> <span class="c1">-- ...</span>
2240 </span></span><span class="line"><span class="cl"><span class="p">}</span>
2241 </span></span></code></pre><p>This works, but it should be pretty clear the limitations of using <code>./.</code>: this
2242 will only work if you open your editor in your <a href="https://github.com/thiagokokada/nix-configs">Nix configuration
2243 repository</a>. For any other
2244 repository, <code>nixosConfigurations.miku-nixos</code> or <code>homeConfigurations.home-linux</code>
2245 will not exist and the completion will not work.</p>
2246 <p>It may look like this is easy to fix if you have
2247 <a href="https://nixos.org/manual/nixos/unstable/options#opt-_module.args"><code>specialArgs</code></a>
2248 (or <code>extraSpecialArgs</code> in Home-Manager) set to have your
2249 Flakes inputs, but:</p>
2250 <pre class="chroma"><code><span class="line"><span class="cl"><span class="c1"># By the way, ${self} does not exist in the Flake output by default, you need</span>
2251 </span></span><span class="line"><span class="cl"><span class="c1"># to explicit add `inherit self` to your outputs:</span>
2252 </span></span><span class="line"><span class="cl"><span class="c1"># https://discourse.nixos.org/t/who-is-self-in-flake-outputs/31859/4</span>
2253 </span></span><span class="line"><span class="cl"><span class="n">nix-repl</span><span class="o">&gt;</span> <span class="p">(</span><span class="nb">builtins</span><span class="o">.</span><span class="n">getFlake</span> <span class="s2">&#34;git+file://</span><span class="si">${</span><span class="n">self</span><span class="si">}</span><span class="s2">&#34;</span><span class="p">)</span><span class="o">.</span><span class="n">nixosConfigurations</span><span class="o">.</span><span class="n">miku-linux</span><span class="o">.</span><span class="n">options</span>
2254 </span></span><span class="line"><span class="cl"><span class="n">error</span><span class="p">:</span>
2255 </span></span><span class="line"><span class="cl"> <span class="err">…</span> <span class="n">while</span> <span class="n">calling</span> <span class="n">the</span> <span class="err">&#39;</span><span class="n">getFlake&#39;</span> <span class="n">builtin</span>
2256 </span></span><span class="line"><span class="cl"> <span class="n">at</span> <span class="err">«</span><span class="n">string</span><span class="err">»</span><span class="p">:</span><span class="mi">1</span><span class="p">:</span><span class="mi">2</span><span class="p">:</span>
2257 </span></span><span class="line"><span class="cl"> <span class="mi">1</span><span class="err">|</span> <span class="p">(</span><span class="nb">builtins</span><span class="o">.</span><span class="n">getFlake</span> <span class="s2">&#34;git+file://</span><span class="si">${</span><span class="n">self</span><span class="si">}</span><span class="s2">&#34;</span><span class="p">)</span>
2258 </span></span><span class="line"><span class="cl"> <span class="err">|</span> <span class="err">^</span>
2259 </span></span><span class="line"><span class="cl">
2260 </span></span><span class="line"><span class="cl"> <span class="err">…</span> <span class="n">while</span> <span class="n">evaluating</span> <span class="n">the</span> <span class="n">argument</span> <span class="n">passed</span> <span class="n">to</span> <span class="nb">builtins</span><span class="o">.</span><span class="n">getFlake</span>
2261 </span></span><span class="line"><span class="cl">
2262 </span></span><span class="line"><span class="cl"> <span class="n">error</span><span class="p">:</span> <span class="n">the</span> <span class="n">string</span> <span class="err">&#39;</span><span class="sd">git+file:///nix/store/avr1lcmznj8ghynh5vj1kakgfdf0zrxx-source&#39;</span> <span class="n">is</span> <span class="n">not</span> <span class="n">allowed</span> <span class="n">to</span> <span class="n">refer</span> <span class="n">to</span> <span class="n">a</span> <span class="n">store</span> <span class="n">path</span> <span class="p">(</span><span class="n">such</span> <span class="n">as</span> <span class="err">&#39;</span><span class="n">avr1lcmznj8ghynh5vj1kakgfdf0zrxx-source&#39;</span><span class="p">)</span>
2263 </span></span></code></pre><p>Well, it was worth a try. Another option would be to:</p>
2264 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">(</span><span class="nb">builtins</span><span class="o">.</span><span class="n">getFlake</span> <span class="s2">&#34;github:thiagokokada/nix-configs&#34;</span><span class="p">)</span><span class="o">.</span><span class="n">nixosConfigurations</span><span class="o">.</span><span class="n">miku-linux</span><span class="o">.</span><span class="n">options</span>
2265 </span></span><span class="line"><span class="cl"><span class="c1"># Or even something like this</span>
2266 </span></span><span class="line"><span class="cl"><span class="c1"># However, using ${rev} means this wouldn&#39;t work in dirty Flake repos, since</span>
2267 </span></span><span class="line"><span class="cl"><span class="c1"># ${rev} is not set in those cases</span>
2268 </span></span><span class="line"><span class="cl"><span class="p">(</span><span class="nb">builtins</span><span class="o">.</span><span class="n">getFlake</span> <span class="s2">&#34;github:thiagokokada/nix-configs/</span><span class="si">${</span><span class="n">rev</span><span class="si">}</span><span class="s2">&#34;</span><span class="p">)</span><span class="o">.</span><span class="n">nixosConfigurations</span><span class="o">.</span><span class="n">miku-linux</span><span class="o">.</span><span class="n">options</span>
2269 </span></span></code></pre><p>But while it works, it is slow, because it needs network to evaluate (and it is
2270 impure, since there is no <code>flake.lock</code>).</p>
2271 <p>The default configuration for <code>nixd</code> makes NixOS completion work even outside
2272 of my configuration repo, and it is fast. How? I decided to take a look at the
2273 <code>nixd</code> source code and found
2274 <a href="https://github.com/nix-community/nixd/blob/d938026c55c7c36a6e79afd9627459160b4924ed/nixd/lib/Controller/LifeTime.cpp#L33C11-L35C76">this</a>
2275 (formatted here for legibility):</p>
2276 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">(</span>
2277 </span></span><span class="line"><span class="cl"> <span class="k">let</span>
2278 </span></span><span class="line"><span class="cl"> <span class="n">pkgs</span> <span class="o">=</span> <span class="kn">import</span> <span class="sr">&lt;nixpkgs&gt;</span> <span class="p">{</span> <span class="p">};</span>
2279 </span></span><span class="line"><span class="cl"> <span class="k">in</span>
2280 </span></span><span class="line"><span class="cl"> <span class="p">(</span><span class="n">pkgs</span><span class="o">.</span><span class="n">lib</span><span class="o">.</span><span class="n">evalModules</span> <span class="p">{</span>
2281 </span></span><span class="line"><span class="cl"> <span class="n">modules</span> <span class="o">=</span> <span class="p">(</span><span class="kn">import</span> <span class="sr">&lt;nixpkgs/nixos/modules/module-list.nix&gt;</span><span class="p">)</span> <span class="o">++</span> <span class="p">[</span>
2282 </span></span><span class="line"><span class="cl"> <span class="p">({</span> <span class="o">...</span> <span class="p">}:</span> <span class="p">{</span> <span class="n">nixpkgs</span><span class="o">.</span><span class="n">hostPlatform</span> <span class="o">=</span> <span class="nb">builtins</span><span class="o">.</span><span class="n">currentSystem</span><span class="p">;</span> <span class="p">})</span>
2283 </span></span><span class="line"><span class="cl"> <span class="p">];</span>
2284 </span></span><span class="line"><span class="cl"> <span class="p">})</span>
2285 </span></span><span class="line"><span class="cl"><span class="p">)</span><span class="o">.</span><span class="n">options</span>
2286 </span></span></code></pre><p>Interesting, so they're manually loading the modules using <code>evalModules</code>. As I
2287 said above, it depends in <code>NIX_PATH</code> being correctly set. Can we fix this to
2288 use our Flake inputs instead? After some tries in the Nix REPL, I got the
2289 following:</p>
2290 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">(</span>
2291 </span></span><span class="line"><span class="cl"> <span class="k">let</span>
2292 </span></span><span class="line"><span class="cl"> <span class="n">pkgs</span> <span class="o">=</span> <span class="kn">import</span> <span class="s2">&#34;</span><span class="si">${</span><span class="n">inputs</span><span class="o">.</span><span class="n">nixpkgs</span><span class="si">}</span><span class="s2">&#34;</span> <span class="p">{</span> <span class="p">};</span>
2293 </span></span><span class="line"><span class="cl"> <span class="k">in</span>
2294 </span></span><span class="line"><span class="cl"> <span class="p">(</span><span class="n">pkgs</span><span class="o">.</span><span class="n">lib</span><span class="o">.</span><span class="n">evalModules</span> <span class="p">{</span>
2295 </span></span><span class="line"><span class="cl"> <span class="n">modules</span> <span class="o">=</span> <span class="p">(</span><span class="kn">import</span> <span class="s2">&#34;</span><span class="si">${</span><span class="n">inputs</span><span class="o">.</span><span class="n">nixpkgs</span><span class="si">}</span><span class="s2">/nixos/modules/module-list.nix&#34;</span><span class="p">)</span> <span class="o">++</span> <span class="p">[</span>
2296 </span></span><span class="line"><span class="cl"> <span class="p">({</span> <span class="o">...</span> <span class="p">}:</span> <span class="p">{</span> <span class="n">nixpkgs</span><span class="o">.</span><span class="n">hostPlatform</span> <span class="o">=</span> <span class="nb">builtins</span><span class="o">.</span><span class="n">currentSystem</span><span class="p">;</span> <span class="p">})</span>
2297 </span></span><span class="line"><span class="cl"> <span class="p">];</span>
2298 </span></span><span class="line"><span class="cl"> <span class="p">})</span>
2299 </span></span><span class="line"><span class="cl"><span class="p">)</span><span class="o">.</span><span class="n">options</span>
2300 </span></span></code></pre><p>So we can adapt this to the neovim configuration:</p>
2301 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
2302 </span></span><span class="line"><span class="cl"> <span class="n">options</span> <span class="o">=</span> <span class="p">{</span>
2303 </span></span><span class="line"><span class="cl"> <span class="n">nixos</span> <span class="o">=</span> <span class="p">{</span>
2304 </span></span><span class="line"><span class="cl"> <span class="n">expr</span> <span class="o">=</span> <span class="s1">&#39;(let pkgs = import &#34;${inputs.nixpkgs}&#34; { }; in (pkgs.lib.evalModules { modules = (import &#34;${inputs.nixpkgs}/nixos/modules/module-list.nix&#34;) ++ [ ({...}: { nixpkgs.hostPlatform = builtins.currentSystem;} ) ] ; })).options&#39;</span><span class="p">,</span>
2305 </span></span><span class="line"><span class="cl"> <span class="p">},</span>
2306 </span></span><span class="line"><span class="cl"> <span class="p">},</span>
2307 </span></span><span class="line"><span class="cl"><span class="p">}</span>
2308 </span></span></code></pre><p>This was easy. But the main issue is Home-Manager. How can we fix it? I needed
2309 to take a look at the Home-Manager <a href="https://github.com/nix-community/home-manager/blob/master/docs/default.nix#L161-L169">source
2310 code</a>
2311 to find the answer:</p>
2312 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">(</span>
2313 </span></span><span class="line"><span class="cl"> <span class="k">let</span>
2314 </span></span><span class="line"><span class="cl"> <span class="n">pkgs</span> <span class="o">=</span> <span class="kn">import</span> <span class="s2">&#34;</span><span class="si">${</span><span class="n">inputs</span><span class="o">.</span><span class="n">nixpkgs</span><span class="si">}</span><span class="s2">&#34;</span> <span class="p">{</span> <span class="p">};</span>
2315 </span></span><span class="line"><span class="cl"> <span class="n">lib</span> <span class="o">=</span> <span class="kn">import</span> <span class="s2">&#34;</span><span class="si">${</span><span class="n">inputs</span><span class="o">.</span><span class="n">home-manager</span><span class="si">}</span><span class="s2">/modules/lib/stdlib-extended.nix&#34;</span> <span class="n">pkgs</span><span class="o">.</span><span class="n">lib</span><span class="p">;</span>
2316 </span></span><span class="line"><span class="cl"> <span class="k">in</span>
2317 </span></span><span class="line"><span class="cl"> <span class="p">(</span><span class="n">lib</span><span class="o">.</span><span class="n">evalModules</span> <span class="p">{</span>
2318 </span></span><span class="line"><span class="cl"> <span class="n">modules</span> <span class="o">=</span> <span class="p">(</span><span class="kn">import</span> <span class="s2">&#34;</span><span class="si">${</span><span class="n">inputs</span><span class="o">.</span><span class="n">home-manager</span><span class="si">}</span><span class="s2">/modules/modules.nix&#34;</span><span class="p">)</span> <span class="p">{</span>
2319 </span></span><span class="line"><span class="cl"> <span class="k">inherit</span> <span class="n">lib</span> <span class="n">pkgs</span><span class="p">;</span>
2320 </span></span><span class="line"><span class="cl"> <span class="n">check</span> <span class="o">=</span> <span class="no">false</span><span class="p">;</span>
2321 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
2322 </span></span><span class="line"><span class="cl"> <span class="p">})</span>
2323 </span></span><span class="line"><span class="cl"><span class="p">)</span><span class="o">.</span><span class="n">options</span>
2324 </span></span></code></pre><p>The interesting part is: Home-Manager has its own extension of the module
2325 system (including <code>evalModules</code>). This includes e.g.: extra types used in
2326 Home-Manager only. Also, we need to disable <code>checks</code>, otherwise we will hit
2327 some validations (e.g.: missing <code>stateVersion</code>). I am not sure if this causes
2328 any issue for module completion yet, I may set it in the future.</p>
2329 <p>And for the final result:</p>
2330 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
2331 </span></span><span class="line"><span class="cl"> <span class="n">nixpkgs</span> <span class="o">=</span> <span class="p">{</span>
2332 </span></span><span class="line"><span class="cl"> <span class="n">expr</span> <span class="o">=</span> <span class="s1">&#39;import &#34;${flake.inputs.nixpkgs}&#34; { }&#39;</span><span class="p">,</span>
2333 </span></span><span class="line"><span class="cl"> <span class="p">},</span>
2334 </span></span><span class="line"><span class="cl"> <span class="n">options</span> <span class="o">=</span> <span class="p">{</span>
2335 </span></span><span class="line"><span class="cl"> <span class="n">nixos</span> <span class="o">=</span> <span class="p">{</span>
2336 </span></span><span class="line"><span class="cl"> <span class="n">expr</span> <span class="o">=</span> <span class="s1">&#39;(let pkgs = import &#34;${inputs.nixpkgs}&#34; { }; in (pkgs.lib.evalModules { modules = (import &#34;${inputs.nixpkgs}/nixos/modules/module-list.nix&#34;) ++ [ ({...}: { nixpkgs.hostPlatform = builtins.currentSystem;} ) ] ; })).options&#39;</span><span class="p">,</span>
2337 </span></span><span class="line"><span class="cl"> <span class="p">},</span>
2338 </span></span><span class="line"><span class="cl"> <span class="n">home_manager</span> <span class="o">=</span> <span class="p">{</span>
2339 </span></span><span class="line"><span class="cl"> <span class="n">expr</span> <span class="o">=</span> <span class="s1">&#39;(let pkgs = import &#34;${inputs.nixpkgs}&#34; { }; lib = import &#34;${inputs.home-manager}/modules/lib/stdlib-extended.nix&#34; pkgs.lib; in (lib.evalModules { modules = (import &#34;${inputs.home-manager}/modules/modules.nix&#34;) { inherit lib pkgs; check = false; }; })).options&#39;</span><span class="p">,</span>
2340 </span></span><span class="line"><span class="cl"> <span class="p">},</span>
2341 </span></span><span class="line"><span class="cl"> <span class="p">},</span>
2342 </span></span><span class="line"><span class="cl"><span class="p">}</span>
2343 </span></span></code></pre><p>Yes, it is quite a mouthful, but it makes module completion work in any
2344 repository, as long as you're using Flakes. And it is fast, since it doesn't
2345 need any network access. Since we are already here, let's define <code>nixpkgs</code> to
2346 not depend in the <code>NIX_PATH</code> being set too.</p></description><pubDate>Sun, 04 Aug 2024 00:00:00 +0000</pubDate><guid>https://kokada.dev/blog/make-nixd-module-completion-to-work-anywhere-with-flakes/</guid></item><item><title>Troubleshooting: ZSH lag and solutions with Nix</title><link>https://kokada.dev/blog/troubleshoting-zsh-lag-and-solutions-with-nix/</link><description><p>Inspired by this <a href="https://lock.cmpxchg8b.com/slowterm.html">blog post from
2347 Tavis</a>, I decided to document my own
2348 recent journey of reducing terminal (ZSH) lag startup. This post is way less
2349 interesting than the one from Tavis that uses a debugger to patch applications
2350 on the fly, but should still be interesting for some. And it also shows how
2351 powerful Nix can be for some things.</p>
2352 <p>For context, I have basically 3 systems where I interact with terminal
2353 frequently:</p>
2354 <ul>
2355 <li><a href="https://www.notebookcheck.net/Lenovo-ThinkPad-P14s-Gen-1-Laptop-Review-AMD-workstation-sans-dedicated-GPU.528746.0.html">Thinkpad P14s Gen 1</a>
2356 running NixOS, with a reasonable fast CPU (<a href="https://www.notebookcheck.net/AMD-Ryzen-7-PRO-4750U-Processor-Benchmarks-and-Specs.464396.0.html">AMD Ryzen 7 PRO
2357 4750U</a>)
2358 and disk (NVMe)</li>
2359 <li><a href="https://everymac.com/systems/apple/macbook_pro/specs/macbook-pro-m1-pro-10-core-cpu-16-core-gpu-16-2021-specs.html">MacBook
2360 Pro</a>
2361 with a really fast CPU (M1 Pro) and disk, but of course running macOS, being
2362 phased-out since this is a job owned machine and I am changing jobs right
2363 now, but should be replaced with another one soon™</li>
2364 <li><a href="https://chromeunboxed.com/lenovo-chromebook-duet-3-review-perfect-sequel">Chromebook Duet
2365 3</a>
2366 running ChromeOS, with slow CPU (<a href="https://www.notebookcheck.net/Qualcomm-Snapdragon-7c-Gen-2-Processor-Benchmarks-and-Specs.554426.0.html">Snapdragon 7c Gen
2367 2</a>)
2368 and disk (eMMC, really?)</li>
2369 </ul>
2370 <p>My experience is similar to Tavis, at around 300ms of startup time I don't care
2371 too much, but around 500ms+ is where I start to notice. I never had any issues
2372 with startup time in NixOS itself (I had issues with macOS before, but it was
2373 not actually the fault of macOS), but in the Chromebook it was awful: 600ms+
2374 with <a href="https://www.instabug.com/blog/understanding-cold-hot-and-warm-app-launch-time">hot
2375 start</a>,
2376 while cold start it could take multiple seconds.</p>
2377 <p>We can check how long ZSH takes to start by using:</p>
2378 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> <span class="nb">time</span> zsh -ci <span class="nb">exit</span>
2379 </span></span><span class="line"><span class="cl"><span class="go">zsh -ic exit 0.04s user 0.10s system 100% cpu 0.143 total
2380 </span></span></span></code></pre><p>The <code>-i</code> flag here is important, because we are interested in the interactive
2381 use of ZSH. Without this flag ZSH will ignore your <code>~/.zshrc</code> file, and the
2382 results will be meaningless.</p>
2383 <p>To do a more interesting benchmark, we can use
2384 <a href="https://github.com/sharkdp/hyperfine"><code>hyperfine</code></a>:</p>
2385 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> hyperfine <span class="s2">&#34;zsh -ic exit&#34;</span>
2386 </span></span><span class="line"><span class="cl"><span class="go">Benchmark 1: zsh -ic exit
2387 </span></span></span><span class="line"><span class="cl"><span class="go"> Time (mean ± σ): 145.4 ms ± 4.2 ms [User: 49.8 ms, System: 97.3 ms]
2388 </span></span></span><span class="line"><span class="cl"><span class="go"> Range (min … max): 138.6 ms … 155.3 ms 19 runs
2389 </span></span></span></code></pre><p>Hyperfine will run the command multiple times and take care of things like
2390 shell startup time. A really great tool to have in your toolbox by the way, but
2391 I digress.</p>
2392 <p>So let's do a little time travelling. Going back to commit
2393 <a href="https://github.com/thiagokokada/nix-configs/tree/b12757f90889653e359a1ab0a8cfd2f90cfabf31"><code>b12757f</code></a>
2394 from <a href="https://github.com/thiagokokada/nix-configs/">nix-configs</a>. Running
2395 <code>hyperfine</code> like above from my NixOS laptop, we have:</p>
2396 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> hyperfine <span class="s2">&#34;zsh -ic exit&#34;</span>
2397 </span></span><span class="line"><span class="cl"><span class="go">Benchmark 1: zsh -ic exit
2398 </span></span></span><span class="line"><span class="cl"><span class="go"> Time (mean ± σ): 218.6 ms ± 5.1 ms [User: 70.6 ms, System: 151.5 ms]
2399 </span></span></span><span class="line"><span class="cl"><span class="go"> Range (min … max): 210.3 ms … 227.0 ms 13 runs
2400 </span></span></span></code></pre><p>This doesn't look that bad, but let's see the same commit in my Chromebook:</p>
2401 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> hyperfine <span class="s2">&#34;zsh -ic exit&#34;</span>
2402 </span></span><span class="line"><span class="cl"><span class="go">Benchmark 1: zsh -ic exit
2403 </span></span></span><span class="line"><span class="cl"><span class="go"> Time (mean ± σ): 679.7 ms ± 40.2 ms [User: 230.8 ms, System: 448.5 ms]
2404 </span></span></span><span class="line"><span class="cl"><span class="go"> Range (min … max): 607.3 ms … 737.0 ms 10 runs
2405 </span></span></span></code></pre><p>Yikes, this is much worse. And those are the results after I retried the
2406 benchmark (so it is a hot start). The cold start times were above 3s. So let's
2407 investigate what is happening here. We can profile what is taking time during
2408 the startup of ZSH using <a href="https://www.bigbinary.com/blog/zsh-profiling">zprof</a>.
2409 You can add the following in your <code>~/.zshrc</code>:</p>
2410 <pre class="chroma"><code><span class="line"><span class="cl"><span class="c1"># At the top of your ~/.zshrc file</span>
2411 </span></span><span class="line"><span class="cl">zmodload zsh/zprof
2412 </span></span><span class="line"><span class="cl">
2413 </span></span><span class="line"><span class="cl"><span class="c1"># ...</span>
2414 </span></span><span class="line"><span class="cl">
2415 </span></span><span class="line"><span class="cl"><span class="c1"># At the end of your ~/.zshrc file</span>
2416 </span></span><span class="line"><span class="cl">zprof
2417 </span></span></code></pre><p>Or if using Home-Manager, use the
2418 <a href="https://nix-community.github.io/home-manager/options.xhtml#opt-programs.zsh.zprof.enable"><code>programs.zsh.zprof.enable</code></a>
2419 option. Once we restart ZSH, we will have something like:</p>
2420 <pre class="chroma"><code><span class="line"><span class="cl"><span class="go">num calls time self name
2421 </span></span></span><span class="line"><span class="cl"><span class="go">-----------------------------------------------------------------------------------
2422 </span></span></span><span class="line"><span class="cl"><span class="go"> 1) 1 36.91 36.91 34.29% 30.47 30.47 28.31% (anon) [/home/thiagoko/.zsh/plugins/zim-completion/init.zsh:13]
2423 </span></span></span><span class="line"><span class="cl"><span class="go"> 2) 1 25.43 25.43 23.63% 25.43 25.43 23.63% (anon) [/home/thiagoko/.zsh/plugins/zim-ssh/init.zsh:6]
2424 </span></span></span><span class="line"><span class="cl"><span class="go"> 3) 1 22.00 22.00 20.45% 21.92 21.92 20.36% _zsh_highlight_load_highlighters
2425 </span></span></span><span class="line"><span class="cl"><span class="go"> 4) 1 12.32 12.32 11.45% 12.32 12.32 11.45% autopair-init
2426 </span></span></span><span class="line"><span class="cl"><span class="go"> 5) 1 6.44 6.44 5.98% 6.44 6.44 5.98% compinit
2427 </span></span></span><span class="line"><span class="cl"><span class="go"> 6) 1 3.56 3.56 3.31% 3.48 3.48 3.23% prompt_pure_state_setup
2428 </span></span></span><span class="line"><span class="cl"><span class="go"> 7) 2 3.79 1.89 3.52% 2.85 1.43 2.65% async
2429 </span></span></span><span class="line"><span class="cl"><span class="go"> 8) 1 0.93 0.93 0.87% 0.93 0.93 0.87% async_init
2430 </span></span></span><span class="line"><span class="cl"><span class="go"> 9) 6 0.93 0.15 0.86% 0.93 0.15 0.86% is-at-least
2431 </span></span></span><span class="line"><span class="cl"><span class="go">10) 6 0.67 0.11 0.63% 0.67 0.11 0.63% add-zle-hook-widget
2432 </span></span></span><span class="line"><span class="cl"><span class="go">11) 1 8.25 8.25 7.66% 0.61 0.61 0.57% prompt_pure_setup
2433 </span></span></span><span class="line"><span class="cl"><span class="go">12) 1 0.40 0.40 0.37% 0.40 0.40 0.37% (anon) [/nix/store/p1zqypy7600fvfyl1v571bljx2l8zhay-zsh-autosuggestions-0.7.0/share/zsh-autosuggestions/zsh-autosuggestions.zsh:458]
2434 </span></span></span><span class="line"><span class="cl"><span class="go">13) 5 0.31 0.06 0.29% 0.31 0.06 0.29% add-zsh-hook
2435 </span></span></span><span class="line"><span class="cl"><span class="go">14) 1 0.60 0.60 0.56% 0.29 0.29 0.27% (anon) [/home/thiagoko/.zsh/plugins/zim-input/init.zsh:5]
2436 </span></span></span><span class="line"><span class="cl"><span class="go">15) 1 0.21 0.21 0.20% 0.21 0.21 0.20% compdef
2437 </span></span></span><span class="line"><span class="cl"><span class="go">16) 1 0.10 0.10 0.09% 0.10 0.10 0.09% _zsh_highlight__function_is_autoload_stub_p
2438 </span></span></span><span class="line"><span class="cl"><span class="go">17) 1 0.26 0.26 0.24% 0.08 0.08 0.08% _zsh_highlight__function_callable_p
2439 </span></span></span><span class="line"><span class="cl"><span class="go">18) 1 0.08 0.08 0.08% 0.08 0.08 0.08% prompt_pure_is_inside_container
2440 </span></span></span><span class="line"><span class="cl"><span class="go">19) 1 0.07 0.07 0.07% 0.07 0.07 0.07% _zsh_highlight__is_function_p
2441 </span></span></span><span class="line"><span class="cl"><span class="go">20) 1 0.01 0.01 0.01% 0.01 0.01 0.01% __wezterm_install_bash_prexec
2442 </span></span></span><span class="line"><span class="cl"><span class="go">21) 1 0.00 0.00 0.00% 0.00 0.00 0.00% _zsh_highlight_bind_widgets
2443 </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="gp">#</span> ...
2444 </span></span></code></pre><p>I ommited some output for brevit. The first 2 things that shows are from the
2445 <a href="https://github.com/zimfw/zimfw">zimfw</a>, the framework that I use to configure
2446 my ZSH (similar to Oh-My-Zsh). I actually don't use <code>zimfw</code> directly, instead I
2447 just load some modules that I find useful, like the <code>zim-completion</code> and
2448 <code>zim-ssh</code> that we can see above. By the way, Zim is generally really well
2449 optimised for startup time, but those 2 modules are kind slow.</p>
2450 <p>For <a href="https://github.com/zimfw/completion"><code>zim-completion</code></a>, after taking a
2451 look at it, there isn't much I could do. It seems that the reason
2452 <code>zim-completion</code> takes so long during startup is because it is trying to decide
2453 if it needs to recompile the completions (and replacing it with just a naive
2454 <code>autoload -U compinit &amp;&amp; compinit</code> is even worse for startup performance). I
2455 may eventually replace it for something else, but I really like what Zim brings
2456 here, so I decided to not touch it for now.</p>
2457 <p>However <a href="https://github.com/zimfw/ssh"><code>zim-ssh</code></a> is another history. The only
2458 reason I used it is to start a <code>ssh-agent</code> and keep it between multiple ZSH
2459 sessions. It shouldn't have that much influence in startup time. So I took a
2460 look the code (since it is small, I am reproducing it here):</p>
2461 <pre class="chroma"><code><span class="line"><span class="cl"><span class="c1">#</span>
2462 </span></span><span class="line"><span class="cl"><span class="c1"># Set up ssh-agent</span>
2463 </span></span><span class="line"><span class="cl"><span class="c1">#</span>
2464 </span></span><span class="line"><span class="cl">
2465 </span></span><span class="line"><span class="cl"><span class="c1"># Don&#39;t do anything unless we can actually use ssh-agent</span>
2466 </span></span><span class="line"><span class="cl"><span class="o">((</span> <span class="si">${</span><span class="p">+commands[ssh-agent]</span><span class="si">}</span> <span class="o">))</span> <span class="o">&amp;&amp;</span> <span class="o">()</span> <span class="o">{</span>
2467 </span></span><span class="line"><span class="cl"> ssh-add -l <span class="p">&amp;</span>&gt;/dev/null
2468 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="o">((</span> ? <span class="o">==</span> <span class="m">2</span> <span class="o">))</span><span class="p">;</span> <span class="k">then</span>
2469 </span></span><span class="line"><span class="cl"> <span class="c1"># Unable to contact the authentication agent</span>
2470 </span></span><span class="line"><span class="cl">
2471 </span></span><span class="line"><span class="cl"> <span class="c1"># Load stored agent connection info</span>
2472 </span></span><span class="line"><span class="cl"> <span class="nb">local</span> -r <span class="nv">ssh_env</span><span class="o">=</span><span class="si">${</span><span class="nv">HOME</span><span class="si">}</span>/.ssh-agent
2473 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="o">[[</span> -r <span class="si">${</span><span class="nv">ssh_env</span><span class="si">}</span> <span class="o">]]</span> <span class="nb">source</span> <span class="si">${</span><span class="nv">ssh_env</span><span class="si">}</span> &gt;/dev/null
2474 </span></span><span class="line"><span class="cl">
2475 </span></span><span class="line"><span class="cl"> ssh-add -l <span class="p">&amp;</span>&gt;/dev/null
2476 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="o">((</span> ? <span class="o">==</span> <span class="m">2</span> <span class="o">))</span><span class="p">;</span> <span class="k">then</span>
2477 </span></span><span class="line"><span class="cl"> <span class="c1"># Start agent and store agent connection info</span>
2478 </span></span><span class="line"><span class="cl"> <span class="o">(</span><span class="nb">umask</span> 066<span class="p">;</span> ssh-agent &gt;! <span class="si">${</span><span class="nv">ssh_env</span><span class="si">}</span><span class="o">)</span>
2479 </span></span><span class="line"><span class="cl"> <span class="nb">source</span> <span class="si">${</span><span class="nv">ssh_env</span><span class="si">}</span> &gt;/dev/null
2480 </span></span><span class="line"><span class="cl"> <span class="k">fi</span>
2481 </span></span><span class="line"><span class="cl"> <span class="k">fi</span>
2482 </span></span><span class="line"><span class="cl">
2483 </span></span><span class="line"><span class="cl"> <span class="c1"># Load identities</span>
2484 </span></span><span class="line"><span class="cl"> ssh-add -l <span class="p">&amp;</span>&gt;/dev/null
2485 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="o">((</span> ? <span class="o">==</span> <span class="m">1</span> <span class="o">))</span><span class="p">;</span> <span class="k">then</span>
2486 </span></span><span class="line"><span class="cl"> <span class="nb">local</span> -a zssh_ids
2487 </span></span><span class="line"><span class="cl"> zstyle -a <span class="s1">&#39;:zim:ssh&#39;</span> ids <span class="s1">&#39;zssh_ids&#39;</span>
2488 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="o">((</span> <span class="si">${#</span><span class="nv">zssh_ids</span><span class="si">}</span> <span class="o">))</span><span class="p">;</span> <span class="k">then</span>
2489 </span></span><span class="line"><span class="cl"> ssh-add <span class="si">${</span><span class="nv">HOME</span><span class="si">}</span>/.ssh/<span class="si">${</span><span class="p">^zssh_ids</span><span class="si">}</span> 2&gt;/dev/null
2490 </span></span><span class="line"><span class="cl"> <span class="k">else</span>
2491 </span></span><span class="line"><span class="cl"> ssh-add 2&gt;/dev/null
2492 </span></span><span class="line"><span class="cl"> <span class="k">fi</span>
2493 </span></span><span class="line"><span class="cl"> <span class="k">fi</span>
2494 </span></span><span class="line"><span class="cl"><span class="o">}</span>
2495 </span></span></code></pre><p>Well, this is bad. Let's assume the common path, where the <code>ssh-agent</code> is
2496 already running but you open a new shell instance (that doesn't have the
2497 connection info yet so it will need to load). This will run <code>ssh-add</code> at 4
2498 times. How long does <code>ssh-add</code> takes to run?</p>
2499 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> hyperfine -Ni <span class="s2">&#34;ssh-add -l&#34;</span>
2500 </span></span><span class="line"><span class="cl"><span class="go">Benchmark 1: ssh-add -l
2501 </span></span></span><span class="line"><span class="cl"><span class="go"> Time (mean ± σ): 4.6 ms ± 1.1 ms [User: 2.0 ms, System: 2.0 ms]
2502 </span></span></span><span class="line"><span class="cl"><span class="go"> Range (min … max): 3.4 ms … 8.7 ms 619 runs
2503 </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="err">
2504 </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="go"> Warning: Ignoring non-zero exit code.
2505 </span></span></span></code></pre><p>For those curious, <code>-N</code> disables the Shell usage, that works better when the
2506 command being tested is too fast.</p>
2507 <p>In average we have 4x4ms=16ms of startup time. But keep in mind the worst case
2508 can be much worse. The question is, how can we improve the situation here?</p>
2509 <p>After taking a look, I decided to write my own code, based in some ideas stolen
2510 from <a href="https://github.com/ohmyzsh/ohmyzsh/blob/67581c53c6458566e174620361e84b364b9034d2/plugins/ssh-agent/ssh-agent.plugin.zsh">Oh-My-Zsh ssh-agent
2511 plugin</a>.
2512 Here is final version of my
2513 <a href="https://github.com/thiagokokada/nix-configs/blob/e45a888f2bf3ce5644c3966f0b6371414d0291e2/home-manager/cli/ssh/ssh-agent.zsh">code</a>:</p>
2514 <pre class="chroma"><code><span class="line"><span class="cl">zmodload zsh/net/socket
2515 </span></span><span class="line"><span class="cl">
2516 </span></span><span class="line"><span class="cl">_check_agent<span class="o">(){</span>
2517 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="o">[[</span> -S <span class="s2">&#34;</span><span class="nv">$SSH_AUTH_SOCK</span><span class="s2">&#34;</span> <span class="o">]]</span> <span class="o">&amp;&amp;</span> zsocket <span class="s2">&#34;</span><span class="nv">$SSH_AUTH_SOCK</span><span class="s2">&#34;</span> 2&gt;/dev/null<span class="p">;</span> <span class="k">then</span>
2518 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="m">0</span>
2519 </span></span><span class="line"><span class="cl"> <span class="k">fi</span>
2520 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="m">1</span>
2521 </span></span><span class="line"><span class="cl"><span class="o">}</span>
2522 </span></span><span class="line"><span class="cl">
2523 </span></span><span class="line"><span class="cl">_start_agent<span class="o">()</span> <span class="o">{</span>
2524 </span></span><span class="line"><span class="cl"> <span class="c1"># Test if $SSH_AUTH_SOCK is visible, in case we start e.g.: ssh-agent via</span>
2525 </span></span><span class="line"><span class="cl"> <span class="c1"># systemd service</span>
2526 </span></span><span class="line"><span class="cl"> <span class="k">if</span> _check_agent<span class="p">;</span> <span class="k">then</span>
2527 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="m">0</span>
2528 </span></span><span class="line"><span class="cl"> <span class="k">fi</span>
2529 </span></span><span class="line"><span class="cl">
2530 </span></span><span class="line"><span class="cl"> <span class="c1"># Get the filename to store/lookup the environment from</span>
2531 </span></span><span class="line"><span class="cl"> <span class="nb">local</span> -r <span class="nv">ssh_env_cache</span><span class="o">=</span><span class="s2">&#34;</span><span class="nv">$HOME</span><span class="s2">/.ssh-agent&#34;</span>
2532 </span></span><span class="line"><span class="cl">
2533 </span></span><span class="line"><span class="cl"> <span class="c1"># Check if ssh-agent is already running</span>
2534 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="o">[[</span> -f <span class="s2">&#34;</span><span class="nv">$ssh_env_cache</span><span class="s2">&#34;</span> <span class="o">]]</span><span class="p">;</span> <span class="k">then</span>
2535 </span></span><span class="line"><span class="cl"> <span class="nb">source</span> <span class="s2">&#34;</span><span class="nv">$ssh_env_cache</span><span class="s2">&#34;</span> &gt; /dev/null
2536 </span></span><span class="line"><span class="cl">
2537 </span></span><span class="line"><span class="cl"> <span class="c1"># Test if $SSH_AUTH_SOCK is visible, e.g.: the ssh-agent is still alive</span>
2538 </span></span><span class="line"><span class="cl"> <span class="k">if</span> _check_agent<span class="p">;</span> <span class="k">then</span>
2539 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="m">0</span>
2540 </span></span><span class="line"><span class="cl"> <span class="k">fi</span>
2541 </span></span><span class="line"><span class="cl"> <span class="k">fi</span>
2542 </span></span><span class="line"><span class="cl">
2543 </span></span><span class="line"><span class="cl"> <span class="c1"># start ssh-agent and setup environment</span>
2544 </span></span><span class="line"><span class="cl"> <span class="o">(</span>
2545 </span></span><span class="line"><span class="cl"> <span class="nb">umask</span> <span class="m">066</span>
2546 </span></span><span class="line"><span class="cl"> ssh-agent -s &gt;! <span class="s2">&#34;</span><span class="nv">$ssh_env_cache</span><span class="s2">&#34;</span>
2547 </span></span><span class="line"><span class="cl"> <span class="o">)</span>
2548 </span></span><span class="line"><span class="cl"> <span class="nb">source</span> <span class="s2">&#34;</span><span class="nv">$ssh_env_cache</span><span class="s2">&#34;</span> &gt; /dev/null
2549 </span></span><span class="line"><span class="cl"><span class="o">}</span>
2550 </span></span><span class="line"><span class="cl">
2551 </span></span><span class="line"><span class="cl">_start_agent
2552 </span></span><span class="line"><span class="cl">unfunction _check_agent _start_agent
2553 </span></span></code></pre><p>The idea here is simple: using
2554 <a href="https://zsh.sourceforge.io/Doc/Release/Zsh-Modules.html#The-zsh_002fnet_002fsocket-Module"><code>zsocket</code></a>
2555 module from ZSH itself to check if the <code>ssh-agent</code> is working instead of
2556 executing <code>ssh-add -l</code>. The only case we run any program now is to start the
2557 agent itself if needed. Let's run <code>hyperfine</code> again:</p>
2558 <pre><code>$ hyperfine &quot;zsh -ic exit&quot;
2559 Benchmark 1: zsh -ic exit
2560 Time (mean ± σ): 188.3 ms ± 8.2 ms [User: 61.1 ms, System: 130.0 ms]
2561 Range (min … max): 170.9 ms … 198.4 ms 16 runs
2562 </code></pre>
2563 <p>Got a good improvement here already. Let's see <code>zprof</code> again:</p>
2564 <pre class="chroma"><code><span class="line"><span class="cl"><span class="go">num calls time self name
2565 </span></span></span><span class="line"><span class="cl"><span class="go">-----------------------------------------------------------------------------------
2566 </span></span></span><span class="line"><span class="cl"><span class="go"> 1) 1 41.23 41.23 48.66% 33.52 33.52 39.56% (anon) [/home/thiagoko/.zsh/plugins/zim-completion/init.zsh:13]
2567 </span></span></span><span class="line"><span class="cl"><span class="go"> 2) 1 22.23 22.23 26.24% 22.12 22.12 26.10% _zsh_highlight_load_highlighters
2568 </span></span></span><span class="line"><span class="cl"><span class="go"> 3) 1 8.90 8.90 10.51% 8.90 8.90 10.51% Gautopair-init
2569 </span></span></span><span class="line"><span class="cl"><span class="go"> 4) 1 7.71 7.71 9.10% 7.71 7.71 9.10% compinit
2570 </span></span></span><span class="line"><span class="cl"><span class="go"> 5) 1 5.74 5.74 6.77% 5.60 5.60 6.60% prompt_pure_state_setup
2571 </span></span></span><span class="line"><span class="cl"><span class="go"> 6) 6 1.19 0.20 1.41% 1.19 0.20 1.41% add-zle-hook-widget
2572 </span></span></span><span class="line"><span class="cl"><span class="go"> 7) 2 1.97 0.99 2.33% 1.14 0.57 1.34% async
2573 </span></span></span><span class="line"><span class="cl"><span class="go"> 8) 6 0.87 0.15 1.03% 0.87 0.15 1.03% is-at-least
2574 </span></span></span><span class="line"><span class="cl"><span class="go"> 9) 1 0.84 0.84 0.99% 0.84 0.84 0.99% async_init
2575 </span></span></span><span class="line"><span class="cl"><span class="go">10) 1 9.30 9.30 10.97% 0.72 0.72 0.84% prompt_pure_setup
2576 </span></span></span><span class="line"><span class="cl"><span class="go">11) 5 0.63 0.13 0.75% 0.63 0.13 0.75% add-zsh-hook
2577 </span></span></span><span class="line"><span class="cl"><span class="go">12) 1 0.41 0.41 0.48% 0.41 0.41 0.48% _start_agent
2578 </span></span></span><span class="line"><span class="cl"><span class="go">13) 1 0.31 0.31 0.37% 0.31 0.31 0.37% (anon) [/nix/store/p1zqypy7600fvfyl1v571bljx2l8zhay-zsh-autosuggestions-0.7.0/share/zsh-autosuggestions/zsh-autosuggestions.zsh:458]
2579 </span></span></span><span class="line"><span class="cl"><span class="go">14) 1 0.55 0.55 0.64% 0.24 0.24 0.28% (anon) [/home/thiagoko/.zsh/plugins/zim-input/init.zsh:5]
2580 </span></span></span><span class="line"><span class="cl"><span class="go">15) 1 0.14 0.14 0.16% 0.14 0.14 0.16% prompt_pure_is_inside_container
2581 </span></span></span><span class="line"><span class="cl"><span class="go">16) 1 0.14 0.14 0.16% 0.14 0.14 0.16% compdef
2582 </span></span></span><span class="line"><span class="cl"><span class="go">17) 1 0.09 0.09 0.11% 0.09 0.09 0.11% _zsh_highlight__function_is_autoload_stub_p
2583 </span></span></span><span class="line"><span class="cl"><span class="go">18) 1 0.25 0.25 0.29% 0.08 0.08 0.09% _zsh_highlight__function_callable_p
2584 </span></span></span><span class="line"><span class="cl"><span class="go">19) 1 0.07 0.07 0.09% 0.07 0.07 0.09% _zsh_highlight__is_function_p
2585 </span></span></span><span class="line"><span class="cl"><span class="go">20) 1 0.01 0.01 0.01% 0.01 0.01 0.01% __wezterm_install_bash_prexec
2586 </span></span></span><span class="line"><span class="cl"><span class="go">21) 1 0.01 0.01 0.01% 0.01 0.01 0.01% _zsh_highlight_bind_widgets
2587 </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="gp">#</span> ...
2588 </span></span></code></pre><p>Well, there is nothing interesting here anymore. I mean, <code>zim-completion</code> is
2589 still the main culprit, but nothing to do for now. Instead of looking at
2590 <code>zproof</code>, let's take a look at my <code>~/.zshrc</code> instead:</p>
2591 <pre class="chroma"><code><span class="line"><span class="cl"><span class="c1"># ...</span>
2592 </span></span><span class="line"><span class="cl"><span class="k">if</span> <span class="o">[[</span> <span class="nv">$options</span><span class="o">[</span>zle<span class="o">]</span> <span class="o">=</span> on <span class="o">]]</span><span class="p">;</span> <span class="k">then</span>
2593 </span></span><span class="line"><span class="cl"> <span class="nb">eval</span> <span class="s2">&#34;</span><span class="k">$(</span>/nix/store/sk6wsgp4h477baxypksz9rl8ldwwh9yg-fzf-0.54.0/bin/fzf --zsh<span class="k">)</span><span class="s2">&#34;</span>
2594 </span></span><span class="line"><span class="cl"><span class="k">fi</span>
2595 </span></span><span class="line"><span class="cl">
2596 </span></span><span class="line"><span class="cl"><span class="c1"># ...</span>
2597 </span></span><span class="line"><span class="cl">/nix/store/x3yblr73r5x76dmaanjk3333mvzxc49r-any-nix-shell-1.2.1/bin/any-nix-shell zsh <span class="p">|</span> <span class="nb">source</span> /dev/stdin
2598 </span></span><span class="line"><span class="cl">
2599 </span></span><span class="line"><span class="cl"><span class="c1"># ...</span>
2600 </span></span><span class="line"><span class="cl"><span class="nb">eval</span> <span class="s2">&#34;</span><span class="k">$(</span>/nix/store/330d6k81flfs6w46b44afmncxk57qggv-zoxide-0.9.4/bin/zoxide init zsh <span class="k">)</span><span class="s2">&#34;</span>
2601 </span></span><span class="line"><span class="cl">
2602 </span></span><span class="line"><span class="cl"><span class="c1"># ...</span>
2603 </span></span><span class="line"><span class="cl"><span class="nb">eval</span> <span class="s2">&#34;</span><span class="k">$(</span>/nix/store/8l9j9kdv9m0z0s30lp4yvrc9s5bcbgmx-direnv-2.34.0/bin/direnv hook zsh<span class="k">)</span><span class="s2">&#34;</span>
2604 </span></span></code></pre><p>So you see, starting all those programs during ZSH startup can hurt the shell
2605 startup considerable. Not necessary for commands fast like <code>fzf</code> (that is
2606 written in Go), but let's see
2607 <a href="https://github.com/haslersn/any-nix-shell"><code>any-nix-shell</code></a>, that is written
2608 in shell script:</p>
2609 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> hyperfine <span class="s2">&#34;any-nix-shell zsh&#34;</span>
2610 </span></span><span class="line"><span class="cl"><span class="go">Benchmark 1: any-nix-shell zsh
2611 </span></span></span><span class="line"><span class="cl"><span class="go"> Time (mean ± σ): 16.0 ms ± 1.8 ms [User: 5.6 ms, System: 10.5 ms]
2612 </span></span></span><span class="line"><span class="cl"><span class="go"> Range (min … max): 11.3 ms … 20.3 ms 143 runs
2613 </span></span></span></code></pre><p>This is bad, consistently bad actually. Even for commands that are fast, keep
2614 in mind that there is a difference between the cold and hot start again. For
2615 example, <code>fzf</code>:</p>
2616 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> hyperfine -N <span class="s2">&#34;fzf --zsh&#34;</span>
2617 </span></span><span class="line"><span class="cl"><span class="go">Benchmark 1: fzf --zsh
2618 </span></span></span><span class="line"><span class="cl"><span class="go"> Time (mean ± σ): 2.9 ms ± 0.9 ms [User: 0.6 ms, System: 2.3 ms]
2619 </span></span></span><span class="line"><span class="cl"><span class="go"> Range (min … max): 1.7 ms … 6.8 ms 1113 runs
2620 </span></span></span></code></pre><p>See the range? While 1.7ms is something that is probably difficult to notice,
2621 6.8ms can be noticiable, especially if this accumulates with other slow
2622 starting apps.</p>
2623 <p>And the thing is, all those commands are just generating in the end a fixed
2624 output, at least for the current version of the program. Can we pre-generate
2625 them instead? If using Nix, of course we can:</p>
2626 <pre class="chroma"><code><span class="line"><span class="cl"><span class="c1"># You need to disable the default integration</span>
2627 </span></span><span class="line"><span class="cl"><span class="n">programs</span><span class="o">.</span><span class="n">direnv</span><span class="o">.</span><span class="n">enableZshIntegration</span> <span class="err">=</span> <span class="no">false</span><span class="p">;</span>
2628 </span></span><span class="line"><span class="cl"><span class="n">programs</span><span class="o">.</span><span class="n">fzf</span><span class="o">.</span><span class="n">enableZshIntegration</span> <span class="err">=</span> <span class="no">false</span><span class="p">;</span>
2629 </span></span><span class="line"><span class="cl"><span class="n">programs</span><span class="o">.</span><span class="n">zoxide</span><span class="o">.</span><span class="n">enableZshIntegration</span> <span class="err">=</span> <span class="no">false</span><span class="p">;</span>
2630 </span></span><span class="line"><span class="cl">
2631 </span></span><span class="line"><span class="cl"><span class="n">programs</span><span class="o">.</span><span class="n">zsh</span><span class="o">.</span><span class="n">initExtra</span> <span class="err">=</span>
2632 </span></span><span class="line"><span class="cl"> <span class="c1"># bash</span>
2633 </span></span><span class="line"><span class="cl"> <span class="s1">&#39;&#39;
2634 </span></span></span><span class="line"><span class="cl"><span class="s1"> # any-nix-shell
2635 </span></span></span><span class="line"><span class="cl"><span class="s1"> source </span><span class="si">${</span>
2636 </span></span><span class="line"><span class="cl"> <span class="n">pkgs</span><span class="o">.</span><span class="n">runCommand</span> <span class="s2">&#34;any-nix-shell-zsh&#34;</span> <span class="p">{</span> <span class="p">}</span> <span class="s1">&#39;&#39;
2637 </span></span></span><span class="line"><span class="cl"><span class="s1"> </span><span class="si">${</span><span class="n">lib</span><span class="o">.</span><span class="n">getExe</span> <span class="n">pkgs</span><span class="o">.</span><span class="n">any-nix-shell</span><span class="si">}</span><span class="s1"> zsh &gt; $out
2638 </span></span></span><span class="line"><span class="cl"><span class="s1"> &#39;&#39;</span>
2639 </span></span><span class="line"><span class="cl"> <span class="si">}</span><span class="s1">
2640 </span></span></span><span class="line"><span class="cl"><span class="s1">
2641 </span></span></span><span class="line"><span class="cl"><span class="s1"> # fzf
2642 </span></span></span><span class="line"><span class="cl"><span class="s1"> source </span><span class="si">${</span><span class="n">config</span><span class="o">.</span><span class="n">programs</span><span class="o">.</span><span class="n">fzf</span><span class="o">.</span><span class="n">package</span><span class="si">}</span><span class="s1">/share/fzf/completion.zsh
2643 </span></span></span><span class="line"><span class="cl"><span class="s1"> source </span><span class="si">${</span><span class="n">config</span><span class="o">.</span><span class="n">programs</span><span class="o">.</span><span class="n">fzf</span><span class="o">.</span><span class="n">package</span><span class="si">}</span><span class="s1">/share/fzf/key-bindings.zsh
2644 </span></span></span><span class="line"><span class="cl"><span class="s1">
2645 </span></span></span><span class="line"><span class="cl"><span class="s1"> # zoxide
2646 </span></span></span><span class="line"><span class="cl"><span class="s1"> source </span><span class="si">${</span>
2647 </span></span><span class="line"><span class="cl"> <span class="n">pkgs</span><span class="o">.</span><span class="n">runCommand</span> <span class="s2">&#34;zoxide-init-zsh&#34;</span> <span class="p">{</span> <span class="p">}</span> <span class="s1">&#39;&#39;
2648 </span></span></span><span class="line"><span class="cl"><span class="s1"> </span><span class="si">${</span><span class="n">lib</span><span class="o">.</span><span class="n">getExe</span> <span class="n">config</span><span class="o">.</span><span class="n">programs</span><span class="o">.</span><span class="n">zoxide</span><span class="o">.</span><span class="n">package</span><span class="si">}</span><span class="s1"> init zsh &gt; $out
2649 </span></span></span><span class="line"><span class="cl"><span class="s1"> &#39;&#39;</span>
2650 </span></span><span class="line"><span class="cl"> <span class="si">}</span><span class="s1">
2651 </span></span></span><span class="line"><span class="cl"><span class="s1">
2652 </span></span></span><span class="line"><span class="cl"><span class="s1"> # direnv
2653 </span></span></span><span class="line"><span class="cl"><span class="s1"> source </span><span class="si">${</span>
2654 </span></span><span class="line"><span class="cl"> <span class="n">pkgs</span><span class="o">.</span><span class="n">runCommand</span> <span class="s2">&#34;direnv-hook-zsh&#34;</span> <span class="p">{</span> <span class="p">}</span> <span class="s1">&#39;&#39;
2655 </span></span></span><span class="line"><span class="cl"><span class="s1"> </span><span class="si">${</span><span class="n">lib</span><span class="o">.</span><span class="n">getExe</span> <span class="n">config</span><span class="o">.</span><span class="n">programs</span><span class="o">.</span><span class="n">direnv</span><span class="o">.</span><span class="n">package</span><span class="si">}</span><span class="s1"> hook zsh &gt; $out
2656 </span></span></span><span class="line"><span class="cl"><span class="s1"> &#39;&#39;</span>
2657 </span></span><span class="line"><span class="cl"> <span class="si">}</span><span class="s1">
2658 </span></span></span><span class="line"><span class="cl"><span class="s1"> &#39;&#39;</span><span class="p">;</span>
2659 </span></span></code></pre><p>So we can use <code>pkgs.runCommand</code> to run those commands during build time and
2660 <code>source</code> the result. <code>fzf</code> actually doesn't need this since we have the files
2661 already generated in the package. I think this is one of those things that
2662 really shows the power of Nix: I wouldn't do something similar if I didn't use
2663 Nix because the risk of breaking something later is big (e.g.: forgetting to
2664 update the generated files), but Nix makes those things trivial.</p>
2665 <p>Let's run <code>hyperfine</code> again:</p>
2666 <pre><code>$ hyperfine &quot;zsh -ic exit&quot;
2667 Benchmark 1: zsh -ic exit
2668 Time (mean ± σ): 162.3 ms ± 4.9 ms [User: 52.7 ms, System: 111.1 ms]
2669 Range (min … max): 153.0 ms … 173.4 ms 19 runs
2670 </code></pre>
2671 <p>Another good improvement. The last change I did is switching between
2672 <a href="https://github.com/zsh-users/zsh-syntax-highlighting"><code>zsh-syntax-highlighting</code></a>
2673 to
2674 <a href="https://github.com/zdharma-continuum/fast-syntax-highlighting"><code>zsh-fast-syntax-highlighting</code></a>,
2675 that is supposed to be faster and have better highlighting too. I got that from
2676 <code>_zsh_highlight_load_highlighters</code> using 26% of the time from my <code>zprof</code> above.
2677 And for the final <code>hyperfine</code> in my laptop:</p>
2678 <pre><code>$ hyperfine &quot;zsh -ic exit&quot;
2679 Benchmark 1: zsh -ic exit
2680 Time (mean ± σ): 138.3 ms ± 7.1 ms [User: 47.5 ms, System: 91.9 ms]
2681 Range (min … max): 123.8 ms … 157.9 ms 21 runs
2682 </code></pre>
2683 <p>A ~36% improvement, not bad. Let's see how it fares in my Chromebook:</p>
2684 <pre><code>$ hyperfine &quot;zsh -ic exit&quot;
2685 Benchmark 1: zsh -ic exit
2686 Time (mean ± σ): 278.2 ms ± 46.9 ms [User: 88.0 ms, System: 184.8 ms]
2687 Range (min … max): 204.7 ms … 368.5 ms 11 runs
2688 </code></pre>
2689 <p>An even more impressive ~59% improvement. And yes, the shell startup now feels
2690 much better.</p></description><pubDate>Thu, 01 Aug 2024 00:00:00 +0000</pubDate><guid>https://kokada.dev/blog/troubleshoting-zsh-lag-and-solutions-with-nix/</guid></item><item><title>Quick bits: realise Nix symlinks</title><link>https://kokada.dev/blog/quick-bits-realise-nix-symlinks/</link><description><p>When you are using Nix, especially with
2691 <a href="https://github.com/nix-community/home-manager/">Home-Manager</a>, there are times
2692 when you want to test something or maybe debug some issue in your
2693 configuration. Those times it would be really convenient if you could avoid a
2694 rebuild of your Home-Manager configuration, since this takes time until
2695 evaluation and activation.</p>
2696 <p>For those times I have this small script in my Nix configuration called
2697 <code>realise-symlinks</code>, that is defined as:</p>
2698 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span> <span class="n">pkgs</span><span class="o">,</span> <span class="o">...</span> <span class="p">}:</span>
2699 </span></span><span class="line"><span class="cl"><span class="k">let</span>
2700 </span></span><span class="line"><span class="cl"> <span class="n">realise-symlink</span> <span class="o">=</span> <span class="n">pkgs</span><span class="o">.</span><span class="n">writeShellApplication</span> <span class="p">{</span>
2701 </span></span><span class="line"><span class="cl"> <span class="n">name</span> <span class="o">=</span> <span class="s2">&#34;realise-symlink&#34;</span><span class="p">;</span>
2702 </span></span><span class="line"><span class="cl"> <span class="n">runtimeInputs</span> <span class="o">=</span> <span class="k">with</span> <span class="n">pkgs</span><span class="p">;</span> <span class="p">[</span> <span class="n">coreutils</span> <span class="p">];</span>
2703 </span></span><span class="line"><span class="cl"> <span class="n">text</span> <span class="o">=</span> <span class="s1">&#39;&#39;
2704 </span></span></span><span class="line"><span class="cl"><span class="s1"> for file in &#34;$@&#34;; do
2705 </span></span></span><span class="line"><span class="cl"><span class="s1"> if [[ -L &#34;$file&#34; ]]; then
2706 </span></span></span><span class="line"><span class="cl"><span class="s1"> if [[ -d &#34;$file&#34; ]]; then
2707 </span></span></span><span class="line"><span class="cl"><span class="s1"> tmpdir=&#34;</span><span class="se">&#39;&#39;$</span><span class="s1">{file}.tmp&#34;
2708 </span></span></span><span class="line"><span class="cl"><span class="s1"> mkdir -p &#34;$tmpdir&#34;
2709 </span></span></span><span class="line"><span class="cl"><span class="s1"> cp --verbose --recursive &#34;$file&#34;/* &#34;$tmpdir&#34;
2710 </span></span></span><span class="line"><span class="cl"><span class="s1"> unlink &#34;$file&#34;
2711 </span></span></span><span class="line"><span class="cl"><span class="s1"> mv &#34;$tmpdir&#34; &#34;$file&#34;
2712 </span></span></span><span class="line"><span class="cl"><span class="s1"> chmod --changes --recursive +w &#34;$file&#34;
2713 </span></span></span><span class="line"><span class="cl"><span class="s1"> else
2714 </span></span></span><span class="line"><span class="cl"><span class="s1"> cp --verbose --remove-destination &#34;$(readlink &#34;$file&#34;)&#34; &#34;$file&#34;
2715 </span></span></span><span class="line"><span class="cl"><span class="s1"> chmod --changes +w &#34;$file&#34;
2716 </span></span></span><span class="line"><span class="cl"><span class="s1"> fi
2717 </span></span></span><span class="line"><span class="cl"><span class="s1"> else
2718 </span></span></span><span class="line"><span class="cl"><span class="s1"> &gt;&amp;2 echo &#34;Not a symlink: $file&#34;
2719 </span></span></span><span class="line"><span class="cl"><span class="s1"> exit 1
2720 </span></span></span><span class="line"><span class="cl"><span class="s1"> fi
2721 </span></span></span><span class="line"><span class="cl"><span class="s1"> done
2722 </span></span></span><span class="line"><span class="cl"><span class="s1"> &#39;&#39;</span><span class="p">;</span>
2723 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
2724 </span></span><span class="line"><span class="cl"><span class="k">in</span>
2725 </span></span><span class="line"><span class="cl"><span class="p">{</span>
2726 </span></span><span class="line"><span class="cl"> <span class="n">home</span><span class="o">.</span><span class="n">packages</span> <span class="o">=</span> <span class="p">[</span> <span class="n">realise-symlink</span> <span class="p">];</span>
2727 </span></span><span class="line"><span class="cl"><span class="p">}</span>
2728 </span></span></code></pre><p>The idea of this script is that you can call it against a symlink against Nix
2729 store and it will realise, e.g.: convert to an &quot;actual&quot; file, e.g.:</p>
2730 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> ls -lah .zshrc
2731 </span></span><span class="line"><span class="cl"><span class="go">lrwxrwxrwx 1 thiagoko users 69 Aug 1 00:10 .zshrc -&gt; /nix/store/glz018yyh0qfqc9lywx1yhr7c3l96lv7-home-manager-files/.zshrc
2732 </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="err">
2733 </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="gp">$</span> realise-symlink .zshrc
2734 </span></span><span class="line"><span class="cl"><span class="go">removed &#39;.zshrc&#39;
2735 </span></span></span><span class="line"><span class="cl"><span class="go">&#39;/nix/store/glz018yyh0qfqc9lywx1yhr7c3l96lv7-home-manager-files/.zshrc&#39; -&gt; &#39;.zshrc&#39;
2736 </span></span></span><span class="line"><span class="cl"><span class="go">mode of &#39;.zshrc&#39; changed from 0444 (r--r--r--) to 0644 (rw-r--r--)
2737 </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="err">
2738 </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="gp">$</span> ls -lah .zshrc
2739 </span></span><span class="line"><span class="cl"><span class="go">-rw-r--r-- 1 thiagoko users 5.8K Aug 1 00:16 .zshrc
2740 </span></span></span></code></pre><p>It also add write permissions to the resulting file, to make it easier to edit.
2741 By the way, it also works with directories:</p>
2742 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> ls -lah zim-completion
2743 </span></span><span class="line"><span class="cl"><span class="go">lrwxrwxrwx 1 thiagoko users 90 Aug 1 00:10 zim-completion -&gt; /nix/store/glz018yyh0qfqc9lywx1yhr7c3l96lv7-home-manager-files/.zsh/plugins/zim-completion
2744 </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="err">
2745 </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="gp">$</span> realise-symlink zim-completion
2746 </span></span><span class="line"><span class="cl"><span class="go">&#39;zim-completion/init.zsh&#39; -&gt; &#39;zim-completion.tmp/init.zsh&#39;
2747 </span></span></span><span class="line"><span class="cl"><span class="go">&#39;zim-completion/init.zsh.zwc&#39; -&gt; &#39;zim-completion.tmp/init.zsh.zwc&#39;
2748 </span></span></span><span class="line"><span class="cl"><span class="go">&#39;zim-completion/LICENSE&#39; -&gt; &#39;zim-completion.tmp/LICENSE&#39;
2749 </span></span></span><span class="line"><span class="cl"><span class="go">&#39;zim-completion/README.md&#39; -&gt; &#39;zim-completion.tmp/README.md&#39;
2750 </span></span></span><span class="line"><span class="cl"><span class="go">mode of &#39;zim-completion/init.zsh&#39; changed from 0444 (r--r--r--) to 0644 (rw-r--r--)
2751 </span></span></span><span class="line"><span class="cl"><span class="go">mode of &#39;zim-completion/init.zsh.zwc&#39; changed from 0444 (r--r--r--) to 0644 (rw-r--r--)
2752 </span></span></span><span class="line"><span class="cl"><span class="go">mode of &#39;zim-completion/LICENSE&#39; changed from 0444 (r--r--r--) to 0644 (rw-r--r--)
2753 </span></span></span><span class="line"><span class="cl"><span class="go">mode of &#39;zim-completion/README.md&#39; changed from 0444 (r--r--r--) to 0644 (rw-r--r--)
2754 </span></span></span><span class="line"><span class="cl"><span class="go"></span><span class="err">
2755 </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="gp">$</span> ls -lah zim-completion
2756 </span></span><span class="line"><span class="cl"><span class="go">total 28K
2757 </span></span></span><span class="line"><span class="cl"><span class="go">drwxr-xr-x 1 thiagoko users 72 Aug 1 00:18 .
2758 </span></span></span><span class="line"><span class="cl"><span class="go">drwxr-xr-x 1 thiagoko users 130 Aug 1 00:18 ..
2759 </span></span></span><span class="line"><span class="cl"><span class="go">-rw-r--r-- 1 thiagoko users 5.3K Aug 1 00:18 init.zsh
2760 </span></span></span><span class="line"><span class="cl"><span class="go">-rw-r--r-- 1 thiagoko users 12K Aug 1 00:18 init.zsh.zwc
2761 </span></span></span><span class="line"><span class="cl"><span class="go">-rw-r--r-- 1 thiagoko users 1.3K Aug 1 00:18 LICENSE
2762 </span></span></span><span class="line"><span class="cl"><span class="go">-rw-r--r-- 1 thiagoko users 2.2K Aug 1 00:18 README.md
2763 </span></span></span></code></pre><p>After you finish whatever you are testing, to return to your configuration you
2764 can just delete those files and re-run your Home-Manager activation:</p>
2765 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> rm -rf .zshrc
2766 </span></span><span class="line"><span class="cl"><span class="err">
2767 </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="gp">$</span> sudo systemctl restart home-manager-&lt;user&gt;.service <span class="c1"># or `home-manager switch`</span>
2768 </span></span><span class="line"><span class="cl"><span class="err">
2769 </span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="gp">$</span> ls -lah .zshrc
2770 </span></span><span class="line"><span class="cl"><span class="go">lrwxrwxrwx 1 thiagoko users 69 Aug 1 00:20 .zshrc -&gt; /nix/store/glz018yyh0qfqc9lywx1yhr7c3l96lv7-home-manager-files/.zshrc
2771 </span></span></span></code></pre><p>It even works with system files (i.e.: NixOS):</p>
2772 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> sudo realise-symlink /etc/nix/nix.conf
2773 </span></span><span class="line"><span class="cl"><span class="go">[sudo] password for thiagoko:
2774 </span></span></span><span class="line"><span class="cl"><span class="go">removed &#39;nix.conf&#39;
2775 </span></span></span><span class="line"><span class="cl"><span class="go">&#39;/etc/static/nix/nix.conf&#39; -&gt; &#39;nix.conf&#39;
2776 </span></span></span><span class="line"><span class="cl"><span class="go">mode of &#39;nix.conf&#39; changed from 0444 (r--r--r--) to 0644 (rw-r--r--)
2777 </span></span></span></code></pre><p>But I never needed for this case since it is more rare to me to experiment with
2778 OS level configuration.</p></description><pubDate>Thu, 01 Aug 2024 00:00:00 +0000</pubDate><guid>https://kokada.dev/blog/quick-bits-realise-nix-symlinks/</guid></item><item><title>Generating YAML files with Nix</title><link>https://kokada.dev/blog/generating-yaml-files-with-nix/</link><description><p>I hate YAML. Instead of writing an essay on why I hate YAML, I can just link to
2779 <a href="https://noyaml.com/">noyaml.com</a>. In my personal projects I will never use it,
2780 preferring either JSON, <a href="https://toml.io/en/">TOML</a> or even plain old
2781 <a href="https://en.wikipedia.org/wiki/INI_file">INI</a> files depending on the use case.
2782 However the ship has sailed already, there are tons of projects everywhere that
2783 uses YAML: from most CI systems (<a href="https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions">GitHub
2784 Actions</a>,
2785 <a href="https://circleci.com/docs/introduction-to-yaml-configurations/">CircleCI</a>,
2786 <a href="https://docs.travis-ci.com/user/build-config-yaml">Travis</a>, <em>et tu</em>
2787 <a href="https://man.sr.ht/builds.sr.ht/">builds.sr.ht</a>), to
2788 <a href="https://kubernetes.io/docs/concepts/overview/working-with-objects/">Kubernetes</a>,
2789 or in almost every
2790 <a href="https://guides.rubyonrails.org/configuring.html#configuring-a-database">Rails</a>
2791 application.</p>
2792 <p>One way to avoid at least some issues with the language is to write YAML in
2793 another language. I will show my solution in one of my <a href="https://github.com/thiagokokada/nix-configs/">personal
2794 repositories</a>, writing Nix to
2795 generate GitHub Actions configuration files. Bonus points for validating the
2796 result against the schema of GitHub Actions, so the famous &quot;this is supposed to
2797 be string instead of a list of strings&quot; is gone.</p>
2798 <p>Let's start with the basics: YAML is supposed to be a <a href="https://stackoverflow.com/a/1729545">superset of
2799 JSON</a>. What that means is that a JSON file
2800 <a href="https://yaml.org/spec/1.2-old/spec.html#id2759572">can be parsed</a> by a YAML
2801 parser. And Nix itself generates JSON natively, after all, Nix can be imagined
2802 as <a href="https://nix.dev/tutorials/nix-language.html">&quot;JSON with functions&quot;</a>.</p>
2803 <p>To make things easier, I will assume that you have the <code>nix-commands</code> and
2804 <code>flakes</code> enabled as <code>experimental-features</code> in your Nix configuration. If not,
2805 go <a href="https://wiki.nixos.org/wiki/Flakes">here</a>.</p>
2806 <p>Using the <code>nix eval</code> command, we can generate a JSON expression from Nix by:</p>
2807 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> nix <span class="nb">eval</span> --expr <span class="s1">&#39;{ foo = &#34;bar&#34;; }&#39;</span> --json
2808 </span></span><span class="line"><span class="cl"><span class="go">{&#34;foo&#34;:&#34;bar&#34;}
2809 </span></span></span></code></pre><p>However, typing long excerpts of Nix code inside the console would be
2810 impractical. We can write the following code inside a <code>foo.nix</code> file instead:</p>
2811 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
2812 </span></span><span class="line"><span class="cl"> <span class="n">foo</span> <span class="o">=</span> <span class="s2">&#34;bar&#34;</span><span class="p">;</span>
2813 </span></span><span class="line"><span class="cl"><span class="p">}</span>
2814 </span></span></code></pre><p>And:</p>
2815 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> nix <span class="nb">eval</span> --file foo.nix --json
2816 </span></span><span class="line"><span class="cl"><span class="go">{&#34;foo&#34;:&#34;bar&#34;}
2817 </span></span></span></code></pre><p>While you can use a JSON output as an input for YAML parsers, it is probably
2818 not the <a href="https://metacpan.org/pod/JSON::XS#JSON-and-YAML">best idea</a>. Sadly (or
2819 maybe not), Nix has no native functionality to export data to YAML. However,
2820 since we are using Nix, it is trivial to use <code>nixpkgs</code> to use some program to
2821 convert from JSON to YAML.</p>
2822 <p>To start, let's create a new directory, move our <code>foo.nix</code> file to it, create a
2823 new <code>flake.nix</code> file and put the following contents:</p>
2824 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
2825 </span></span><span class="line"><span class="cl"> <span class="n">description</span> <span class="o">=</span> <span class="s2">&#34;Generate YAML files with Nix&#34;</span><span class="p">;</span>
2826 </span></span><span class="line"><span class="cl">
2827 </span></span><span class="line"><span class="cl"> <span class="n">inputs</span> <span class="o">=</span> <span class="p">{</span>
2828 </span></span><span class="line"><span class="cl"> <span class="n">nixpkgs</span><span class="o">.</span><span class="n">url</span> <span class="o">=</span> <span class="s2">&#34;github:NixOS/nixpkgs/nixos-unstable&#34;</span><span class="p">;</span>
2829 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
2830 </span></span><span class="line"><span class="cl">
2831 </span></span><span class="line"><span class="cl"> <span class="n">outputs</span> <span class="o">=</span> <span class="p">{</span> <span class="n">nixpkgs</span><span class="o">,</span> <span class="o">...</span> <span class="p">}:</span>
2832 </span></span><span class="line"><span class="cl"> <span class="p">{</span>
2833 </span></span><span class="line"><span class="cl"> <span class="n">packages</span><span class="o">.</span><span class="n">x86_64-linux</span> <span class="o">=</span>
2834 </span></span><span class="line"><span class="cl"> <span class="k">let</span>
2835 </span></span><span class="line"><span class="cl"> <span class="k">inherit</span> <span class="p">(</span><span class="n">nixpkgs</span><span class="p">)</span> <span class="n">lib</span><span class="p">;</span>
2836 </span></span><span class="line"><span class="cl"> <span class="n">pkgs</span> <span class="o">=</span> <span class="kn">import</span> <span class="n">nixpkgs</span> <span class="p">{</span> <span class="n">system</span> <span class="o">=</span> <span class="s2">&#34;x86_64-linux&#34;</span><span class="p">;</span> <span class="p">};</span>
2837 </span></span><span class="line"><span class="cl"> <span class="k">in</span>
2838 </span></span><span class="line"><span class="cl"> <span class="p">{</span>
2839 </span></span><span class="line"><span class="cl"> <span class="n">toYAML</span> <span class="o">=</span> <span class="n">pkgs</span><span class="o">.</span><span class="n">runCommand</span> <span class="s2">&#34;toYAML&#34;</span> <span class="p">{</span>
2840 </span></span><span class="line"><span class="cl"> <span class="n">buildInputs</span> <span class="o">=</span> <span class="k">with</span> <span class="n">pkgs</span><span class="p">;</span> <span class="p">[</span> <span class="n">yj</span> <span class="p">];</span>
2841 </span></span><span class="line"><span class="cl"> <span class="n">json</span> <span class="o">=</span> <span class="nb">builtins</span><span class="o">.</span><span class="n">toJSON</span> <span class="p">(</span><span class="kn">import</span> <span class="sr">./go.nix</span><span class="p">);</span>
2842 </span></span><span class="line"><span class="cl"> <span class="n">passAsFile</span> <span class="o">=</span> <span class="p">[</span> <span class="s2">&#34;json&#34;</span> <span class="p">];</span> <span class="c1"># will be available as `$jsonPath`</span>
2843 </span></span><span class="line"><span class="cl"> <span class="p">}</span> <span class="s1">&#39;&#39;
2844 </span></span></span><span class="line"><span class="cl"><span class="s1"> mkdir -p $out
2845 </span></span></span><span class="line"><span class="cl"><span class="s1"> yj -jy &lt; &#34;$jsonPath&#34; &gt; $out/go.yaml
2846 </span></span></span><span class="line"><span class="cl"><span class="s1"> &#39;&#39;</span><span class="p">;</span>
2847 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
2848 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
2849 </span></span><span class="line"><span class="cl"><span class="p">}</span>
2850 </span></span></code></pre><p>We are loading the <code>./foo.nix</code> as a Nix file, converting it to JSON with
2851 <code>builtins.toJSON</code> function, and finally, using <code>pkgs.runCommand</code> and its
2852 <code>passAsFile</code> option to load the contents of the JSON file into
2853 <a href="https://github.com/sclevine/yj">yj</a>, that converts between serialisation
2854 formats (<code>-jy</code> flag means &quot;JSON to YAML&quot;). The reason I choose <code>yj</code> is mostly
2855 because it is a single binary Go program, but you can use whatever you prefer.</p>
2856 <p>By the way, there is a
2857 <a href="https://github.com/NixOS/nixpkgs/blob/9f918d616c5321ad374ae6cb5ea89c9e04bf3e58/lib/generators.nix#L805"><code>lib.generators.toYAML</code></a>
2858 inside <code>nixpkgs.lib</code>, but as of the day of this post it only calls
2859 <code>lib.strings.toJSON</code> (that in turn, calls <code>builtins.toJSON</code>). So it doesn't
2860 really help here. Another option would be <code>pkgs.formats.yaml.generate</code>, that
2861 converts between formats, but it calls
2862 <a href="https://github.com/NixOS/nixpkgs/blob/008ceae1a2b47a84d7aa01e55f8468272c70b9ee/pkgs/pkgs-lib/formats.nix#L77-L84"><code>remarshal</code></a>
2863 (in Python), so not my favorite choice.</p>
2864 <p>If we run the following commands, we can see the result:</p>
2865 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> nix build .#packages.x86_64-linux.toYAML
2866 </span></span><span class="line"><span class="cl"><span class="gp">$</span> cat result/foo.yaml
2867 </span></span><span class="line"><span class="cl"><span class="go">foo: bar
2868 </span></span></span></code></pre><p>That is the basic idea. To have a more realistic example, let's convert the
2869 <a href="https://github.com/thiagokokada/blog/blob/4e3f25485c6682f3e066b219df2290934bc0d256/.github/workflows/go.yml"><code>go.yml</code></a>,
2870 that builds this blog, to Nix:</p>
2871 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
2872 </span></span><span class="line"><span class="cl"> <span class="n">name</span> <span class="o">=</span> <span class="s2">&#34;Go&#34;</span><span class="p">;</span>
2873 </span></span><span class="line"><span class="cl"> <span class="n">on</span><span class="o">.</span><span class="n">push</span><span class="o">.</span><span class="n">branches</span> <span class="o">=</span> <span class="p">[</span> <span class="s2">&#34;main&#34;</span> <span class="p">];</span>
2874 </span></span><span class="line"><span class="cl">
2875 </span></span><span class="line"><span class="cl"> <span class="n">jobs</span> <span class="o">=</span> <span class="p">{</span>
2876 </span></span><span class="line"><span class="cl"> <span class="n">build</span> <span class="o">=</span> <span class="p">{</span>
2877 </span></span><span class="line"><span class="cl"> <span class="n">runs-on</span> <span class="o">=</span> <span class="s2">&#34;ubuntu-latest&#34;</span><span class="p">;</span>
2878 </span></span><span class="line"><span class="cl"> <span class="n">permissions</span><span class="o">.</span><span class="n">contents</span> <span class="o">=</span> <span class="s2">&#34;write&#34;</span><span class="p">;</span>
2879 </span></span><span class="line"><span class="cl"> <span class="n">steps</span> <span class="o">=</span> <span class="p">[</span>
2880 </span></span><span class="line"><span class="cl"> <span class="p">{</span> <span class="n">uses</span> <span class="o">=</span> <span class="s2">&#34;actions/checkout@v4&#34;</span><span class="p">;</span> <span class="p">}</span>
2881 </span></span><span class="line"><span class="cl"> <span class="p">{</span>
2882 </span></span><span class="line"><span class="cl"> <span class="n">name</span> <span class="o">=</span> <span class="s2">&#34;Set up Go&#34;</span><span class="p">;</span>
2883 </span></span><span class="line"><span class="cl"> <span class="n">uses</span> <span class="o">=</span> <span class="s2">&#34;actions/checkout@v4&#34;</span><span class="p">;</span>
2884 </span></span><span class="line"><span class="cl"> <span class="s2">&#34;with&#34;</span><span class="o">.</span><span class="n">go-version</span> <span class="o">=</span> <span class="s2">&#34;1.21&#34;</span><span class="p">;</span>
2885 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
2886 </span></span><span class="line"><span class="cl"> <span class="p">{</span>
2887 </span></span><span class="line"><span class="cl"> <span class="n">name</span> <span class="o">=</span> <span class="s2">&#34;Update&#34;</span><span class="p">;</span>
2888 </span></span><span class="line"><span class="cl"> <span class="n">run</span> <span class="o">=</span> <span class="s2">&#34;make&#34;</span><span class="p">;</span>
2889 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
2890 </span></span><span class="line"><span class="cl"> <span class="p">{</span>
2891 </span></span><span class="line"><span class="cl"> <span class="n">name</span> <span class="o">=</span> <span class="s2">&#34;Publish&#34;</span><span class="p">;</span>
2892 </span></span><span class="line"><span class="cl"> <span class="n">run</span> <span class="o">=</span> <span class="s2">&#34;make publish&#34;</span><span class="p">;</span>
2893 </span></span><span class="line"><span class="cl"> <span class="n">env</span><span class="o">.</span><span class="n">MATAROA_TOKEN</span> <span class="o">=</span> <span class="s1">&#39;&#39;</span><span class="se">&#39;&#39;$</span><span class="s1">{{ secrets.MATAROA_TOKEN }}&#39;&#39;</span><span class="p">;</span>
2894 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
2895 </span></span><span class="line"><span class="cl"> <span class="p">{</span>
2896 </span></span><span class="line"><span class="cl"> <span class="n">name</span> <span class="o">=</span> <span class="s2">&#34;Commit&#34;</span><span class="p">;</span>
2897 </span></span><span class="line"><span class="cl"> <span class="n">uses</span> <span class="o">=</span> <span class="s2">&#34;stefanzweifel/git-auto-commit-action@v5&#34;</span><span class="p">;</span>
2898 </span></span><span class="line"><span class="cl"> <span class="s2">&#34;with&#34;</span><span class="o">.</span><span class="n">commit_message</span> <span class="o">=</span> <span class="s2">&#34;README/rss:update&#34;</span><span class="p">;</span>
2899 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
2900 </span></span><span class="line"><span class="cl"> <span class="p">];</span>
2901 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
2902 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
2903 </span></span><span class="line"><span class="cl"><span class="p">}</span>
2904 </span></span></code></pre><p>Some interesting things to highlight: <code>with</code> is a reserved word in Nix, so we
2905 need to quote it. Not a problem, but something to be aware. And the template
2906 string in GitHub Actions uses the same <code>${}</code> that Nix uses, so we need to
2907 escape.</p>
2908 <p>And after running the following commands:</p>
2909 <pre><code>$ nix build .#packages.x86_64-linux.toYAML
2910 $ cat result/go.yaml
2911 jobs:
2912 build:
2913 permissions:
2914 contents: write
2915 runs-on: ubuntu-latest
2916 steps:
2917 - uses: actions/checkout@v4
2918 - name: Set up Go
2919 uses: actions/checkout@v4
2920 with:
2921 go-version: &quot;1.21&quot;
2922 - name: Update
2923 run: make
2924 - env:
2925 MATAROA_TOKEN: ${{ secrets.MATAROA_TOKEN }}
2926 name: Publish
2927 run: make publish
2928 - name: Commit
2929 uses: stefanzweifel/git-auto-commit-action@v5
2930 with:
2931 commit_message: README/rss:update
2932 name: Go
2933 &quot;on&quot;:
2934 push:
2935 branches:
2936 - main
2937 </code></pre>
2938 <p>Yes, the keys are not in the same order as we defined, since Nix, like most
2939 programming languages (with the exception of
2940 <a href="https://mail.python.org/pipermail/python-dev/2017-December/151283.html">Python</a>),
2941 do not guarantee the insertion order in maps/dicts/attrsets/whatever. But I
2942 really hope whatever is consuming your YAML is not relying in the order the
2943 keys are defined (this would be more cursed than YAML already is).</p>
2944 <p>So that is basically it. For the bonus points that I talked at the start of the
2945 post, we can modify <code>pkgs.runCommand</code> to run some kind of validator. I use
2946 <a href="https://github.com/mpalmer/action-validator"><code>action-validator</code></a>, one that I
2947 particularly packaged in
2948 <a href="https://github.com/NixOS/nixpkgs/pull/260217">nixpkgs</a> to use in those cases.
2949 But you could use e.g.: a validator of Kubernetes YAML. Or a generic YAML lint
2950 like this <a href="https://github.com/adrienverge/yamllint">one</a>. The possibilities are
2951 endless.</p>
2952 <p>Let's modify our <code>flake.nix</code> to add the validation:</p>
2953 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
2954 </span></span><span class="line"><span class="cl"> <span class="c1"># ...</span>
2955 </span></span><span class="line"><span class="cl"> <span class="n">outputs</span> <span class="o">=</span> <span class="p">{</span> <span class="n">nixpkgs</span><span class="o">,</span> <span class="o">...</span> <span class="p">}:</span>
2956 </span></span><span class="line"><span class="cl"> <span class="p">{</span>
2957 </span></span><span class="line"><span class="cl"> <span class="n">packages</span><span class="o">.</span><span class="n">x86_64-linux</span> <span class="o">=</span>
2958 </span></span><span class="line"><span class="cl"> <span class="k">let</span>
2959 </span></span><span class="line"><span class="cl"> <span class="k">inherit</span> <span class="p">(</span><span class="n">nixpkgs</span><span class="p">)</span> <span class="n">lib</span><span class="p">;</span>
2960 </span></span><span class="line"><span class="cl"> <span class="n">pkgs</span> <span class="o">=</span> <span class="kn">import</span> <span class="n">nixpkgs</span> <span class="p">{</span> <span class="n">system</span> <span class="o">=</span> <span class="s2">&#34;x86_64-linux&#34;</span><span class="p">;</span> <span class="p">};</span>
2961 </span></span><span class="line"><span class="cl"> <span class="k">in</span>
2962 </span></span><span class="line"><span class="cl"> <span class="p">{</span>
2963 </span></span><span class="line"><span class="cl"> <span class="n">toYAML</span> <span class="o">=</span> <span class="n">pkgs</span><span class="o">.</span><span class="n">runCommand</span> <span class="s2">&#34;toYAML&#34;</span> <span class="p">{</span>
2964 </span></span><span class="line"><span class="cl"> <span class="n">buildInputs</span> <span class="o">=</span> <span class="k">with</span> <span class="n">pkgs</span><span class="p">;</span> <span class="p">[</span> <span class="n">action-validator</span> <span class="n">yj</span> <span class="p">];</span>
2965 </span></span><span class="line"><span class="cl"> <span class="n">json</span> <span class="o">=</span> <span class="nb">builtins</span><span class="o">.</span><span class="n">toJSON</span> <span class="p">(</span><span class="kn">import</span> <span class="sr">./go.nix</span><span class="p">);</span>
2966 </span></span><span class="line"><span class="cl"> <span class="n">passAsFile</span> <span class="o">=</span> <span class="p">[</span> <span class="s2">&#34;json&#34;</span> <span class="p">];</span>
2967 </span></span><span class="line"><span class="cl"> <span class="p">}</span> <span class="s1">&#39;&#39;
2968 </span></span></span><span class="line"><span class="cl"><span class="s1"> mkdir -p $out
2969 </span></span></span><span class="line"><span class="cl"><span class="s1"> yj -jy &lt; &#34;$jsonPath&#34; &gt; $out/go.yaml
2970 </span></span></span><span class="line"><span class="cl"><span class="s1"> action-validator -v $out/go.yaml
2971 </span></span></span><span class="line"><span class="cl"><span class="s1"> &#39;&#39;</span><span class="p">;</span>
2972 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
2973 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
2974 </span></span><span class="line"><span class="cl"><span class="p">}</span>
2975 </span></span></code></pre><p>And let's add an error in our <code>go.nix</code> file:</p>
2976 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gh">diff --git a/go.nix b/go.nix
2977 </span></span></span><span class="line"><span class="cl"><span class="gh">index 25e0596..8c00033 100644
2978 </span></span></span><span class="line"><span class="cl"><span class="gh"></span><span class="gd">--- a/go.nix
2979 </span></span></span><span class="line"><span class="cl"><span class="gd"></span><span class="gi">+++ b/go.nix
2980 </span></span></span><span class="line"><span class="cl"><span class="gi"></span><span class="gu">@@ -5,7 +5,7 @@
2981 </span></span></span><span class="line"><span class="cl"><span class="gu"></span> jobs = {
2982 </span></span><span class="line"><span class="cl"> build = {
2983 </span></span><span class="line"><span class="cl"> runs-on = &#34;ubuntu-latest&#34;;
2984 </span></span><span class="line"><span class="cl"><span class="gd">- permissions.contents = &#34;write&#34;;
2985 </span></span></span><span class="line"><span class="cl"><span class="gd"></span><span class="gi">+ permissions.contents = [ &#34;write&#34; ];
2986 </span></span></span><span class="line"><span class="cl"><span class="gi"></span> steps = [
2987 </span></span><span class="line"><span class="cl"> { uses = &#34;actions/checkout@v4&#34;; }
2988 </span></span><span class="line"><span class="cl"> {
2989 </span></span></code></pre><p>Finally, let's try to build our YAML file again:</p>
2990 <pre><code>$ nix build .#packages.x86_64-linux.toYAML
2991 error: builder for '/nix/store/j8wr6j1pvyf986sf74hqw8k31lvlzac5-toYAML.drv' failed with exit code 1;
2992 last 25 log lines:
2993 &gt; &quot;Additional property 'runs-on' is not allowed&quot;,
2994 &gt; ),
2995 &gt; path: &quot;/jobs/build&quot;,
2996 &gt; title: &quot;Property conditions are not met&quot;,
2997 &gt; },
2998 &gt; Properties {
2999 &gt; code: &quot;properties&quot;,
3000 &gt; detail: Some(
3001 &gt; &quot;Additional property 'steps' is not allowed&quot;,
3002 &gt; ),
3003 &gt; path: &quot;/jobs/build&quot;,
3004 &gt; title: &quot;Property conditions are not met&quot;,
3005 &gt; },
3006 &gt; Required {
3007 &gt; code: &quot;required&quot;,
3008 &gt; detail: None,
3009 &gt; path: &quot;/jobs/build/uses&quot;,
3010 &gt; title: &quot;This property is required&quot;,
3011 &gt; },
3012 &gt; ],
3013 &gt; },
3014 &gt; ],
3015 &gt; },
3016 &gt; ],
3017 &gt; }
3018 For full logs, run 'nix log /nix/store/j8wr6j1pvyf986sf74hqw8k31lvlzac5-toYAML.drv'.
3019 </code></pre>
3020 <p>Yes, the output of <code>action-validator</code> is awfully verbose, but it is still
3021 better than making <a href="https://x.com/eric_sink/status/1430954572848287744">&quot;8 commits/push in one
3022 hour&quot;</a>.</p>
3023 <p>If you are interested in how a more advantage usage of this technique is,
3024 including usage of functions and constants to share common steps between
3025 different actions, please take a look at the
3026 <a href="https://github.com/thiagokokada/nix-configs/tree/master/actions">actions</a>
3027 (<a href="https://github.com/thiagokokada/nix-configs/tree/fbbc8f935cc7f774f691196e1612bbf7dd2054b5/actions">permalink</a>)
3028 in my <code>nix-config</code> repository.</p></description><pubDate>Wed, 31 Jul 2024 00:00:00 +0000</pubDate><guid>https://kokada.dev/blog/generating-yaml-files-with-nix/</guid></item><item><title>First impressions: FPGBC</title><link>https://kokada.dev/blog/first-impressions-fpgbc/</link><description><p>Here is something for nostalgia: I just put together a <a href="https://en.wikipedia.org/wiki/Game_Boy_Color">Game Boy
3029 Color</a> made of completely new
3030 parts for a friend: here is the
3031 <a href="https://funnyplaying.com/products/fpgbc-kit">FPGBC</a>.</p>
3032 <p>The <em>FP</em> part of the name comes from
3033 <a href="https://en.wikipedia.org/wiki/Field-programmable_gate_array">FPGA</a>, because
3034 instead of software emulation this device use FPGA to reproduce the device.
3035 While I am not convinced that FPGA is necessary more accurate than a good
3036 software emulator, one advantage of FPGA is the (possible) lower input latency
3037 thanks to the avoidance of complexity to handle the user input (e.g.: the
3038 Operational System). A quick playthrough against <a href="https://en.wikipedia.org/wiki/Motocross_Maniacs">Motocross
3039 Maniacs</a> seems to be fine, but
3040 I can't see much difference from my <a href="https://retrogamecorps.com/2022/05/15/miyoo-mini-v2-guide/">Miyoo
3041 Mini+</a> (I will do
3042 more comparisons between the two devices later), that is a software emulation
3043 device.</p>
3044 <p>But I think focusing in accuracy is wrong, the main reason of getting a device
3045 like this one is for nostalgia, and this definitely hit the mark. The quality
3046 of the case is as good as I remember the original, and most of the details are
3047 replicate perfectly, including reproduction stickers in the back of the device.
3048 The only differences that I can find is the usage of USB-C port for charging in
3049 place of the barrel jack power adapter (thanks!), and the fact that the screen
3050 bezels are smaller compared to the original (because the screen is bigger) and
3051 doesn't include the Game Boy Color logo (that is fine in my opinion, since it
3052 would look weird in the fine bezels). It even has a supposedly working <a href="https://en.wikipedia.org/wiki/Game_Link_Cable">Link
3053 Cable</a> (I don't have another
3054 Game Boy to test). Sadly it is missing the infrared sensor, but the usage of
3055 that was pretty limited anyway.</p>
3056 <p><a href="https://github.com/thiagokokada/blog/raw/main/posts/2024-07-30/PXL_20240729_175245569.jpg"><img src="https://github.com/thiagokokada/blog/raw/main/posts/2024-07-30/PXL_20240729_175245569.jpg" alt="FPGBC running Tetris."></a></p>
3057 <p><a href="https://github.com/thiagokokada/blog/raw/main/posts/2024-07-30/PXL_20240729_175131157.jpg"><img src="https://github.com/thiagokokada/blog/raw/main/posts/2024-07-30/PXL_20240729_175131157.jpg" alt="Back of FPGBC. It includes even reproduction stickers of the original."></a></p>
3058 <p>So how well does it work? I can't say for sure. I don't have any original games
3059 with me, so I am relying in backups and a
3060 <a href="https://en.wikipedia.org/wiki/Flashcard">flashcard</a> for now. Many games that I
3061 tested works fine, a few of them have graphical issues that can be fixed in the
3062 menu (more about it later), and some of them doesn't boot. But I don't know if
3063 the issue with the games not booting are because of the roms, the flashcard or
3064 the device itself.</p>
3065 <p>By the way, the flashcard I am using is a cheap knockoff of an <a href="https://gbatemp.net/review/everdrive-gb.141/">Everdrive
3066 GB</a>. This FPGBC came with
3067 firmware v1.09, while there is an update available for v1.10 in the
3068 <a href="https://funnyplaying.com/products/fpgbc-kit">website</a>. I had an weird issue in
3069 the new firmware where no games would boot with this knockoff Everdrive so I
3070 had to go back to v1.09, but again, I am not sure if the issue was fact that I
3071 am using a knockoff device or this would happen with an original Everdrive GB.
3072 If you are going to buy a proper Everdrive, you probably wouldn't get a
3073 Everdrive GB anyway since it is discontinued, and it seems the <a href="https://www.reddit.com/r/Gameboy/comments/1atwjh3/fpgbc_everdrive_compatibility/">newer
3074 versions</a>
3075 have better compatibility with FPGBC.</p>
3076 <p>Sadly that the update didn't work, since there is this
3077 <a href="https://github.com/makhowastaken/GWGBC_FW">repository</a> that patches the
3078 firmware to boot the original logo instead of the ugly FPGBC one. And yes, for
3079 some reason the v1.09 firmware from this repository still doesn't work with my
3080 knockoff Everdrive.</p>
3081 <p>By the way, it seems the device is not easy to brick: I borked the firmware
3082 update process once while trying to downgrade back to v1.09, resulting in a
3083 black screen when I turned on the console. But just connecting the device to
3084 the computer and powering on, I could flash the firmware again and the device
3085 came back to life.</p>
3086 <p>About the features of the device: if you press the volume button (yes, you can
3087 press it now), it opens the following menu:</p>
3088 <p><a href="https://github.com/thiagokokada/blog/raw/main/posts/2024-07-30/PXL_20240729_210604830.jpg"><img src="https://github.com/thiagokokada/blog/raw/main/posts/2024-07-30/PXL_20240729_210604830.jpg" alt="FPGBC menu."></a></p>
3089 <p>The first 2 options are the LCD backlight (<code>BKLT</code>) and volume (<code>VOL</code>). I didn't talk about
3090 those, but the LCD screen seems to be IPS, and the quality is really good, and
3091 also looks bright enough to play even under bad lightining conditions. And the
3092 speaker has good quality, the sound is better than I remember, but sadly the
3093 maximum volume is kind low. Still should be enough for playing in a quiet room.</p>
3094 <p><code>DISPMOD</code> is probably the most controversial option: it allow you to set which
3095 scale you want. Anything with <code>EMU</code> at the end means emulating the original
3096 colors, and as far I remember it gets really close. You can also chose betwen
3097 <code>X4</code>, <code>X4P</code> and <code>FUL</code>, the last one is the one shown in the photos where the
3098 image fills the whole screen at the cost of non-integer scaling. <code>X4</code> is
3099 integer scaling, however the image doesn't fill the whole screen. The <code>X4P</code>
3100 also includes a pixel effect that makes the image closer than the original
3101 screen. It actually looks good, but the fact that I chose a white border for
3102 this FPGBC makes the border really distracting. Maybe the black one is a better
3103 choice if you want integer scale.</p>
3104 <p><code>CORE</code> is simple: you can choose between <code>GB</code> (Game Boy) or <code>GBC</code> (Game Boy
3105 Color). For those who don't know, you can run Game Boy games in Game Boy Color
3106 and they will be automatically colorised. Some people don't like this and
3107 prefer the colors of <code>GB</code>, so you have this option. The <code>GB_PALETTE</code> allows you
3108 to chose the color in GB mode, for example, the green-ish colors from the
3109 original Game Boy or the blue-ish colors from <a href="https://nintendo.fandom.com/wiki/Game_Boy_Light">Game Boy
3110 Light</a>. And yes, you can
3111 choose the color palette for Game Boy games running in <code>GBC</code> mode by pressing a
3112 <a href="https://gbstudiocentral.com/tips/game-boy-color-modes/">button combination</a> at
3113 the boot screen, but it seems not working in my unit and again, not sure if the
3114 fault is my knockoff Everdrive.</p>
3115 <p><code>FRAME_MIX</code> basically is an option that makes some effects, like transparency
3116 in <a href="https://en.wikipedia.org/wiki/Wave_Race">Wave Race</a>, to work at the cost of
3117 introducing blurriness. The reason for this is that those effects depends in
3118 the fact that the Game Boy screen was slow refresh, so you could rely on it by
3119 rapidly changing pixels to create some interesting effects, but sadly those
3120 effects doesn't work well in modern displays.</p>
3121 <p><code>GB_CLRFIX</code> is the option I mentioned before, where some Game Boy games just
3122 get completely wrong colors for some reason, e.g.: <a href="https://en.wikipedia.org/wiki/The_Addams_Family_(video_game)">The Addams
3123 Family</a>. Turning
3124 on fixes those games, but I am not sure if this option breaks other games.</p>
3125 <p>Finally, <code>SPD</code> allows you to increase or decrease the CPU clock, slowing or
3126 speeding up the games (including the sound). The result can be hilarious, so I
3127 think this is a nice addition to the features. Sadly you can't know what the
3128 default speed is, so you need to rely on sound to adjust back to the default.</p>
3129 <p>So in the end, can I recommend a FPGBC? I am not sure. If you want a device to
3130 play games, I still think something like a Miyoo Mini+ is a better choice. Not
3131 only you will have access to more games from different platforms, you also
3132 don't need to rely on flashcards or cartridges. Also it has way more features
3133 than FPGBC, like wireless multiplayer,
3134 <a href="https://retroachievements.org/">RetroArchivements</a> and save states.</p>
3135 <p>But the actual reason to get a FPGBC is nostalgia, and for that I think the
3136 FPGBC is difficult to beat. The price of the <a href="https://funnyplaying.com/products/fpgbc-kit">kit to
3137 assemble</a> ($69.90) is cheaper than
3138 most Game Boy's in good condition you can find in eBay, and you get for that
3139 price a rechargable battery, an amazing quality screen, the PCB and the
3140 speaker. You need to buy separately the case and the buttons, but in total you
3141 will still end up paying less, and allows you to fully customise your build.
3142 And the result device is not only in mint condition, it is really convenient
3143 too: recharging batteries (via USB-C even) is much more convenient than buying
3144 AA batteries, and the screen not only is better but it even has backlight. You
3145 can also buy a fully built console for
3146 <a href="https://funnyplaying.com/products/fpgbc-console">$99.00</a>, but you have less
3147 options of customisation.</p>
3148 <p>This is the classic case of do what I say, don't do what I do. This FPGBC is a
3149 gift, and I will buy another one soon. Can't wait to play <a href="https://en.wikipedia.org/wiki/Pok%C3%A9mon_Gold_and_Silver">Pokémon
3150 Gold</a> in (almost)
3151 original hardware again.</p>
3152 <p><a href="https://github.com/thiagokokada/blog/raw/main/posts/2024-07-30/PXL_20240729_123847458.jpg"><img src="https://github.com/thiagokokada/blog/raw/main/posts/2024-07-30/PXL_20240729_123847458.jpg" alt="The kit before assemble."></a></p></description><pubDate>Tue, 30 Jul 2024 00:00:00 +0000</pubDate><guid>https://kokada.dev/blog/first-impressions-fpgbc/</guid></item><item><title>Quick bits: why you should automate everything</title><link>https://kokada.dev/blog/quick-bits-why-you-should-automate-everything/</link><description><p>If everything works as expected this blog post should appear in <a href="https://kokada.capivaras.dev/">in my
3153 blog</a> without I ever touching the
3154 <a href="https://capivaras.dev">capivaras.dev</a> website. I rewrote my <a href="https://kokada.dev/blog/using-github-as-a-bad-blog-platform">previous Python
3155 script</a> to Go
3156 (<a href="https://github.com/thiagokokada/blog/blob/3c39e0f7cd58b1af885f69871490b05bf6fc7d99/blog.go">permalink</a>)
3157 since my attempt to generate proper description to the RSS feed resulted in
3158 slow startup times (not because of Python, but because of my usage of
3159 <code>nix-shell</code> since I didn't want to deal with
3160 <a href="https://docs.python.org/3/library/venv.html">venv</a> or anything to manage my
3161 Python dependencies).</p>
3162 <p>My previous workflow of this blog already involved me writing the texts in
3163 <a href="https://neovim.io/">neovim</a>, copying and pasting the result in the
3164 <a href="https://capivaras.dev">capivaras.dev</a> website and publishing. This was not
3165 that bad, except that it seems I have a heavy tendency of editing my posts
3166 multiple times. Copying and pasting data between neovim and the website became
3167 tedious, so I decided to give up and automate the whole process.</p>
3168 <p><a href="https://mataroa.blog/">Mataroa</a> (the blog platform
3169 <a href="https://capivaras.dev">capivaras.dev</a> run) has a reasonable good
3170 <a href="https://mataroa.blog/api/docs/">API</a>, and it only took a few hours to get a
3171 version of publishing working (it would take less if
3172 <a href="https://www.djangoproject.com/">Django</a>, the framework Mataroa is written, did
3173 not have a weird behavior with URLs missing a trailing <code>/</code>). An additional few
3174 lines of
3175 <a href="https://github.com/thiagokokada/blog/blob/51b20612335c7f4312a51a0f436235b4b701ce8b/.github/workflows/go.yml">YAML</a>
3176 to make GitHub Actions trigger a pipeline and now I should never have to
3177 manually update my blog again.</p>
3178 <p>I could have not done this. I mean, I probably wasted more time writing an
3179 automation than I actually wasted publishing manually. But the manual process
3180 is easy to miss, and I already did multiple mistakes publishing in the manual
3181 method. For example, when writing the Markdown files, each post is written in a
3182 particular format, where the first header is considered the title, so I need to
3183 remove it from the contents during publication. But of course, this is easy to
3184 miss, and I had to fix this multiple times already.</p>
3185 <p>So yes, I think this is a good lesson on why you should automate everything. It
3186 is more than just about <a href="https://xkcd.com/1205/">time savings</a>, it is about
3187 reducing mistakes and even documenting (even if poorly) a process. I mean, the
3188 code I wrote is not that great, but I can definitely rely on it in the future
3189 to remember what I need to do. It will be much faster than trying to learn from
3190 scratch again.</p></description><pubDate>Mon, 29 Jul 2024 00:00:00 +0000</pubDate><guid>https://kokada.dev/blog/quick-bits-why-you-should-automate-everything/</guid></item><item><title>Go, a reasonable good language</title><link>https://kokada.dev/blog/go-a-reasonable-good-language/</link><description><p>Go was one of the languages that I always was interested to learn, but never
3191 got the hang of it. I first got interested in the language when I was in my
3192 first job, between 2016-2018. At the time the language was a completely
3193 different beast: no modules, no generics, no easy way to error wrap yet, etc.</p>
3194 <p>Go forward 2023 (no pun indented), I wrote my <a href="https://github.com/thiagokokada/twenty-twenty-twenty/">first project in
3195 Go</a>, wrote some scripts
3196 at <code>$CURRENT_JOB</code> in the language, and now wrote <a href="https://github.com/thiagokokada/hyprland-go/">my first
3197 library</a> (with an <a href="https://kokada.dev/blog/writing-nixos-tests-for-fun-and-profit">interesting
3198 way to run
3199 CI</a>). I am also
3200 writing more
3201 <a href="https://github.com/thiagokokada/nix-configs/blob/8c559527ed12e1d4f57a3fc5c72630b956f4c290/home-manager/desktop/wayland/hyprland/hyprtabs/hyprtabs.go">scripts</a>
3202 in the language, where I would prefer to use Bash or Python before. Heck, even
3203 this blog is automatically published with a <a href="https://kokada.dev/blog/quick-bits-why-you-should-automate-everything">Go
3204 script</a>,
3205 that used to be a <a href="https://kokada.dev/blog/using-github-as-a-bad-blog-platform">Python
3206 one</a> before. I can
3207 say that nowadays it is another language in my toolbox, and while it is still a
3208 love and hate relationship, recently it is more about love and less about hate.</p>
3209 <p>The points that I love about Go is probably obvious for some, but still
3210 interesting to talk about anyway. The fact that the language generates static
3211 binaries by default and have fast compilation times is something that I
3212 apreciate since I first heard about the language, and now that I am using the
3213 language frequently, are points thatI appreciate even more. Something about
3214 getting almost instant feedback after changing a line of code and running <code>go run</code> (even with its quirks) are great for the developer experience. This is the
3215 main reason why I am using the language more frequently for scripts.</p>
3216 <p>Then we have the fast startup times. I am kind of sensitive to latency,
3217 especially of command line utilities that need to answer fast when I expect
3218 them to be fast (e.g.: <code>foo --help</code>). This is one part where I could have
3219 issues in Python, especially for more complex programs, but in Go it is rarely
3220 an issue.</p>
3221 <p>Modules are also fantastic. It is not without its weirdness (like everything in
3222 Go ecossystem), but the fact that it is so easy to add and manage dependencies
3223 in a project using only the <code>go</code> CLI is great. I also like that it generates a
3224 hash of every dependency, making it reproducible (well, probably not at Nix
3225 level, but still reproducible).</p>
3226 <p>Since I started to talk about <code>go</code> CLI, what a great tool! The fact that you
3227 can manage dependencies, generate documentation, format code, lint, run
3228 tests/<a href="https://blog.logrocket.com/benchmarking-golang-improve-function-performance/">benchmarks</a>/<a href="https://go.dev/doc/security/fuzz/">fuzzing</a>,
3229 check code for <a href="https://go.dev/doc/articles/race_detector">races</a> etc., all
3230 with just the &quot;compiler&quot; for the language is excelent. Still probably one of
3231 the best developer experiences I know in any programming language.</p>
3232 <p>I will not even talk about the things that everyone talks about Go, like
3233 <a href="https://go.dev/doc/effective_go#goroutines">goroutines</a>, because I just don't
3234 think I can add anything interesting to the topic.</p>
3235 <p>Now for the parts that I like less, the test part still quirks me that it is
3236 not based in assertions, but thankfully it is easy to write assertions with
3237 generics nowadays:</p>
3238 <pre class="chroma"><code><span class="line"><span class="cl"><span class="kd">func</span> <span class="nx">Equal</span><span class="p">[</span><span class="nx">T</span> <span class="nx">comparable</span><span class="p">](</span><span class="nx">t</span> <span class="o">*</span><span class="nx">testing</span><span class="p">.</span><span class="nx">T</span><span class="p">,</span> <span class="nx">got</span><span class="p">,</span> <span class="nx">want</span> <span class="nx">T</span><span class="p">)</span> <span class="p">{</span>
3239 </span></span><span class="line"><span class="cl"> <span class="nx">t</span><span class="p">.</span><span class="nf">Helper</span><span class="p">()</span>
3240 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="nx">got</span> <span class="o">!=</span> <span class="nx">want</span> <span class="p">{</span>
3241 </span></span><span class="line"><span class="cl"> <span class="nx">t</span><span class="p">.</span><span class="nf">Errorf</span><span class="p">(</span><span class="s">&#34;got: %#v, want: %#v&#34;</span><span class="p">,</span> <span class="nx">got</span><span class="p">,</span> <span class="nx">want</span><span class="p">)</span>
3242 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
3243 </span></span><span class="line"><span class="cl"><span class="p">}</span>
3244 </span></span><span class="line"><span class="cl">
3245 </span></span><span class="line"><span class="cl"><span class="kd">func</span> <span class="nx">GreaterOrEqual</span><span class="p">[</span><span class="nx">T</span> <span class="nx">cmp</span><span class="p">.</span><span class="nx">Ordered</span><span class="p">](</span><span class="nx">t</span> <span class="o">*</span><span class="nx">testing</span><span class="p">.</span><span class="nx">T</span><span class="p">,</span> <span class="nx">actual</span><span class="p">,</span> <span class="nx">expected</span> <span class="nx">T</span><span class="p">)</span> <span class="p">{</span>
3246 </span></span><span class="line"><span class="cl"> <span class="nx">t</span><span class="p">.</span><span class="nf">Helper</span><span class="p">()</span>
3247 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="nx">actual</span> <span class="p">&lt;</span> <span class="nx">expected</span> <span class="p">{</span>
3248 </span></span><span class="line"><span class="cl"> <span class="nx">t</span><span class="p">.</span><span class="nf">Errorf</span><span class="p">(</span><span class="s">&#34;got: %v; want: &gt;=%v&#34;</span><span class="p">,</span> <span class="nx">actual</span><span class="p">,</span> <span class="nx">expected</span><span class="p">)</span>
3249 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
3250 </span></span><span class="line"><span class="cl"><span class="p">}</span>
3251 </span></span><span class="line"><span class="cl">
3252 </span></span><span class="line"><span class="cl"><span class="c1">// etc...
3253 </span></span></span></code></pre><p>Just one of those things that I end up re-writing in every project. Yes, I know
3254 about <a href="https://github.com/stretchr/testify">testify</a> and other assertion
3255 libraries, but quoting <a href="https://www.youtube.com/watch?v=PAAkCSZUG1c&amp;t=568s">Rob
3256 Pike</a> here, &quot;a little
3257 copying is better than a little dependency&quot;. As long the code you write is
3258 trivial, it is better to duplicate the code than try to import a dependency.</p>
3259 <p>About another piece of code that generics allows me to write and I always end
3260 up re-writing in every project is the <code>must*</code> family of functions:</p>
3261 <pre class="chroma"><code><span class="line"><span class="cl"><span class="kd">func</span> <span class="nf">must</span><span class="p">(</span><span class="nx">err</span> <span class="kt">error</span><span class="p">)</span> <span class="p">{</span>
3262 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="nx">err</span> <span class="o">!=</span> <span class="kc">nil</span> <span class="p">{</span>
3263 </span></span><span class="line"><span class="cl"> <span class="nb">panic</span><span class="p">(</span><span class="nx">err</span><span class="p">)</span>
3264 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
3265 </span></span><span class="line"><span class="cl"><span class="p">}</span>
3266 </span></span><span class="line"><span class="cl">
3267 </span></span><span class="line"><span class="cl"><span class="kd">func</span> <span class="nx">must1</span><span class="p">[</span><span class="nx">T</span> <span class="nx">any</span><span class="p">](</span><span class="nx">v</span> <span class="nx">T</span><span class="p">,</span> <span class="nx">err</span> <span class="kt">error</span><span class="p">)</span> <span class="nx">T</span> <span class="p">{</span>
3268 </span></span><span class="line"><span class="cl"> <span class="nf">must</span><span class="p">(</span><span class="nx">err</span><span class="p">)</span>
3269 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="nx">v</span>
3270 </span></span><span class="line"><span class="cl"><span class="p">}</span>
3271 </span></span><span class="line"><span class="cl">
3272 </span></span><span class="line"><span class="cl"><span class="kd">func</span> <span class="nx">must2</span><span class="p">[</span><span class="nx">T1</span> <span class="nx">any</span><span class="p">,</span> <span class="nx">T2</span> <span class="nx">any</span><span class="p">](</span><span class="nx">v1</span> <span class="nx">T1</span><span class="p">,</span> <span class="nx">v2</span> <span class="nx">T2</span><span class="p">,</span> <span class="nx">err</span> <span class="kt">error</span><span class="p">)</span> <span class="nx">T</span> <span class="p">{</span>
3273 </span></span><span class="line"><span class="cl"> <span class="nf">must</span><span class="p">(</span><span class="nx">err</span><span class="p">)</span>
3274 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="nx">v1</span><span class="p">,</span> <span class="nx">v2</span>
3275 </span></span><span class="line"><span class="cl"><span class="p">}</span>
3276 </span></span><span class="line"><span class="cl">
3277 </span></span><span class="line"><span class="cl"><span class="c1">// must3, must4, etc...
3278 </span></span></span></code></pre><p>Those functions are so useful, especially for scripts where I generally don't
3279 want to handle each error: if I have an error, I want the program to halt and
3280 print a stack trace (exactly as I would have with a language with exceptions).
3281 It basically allow me to convert code from:</p>
3282 <pre class="chroma"><code><span class="line"><span class="cl"><span class="nx">contents</span><span class="p">,</span> <span class="nx">err</span> <span class="o">:=</span> <span class="nx">os</span><span class="p">.</span><span class="nf">ReadFile</span><span class="p">(</span><span class="s">&#34;file&#34;</span><span class="p">)</span>
3283 </span></span><span class="line"><span class="cl"><span class="k">if</span> <span class="nx">err</span> <span class="o">!=</span> <span class="kc">nil</span> <span class="p">{</span>
3284 </span></span><span class="line"><span class="cl"> <span class="nb">panic</span><span class="p">(</span><span class="nx">err</span><span class="p">)</span>
3285 </span></span><span class="line"><span class="cl"><span class="p">}</span>
3286 </span></span></code></pre><p>To:</p>
3287 <pre class="chroma"><code><span class="line"><span class="cl"><span class="nx">contents</span> <span class="o">:=</span> <span class="nf">must1</span><span class="p">(</span><span class="nx">os</span><span class="p">.</span><span class="nf">ReadFile</span><span class="p">(</span><span class="s">&#34;file&#34;</span><span class="p">))</span>
3288 </span></span></code></pre><p>This brings Go closer to Python to me, and I think for scripts this is
3289 something great.</p>
3290 <p>Finally, for the things that I hate, well the biggest one currently is the lack
3291 of nullability (or in Go terms,
3292 <a href="https://github.com/golang/go/issues/49202">nillability</a>). After using
3293 languages that has it, like Kotlin, or even something like
3294 <a href="https://www.mypy-lang.org/">mypy</a>, this is one of those things that completely
3295 changes the developer experience. I also still don't like the error handling
3296 (but <code>must*</code> goes far by improving the situation, when it is possible to use
3297 it), especially because it is easy to lose context on it:</p>
3298 <pre class="chroma"><code><span class="line"><span class="cl"><span class="c1">// bad
3299 </span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="kd">func</span> <span class="nf">readFileContents</span><span class="p">(</span><span class="nx">file</span><span class="p">)</span> <span class="p">([]</span><span class="kt">byte</span><span class="p">,</span> <span class="kt">error</span><span class="p">)</span> <span class="p">{</span>
3300 </span></span><span class="line"><span class="cl"> <span class="nx">contents</span><span class="p">,</span> <span class="nx">err</span> <span class="o">:=</span> <span class="nx">os</span><span class="p">.</span><span class="nf">ReadFile</span><span class="p">(</span><span class="nx">file</span><span class="p">)</span>
3301 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="nx">err</span> <span class="o">!=</span> <span class="kc">nil</span> <span class="p">{</span>
3302 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="kc">nil</span><span class="p">,</span> <span class="nx">err</span>
3303 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
3304 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="nx">contents</span><span class="p">,</span> <span class="kc">nil</span>
3305 </span></span><span class="line"><span class="cl"><span class="p">}</span>
3306 </span></span><span class="line"><span class="cl">
3307 </span></span><span class="line"><span class="cl"><span class="c1">// good
3308 </span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="kd">func</span> <span class="nf">readFileContents</span><span class="p">(</span><span class="nx">file</span><span class="p">)</span> <span class="p">([]</span><span class="kt">byte</span><span class="p">,</span> <span class="kt">error</span><span class="p">)</span> <span class="p">{</span>
3309 </span></span><span class="line"><span class="cl"> <span class="nx">contents</span><span class="p">,</span> <span class="nx">err</span> <span class="o">:=</span> <span class="nx">os</span><span class="p">.</span><span class="nf">ReadFile</span><span class="p">(</span><span class="nx">file</span><span class="p">)</span>
3310 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="nx">err</span> <span class="o">!=</span> <span class="kc">nil</span> <span class="p">{</span>
3311 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="kc">nil</span><span class="p">,</span> <span class="nx">fmt</span><span class="p">.</span><span class="nf">Errorf</span><span class="p">(</span><span class="s">&#34;readFileContents: error while reading a file: %w&#34;</span><span class="p">,</span> <span class="nx">err</span><span class="p">)</span>
3312 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
3313 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="nx">contents</span><span class="p">,</span> <span class="kc">nil</span>
3314 </span></span><span class="line"><span class="cl"><span class="p">}</span>
3315 </span></span></code></pre><p>I also have some grips about the mutate everything approach of the language. I
3316 prefer immutability by default, but I find that in general as long as you split
3317 your functions at a reasonable size it is generally fine.</p>
3318 <p>And for a language that is supposed to be straightforward, it is strange on how
3319 much magic the language relies on, in the form of things like <a href="https://go.dev/doc/modules/layout"><code>internal</code> and
3320 <code>main</code></a> packages, name capitalisation to
3321 indicate visibility (<code>private</code> vs <code>Public</code>), conditional compiling by filenames
3322 (e.g.: <code>foo_amd64.go</code>, <code>bar_linux.go</code>), magic comments (e.g.: <code>//go:build</code>),
3323 etc.</p>
3324 <p>I expect to write more Go code going forward. Not because it is the perfect
3325 language or whatever, but just because it a is language that has some really
3326 good qualities that makes the language attractive even with the issues that I
3327 have. That makes it a reasonable good language, and at least for me this is
3328 good enough.</p></description><pubDate>Mon, 29 Jul 2024 00:00:00 +0000</pubDate><guid>https://kokada.dev/blog/go-a-reasonable-good-language/</guid></item><item><title>Quick bits: nix-shell is cursed</title><link>https://kokada.dev/blog/quick-bits-nix-shell-is-cursed/</link><description><p>The other day I had to run a PHP project in my machine. I have no idea how PHP
3329 ecosystem work, I just wanted to get it to run.</p>
3330 <p>The easiest way to get a script to run if you use Nix is to use <code>nix-shell</code>. As
3331 many of you probably know, you can add <code>nix-shell</code> as a shebang in your scripts
3332 to run them as <code>./script</code>. This was a PHP script so I wanted to do the same.
3333 Easy right?</p>
3334 <pre class="chroma"><code><span class="line"><span class="cl"><span class="c1">#!/use/bin/env nix-shell
3335 </span></span></span><span class="line"><span class="cl"><span class="c1">#!nix-shell -i php -p php83
3336 </span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="o">&lt;?</span><span class="nx">php</span>
3337 </span></span><span class="line"><span class="cl"><span class="k">declare</span><span class="p">(</span><span class="nx">strict_types</span><span class="o">=</span><span class="mi">1</span><span class="p">);</span>
3338 </span></span></code></pre><p>And:</p>
3339 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> ./index.php
3340 </span></span><span class="line"><span class="cl"><span class="go">Fatal error: strict_types declaration must be the very first statement in the script in index.php on line 4
3341 </span></span></span></code></pre><p>So it seems that <code>declare(strict_types=1)</code> needs to be the first line in a PHP
3342 script if used. I removed <code>declare(strict_types=1)</code> and while the script works,
3343 I don't have enough expertise in PHP to know if this would be safe or not.</p>
3344 <p>I decided to try something that initially looked really dumb:</p>
3345 <pre class="chroma"><code><span class="line"><span class="cl"><span class="c1">#!/use/bin/env nix-shell
3346 </span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="o">&lt;?</span><span class="nx">php</span>
3347 </span></span><span class="line"><span class="cl"><span class="k">declare</span><span class="p">(</span><span class="nx">strict_types</span><span class="o">=</span><span class="mi">1</span><span class="p">);</span>
3348 </span></span><span class="line"><span class="cl"><span class="c1">#!nix-shell -i php -p php83
3349 </span></span></span></code></pre><p>And:</p>
3350 <pre class="chroma"><code><span class="line"><span class="cl"><span class="gp">$</span> ./index.php
3351 </span></span><span class="line"><span class="cl"><span class="go">Works
3352 </span></span></span></code></pre><p>Wat? I mean, it is not dumb if it works, but this at least looks cursed.</p>
3353 <p>Eventually I found this
3354 <a href="https://github.com/NixOS/nix/issues/2570#issuecomment-446220517">comment</a> in a
3355 Nix issue talking about cases where <code>nix-shell</code> shebang doesn't work. It looks
3356 like the classic case of a <a href="https://github.com/NixOS/nix/issues/2570#issuecomment-446222206">bug that becomes a
3357 feature</a>.</p>
3358 <p><em>Update:</em> after posting this in
3359 <a href="https://lobste.rs/s/gkcgza/quick_bits_nix_shell_is_cursed">Lobte.rs</a>, it seems
3360 someone decided to open a <a href="https://github.com/NixOS/nix/pull/11202">Pull
3361 Request</a> to document this behavior.
3362 Also the equivalent for the new <a href="https://nix.dev/manual/nix/2.23/command-ref/new-cli/nix#shebang-interpreter">nix
3363 CLI</a>
3364 explicitly documents this behavior:</p>
3365 <blockquote>
3366 <p>Note that the <code>#! nix</code> lines don't need to follow after the first line, to
3367 accomodate other interpreters.</p>
3368 </blockquote></description><pubDate>Sat, 27 Jul 2024 00:00:00 +0000</pubDate><guid>https://kokada.dev/blog/quick-bits-nix-shell-is-cursed/</guid></item><item><title>Using GitHub as a (bad) blog platform</title><link>https://kokada.dev/blog/using-github-as-a-bad-blog-platform/</link><description><p>I finally started a new blog, thanks to the offer of
3369 <a href="https://gluer.org/">@ratsclub</a> to give me free access to
3370 <a href="https://capivaras.dev/">capivaras.dev</a>. But considering how small this blog
3371 platform is supposed to be, I want to have at least somewhere to have a backup
3372 of my posts. I know <a href="https://mataroa.blog/">Mataroa</a>, the blog platform that
3373 <a href="https://capivaras.dev/">capivaras.dev</a> runs, has automatic e-mail backups, but
3374 I want something more reliable.</p>
3375 <p>I am writing all my posts in Markdown (the format that <a href="https://1.mataroa.blog/guides/markdown/">Mataroa
3376 supports</a>) files inside
3377 <a href="https://neovim.io/">neovim</a> anyway, so why not store all my Markdown files in
3378 Git? So this is what I did, I now have an unofficial mirror in
3379 <a href="https://github.com/thiagokokada/blog">GitHub</a>.</p>
3380 <p>While I am here, why not overcomplicate? Can I make an usable blog platform
3381 from GitHub? And by that I don't mean GitHub pages, the repository itself. I
3382 mean, it already renders Markdown files by default, so no need to do anything
3383 in that space. To reach feature parity with
3384 <a href="https://capivaras.dev/">capivaras.dev</a>, I only need to have an index and RSS
3385 (since comments are not supported anyway). No need for newsletter since GitHub
3386 has a <a href="https://docs.github.com/en/account-and-profile/managing-subscriptions-and-notifications-on-github/managing-subscriptions-for-activity-on-github/viewing-your-subscriptions">watch
3387 feature</a>
3388 already.</p>
3389 <p>After a couple of hours hacking a Python script, you can see the result of this
3390 monstrosity <a href="https://github.com/thiagokokada/blog">here</a>. The script, called
3391 <code>gen_blog.py</code>, is available at the same repository (here is a
3392 <a href="https://github.com/thiagokokada/blog/blob/c8986d1ab1b94c0986fd814629bb8eb4034fb6e7/gen_blog.py">permalink</a>).
3393 It automatically generates an index at
3394 <a href="https://github.com/thiagokokada/blog/blob/main/README.md"><code>README.md</code></a> with
3395 each blog post and a
3396 <a href="https://raw.githubusercontent.com/thiagokokada/blog/main/rss.xml"><code>rss.xml</code></a>
3397 file at the root of the repository.</p>
3398 <p>Instead of trying to explain the code, I am going to explain the general idea,
3399 because I think that if you want to replicate this idea it is better to rewrite
3400 it in a way that you understand. It shouldn't take more than 2 hours in any
3401 decent programming language. But if you really want, the script itself is
3402 licensed in <a href="https://en.wikipedia.org/wiki/WTFPL">WTFPL</a> license. The code only
3403 uses Python 3's standard library and should work in any relatively recent
3404 version (anything newer than 3.9 should work).</p>
3405 <p>So the idea is basically to organise the repository and the Markdown files in a
3406 easy way that makes it trivial to parse in a deterministic way. For example, my
3407 repository is organised in the following way:</p>
3408 <pre><code>.
3409 ├── 2024-07-26
3410 │ ├── 01-writing-nixos-tests-for-fun-and-profit.md
3411 │ └── 02-using-github-as-a-bad-blog-platform.md &lt;- this file
3412 ├── gen_blog.py
3413 ├── README.md
3414 └── rss.xml
3415 </code></pre>
3416 <p>Each day that you write a new blog post will be on its own directory. This is
3417 nice because Markdown files may include extra files than the posts themselves,
3418 e.g.: images, and this organisation make it trivial to organise everything.</p>
3419 <p>Each post has its own Markdown file. I put a two digit number before each post,
3420 to ensure that when publishing multiple posts at the same day I keep them in
3421 the same order of publishing. But if you don't care about it, you can just name
3422 the files whatever you want.</p>
3423 <p>Also, I am assuming that each Markdown file has a header starting with <code># </code>,
3424 and that is the title of the blog post.</p>
3425 <p>Using the above organisation, I have this function that scrapes the repository
3426 and collect the necessary information to generate the index and RSS files:</p>
3427 <pre class="chroma"><code><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">grab_posts</span><span class="p">(</span><span class="n">pwd</span><span class="p">:</span> <span class="n">Path</span><span class="p">):</span>
3428 </span></span><span class="line"><span class="cl"> <span class="n">posts</span> <span class="o">=</span> <span class="n">defaultdict</span><span class="p">(</span><span class="nb">list</span><span class="p">)</span>
3429 </span></span><span class="line"><span class="cl">
3430 </span></span><span class="line"><span class="cl"> <span class="k">for</span> <span class="nb">dir</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">pwd</span><span class="o">.</span><span class="n">iterdir</span><span class="p">(),</span> <span class="n">reverse</span><span class="o">=</span><span class="kc">True</span><span class="p">):</span>
3431 </span></span><span class="line"><span class="cl"> <span class="c1"># Ignore non-directories or hidden files</span>
3432 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="ow">not</span> <span class="nb">dir</span><span class="o">.</span><span class="n">is_dir</span><span class="p">()</span> <span class="ow">or</span> <span class="nb">dir</span><span class="o">.</span><span class="n">name</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="s2">&#34;.&#34;</span><span class="p">:</span>
3433 </span></span><span class="line"><span class="cl"> <span class="k">continue</span>
3434 </span></span><span class="line"><span class="cl">
3435 </span></span><span class="line"><span class="cl"> <span class="c1"># Try to parse date from directory name</span>
3436 </span></span><span class="line"><span class="cl"> <span class="k">try</span><span class="p">:</span>
3437 </span></span><span class="line"><span class="cl"> <span class="n">date</span> <span class="o">=</span> <span class="n">datetime</span><span class="o">.</span><span class="n">strptime</span><span class="p">(</span><span class="nb">dir</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="s2">&#34;%Y-%m-</span><span class="si">%d</span><span class="s2">&#34;</span><span class="p">)</span>
3438 </span></span><span class="line"><span class="cl"> <span class="k">except</span> <span class="ne">ValueError</span><span class="p">:</span>
3439 </span></span><span class="line"><span class="cl"> <span class="nb">print</span><span class="p">(</span><span class="sa">f</span><span class="s2">&#34;WARN: ignoring non-date directory: </span><span class="si">{</span><span class="nb">dir</span><span class="si">}</span><span class="s2">&#34;</span><span class="p">,</span> <span class="n">file</span><span class="o">=</span><span class="n">sys</span><span class="o">.</span><span class="n">stderr</span><span class="p">)</span>
3440 </span></span><span class="line"><span class="cl"> <span class="k">continue</span>
3441 </span></span><span class="line"><span class="cl">
3442 </span></span><span class="line"><span class="cl"> <span class="c1"># Iterate between the files in the date directory</span>
3443 </span></span><span class="line"><span class="cl"> <span class="k">for</span> <span class="n">post</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="nb">dir</span><span class="o">.</span><span class="n">iterdir</span><span class="p">(),</span> <span class="n">reverse</span><span class="o">=</span><span class="kc">True</span><span class="p">):</span>
3444 </span></span><span class="line"><span class="cl"> <span class="c1"># Ignore non-markdown files or hidden files (draft)</span>
3445 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="ow">not</span> <span class="n">post</span><span class="o">.</span><span class="n">suffix</span> <span class="o">==</span> <span class="s2">&#34;.md&#34;</span> <span class="ow">or</span> <span class="n">post</span><span class="o">.</span><span class="n">name</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="s2">&#34;.&#34;</span><span class="p">:</span>
3446 </span></span><span class="line"><span class="cl"> <span class="k">continue</span>
3447 </span></span><span class="line"><span class="cl">
3448 </span></span><span class="line"><span class="cl"> <span class="c1"># Grab the first H1 section to parse as title</span>
3449 </span></span><span class="line"><span class="cl"> <span class="n">text</span> <span class="o">=</span> <span class="n">post</span><span class="o">.</span><span class="n">read_text</span><span class="p">()</span>
3450 </span></span><span class="line"><span class="cl"> <span class="n">mTitle</span> <span class="o">=</span> <span class="n">re</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="sa">r</span><span class="s2">&#34;# (?P&lt;title&gt;.*)\r?\n&#34;</span><span class="p">,</span> <span class="n">text</span><span class="p">)</span>
3451 </span></span><span class="line"><span class="cl"> <span class="k">if</span> <span class="n">mTitle</span> <span class="ow">and</span> <span class="p">(</span><span class="n">title</span> <span class="o">:=</span> <span class="n">mTitle</span><span class="o">.</span><span class="n">groupdict</span><span class="p">()</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&#34;title&#34;</span><span class="p">)):</span>
3452 </span></span><span class="line"><span class="cl"> <span class="n">posts</span><span class="p">[</span><span class="n">date</span><span class="p">]</span><span class="o">.</span><span class="n">append</span><span class="p">({</span><span class="s2">&#34;title&#34;</span><span class="p">:</span> <span class="n">title</span><span class="p">,</span> <span class="s2">&#34;file&#34;</span><span class="p">:</span> <span class="nb">str</span><span class="p">(</span><span class="n">post</span><span class="p">)})</span>
3453 </span></span><span class="line"><span class="cl"> <span class="k">else</span><span class="p">:</span>
3454 </span></span><span class="line"><span class="cl"> <span class="nb">print</span><span class="p">(</span><span class="sa">f</span><span class="s2">&#34;WARN: did not find title for file: </span><span class="si">{</span><span class="n">post</span><span class="si">}</span><span class="s2">&#34;</span><span class="p">,</span> <span class="n">file</span><span class="o">=</span><span class="n">sys</span><span class="o">.</span><span class="n">stderr</span><span class="p">)</span>
3455 </span></span><span class="line"><span class="cl">
3456 </span></span><span class="line"><span class="cl"> <span class="k">return</span> <span class="n">posts</span>
3457 </span></span></code></pre><p>Some interesting tidbits: if a Markdown file has a <code>.</code> at the start I assume it
3458 is a draft post, and ignore it from my scraper. I added a bunch of <code>WARN</code>
3459 prints to make sure that the me in the future doesn't do anything dumb. Also,
3460 sorting in reverse since reverse chronological order is the one most people
3461 expect in blogs (i.e.: more recent blog posts at top).</p>
3462 <p>After running the function above, I have a resulting dictionary that I can use
3463 to generate either a <code>README.md</code> file or Markdown:</p>
3464 <pre class="chroma"><code><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">gen_readme</span><span class="p">(</span><span class="n">posts</span><span class="p">):</span>
3465 </span></span><span class="line"><span class="cl"> <span class="n">titles</span> <span class="o">=</span> <span class="p">[]</span>
3466 </span></span><span class="line"><span class="cl">
3467 </span></span><span class="line"><span class="cl"> <span class="k">for</span> <span class="n">date</span><span class="p">,</span> <span class="n">dayPosts</span> <span class="ow">in</span> <span class="n">posts</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
3468 </span></span><span class="line"><span class="cl"> <span class="k">for</span> <span class="n">post</span> <span class="ow">in</span> <span class="n">dayPosts</span><span class="p">:</span>
3469 </span></span><span class="line"><span class="cl"> <span class="c1"># This creates a relative link to the Markdown file, .e.g.:</span>
3470 </span></span><span class="line"><span class="cl"> <span class="c1"># ./02-using-github-as-a-bad-blog-platform.md</span>
3471 </span></span><span class="line"><span class="cl"> <span class="n">link</span> <span class="o">=</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="s2">&#34;.&#34;</span><span class="p">,</span> <span class="n">post</span><span class="p">[</span><span class="s2">&#34;file&#34;</span><span class="p">])</span>
3472 </span></span><span class="line"><span class="cl"> <span class="c1"># This formats the title, e.g.:</span>
3473 </span></span><span class="line"><span class="cl"> <span class="c1"># - [Using GitHub as a (bad) blog platform](./2024-07-26/02-using-github-as-a-bad-blog-platform.md) - 2024-07-26</span>
3474 </span></span><span class="line"><span class="cl"> <span class="n">title</span> <span class="o">=</span> <span class="n">date</span><span class="o">.</span><span class="n">strftime</span><span class="p">(</span><span class="sa">f</span><span class="s2">&#34;- [</span><span class="si">{</span><span class="n">post</span><span class="p">[</span><span class="s1">&#39;title&#39;</span><span class="p">]</span><span class="si">}</span><span class="s2">](</span><span class="si">{</span><span class="n">link</span><span class="si">}</span><span class="s2">) - %Y-%m-%d&#34;</span><span class="p">)</span>
3475 </span></span><span class="line"><span class="cl"> <span class="c1"># This appends to the list to generate the content later</span>
3476 </span></span><span class="line"><span class="cl"> <span class="n">titles</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">title</span><span class="p">)</span>
3477 </span></span><span class="line"><span class="cl">
3478 </span></span><span class="line"><span class="cl"> <span class="c1"># README_TEMPLATE is a string with the static part of the README</span>
3479 </span></span><span class="line"><span class="cl"> <span class="nb">print</span><span class="p">(</span><span class="n">README_TEMPLATE</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">posts</span><span class="o">=</span><span class="s2">&#34;</span><span class="se">\n</span><span class="s2">&#34;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">titles</span><span class="p">)))</span>
3480 </span></span><span class="line"><span class="cl">
3481 </span></span><span class="line"><span class="cl">
3482 </span></span><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">gen_rss</span><span class="p">(</span><span class="n">posts</span><span class="p">):</span>
3483 </span></span><span class="line"><span class="cl"> <span class="c1"># Got most of the specification from here:</span>
3484 </span></span><span class="line"><span class="cl"> <span class="c1"># https://www.w3schools.com/XML/xml_rss.asp</span>
3485 </span></span><span class="line"><span class="cl"> <span class="n">rss</span> <span class="o">=</span> <span class="n">ET</span><span class="o">.</span><span class="n">Element</span><span class="p">(</span><span class="s2">&#34;rss&#34;</span><span class="p">,</span> <span class="n">version</span><span class="o">=</span><span class="s2">&#34;2.0&#34;</span><span class="p">)</span>
3486 </span></span><span class="line"><span class="cl">
3487 </span></span><span class="line"><span class="cl"> <span class="c1"># Here are the RSS metadata for the blog itself</span>
3488 </span></span><span class="line"><span class="cl"> <span class="n">channel</span> <span class="o">=</span> <span class="n">ET</span><span class="o">.</span><span class="n">SubElement</span><span class="p">(</span><span class="n">rss</span><span class="p">,</span> <span class="s2">&#34;channel&#34;</span><span class="p">)</span>
3489 </span></span><span class="line"><span class="cl"> <span class="n">ET</span><span class="o">.</span><span class="n">SubElement</span><span class="p">(</span><span class="n">channel</span><span class="p">,</span> <span class="s2">&#34;title&#34;</span><span class="p">)</span><span class="o">.</span><span class="n">text</span> <span class="o">=</span> <span class="s2">&#34;kokada&#39;s blog&#34;</span>
3490 </span></span><span class="line"><span class="cl"> <span class="n">ET</span><span class="o">.</span><span class="n">SubElement</span><span class="p">(</span><span class="n">channel</span><span class="p">,</span> <span class="s2">&#34;link&#34;</span><span class="p">)</span><span class="o">.</span><span class="n">text</span> <span class="o">=</span> <span class="s2">&#34;https://github.com/thiagokokada/blog&#34;</span>
3491 </span></span><span class="line"><span class="cl"> <span class="n">ET</span><span class="o">.</span><span class="n">SubElement</span><span class="p">(</span><span class="n">channel</span><span class="p">,</span> <span class="s2">&#34;description&#34;</span><span class="p">)</span><span class="o">.</span><span class="n">text</span> <span class="o">=</span> <span class="s2">&#34;dd if=/dev/urandom of=/dev/brain0&#34;</span>
3492 </span></span><span class="line"><span class="cl">
3493 </span></span><span class="line"><span class="cl"> <span class="c1"># You create one item for each blog post</span>
3494 </span></span><span class="line"><span class="cl"> <span class="k">for</span> <span class="n">date</span><span class="p">,</span> <span class="n">dayPost</span> <span class="ow">in</span> <span class="n">posts</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
3495 </span></span><span class="line"><span class="cl"> <span class="k">for</span> <span class="n">post</span> <span class="ow">in</span> <span class="n">dayPost</span><span class="p">:</span>
3496 </span></span><span class="line"><span class="cl"> <span class="n">item</span> <span class="o">=</span> <span class="n">ET</span><span class="o">.</span><span class="n">SubElement</span><span class="p">(</span><span class="n">channel</span><span class="p">,</span> <span class="s2">&#34;item&#34;</span><span class="p">)</span>
3497 </span></span><span class="line"><span class="cl"> <span class="n">link</span> <span class="o">=</span> <span class="n">urljoin</span><span class="p">(</span><span class="n">RSS_POST_LINK_PREFIX</span><span class="p">,</span> <span class="n">post</span><span class="p">[</span><span class="s2">&#34;file&#34;</span><span class="p">])</span>
3498 </span></span><span class="line"><span class="cl"> <span class="n">ET</span><span class="o">.</span><span class="n">SubElement</span><span class="p">(</span><span class="n">item</span><span class="p">,</span> <span class="s2">&#34;title&#34;</span><span class="p">)</span><span class="o">.</span><span class="n">text</span> <span class="o">=</span> <span class="n">post</span><span class="p">[</span><span class="s2">&#34;title&#34;</span><span class="p">]</span>
3499 </span></span><span class="line"><span class="cl"> <span class="n">ET</span><span class="o">.</span><span class="n">SubElement</span><span class="p">(</span><span class="n">item</span><span class="p">,</span> <span class="s2">&#34;guid&#34;</span><span class="p">)</span><span class="o">.</span><span class="n">text</span> <span class="o">=</span> <span class="n">link</span>
3500 </span></span><span class="line"><span class="cl"> <span class="n">ET</span><span class="o">.</span><span class="n">SubElement</span><span class="p">(</span><span class="n">item</span><span class="p">,</span> <span class="s2">&#34;link&#34;</span><span class="p">)</span><span class="o">.</span><span class="n">text</span> <span class="o">=</span> <span class="n">link</span>
3501 </span></span><span class="line"><span class="cl"> <span class="n">ET</span><span class="o">.</span><span class="n">SubElement</span><span class="p">(</span><span class="n">item</span><span class="p">,</span> <span class="s2">&#34;pubDate&#34;</span><span class="p">)</span><span class="o">.</span><span class="n">text</span> <span class="o">=</span> <span class="n">date</span><span class="o">.</span><span class="n">strftime</span><span class="p">(</span><span class="s1">&#39;</span><span class="si">%a</span><span class="s1">, </span><span class="si">%d</span><span class="s1"> %b %Y %H:%M:%S GMT&#39;</span><span class="p">)</span>
3502 </span></span><span class="line"><span class="cl">
3503 </span></span><span class="line"><span class="cl"> <span class="c1"># Generate the XML and indent</span>
3504 </span></span><span class="line"><span class="cl"> <span class="n">tree</span> <span class="o">=</span> <span class="n">ET</span><span class="o">.</span><span class="n">ElementTree</span><span class="p">(</span><span class="n">rss</span><span class="p">)</span>
3505 </span></span><span class="line"><span class="cl"> <span class="n">ET</span><span class="o">.</span><span class="n">indent</span><span class="p">(</span><span class="n">tree</span><span class="p">,</span> <span class="n">space</span><span class="o">=</span><span class="s2">&#34;</span><span class="se">\t</span><span class="s2">&#34;</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>
3506 </span></span><span class="line"><span class="cl"> <span class="n">tree</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="s2">&#34;rss.xml&#34;</span><span class="p">,</span> <span class="n">xml_declaration</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">encoding</span><span class="o">=</span><span class="s2">&#34;UTF-8&#34;</span><span class="p">)</span>
3507 </span></span></code></pre><p>To publish a new Post, a basically write a Markdown file, run `./gen_readme.py</p>
3508 <blockquote>
3509 <p>README.md` at the root of the repository, and see the magic happen.</p>
3510 </blockquote>
3511 <p>It works much better than I initially anticipated. The <code>README.md</code> is properly
3512 populated with the titles and links. The RSS is kind of empty since it has no
3513 description, but it seems to work fine (at least in
3514 <a href="https://www.inoreader.com/">Inoreader</a>, my RSS reader of choice). I can
3515 probably fill the post description with more information if I really want, but
3516 it is enough for now (update: it is working now, you just need to render the
3517 Markdown as HTML and escape the tags;
3518 <a href="https://github.com/thiagokokada/blog/blob/9506051cc3d49f203304174b335ff62de9d17a05/gen_blog.py">permalink</a>
3519 for the updated script). Not sure who is that interested in my writing that
3520 will want to use this RSS feed instead the one available in
3521 <a href="https://kokada.capivaras.dev/rss/">capivaras.dev</a> anyway.</p>
3522 <p>Also, while I am using GitHub here, the same idea would work in GitLab, Gitea,
3523 sr.ht or whatever. As long as your source hub supports Markdown files it should
3524 work.</p>
3525 <p>So that is it. I am not saying this is a good idea for your primary blog
3526 platform or whatever, and I still prefer to publish to a platform that doesn't
3527 track users or have tons of JavaScript or whatever. But if you want a backup of
3528 your posts and you are already writing Markdown anyway, well, there are worse
3529 ways to do it I think.</p>
3530 <p><em>Update:</em> I rewrote the script again using Go
3531 (<a href="https://github.com/thiagokokada/blog/blob/3ece3853e7dd32977e7ec5238d2955e530ca4beb/blog.go">permalink</a>).
3532 The reason for it is because when I started rendering Markdown (for
3533 descriptions) the Python version got quite slow (not the fault of Python
3534 itself, mostly because of the usage of <code>nix-shell</code> to manage dependencies;
3535 something that Go doesn't need). Took about half an hour, showing how easy it
3536 is to do the same.</p></description><pubDate>Fri, 26 Jul 2024 00:00:00 +0000</pubDate><guid>https://kokada.dev/blog/using-github-as-a-bad-blog-platform/</guid></item><item><title>Writing NixOS tests for fun and profit</title><link>https://kokada.dev/blog/writing-nixos-tests-for-fun-and-profit/</link><description><p>I recently started a <a href="https://github.com/thiagokokada/hyprland-go">new side
3537 project</a> writing an IPC library in
3538 Go for <a href="https://hyprland.org/">Hyprland</a>, a Window Manager for Wayland.</p>
3539 <p>Once I got past the Work-in-Progress phase, I realise I had an issue: I wrote
3540 some tests, but I was running then inside my system running Hyprland. And the
3541 tests themselves were annoying: since they send commands to the current running
3542 Hyprland instance, I was having programs being opened and settings being
3543 changed, because this was the only way to have a reasonable good confidence
3544 that what I was doing was correct. So I need to do like any good developer and
3545 implement a CI, but how?</p>
3546 <p>One approach would be to create something like a mock client and test against
3547 my mock. Since this mock wouldn't need a running Hyprland instance the tests
3548 could run everywhere (even in non-Linux systems!), but they wouldn't be much
3549 useful. Mocks are great for testing business logic, but not really for making
3550 sure everything is working correctly.</p>
3551 <p>I need something more akin to an integration test, but this is tricky. It is
3552 not like I am doing integration with e.g.: PostgreSQL that has thousands of
3553 libraries available to make integration tests easier, I am doing integration
3554 with a Window Manager that is a moving target with multiple breaking changes in
3555 each release. And this is where NixOS tests enter, a way to run tests inside
3556 Virtual Machines configured in Nix.</p>
3557 <p>I am a long time NixOS user and commiter, but I never wrote a NixOS test
3558 outside of <a href="https://github.com/NixOS/nixpkgs">nixpkgs</a> itself. However I knew
3559 it was possible, and after doing a quick reading of the <a href="https://wiki.nixos.org/wiki/NixOS_VM_tests">Wiki
3560 entry</a> about it, I was ready to
3561 start.</p>
3562 <p>The first part is to call <code>pkgs.testers.runNixOSTest</code> and configure the machine
3563 as any other NixOS system, e.g.:</p>
3564 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span> <span class="n">pkgs</span><span class="o">,</span> <span class="o">...</span> <span class="p">}:</span>
3565 </span></span><span class="line"><span class="cl"><span class="n">pkgs</span><span class="o">.</span><span class="n">testers</span><span class="o">.</span><span class="n">runNixOSTest</span> <span class="p">{</span>
3566 </span></span><span class="line"><span class="cl"> <span class="n">name</span> <span class="o">=</span> <span class="s2">&#34;hyprland-go&#34;</span><span class="p">;</span>
3567 </span></span><span class="line"><span class="cl">
3568 </span></span><span class="line"><span class="cl"> <span class="n">nodes</span><span class="o">.</span><span class="n">machine</span> <span class="o">=</span>
3569 </span></span><span class="line"><span class="cl"> <span class="p">{</span> <span class="n">config</span><span class="o">,</span> <span class="n">pkgs</span><span class="o">,</span> <span class="n">lib</span><span class="o">,</span> <span class="o">...</span> <span class="p">}:</span>
3570 </span></span><span class="line"><span class="cl"> <span class="p">{</span>
3571 </span></span><span class="line"><span class="cl"> <span class="c1"># bootloader related configuration</span>
3572 </span></span><span class="line"><span class="cl"> <span class="n">boot</span><span class="o">.</span><span class="n">loader</span><span class="o">.</span><span class="n">systemd-boot</span><span class="o">.</span><span class="n">enable</span> <span class="o">=</span> <span class="no">true</span><span class="p">;</span>
3573 </span></span><span class="line"><span class="cl"> <span class="n">boot</span><span class="o">.</span><span class="n">loader</span><span class="o">.</span><span class="n">efi</span><span class="o">.</span><span class="n">canTouchEfiVariables</span> <span class="o">=</span> <span class="no">true</span><span class="p">;</span>
3574 </span></span><span class="line"><span class="cl">
3575 </span></span><span class="line"><span class="cl"> <span class="c1"># enable hyprland</span>
3576 </span></span><span class="line"><span class="cl"> <span class="n">programs</span><span class="o">.</span><span class="n">hyprland</span><span class="o">.</span><span class="n">enable</span> <span class="o">=</span> <span class="no">true</span><span class="p">;</span>
3577 </span></span><span class="line"><span class="cl">
3578 </span></span><span class="line"><span class="cl"> <span class="c1"># create a user called alice</span>
3579 </span></span><span class="line"><span class="cl"> <span class="n">users</span><span class="o">.</span><span class="n">users</span><span class="o">.</span><span class="n">alice</span> <span class="o">=</span> <span class="p">{</span>
3580 </span></span><span class="line"><span class="cl"> <span class="n">isNormalUser</span> <span class="o">=</span> <span class="no">true</span><span class="p">;</span>
3581 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
3582 </span></span><span class="line"><span class="cl">
3583 </span></span><span class="line"><span class="cl"> <span class="c1"># add some extra packages that we need during tests</span>
3584 </span></span><span class="line"><span class="cl"> <span class="n">environment</span><span class="o">.</span><span class="n">systemPackages</span> <span class="o">=</span> <span class="k">with</span> <span class="n">pkgs</span><span class="p">;</span> <span class="p">[</span> <span class="n">go</span> <span class="n">kitty</span> <span class="p">];</span>
3585 </span></span><span class="line"><span class="cl">
3586 </span></span><span class="line"><span class="cl"> <span class="c1"># auto login as alice</span>
3587 </span></span><span class="line"><span class="cl"> <span class="n">services</span><span class="o">.</span><span class="n">getty</span><span class="o">.</span><span class="n">autologinUser</span> <span class="o">=</span> <span class="s2">&#34;alice&#34;</span><span class="p">;</span>
3588 </span></span><span class="line"><span class="cl">
3589 </span></span><span class="line"><span class="cl"> <span class="c1"># configure VM, increase memory and CPU and enable OpenGL via LLVMPipe</span>
3590 </span></span><span class="line"><span class="cl"> <span class="n">virtualisation</span><span class="o">.</span><span class="n">qemu</span> <span class="o">=</span> <span class="p">{</span>
3591 </span></span><span class="line"><span class="cl"> <span class="n">options</span> <span class="o">=</span> <span class="p">[</span>
3592 </span></span><span class="line"><span class="cl"> <span class="s2">&#34;-smp 2&#34;</span>
3593 </span></span><span class="line"><span class="cl"> <span class="s2">&#34;-m 4G&#34;</span>
3594 </span></span><span class="line"><span class="cl"> <span class="s2">&#34;-vga none&#34;</span>
3595 </span></span><span class="line"><span class="cl"> <span class="s2">&#34;-device virtio-gpu-pci&#34;</span>
3596 </span></span><span class="line"><span class="cl"> <span class="p">];</span>
3597 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
3598 </span></span><span class="line"><span class="cl">
3599 </span></span><span class="line"><span class="cl"> <span class="c1"># Start hyprland at login</span>
3600 </span></span><span class="line"><span class="cl"> <span class="n">programs</span><span class="o">.</span><span class="n">bash</span><span class="o">.</span><span class="n">loginShellInit</span> <span class="o">=</span> <span class="s2">&#34;Hyprland&#34;</span><span class="p">;</span>
3601 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
3602 </span></span><span class="line"><span class="cl">
3603 </span></span><span class="line"><span class="cl"> <span class="n">testScript</span> <span class="o">=</span> <span class="s2">&#34;start_all()&#34;</span><span class="p">;</span>
3604 </span></span><span class="line"><span class="cl"><span class="p">}</span>
3605 </span></span></code></pre><p>A few details that I want to bring to attention. The first one is how easy it
3606 is to setup things like a normal user account, add some extra packages we need
3607 for testing, add Hyprland itself and configure auto-login. I have no idea how
3608 painful it would be to automatise all those steps in e.g.: Ansible, but here we
3609 are in a few lines of Nix code. This is, of course, thanks to all the
3610 contributors to nixpkgs that implement something that help their own use case,
3611 but once combined make it greater than the sum of the parts.</p>
3612 <p>Second is something that I took a while to figure out: how to enable GPU
3613 acceleration inside the VM. You see, Hyprland, different from other Window
3614 Managers, requires OpenGL support. This is basically why the flag <code>-device virtio-gpu-pci</code> is in <code>virtualisation.qemu.options</code>, this enables OpenGL
3615 rendering via LLVMPipe, that while being slow since it is rendered in CPU, is
3616 sufficient for this case.</p>
3617 <p>Putting the above code inside a
3618 <a href="https://wiki.nixos.org/wiki/Flakes"><code>flake.nix</code></a> for reproducibility, I had
3619 something similar to:</p>
3620 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
3621 </span></span><span class="line"><span class="cl"> <span class="n">description</span> <span class="o">=</span> <span class="s2">&#34;Hyprland&#39;s IPC bindings for Go&#34;</span><span class="p">;</span>
3622 </span></span><span class="line"><span class="cl">
3623 </span></span><span class="line"><span class="cl"> <span class="n">inputs</span> <span class="o">=</span> <span class="p">{</span>
3624 </span></span><span class="line"><span class="cl"> <span class="n">nixpkgs</span><span class="o">.</span><span class="n">url</span> <span class="o">=</span> <span class="s2">&#34;github:NixOS/nixpkgs/nixos-unstable&#34;</span><span class="p">;</span>
3625 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
3626 </span></span><span class="line"><span class="cl">
3627 </span></span><span class="line"><span class="cl"> <span class="n">outputs</span> <span class="o">=</span> <span class="p">{</span> <span class="n">nixpkgs</span><span class="o">,</span> <span class="o">...</span> <span class="p">}:</span>
3628 </span></span><span class="line"><span class="cl"> <span class="p">{</span>
3629 </span></span><span class="line"><span class="cl"> <span class="n">checks</span><span class="o">.</span><span class="n">x86_64-linux</span> <span class="o">=</span>
3630 </span></span><span class="line"><span class="cl"> <span class="k">let</span>
3631 </span></span><span class="line"><span class="cl"> <span class="n">pkgs</span> <span class="o">=</span> <span class="kn">import</span> <span class="n">nixpkgs</span> <span class="p">{</span> <span class="n">system</span> <span class="o">=</span> <span class="s2">&#34;x86_64-linux&#34;</span><span class="p">;</span> <span class="p">};</span>
3632 </span></span><span class="line"><span class="cl"> <span class="k">in</span>
3633 </span></span><span class="line"><span class="cl"> <span class="p">{</span>
3634 </span></span><span class="line"><span class="cl"> <span class="n">testVm</span> <span class="o">=</span> <span class="n">pkgs</span><span class="o">.</span><span class="n">testers</span><span class="o">.</span><span class="n">runNixOSTest</span> <span class="p">{</span>
3635 </span></span><span class="line"><span class="cl"> <span class="c1"># the code above</span>
3636 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
3637 </span></span><span class="line"><span class="cl"> <span class="p">}</span>
3638 </span></span><span class="line"><span class="cl"> <span class="p">};</span>
3639 </span></span><span class="line"><span class="cl"><span class="p">}</span>
3640 </span></span></code></pre><p>I can now run <code>nix build .#checks.x86_64-linux.testVm -L</code> to build and run the
3641 VM. However it is not really useful right now, since we didn't add any useful
3642 code in <code>testScript</code>, the core of the NixOS test framework. We can also run
3643 <code>nix build .#checks.x86_64-linux.testVm.driverInteractive</code> and
3644 <code>./result/bin/nixos-test-driver</code>: this will start a Python console where we can
3645 manually play with the VM (try typing <code>start_all()</code> for example).</p>
3646 <p>The <code>testScript</code> is a sequence of Python statements that perform various
3647 actions, such as starting VMs, executing commands in the VMs, and so on. More
3648 about it in the official
3649 <a href="https://nixos.org/manual/nixos/stable/index.html#sec-nixos-tests">documentation</a>.
3650 For our case we can start with something like this:</p>
3651 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
3652 </span></span><span class="line"><span class="cl"> <span class="n">testScript</span> <span class="o">=</span> <span class="cm">/* python */</span> <span class="s1">&#39;&#39;
3653 </span></span></span><span class="line"><span class="cl"><span class="s1"> start_all()
3654 </span></span></span><span class="line"><span class="cl"><span class="s1">
3655 </span></span></span><span class="line"><span class="cl"><span class="s1"> machine.wait_for_unit(&#34;multi-user.target&#34;)
3656 </span></span></span><span class="line"><span class="cl"><span class="s1"> machine.wait_for_file(&#34;/home/alice/test-finished&#34;)
3657 </span></span></span><span class="line"><span class="cl"><span class="s1"> &#39;&#39;</span><span class="p">;</span>
3658 </span></span><span class="line"><span class="cl"><span class="p">}</span>
3659 </span></span></code></pre><p>The first statement, <code>start_all()</code>, starts all VMs, in this case we have only
3660 one, called <code>machine</code>. We send two further commands to <code>machine</code>:
3661 <code>wait_for_unit(&quot;multi-user.target&quot;)</code> and
3662 <code>wait_for_file(&quot;/home/alice/test-finished&quot;)</code>.</p>
3663 <p>The first command waits until systemd's <code>multi-user.target</code> is ready, a good
3664 way to ensure that the system is ready for further commands. The second one we
3665 wait for a file called <code>test-finished</code> to appear in Alice's <code>$HOME</code> (basically,
3666 a canary), but how can we generate this file?</p>
3667 <p>Remember that we added <code>programs.bash.loginShellInit = &quot;Hyprland&quot;</code>, that
3668 automatically starts Hyprland when Alice logs in. We need to modify that
3669 command to run the Go tests from our library. The good thing is that Hyprland
3670 configuration file supports a
3671 <a href="https://wiki.hyprland.org/Configuring/Keywords/#executing"><code>exec-once</code></a>
3672 command that runs a command during Hyprland launch. We can abuse this to launch
3673 a terminal emulator and run our tests:</p>
3674 <pre class="chroma"><code><span class="line"><span class="cl"><span class="p">{</span>
3675 </span></span><span class="line"><span class="cl"> <span class="n">programs</span><span class="o">.</span><span class="n">bash</span><span class="o">.</span><span class="n">loginShellInit</span> <span class="o">=</span>
3676 </span></span><span class="line"><span class="cl"> <span class="k">let</span>
3677 </span></span><span class="line"><span class="cl"> <span class="n">testScript</span> <span class="o">=</span> <span class="n">pkgs</span><span class="o">.</span><span class="n">writeShellScript</span> <span class="s2">&#34;hyprland-go-test&#34;</span> <span class="s1">&#39;&#39;
3678 </span></span></span><span class="line"><span class="cl"><span class="s1"> set -euo pipefail
3679 </span></span></span><span class="line"><span class="cl"><span class="s1">
3680 </span></span></span><span class="line"><span class="cl"><span class="s1"> trap &#39;echo $? &gt; $HOME/test-finished&#39; EXIT # creates the canary when the script finishes
3681 </span></span></span><span class="line"><span class="cl"><span class="s1">
3682 </span></span></span><span class="line"><span class="cl"><span class="s1"> cd </span><span class="si">${</span><span class="sr">./.</span><span class="si">}</span><span class="s1"> # go to the library directory
3683 </span></span></span><span class="line"><span class="cl"><span class="s1"> go test -v ./... &gt; $HOME/test.log 2&gt;&amp;1 # run Go tests
3684 </span></span></span><span class="line"><span class="cl"><span class="s1"> &#39;&#39;</span><span class="p">;</span>
3685 </span></span><span class="line"><span class="cl"> <span class="n">hyprlandConf</span> <span class="o">=</span> <span class="n">pkgs</span><span class="o">.</span><span class="n">writeText</span> <span class="s2">&#34;hyprland.conf&#34;</span> <span class="s1">&#39;&#39;
3686 </span></span></span><span class="line"><span class="cl"><span class="s1"> exec-once = kitty sh -c </span><span class="si">${</span><span class="n">testScript</span><span class="si">}</span><span class="s1">
3687 </span></span></span><span class="line"><span class="cl"><span class="s1"> &#39;&#39;</span><span class="p">;</span>
3688 </span></span><span class="line"><span class="cl"> <span class="k">in</span> <span class="s1">&#39;&#39;
3689 </span></span></span><span class="line"><span class="cl"><span class="s1"> Hyprland --config </span><span class="si">${</span><span class="n">hyprlandConf</span><span class="si">}</span><span class="s1">
3690 </span></span></span><span class="line"><span class="cl"><span class="s1"> &#39;&#39;</span><span class="p">;</span>
3691 </span></span><span class="line"><span class="cl"><span class="p">}</span>
3692 </span></span></code></pre><p>So we are basically creating a custom Hyprland config that starts a
3693 <a href="https://sw.kovidgoyal.net/kitty/">Kitty</a> terminal emulator, that then launches
3694 a shell script that runs the test. Since we have no way to get the results of
3695 the test, we pipe the output to a file that we can collect later (e.g.:
3696 <code>machine.succeded(&quot;cat /home/alice/test.log&quot;)</code>). And once the script exit, we
3697 create the canary file <code>$HOME/test-finished</code>, that allows the <code>testScript</code>
3698 knows that the test finished and it can destroy the VM safely.</p>
3699 <p>If you want to take a look at the final result, it is
3700 <a href="https://github.com/thiagokokada/hyprland-go/blob/v0.0.1/flake.nix">here</a>. This
3701 tests run in any Linux machine that supports KVM, and also works in <a href="https://github.com/thiagokokada/hyprland-go/actions/workflows/nix.yaml">GitHub
3702 Actions</a>
3703 thanks to the the
3704 <a href="https://github.com/DeterminateSystems/nix-installer-action">nix-installer-action</a>.</p>
3705 <p>And now I have a proper CI pipeline in a way that I never imagined would be
3706