nedbatchelder.com_blog.rss.xml - sfeed_tests - sfeed tests and RSS and Atom files
(HTM) git clone git://git.codemadness.org/sfeed_tests
(DIR) Log
(DIR) Files
(DIR) Refs
(DIR) README
(DIR) LICENSE
---
nedbatchelder.com_blog.rss.xml (67201B)
---
1 <?xml version="1.0" encoding="UTF-8"?>
2 <?xml-stylesheet type="text/xsl" href="https://nedbatchelder.com/rssfull2html.xslt" media="screen" ?>
3
4 <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns="http://purl.org/rss/1.0/">
5 <channel rdf:about="https://nedbatchelder.com/blog">
6 <title>Ned Batchelder's blog</title>
7 <link>https://nedbatchelder.com/blog</link>
8 <description>Ned Batchelder's personal blog.</description>
9 <dc:language>en-US</dc:language>
10 <image rdf:resource="https://nedbatchelder.com/pix/rss-banner.gif"/>
11 <items>
12 <rdf:Seq>
13 <rdf:li resource="https://nedbatchelder.com/blog/202509/hobby_hilbert_simplex.html"/><rdf:li resource="https://nedbatchelder.com/blog/202509/testing_is_better_than_dsa.html"/><rdf:li resource="https://nedbatchelder.com/blog/202508/finding_unneeded_pragmas.html"/><rdf:li resource="https://nedbatchelder.com/blog/202508/starting_with_pytests_parametrize.html"/><rdf:li resource="https://nedbatchelder.com/blog/202507/coveragepy_regex_pragmas.html"/><rdf:li resource="https://nedbatchelder.com/blog/202507/coverage_7100_patch.html"/><rdf:li resource="https://nedbatchelder.com/blog/202507/2048_iterators_and_iterables.html"/><rdf:li resource="https://nedbatchelder.com/blog/202506/math_factoid_of_the_day_63.html"/><rdf:li resource="https://nedbatchelder.com/blog/202506/digital_equipment_corporation_no_more.html"/><rdf:li resource="https://nedbatchelder.com/blog/202505/pycon_summer_camp.html"/>
14 </rdf:Seq>
15 </items>
16 </channel>
17 <image rdf:about="https://nedbatchelder.com/pix/rss-banner.gif">
18 <title>Ned Batchelder's blog</title>
19 <link>https://nedbatchelder.com/blog</link>
20 <url>https://nedbatchelder.com/pix/rss-banner.gif</url>
21 </image>
22
23 <item rdf:about="https://nedbatchelder.com/blog/202509/hobby_hilbert_simplex.html">
24 <title>Hobby Hilbert Simplex</title>
25 <link>https://nedbatchelder.com/blog/202509/hobby_hilbert_simplex.html</link>
26
27 <dc:date>2025-09-26T08:14:04-04:00</dc:date>
28 <dc:creator>Ned Batchelder</dc:creator>
29 <description><![CDATA[<p>I saw a generative art piece I liked and wanted to learn how it was made.
30 Starting with the artist’s Kotlin code, I dug into three new algorithms, hacked
31 together some Python code, experimented with alternatives, and learned a lot.
32 Now I can explain it to you.</p><p>It all started with this post by
33 <a href="https://genart.social/@hamoid/115125620138280715" rel="external noopener">aBe on Mastodon</a>:</p><blockquote class="mastodon-post" lang="en" cite="https://genart.social/@hamoid/115125620138280715" data-source="fediverse">
34 <p>I love how these lines separate and reunite. And the fact that I can express this idea in 3 or 4 lines of code.</p><p>For me they’re lives represented by closed paths that end where they started, spending part of the journey together, separating while we go in different directions and maybe reconnecting again in the future.</p><p><a href="https://genart.social/tags/CreativeCoding" rel="nofollow noopener" class="mention hashtag" target="_blank">#<span>CreativeCoding</span></a> <a href="https://genart.social/tags/algorithmicart" rel="nofollow noopener" class="mention hashtag" target="_blank">#<span>algorithmicart</span></a> <a href="https://genart.social/tags/proceduralArt" rel="nofollow noopener" class="mention hashtag" target="_blank">#<span>proceduralArt</span></a> <a href="https://genart.social/tags/OPENRNDR" rel="nofollow noopener" class="mention hashtag" target="_blank">#<span>OPENRNDR</span></a> <a href="https://genart.social/tags/Kotlin" rel="nofollow noopener" class="mention hashtag" target="_blank">#<span>Kotlin</span></a></p>
35 <figure><figure><img src="https://media.hachyderm.io/cache/media_attachments/files/115/125/620/285/265/947/small/5a73d40e6a4a81c1.png" alt="80 wobbly black hobby curves with low opacity. In some places the curves travel together, but sometimes they split in 2 or 3 groups and later reunite. Due to the low opacity, depending on how many curves overlap the result is brighter or darker." width="480" height="480"></figure></figure>
36 <footer>
37 — aBe (@hamoid@genart.social) <a href="https://genart.social/@hamoid/115125620138280715" rel="external noopener"><time datetime="2025-08-31T21:59:13.000Z">8/31/2025, 5:59:13 PM</time></a>
38 </footer>
39 </blockquote><p>The drawing is made by choosing 10 random points, drawing a curve through
40 those points, then slightly scooching the points and drawing another curve.
41 There are 40 curves, each slightly different than the last. Occasionally
42 the next curve makes a jump, which is why they separate and reunite.</p><p>Eventually I made something similar:</p><div class="figurep"><figure><picture><source type="image/webp" srcset="https://nedbatchelder.com/iv/webp/pix/fluidity/repro_139.png.webp"><img src="https://nedbatchelder.com/pix/fluidity/repro_139.png" alt="" width="600" height="600" class="hairline"></picture></figure></div><p>Along the way I had to learn about three techniques I got from the Kotlin
43 code: Hobby curves, Hilbert sorting, and simplex noise.</p><p>Each of these algorithms tries to do something “natural” automatically, so
44 that we can generate art that looks nice without any manual steps.</p><h1 id="h_hobby_curves">Hobby curves<a class="headerlink" aria-label="Link to this header" href="#h_hobby_curves"></a></h1><p>To draw swoopy curves through our random points, we use an algorithm
45 developed by John Hobby as part of Donald Knuth’s Metafont type design system.
46 Jake Low has a <a rel="external noopener" href="https://www.jakelow.com/blog/hobby-curves">great interactive page for playing with Hobby
47 curves</a>, you should try it.</p><p>Here are three examples of Hobby curves through ten random points:</p><div class="figurep"><figure><picture><source type="image/webp" srcset="https://nedbatchelder.com/iv/webp/pix/fluidity/hobby_unsorted.png.webp"><img src="https://nedbatchelder.com/pix/fluidity/hobby_unsorted.png" alt="" width="600" height="200" class="hairline"></picture></figure></div><p>The curves are nice, but kind of a scribble, because we’re joining points
48 together in the order we generated them (shown by the green lines). If you
49 asked a person to connect random points, they wouldn’t jump back and forth
50 across the canvas like this. They would find a nearby point to use next,
51 producing a more natural tour of the set.</p><p>We’re generating everything automatically, so we can’t manually intervene
52 to choose a natural order for the points. Instead we use Hilbert sorting.</p><h1 id="h_hilbert_sorting">Hilbert sorting<a class="headerlink" aria-label="Link to this header" href="#h_hilbert_sorting"></a></h1><p>The Hilbert space-filling fractal visits every square in a 2D grid.
53 <a rel="external noopener" href="https://doc.cgal.org/latest/Spatial_sorting/index.html">Hilbert sorting</a> uses a Hilbert fractal traversing
54 the canvas, and sorts the points by when their square is visited by the fractal.
55 This gives a tour of the points that corresponds more closely to what people
56 expect. Points that are close together in space are likely (but not guaranteed)
57 to be close in the ordering.</p><p>If we sort the points using Hilbert sorting, we get much nicer curves. Here
58 are the same points as last time:</p><div class="figurep"><figure><picture><source type="image/webp" srcset="https://nedbatchelder.com/iv/webp/pix/fluidity/hobby_sorted.png.webp"><img src="https://nedbatchelder.com/pix/fluidity/hobby_sorted.png" alt="" width="600" height="200" class="hairline"></picture></figure></div><p>Here are pairs of the same points, unsorted and sorted side-by-side:</p><div class="figurep"><figure><picture><source type="image/webp" srcset="https://nedbatchelder.com/iv/webp/pix/fluidity/hilbert_compared.png.webp"><img src="https://nedbatchelder.com/pix/fluidity/hilbert_compared.png" alt="" width="400" height="800" class="hairline"></picture></figure></div><p>If you compare closely, the points in each pair are the same, but the sorted
59 points are connected in a better order, producing nicer curves.</p><h1 id="h_simplex_noise">Simplex noise<a class="headerlink" aria-label="Link to this header" href="#h_simplex_noise"></a></h1><p>Choosing random points would be easy to do with a random number generator,
60 but we want the points to move in interesting graceful ways. To do that, we use
61 simplex noise. This is a 2D function (let’s call the inputs u and v) that
62 produces a value from -1 to 1. The important thing is the function is
63 continuous: if you sample it at two (u,v) coordinates that are close together,
64 the results will be close together. But it’s also random: the continuous curves
65 you get are wavy in unpredictable ways. Think of the simplex noise function as
66 a smooth hilly landscape.</p><p>To get an (x,y) point for our drawing, we choose a (u,v) coordinate to
67 produce an x value and a completely different (u,v) coordinate for the y. To
68 get the next (x,y) point, we keep the u values the same and change the v values by
69 just a tiny bit. That makes the (x,y) points move smoothly but interestingly.</p><p>Here are the trails of four points taking 50 steps using this scheme:</p><div class="figurep"><figure><picture><source type="image/webp" srcset="https://nedbatchelder.com/iv/webp/pix/fluidity/point_motion.png.webp"><img src="https://nedbatchelder.com/pix/fluidity/point_motion.png" alt="" width="400" height="400" class="hairline"></picture></figure></div><p>If we use seven points taking five steps, and draw curves through the seven
70 points at each step, we get examples like this:</p><div class="figurep"><figure><picture><source type="image/webp" srcset="https://nedbatchelder.com/iv/webp/pix/fluidity/small_runs.png.webp"><img src="https://nedbatchelder.com/pix/fluidity/small_runs.png" alt="" width="600" height="300" class="hairline"></picture></figure></div><p>I’ve left the points visible, and given them large steps so the lines are
71 very widely spaced to show the motion. Taking out the points and drawing more
72 lines with smaller steps gives us this:</p><div class="figurep"><figure><picture><source type="image/webp" srcset="https://nedbatchelder.com/iv/webp/pix/fluidity/large_runs.png.webp"><img src="https://nedbatchelder.com/pix/fluidity/large_runs.png" alt="" width="600" height="300" class="hairline"></picture></figure></div><p>With 40 lines drawn wider with some transparency, we start to see the smoky
73 fluidity:</p><div class="figurep"><figure><picture><source type="image/webp" srcset="https://nedbatchelder.com/iv/webp/pix/fluidity/larger_runs.png.webp"><img src="https://nedbatchelder.com/pix/fluidity/larger_runs.png" alt="" width="600" height="300" class="hairline"></picture></figure></div><h1 id="h_jumps">Jumps<a class="headerlink" aria-label="Link to this header" href="#h_jumps"></a></h1><p>In his Mastodon post, aBe commented on the separating of the lines as one of
74 the things he liked about this. But why do they do that? If we are moving the
75 points in small increments, why do the curves sometimes make large jumps?</p><p>The first reason is because of Hobby curves. They do a great job drawing a
76 curve through a set of points as a person might. But a downside of the
77 algorithm is sometimes changing a point a small amount makes the entire curve
78 take a different route. If you play around with the interactive examples on
79 <a rel="external noopener" href="https://www.jakelow.com/blog/hobby-curves">Jake Low’s page</a> you will see the curve can unexpectedly
80 take a different shape.</p><p>As we inch our points along, sometimes the Hobby curve jumps.</p><p>The second reason is due to Hilbert sorting. Each of our lines is sorted
81 independently of how the previous line was sorted. If a point’s small motion
82 moves it into a different grid square, it can change the sorting order, which
83 changes the Hobby curve even more.</p><p>If we sort the first line, and then keep that order of points for all the
84 lines, the result has fewer jumps, but the Hobby curves still act
85 unpredictably:</p><div class="figurep"><figure><picture><source type="image/webp" srcset="https://nedbatchelder.com/iv/webp/pix/fluidity/first_line_runs.png.webp"><img src="https://nedbatchelder.com/pix/fluidity/first_line_runs.png" alt="" width="600" height="300" class="hairline"></picture></figure></div><h1 id="h_colophon">Colophon<a class="headerlink" aria-label="Link to this header" href="#h_colophon"></a></h1><p>This was all done with Python, using other people’s implementations of the
86 hard parts:
87 <a href="https://github.com/ltrujello/Hobby_Curve_Algorithm/blob/main/python/hobby.py" rel="external noopener">hobby.py</a>,
88 <a href="https://pypi.org/project/hilbertcurve/" rel="external noopener">hilbertcurve</a>, and
89 <a href="https://pypi.org/project/super-simplex/" rel="external noopener">super-simplex</a>. My code
90 is on GitHub
91 (<a href="https://github.com/nedbat/fluidity" rel="external noopener">nedbat/fluidity</a>), but it’s a
92 mess. Think of it as a woodworking studio with half-finished pieces and wood
93 chips strewn everywhere.</p><p>A lot of the learning and experimentation was in
94 <a href="https://github.com/nedbat/fluidity/blob/main/play.ipynb" rel="external noopener">my Jupyter
95 notebook</a>. Part of the process for work like this is playing around with
96 different values of tweakable parameters and seeds for the random numbers to get
97 the effect you want, either artistic or pedagogical. The notebook shows some of
98 the thumbnail galleries I used to pick the examples to show.</p><p>I went on to play with animations, which led to other learnings, but those
99 will have to wait for another blog post.</p>
100 ]]></description>
101 </item>
102
103 <item rdf:about="https://nedbatchelder.com/blog/202509/testing_is_better_than_dsa.html">
104 <title>Testing is better than DSA</title>
105 <link>https://nedbatchelder.com/blog/202509/testing_is_better_than_dsa.html</link>
106
107 <dc:date>2025-09-22T12:04:08-04:00</dc:date>
108 <dc:creator>Ned Batchelder</dc:creator>
109 <description><![CDATA[<p>I see new learners asking about “DSA” a lot. Data Structures and Algorithms
110 are of course important: considered broadly, they are the two ingredients that
111 make up all programs. But in my opinion, “DSA” as an abstract field of study
112 is over-emphasized.</p><p>I understand why people focus on DSA: it’s a concrete thing to learn about,
113 there are web sites devoted to testing you on it, and most importantly, because
114 job interviews often involve DSA coding questions.</p><p>Before I get to other opinions, let me make clear that anything you can do to
115 help you get a job is a good thing to do. If grinding
116 <a rel="external noopener" href="https://leetcode.com/">leetcode</a> will land you a position, then do it.</p><p>But I hope companies hiring entry-level engineers aren’t asking them to
117 reverse linked lists or balance trees. Asking about techniques that can be
118 memorized ahead of time won’t tell them anything about how well you can work.
119 The stated purpose of those interviews is to see how well you can figure out
120 solutions, in which case memorization will defeat the point.</p><p>The thing new learners don’t understand about DSA is that actual software
121 engineering almost never involves implementing the kinds of algorithms that
122 “DSA” teaches you. Sure, it can be helpful to work through some of these
123 puzzles and see how they are solved, but writing real code just doesn’t involve
124 writing that kind of code.</p><p>Here is what I think in-the-trenches software engineers should know about
125 data structures and algorithms:</p><ul>
126
127 <li>Data structures are ways to organize data. Learn some of the basics: linked
128 list, array, hash table, tree. By “learn” I mean understand what it does
129 and why you might want to use one.</li>
130
131 <li>Different data structures can be used to organize the same data in different
132 ways. Learn some of the trade-offs between structures that are similar.</li>
133
134 <li>Algorithms are ways of manipulating data. I don’t mean named algorithms
135 like Quicksort, but algorithms as any chunk of code that works on data and
136 does something with it.</li>
137
138 <li>How you organize data affects what algorithms you can use to work with the
139 data. Some data structures will be slow for some operations where another
140 structure will be fast.</li>
141
142 <li>Algorithms have a “time complexity” (Big O): <a rel="external noopener" href="/text/bigo.html">how the code
143 slows as the data grows</a>. Get a sense of what this means.</li>
144
145 <li>Python has a number of built-in data structures. Learn how they work, and
146 the time complexity of their operations.</li>
147
148 <li>Learn how to think about your code to understand its time complexity.</li>
149
150 <li>Read a little about more esoteric things like <a rel="external noopener" href="https://systemdesign.one/bloom-filters-explained/">Bloom
151 filters</a>, so you can find them later in the unlikely case you need them.</li>
152
153 </ul><p>Here are some things you don’t need to learn:</p><ul>
154
155 <li>The details of a dozen different sorting algorithms. Look at two to see
156 different ways of approaching the same problem, then move on.</li>
157
158 <li>The names of “important” algorithms. Those have all been implemented for
159 you.</li>
160
161 <li>The answers to all N problems on some quiz web site. You won’t be asked
162 these exact questions, and they won’t come up in your real work. Again: try a
163 few to get a feel for how some algorithms work. The exact answers are not what
164 you need.</li>
165
166 </ul><p>Of course some engineers need to implement hash tables, or sorting algorithms
167 or whatever. We love those engineers: they write libraries we can use off the
168 shelf so we don’t have to implement them ourselves.</p><p>There have been times when I implemented something that felt like An
169 Algorithm (for example, <a href="https://nedbatchelder.com/blog/201707/finding_fuzzy_floats.html">Finding fuzzy floats</a>), but it was
170 more about considering another perspective on my data, looking at the time
171 complexity, and moving operations around to avoid quadratic behavior. It wasn’t
172 opening a textbook to find the famous algorithm that would solve my problem.</p><p>Again: if it will help you get a job, deep-study DSA. But don’t be
173 disappointed when you don’t use it on the job.</p><p>If you want to prepare yourself for a career, and also stand out in job
174 interviews, learn how to write tests:</p><ul>
175
176 <li>This will be a skill you use constantly. Real-world software means writing
177 tests much more than school teaches you to.</li>
178
179 <li>In a job search, testing experience will stand out more than DSA depth. It
180 shows you’ve thought about what it takes to write high-quality software instead
181 of just academic exercises.</li>
182
183 <li>It’s not obvious how to test code well. It’s a puzzle and a problem to
184 solve. If you like figuring out solutions to tricky questions, focus on how to
185 write code so that it can be tested, and how to test it.</li>
186
187 <li>Testing not only gives you more confidence in your code, it helps you write
188 better code in the first place.</li>
189
190 <li>Testing applies everywhere, from tiny bits of code to entire architectures,
191 assisting you in design and implementation at all scales.</li>
192
193 <li>If pursued diligently, testing is an engineering discipline in its own
194 right, with a fascinating array of tools and techniques.</li>
195
196 </ul><p>Less DSA, more testing.</p>
197 ]]></description>
198 </item>
199
200 <item rdf:about="https://nedbatchelder.com/blog/202508/finding_unneeded_pragmas.html">
201 <title>Finding unneeded pragmas</title>
202 <link>https://nedbatchelder.com/blog/202508/finding_unneeded_pragmas.html</link>
203
204 <dc:date>2025-08-24T17:28:12-04:00</dc:date>
205 <dc:creator>Ned Batchelder</dc:creator>
206 <description><![CDATA[<p>To answer a <a rel="external noopener" href="https://github.com/nedbat/coveragepy/issues/251">long-standing coverage.py feature request</a>, I
207 threw together an experiment: a tool to identify lines that have been excluded
208 from coverage, but which were actually executed.</p><p>The program is a standalone file in the coverage.py repo. It is unsupported.
209 I’d like people to try it to see what they think of the idea. Later we can
210 decide what to do with it.</p><p>To try it: copy <a rel="external noopener" href="https://github.com/nedbat/coveragepy/blob/master/lab/warn_executed.py">warn_executed.py</a> from
211 GitHub. Create a .toml file that looks something like this:</p><blockquote class="code"><pre class="toml"><div class="source"><span class="c1"># Regexes that identify excluded lines:</span>
212 <br><span class="n">warn-executed</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">[</span>
213 <br><span class="w">    </span><span class="s2">"pragma: no cover"</span><span class="p">,</span>
214 <br><span class="w">    </span><span class="s2">"raise AssertionError"</span><span class="p">,</span>
215 <br><span class="w">    </span><span class="s2">"pragma: cant happen"</span><span class="p">,</span>
216 <br><span class="w">    </span><span class="s2">"pragma: never called"</span><span class="p">,</span>
217 <br><span class="w">    </span><span class="p">]</span>
218 <br>
219 <br><span class="c1"># Regexes that identify partial branch lines:</span>
220 <br><span class="n">warn-not-partial</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">[</span>
221 <br><span class="w">    </span><span class="s2">"pragma: no branch"</span><span class="p">,</span>
222 <br><span class="w">    </span><span class="p">]</span>
223 <br></div>
224 </pre></blockquote><p>These are exclusion regexes that you’ve used in your coverage runs. The
225 program will print out any line identified by a pattern and that ran during your
226 tests. It might be that you don’t need to exclude the line, because it ran.</p><p>In this file, none of your coverage settings or the default regexes are
227 assumed: you need to explicitly specify all the patterns you want flagged.</p><p>Run the program with Python 3.11 or higher, giving the name of the coverage
228 data file and the name of your new TOML configuration file. It will print the
229 lines that might not need excluding:</p><blockquote class="code"><pre class="shell"><div class="source">$<span class="w"> </span>python3.12<span class="w"> </span>warn_executed.py<span class="w"> </span>.coverage<span class="w"> </span><span>warn.toml</span>
230 <br></div>
231 </pre></blockquote><p>The reason for a new list of patterns instead of just reading the existing
232 coverage settings is that some exclusions are “don’t care” rather than “this
233 will never happen.” For example, I exclude “def __repr__” because some
234 __repr__’s are just to make my debugging easier. I don’t care if the test suite
235 runs them or not. It might run them, so I don’t want it to be a warning that
236 they actually ran.</p><p>This tool is not perfect. For example, I exclude “if TYPE_CHECKING:” because
237 I want that entire clause excluded. But the if-line itself is actually run. If
238 I include that pattern in the warn-executed list, it will flag all of those
239 lines. Maybe I’m forgetting a way to do this: it would be good to have a way to
240 exclude the body of the if clause while understanding that the if-line itself is
241 executed.</p><p>Give <a rel="external noopener" href="https://github.com/nedbat/coveragepy/blob/master/lab/warn_executed.py">warn_executed.py</a> a try and comment on
242 <a rel="external noopener" href="https://github.com/nedbat/coveragepy/issues/251">the issue</a> about what you think of it.</p>
243 ]]></description>
244 </item>
245
246 <item rdf:about="https://nedbatchelder.com/blog/202508/starting_with_pytests_parametrize.html">
247 <title>Starting with pytest’s parametrize</title>
248 <link>https://nedbatchelder.com/blog/202508/starting_with_pytests_parametrize.html</link>
249
250 <dc:date>2025-08-13T06:14:46-04:00</dc:date>
251 <dc:creator>Ned Batchelder</dc:creator>
252 <description><![CDATA[<p>Writing tests can be difficult and repetitive. Pytest has a feature called
253 parametrize that can make it reduce duplication, but it can be hard to
254 understand if you are new to the testing world. It’s not as complicated as it
255 seems.</p><p>Let’s say you have a function called <code>add_nums()</code> that adds up a list of
256 numbers, and you want to write tests for it. Your tests might look like
257 this:</p><blockquote class="code"><pre class="python"><div class="source"><span class="k">def</span><span class="w"> </span><span class="nf">test_123</span><span class="p">():</span>
258 <br>    <span class="k">assert</span> <span class="n">add_nums</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">])</span> <span class="o">==</span> <span class="mi">6</span>
259 <br>
260 <br><span class="k">def</span><span class="w"> </span><span class="nf">test_negatives</span><span class="p">():</span>
261 <br>    <span class="k">assert</span> <span class="n">add_nums</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="o">-</span><span class="mi">3</span><span class="p">])</span> <span class="o">==</span> <span class="mi">0</span>
262 <br>
263 <br><span class="k">def</span><span class="w"> </span><span class="nf">test_empty</span><span class="p">():</span>
264 <br>    <span class="k">assert</span> <span class="n">add_nums</span><span class="p">([])</span> <span class="o">==</span> <span class="mi">0</span>
265 <br></div>
266 </pre></blockquote><p>This is great: you’ve tested some behaviors of your <code>add_nums()</code>
267 function. But it’s getting tedious to write out more test cases. The names of the
268 function have to be different from each other, and they don’t mean anything, so
269 it’s extra work for no benefit. The test functions all have the same structure,
270 so you’re repeating uninteresting details. You want to add more cases but it
271 feels like there’s friction that you want to avoid.</p><p>If we look at these functions, they are very similar. In any software, when
272 we have functions that are similar in structure, but differ in some details, we
273 can refactor them to be one function with parameters for the differences. We can
274 do the same for our test functions.</p><p>Here the functions all have the same structure: call <code>add_nums()</code> and
275 assert what the return value should be. The differences are the list we pass to
276 <code>add_nums()</code> and the value we expect it to return. So we can turn those
277 into two parameters in our refactored function:</p><blockquote class="code"><pre class="python"><div class="source"><span class="k">def</span><span class="w"> </span><span class="nf">test_add_nums</span><span class="p">(</span><span class="n">nums</span><span class="p">,</span> <span class="n">expected_total</span><span class="p">):</span>
278 <br>    <span class="k">assert</span> <span class="n">add_nums</span><span class="p">(</span><span class="n">nums</span><span class="p">)</span> <span class="o">==</span> <span class="n">expected_total</span>
279 <br></div>
280 </pre></blockquote><p>Unfortunately, tests aren’t run like regular functions. We write the test
281 functions, but we don’t call them ourselves. That’s the reason the names of the
282 test functions don’t matter. The test runner (pytest) finds functions named
283 <code>test_*</code> and calls them for us. When they have no parameters, pytest can
284 call them directly. But now that our test function has two parameters, we have
285 to give pytest instructions about how to call it.</p><p>To do that, we use the <code>@pytest.mark.parametrize</code> decorator. Using it
286 looks like this:</p><blockquote class="code"><pre class="python"><div class="source"><span class="kn">import</span><span class="w"> </span><span class="nn">pytest</span>
287 <br>
288 <br><span class="nd">@pytest</span><span class="o">.</span><span class="n">mark</span><span class="o">.</span><span class="n">parametrize</span><span class="p">(</span>
289 <br>    <span class="s2">"nums, expected_total"</span><span class="p">,</span>
290 <br>    <span class="p">[</span>
291 <br>        <span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">],</span> <span class="mi">6</span><span class="p">),</span>
292 <br>        <span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="o">-</span><span class="mi">3</span><span class="p">],</span> <span class="mi">0</span><span class="p">),</span>
293 <br>        <span class="p">([],</span> <span class="mi">0</span><span class="p">),</span>
294 <br>    <span class="p">]</span>
295 <br><span class="p">)</span>
296 <br><span class="k">def</span><span class="w"> </span><span class="nf">test_add_nums</span><span class="p">(</span><span class="n">nums</span><span class="p">,</span> <span class="n">expected_total</span><span class="p">):</span>
297 <br>    <span class="k">assert</span> <span class="n">add_nums</span><span class="p">(</span><span class="n">nums</span><span class="p">)</span> <span class="o">==</span> <span class="n">expected_total</span>
298 <br></div>
299 </pre></blockquote><p>There’s a lot going on here, so let’s take it step by step.</p><p>If you haven’t seen a decorator before, it starts with <code>@</code> and is like a
300 prologue to a function definition. It can affect how the function is defined or
301 provide information about the function.</p><p>The parametrize decorator is itself a function call that takes two arguments.
302 The first is a string (“nums, expected_total”) that names the two arguments to
303 the test function. Here the decorator is instructing pytest, “when you call
304 <code>test_add_nums</code>, you will need to provide values for its <code>nums and</code>
305 <code>expected_total parameters</code>.”</p><p>The second argument to <code>parametrize</code> is a list of the values to supply
306 as the arguments. Each element of the list will become one call to our test
307 function. In this example, the list has three tuples, so pytest will call our
308 test function three times. Since we have two parameters to provide, each
309 element of the list is a tuple of two values.</p><p>The first tuple is <code>([1, 2, 3], 6)</code>, so the first time pytest calls
310 test_add_nums, it will call it as test_add_nums([1, 2, 3], 6). All together,
311 pytest will call us three times, like this:</p><blockquote class="code"><pre class="python"><div class="source"><span class="n">test_add_nums</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">],</span> <span class="mi">6</span><span class="p">)</span>
312 <br><span class="n">test_add_nums</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="o">-</span><span class="mi">3</span><span class="p">],</span> <span class="mi">0</span><span class="p">)</span>
313 <br><span class="n">test_add_nums</span><span class="p">([],</span> <span class="mi">0</span><span class="p">)</span>
314 <br></div>
315 </pre></blockquote><p>This will all happen automatically. With our original test functions, when
316 we ran pytest, it showed the results as three passing tests because we had three
317 separate test functions. Now even though we only have one function, it still
318 shows as three passing tests! Each set of values is considered a separate test
319 that can pass or fail independently. This is the main advantage of using
320 parametrize instead of writing three separate assert lines in the body of a
321 simple test function.</p><p>What have we gained?</p><ul>
322
323 <li>We don’t have to write three separate functions with different names.</li>
324
325 <li>We don’t have to repeat the same details in each function (<code>assert</code>,
326 <code>add_nums()</code>, <code>==</code>).</li>
327
328 <li>The differences between the tests (the actual data) are written succinctly
329 all in one place.</li>
330
331 <li>Adding another test case is as simple as adding another line of data to the
332 decorator.</li>
333
334 </ul>
335 ]]></description>
336 </item>
337
338 <item rdf:about="https://nedbatchelder.com/blog/202507/coveragepy_regex_pragmas.html">
339 <title>Coverage.py regex pragmas</title>
340 <link>https://nedbatchelder.com/blog/202507/coveragepy_regex_pragmas.html</link>
341
342 <dc:date>2025-07-28T12:04:12-04:00</dc:date>
343 <dc:creator>Ned Batchelder</dc:creator>
344 <description><![CDATA[<p><a rel="external noopener" href="https://coverage.readthedocs.io/">Coverage.py</a> lets you indicate code to exclude from
345 measurement by adding comments to your Python files. But coverage implements
346 them differently than other similar tools. Rather than having fixed syntax for
347 these comments, they are defined using regexes that you can change or add to.
348 This has been surprisingly powerful.</p><p>The basic behavior: coverage finds lines in your source files that match the
349 regexes. These lines are excluded from measurement, that is, it’s OK if they
350 aren’t executed. If a matched line is part of a multi-line statement the
351 whole multi-line statement is excluded. If a matched line introduces a block of
352 code the entire block is excluded.</p><p>At first, these regexes were just to make it easier to implement the basic
353 “here’s the comment you use” behavior for pragma comments. But it also enabled
354 pragma-less exclusions. You could decide (for example) that you didn’t care to
355 test any <code>__repr__</code> methods. By adding <code>def __repr__</code> as an exclusion
356 regex, all of those methods were automatically excluded from coverage
357 measurement without having to add a comment to each one. Very nice.</p><p>Not only did this let people add custom exclusions in their projects, but
358 it enabled third-party plugins that could configure regexes in other interesting
359 ways:</p><ul>
360
361 <li><a href="https://pypi.org/project/covdefaults/" rel="external noopener">covdefaults</a> adds a
362 bunch of default exclusions, and also platform- and version-specific comment
363 syntaxes.</li>
364
365 <li><a href="https://pypi.org/project/coverage-conditional-plugin/" rel="external noopener">coverage-conditional-plugin</a>
366 gives you a way to create comment syntaxes for entire files, for whether other
367 packages are installed, and so on.</li>
368
369 </ul><p>Then about a year ago, <a rel="external noopener" href="https://github.com/nedbat/coveragepy/pull/1807">Daniel Diniz contributed a
370 change</a> that amped up the power: regexes could match multi-line patterns.
371 This sounds like not that large a change, but it enabled much more powerful
372 exclusions. As a sign, it made it possible to support <a rel="external noopener" href="https://coverage.readthedocs.io/en/latest/changes.html#version-7-6-0-2024-07-11">four
373 different feature requests</a>.</p><p>To make it work, Daniel changed the matching code. Originally, it was a loop
374 over the lines in the source file, checking each line for a match against the
375 regexes. The new code uses the entire source file as the target string, and
376 loops over the matches against that text. Each match is converted into a set of
377 line numbers and added to the results.</p><p>The power comes from being able to use one pattern to match many lines. For
378 example, one of the four feature requests was <a rel="external noopener" href="https://github.com/nedbat/coveragepy/issues/118">how to exclude an
379 entire file</a>. With configurable multi-line regex patterns, you can do this
380 yourself:</p><blockquote class="code"><pre>\A(?s:.*# pragma: exclude file.*)\Z<br></pre></blockquote><p>With this regex, if you put the comment “# pragma: exclude file” in your
381 source file, the entire file will be excluded. The <code>\A</code> and <code>\Z</code>
382 match the start and end of the target text, which remember is the entire file.
383 The <code>(?s:...)</code> means the <a rel="external noopener" href="https://docs.python.org/3/library/re.html#re.S">s/DOTALL</a> flag is in
384 effect, so <code>.</code> can match newlines. This pattern matches the entire source
385 file if the desired pragma is somewhere in the file.</p><p>Another requested feature was <a rel="external noopener" href="https://github.com/nedbat/coveragepy/issues/1803">excluding code between two
386 lines</a>. We can use “# no cover: start” and “# no cover: end” as delimiters
387 with this regex:</p><blockquote class="code"><pre># no cover: start(?s:.*?)# no cover: stop<br></pre></blockquote><p>Here <code>(?s:.*?)</code> means any number of any character at all, but as few as
388 possible. A star in regexes means as many as possible, but star-question-mark
389 means as few as possible. We need the minimal match so that we don’t match from
390 the start of one pair of comments all the way through to the end of a different
391 pair of comments.</p><p>This regex approach is powerful, but is still fairly shallow. For example,
392 either of these two examples would get the wrong lines if you had a string
393 literal with the pragma text in it. There isn’t a regex that skips easily over
394 string literals.</p><p>This kind of difficulty hit home when I added a new default pattern to
395 exclude empty placeholder methods like this:</p><blockquote class="code"><pre class="python"><div class="source"><span class="k">def</span><span class="w"> </span><span class="nf">not_yet</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span> <span class="o">...</span>
396 <br>
397 <br><span class="k">def</span><span class="w"> </span><span class="nf">also_not_this</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
398 <br>    <span class="o">...</span>
399 <br>
400 <br><span class="k">async</span> <span class="k">def</span><span class="w"> </span><span class="nf">definitely_not_this</span><span class="p">(</span>
401 <br>    <span class="bp">self</span><span class="p">,</span>
402 <br>    <span class="n">arg1</span><span class="p">,</span>
403 <br><span class="p">):</span>
404 <br>    <span class="o">...</span>
405 <br></div>
406 </pre></blockquote><p>We can’t just match three dots, because ellipses can be used in other places
407 than empty function bodies. We need to be more delicate. I ended up with:</p><blockquote class="code"><pre>^\s*(((async )?def .*?)?\)(\s*->.*?)?:\s*)?\.\.\.\s*(#|$)<br></pre></blockquote><p>This craziness ensures the ellipsis is part of an (async) def, that the
408 ellipsis appears first in the body (but no docstring allowed, doh!), allows for
409 a comment on the line, and so on. And even with a pattern this complex, it
410 would incorrectly match this contrived line:</p><blockquote class="code"><pre class="python"><div class="source"><span class="k">def</span><span class="w"> </span><span class="nf">f</span><span class="p">():</span> <span class="nb">print</span><span class="p">(</span><span class="s2">"(well): ... #2 false positive!"</span><span class="p">)</span>
411 <br></div>
412 </pre></blockquote><p>So regexes aren’t perfect, but they’re a pretty good balance: flexible and
413 powerful, and will work great on real code even if we can invent weird edge
414 cases where they fail.</p><p>What started as a simple implementation expediency has turned into a powerful
415 configuration option that has done more than I would have thought.</p>
416 ]]></description>
417 </item>
418
419 <item rdf:about="https://nedbatchelder.com/blog/202507/coverage_7100_patch.html">
420 <title>Coverage 7.10.0: patch</title>
421 <link>https://nedbatchelder.com/blog/202507/coverage_7100_patch.html</link>
422
423 <dc:date>2025-07-24T19:03:27-04:00</dc:date>
424 <dc:creator>Ned Batchelder</dc:creator>
425 <description><![CDATA[<p>Years ago I greeted a friend returning from vacation and asked how it had
426 been. She answered, “It was good, I got a lot done!” I understand that feeling.
427 I just had a long vacation myself, and used the time to clean up some old issues
428 and add some new features in <a rel="external noopener" href="https://pypi.org/project/coverage/">coverage.py v7.10</a>.</p><p>The major new feature is a configuration option,
429 <a rel="external noopener" href="https://coverage.readthedocs.io/en/latest/config.html#run-patch"><code>[run] patch</code></a>. With it, you specify named
430 patches that coverage can use to monkey-patch some behavior that gets in the way
431 of coverage measurement.</p><p>The first is <code>subprocess</code>. Coverage works great when you start your
432 program with coverage measurement, but has long had the problem of how to also
433 measure the coverage of sub-processes that your program created. The existing
434 solution had been a complicated two-step process of creating obscure .pth files
435 and setting environment variables. Whole projects appeared on PyPI to handle
436 this for you.</p><p>Now, <code>patch = subprocess</code> will do this for you automatically, and clean
437 itself up when the program ends. It handles sub-processes created by the
438 <a rel="external noopener" href="https://docs.python.org/3/library/subprocess.html#module-subprocess">subprocess</a> module, the
439 <a rel="external noopener" href="https://docs.python.org/3/library/os.html#os.system">os.system()</a> function, and any of the
440 <a rel="external noopener" href="https://docs.python.org/3/library/os.html#os.execl">execv</a> or <a rel="external noopener" href="https://docs.python.org/3/library/os.html#os.spawnl">spawnv</a> families of
441 functions.</p><p>This alone has spurred <a rel="external noopener" href="https://bsky.app/profile/did:plc:yj4vzsbzzkpswr7x5yagzhhx/post/3luqfffiiqk27">one user to exclaim</a>,</p><blockquote><div><p>The latest release of Coverage feels like a Christmas present!
442 The native support for Python subprocesses is so good!</p></div></blockquote><p>Another patch is <code>_exit</code>. This patches
443 <a rel="external noopener" href="https://docs.python.org/3/library/os.html#os._exit">os._exit()</a> so that coverage saves its data before
444 exiting. The os._exit() function is an immediate and abrupt termination of the
445 program, skipping all kinds of registered clean up code. This patch makes it
446 possible to collect coverage data from programs that end this way.</p><p>The third patch is <code>execv</code>. The <a rel="external noopener" href="https://docs.python.org/3/library/os.html#os.execl">execv</a> functions
447 end the current program and replace it with a new program in the same process.
448 The <code>execv</code> patch arranges for coverage to save its data before the
449 current program is ended.</p><p>Now that these patches are available, it seems silly that it’s taken so long.
450 They (mostly) weren’t difficult. I guess it took looking at the old issues,
451 realizing the friction they caused, and thinking up a new way to let users
452 control the patching. Monkey-patching is a bit invasive, so I’ve never wanted
453 to do it implicitly. The patch option gives the user an explicit way to request
454 what they need without having to get into the dirty details themselves.</p><p>Another process-oriented feature was contributed by Arkady Gilinsky: with
455 <code>--save-signal=USR1</code> you can specify a user signal that coverage will
456 attend to. When you send the signal to your running coverage process, it will
457 save the collected data to disk. This gives a way to measure coverage in a
458 long-running process without having to end the process.</p><p>There were some other fixes and features along the way, like better HTML
459 coloring of multi-line statements, and more default exclusions
460 (<code>if TYPE_CHECKING:</code> and <code>...</code>).</p><p>It feels good to finally address some of these pain points. I also closed
461 some stale issues and pull requests. There is more to do, always more to do,
462 but this feels like a real step forward. Give <a rel="external noopener" href="https://coverage.readthedocs.io/en/7.10.0/changes.html#version-7-10-0-2025-07-24">coverage
463 7.10.0</a> a try and let me know how it works for you.</p>
464 ]]></description>
465 </item>
466
467 <item rdf:about="https://nedbatchelder.com/blog/202507/2048_iterators_and_iterables.html">
468 <title>2048: iterators and iterables</title>
469 <link>https://nedbatchelder.com/blog/202507/2048_iterators_and_iterables.html</link>
470
471 <dc:date>2025-07-15T06:52:29-04:00</dc:date>
472 <dc:creator>Ned Batchelder</dc:creator>
473 <description><![CDATA[<p>I wrote a <a rel="external noopener" href="https://github.com/nedbat/odds/blob/master/2048/2048.py">low-tech terminal-based version</a> of the
474 classic <a rel="external noopener" href="https://play2048.co/">2048 game</a> and had some interesting difficulties
475 with iterators along the way.</p><p>2048 has a 4<span class="times">×</span>4 grid with sliding tiles. Because the tiles can slide
476 left or right and up or down, sometimes we want to loop over the rows and
477 columns from 0 to 3, and sometimes from 3 to 0. My first attempt looked like
478 this:</p><blockquote class="code"><pre class="python"><div class="source"><span class="n">N</span> <span class="o">=</span> <span class="mi">4</span>
479 <br><span class="k">if</span> <span class="n">sliding_right</span><span class="p">:</span>
480 <br>    <span class="n">cols</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="n">N</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="o">-</span><span class="mi">1</span><span class="p">)</span>   <span class="c1"># 3 2 1 0</span>
481 <br><span class="k">else</span><span class="p">:</span>
482 <br>    <span class="n">cols</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="n">N</span><span class="p">)</span>             <span class="c1"># 0 1 2 3</span>
483 <br>
484 <br><span class="k">if</span> <span class="n">sliding_down</span><span class="p">:</span>
485 <br>    <span class="n">rows</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="n">N</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="o">-</span><span class="mi">1</span><span class="p">)</span>   <span class="c1"># 3 2 1 0</span>
486 <br><span class="k">else</span><span class="p">:</span>
487 <br>    <span class="n">rows</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="n">N</span><span class="p">)</span>             <span class="c1"># 0 1 2 3</span>
488 <br>
489 <br><span class="k">for</span> <span class="n">row</span> <span class="ow">in</span> <span class="n">rows</span><span class="p">:</span>
490 <br>    <span class="k">for</span> <span class="n">col</span> <span class="ow">in</span> <span class="n">cols</span><span class="p">:</span>
491 <br>        <span class="o">...</span>
492 <br></div>
493 </pre></blockquote><p>This worked, but those counting-down ranges are ugly. Let’s make it
494 nicer:</p><blockquote class="code"><pre class="python"><div class="source"><span class="n">cols</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="n">N</span><span class="p">)</span>                 <span class="c1"># 0 1 2 3</span>
495 <br><span class="k">if</span> <span class="n">sliding_right</span><span class="p">:</span>
496 <br>    <span class="n">cols</span> <span class="o">=</span> <span class="nb">reversed</span><span class="p">(</span><span class="n">cols</span><span class="p">)</span>       <span class="c1"># 3 2 1 0</span>
497 <br>
498 <br><span class="n">rows</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="n">N</span><span class="p">)</span>                 <span class="c1"># 0 1 2 3</span>
499 <br><span class="k">if</span> <span class="n">sliding_down</span><span class="p">:</span>
500 <br>    <span class="n">rows</span> <span class="o">=</span> <span class="nb">reversed</span><span class="p">(</span><span class="n">rows</span><span class="p">)</span>       <span class="c1"># 3 2 1 0</span>
501 <br>
502 <br><span class="k">for</span> <span class="n">row</span> <span class="ow">in</span> <span class="n">rows</span><span class="p">:</span>
503 <br>    <span class="k">for</span> <span class="n">col</span> <span class="ow">in</span> <span class="n">cols</span><span class="p">:</span>
504 <br>        <span class="o">...</span>
505 <br></div>
506 </pre></blockquote><p>Looks cleaner, but it doesn’t work! Can you see why? It took me a bit of
507 debugging to see the light.</p><p><code>range()</code> produces an iterable: something that can be iterated over.
508 Similar but different is that <code>reversed()</code> produces an iterator: something
509 that is already iterating. Some iterables (like ranges) can be used more than
510 once, creating a new iterator each time. But once an iterator like
511 <code>reversed()</code> has been consumed, it is done. Iterating it again will
512 produce no values.</p><p>If “iterable” vs “iterator” is already confusing here’s a quick definition:
513 an iterable is something that can be iterated, that can produce values in a
514 particular order. An iterator tracks the state of an iteration in progress. An
515 analogy: the pages of a book are iterable; a bookmark is an iterator. The
516 English hints at it: an iter-able is able to be iterated at some point, an
517 iterator is actively iterating.</p><p>The outer loop of my double loop was iterating only once over the rows, so
518 the row iteration was fine whether it was going forward or backward. But the
519 columns were being iterated again for each row. If the columns were going
520 forward, they were a range, a reusable iterable, and everything worked fine.</p><p>But if the columns were meant to go backward, they were a one-use-only
521 iterator made by <code>reversed()</code>. The first row would get all the columns,
522 but the other rows would try to iterate using a fully consumed iterator and get
523 nothing.</p><p>The simple fix was to use <code>list()</code> to turn my iterator into a reusable
524 iterable:</p><blockquote class="code"><pre class="python"><div class="source"><span class="n">cols</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">reversed</span><span class="p">(</span><span class="n">cols</span><span class="p">))</span>
525 <br></div>
526 </pre></blockquote><p>The code was slightly less nice, but it worked. An even better fix
527 was to change my doubly nested loop into a single loop:</p><blockquote class="code"><pre class="python"><div class="source"><span class="k">for</span> <span class="n">row</span><span class="p">,</span> <span class="n">col</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">product</span><span class="p">(</span><span class="n">rows</span><span class="p">,</span> <span class="n">cols</span><span class="p">):</span>
528 <br></div>
529 </pre></blockquote><p>That also takes care of the original iterator/iterable problem, so I can get
530 rid of that first fix:</p><blockquote class="code"><pre class="python"><div class="source"><span class="n">cols</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="n">N</span><span class="p">)</span>
531 <br><span class="k">if</span> <span class="n">sliding_right</span><span class="p">:</span>
532 <br>    <span class="n">cols</span> <span class="o">=</span> <span class="nb">reversed</span><span class="p">(</span><span class="n">cols</span><span class="p">)</span>
533 <br>
534 <br><span class="n">rows</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="n">N</span><span class="p">)</span>
535 <br><span class="k">if</span> <span class="n">sliding_down</span><span class="p">:</span>
536 <br>    <span class="n">rows</span> <span class="o">=</span> <span class="nb">reversed</span><span class="p">(</span><span class="n">rows</span><span class="p">)</span>
537 <br>
538 <br><span class="k">for</span> <span class="n">row</span><span class="p">,</span> <span class="n">col</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">product</span><span class="p">(</span><span class="n">rows</span><span class="p">,</span> <span class="n">cols</span><span class="p">):</span>
539 <br>    <span class="o">...</span>
540 <br></div>
541 </pre></blockquote><p>Once I had this working, I wondered why <code>product()</code> solved the
542 iterator/iterable problem. The <a rel="external noopener" href="https://docs.python.org/3/library/itertools.html#itertools.product">docs have a sample Python
543 implementation</a> that shows why: internally, <code>product()</code> is doing just
544 what my <code>list()</code> call did: it makes an explicit iterable from each of the
545 iterables it was passed, then picks values from them to make the pairs. This
546 lets <code>product()</code> accept iterators (like my reversed range) rather than
547 forcing the caller to always pass iterables.</p><p>If your head is spinning from all this iterable / iterator / iteration talk,
548 I don’t blame you. Just now I said, “it makes an explicit iterable from each of
549 the iterables it was passed.” How does that make sense? Well, an iterator is an
550 iterable. So <code>product()</code> can take either a reusable iterable (like a range
551 or a list) or it can take a use-once iterator (like a reversed range). Either
552 way, it populates its own reusable iterables internally.</p><p>Python’s iteration features are powerful but sometimes require careful
553 thinking to get right. Don’t overlook the tools in itertools, and mind your
554 iterators and iterables!</p><p class="bulletsep">• • •</p><p>Some more notes:</p><p>1: Another way to reverse a range: you can slice them!</p><blockquote class="code"><pre class="python"><div class="source"><span class="o">>>></span> <span class="nb">range</span><span class="p">(</span><span class="mi">4</span><span class="p">)</span>
555 <br><span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">4</span><span class="p">)</span>
556 <br><span class="o">>>></span> <span class="nb">range</span><span class="p">(</span><span class="mi">4</span><span class="p">)[::</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span>
557 <br><span class="nb">range</span><span class="p">(</span><span class="mi">3</span><span class="p">,</span> <span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="o">-</span><span class="mi">1</span><span class="p">)</span>
558 <br><span class="o">>>></span> <span class="nb">reversed</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="mi">4</span><span class="p">))</span>
559 <br><span class="o"><</span><span class="n">range_iterator</span> <span class="nb">object</span> <span class="n">at</span> <span class="mh">0x10307cba0</span><span class="o">></span>
560 <br></div>
561 </pre></blockquote><p>It didn’t occur to me to reverse-slice the range, since <code>reversed</code> is
562 right there, but the slice gives you a new reusable range object while reversing
563 the range gives you a use-once iterator.</p><p>2: Why did <code>product()</code> explicitly store the values it would need but
564 <code>reversed</code> did not? Two reasons: first, <code>reversed()</code> depends on the
565 <code>__reversed__</code> dunder method, so it’s up to the original object to decide
566 how to implement it. Ranges know how to produce their values in backward order,
567 so they don’t need to store them all. Second, <code>product()</code> is going to need
568 to use the values from each iterable many times and can’t depend on the
569 iterables being reusable.</p>
570 ]]></description>
571 </item>
572
573 <item rdf:about="https://nedbatchelder.com/blog/202506/math_factoid_of_the_day_63.html">
574 <title>Math factoid of the day: 63</title>
575 <link>https://nedbatchelder.com/blog/202506/math_factoid_of_the_day_63.html</link>
576
577 <dc:date>2025-06-16T00:00:00-04:00</dc:date>
578 <dc:creator>Ned Batchelder</dc:creator>
579 <description><![CDATA[<p>63 is a <a rel="external noopener" href="https://en.wikipedia.org/wiki/Centered_octahedral_number">centered octahedral number</a>. That means if you
580 build an approximation of an octahedron with cubes, one size of octahedron will
581 have 63 cubes.</p><p>In the late 1700’s <a rel="external noopener" href="https://en.wikipedia.org/wiki/Ren%C3%A9_Just_Ha%C3%BCy">René Just Haüy</a> developed a theory
582 about how crystals formed: successive layers of fundamental primitives in
583 orderly arrangements. One of those arrangements was stacking cubes together to
584 make an octahedron.</p><p>Start with one cube:</p><div class="figurep"><img src="https://nedbatchelder.com/code/diagrams/hauy/0.svg" alt="Just one lonely cube"></div><p>Add six more cubes around it, one on each face. Now we have seven:</p><div class="figurep"><img src="https://nedbatchelder.com/code/diagrams/hauy/1.svg" alt="Seven cubes as a crude octahedron"></div><p>Add another layer, adding a cube to touch each visible cube, making 25:</p><div class="figurep"><img src="https://nedbatchelder.com/code/diagrams/hauy/2.svg" alt="25 cubes arranged like an octahedron five cubes wide"></div><p>One more layer and we have a total of 63:</p><div class="figurep"><img src="https://nedbatchelder.com/code/diagrams/hauy/3.svg" alt="63 cubes arranged like an octahedron seven cubes wide"></div><p>The remaining numbers in <a href="https://oeis.org/A001845" rel="external noopener">the sequence</a>
585 less than 10,000 are 129, 231, 377, 575, 833, 1159, 1561, 2047, 2625, 3303,
586 4089, 4991, 6017, 7175, 8473, 9919.</p><p>63 also shows up in the <a rel="external noopener" href="https://en.wikipedia.org/wiki/Delannoy_number">Delannoy numbers</a>: the
587 number of ways to traverse a grid from the lower left corner to upper right
588 using only steps north, east, or northeast. Here are the 63 ways of moving on a
589 3<span class="times">×</span>3 grid:</p><div class="figurep"><img src="https://nedbatchelder.com/code/diagrams/delannoy3.svg" alt="63 different ways to traverse a 3x3 grid"></div><p>(Diagram from <a href="https://en.wikipedia.org/wiki/File:Delannoy3x3.svg" rel="external noopener">Wikipedia</a>)</p><p>In fact, the number of cubes in a Haüy octahedron with N layers is the same
590 as the number of Delannoy steps on a 3<span class="times">×</span>N grid!</p><p>Since the two ideas are both geometric and fairly simple, I would love to
591 find a geometric explanation for the correspondence. The octahedron is
592 three-dimensional, and the Delannoy grids have that tantalizing 3 in them. It
593 seems like there should be a way to convert Haüy coordinates to Delannoy
594 coordinates to show how they relate. But I haven’t found one...</p><p class="bulletsep">• • •</p><p>Colophon: I made the octahedron diagrams by asking Claude to write a
595 <a href="https://nedbatchelder.com/code/diagrams/hauy/hauy_oct.py">Python program</a> to do it.
596 It wasn’t a fast process because it took pushing and prodding to get the
597 diagrams to come out the way I liked. But Claude was very competent, and I
598 could think about the results rather than about projections or color spaces. I
599 could dip into it for 10 minutes at a time over a number of days without having
600 to somehow reconstruct a mental context.</p><p>This kind of casual hobby programming is perfect for AI assistance. I don’t
601 need the code to be perfect or even good, I just want the diagrams to be nice.
602 I don’t have the focus time to learn how to write the program, so I can leave it
603 to an imperfect assistant.</p>
604 ]]></description>
605 </item>
606
607 <item rdf:about="https://nedbatchelder.com/blog/202506/digital_equipment_corporation_no_more.html">
608 <title>Digital Equipment Corporation no more</title>
609 <link>https://nedbatchelder.com/blog/202506/digital_equipment_corporation_no_more.html</link>
610
611 <dc:date>2025-06-09T09:43:53-04:00</dc:date>
612 <dc:creator>Ned Batchelder</dc:creator>
613 <description><![CDATA[<p>Today is the 39-year anniversary of my first day working for
614 <a rel="external noopener" href="https://en.wikipedia.org/wiki/Digital_Equipment_Corporation">Digital Equipment Corporation</a>. It was my first real job in
615 the tech world, two years out of college. <a href="https://nedbatchelder.com/blog/200606/digital_equipment_corporation.html">I wrote
616 about it 19 years ago</a>, but it’s on my mind again.</p><p>More and more, I find that people have never heard of Digital (as we called
617 it) or DEC (as they preferred we didn’t call it but everyone did). It’s
618 something I’ve had to get used to. I try to relate a story from that time, and
619 I find that even experienced engineers with deep knowledge of technologies don’t
620 know of the company.</p><p>I mention this not in a crabby “kids these days” kind of way. It does
621 surprise me, but I’m taking it as a learning opportunity. If there’s a lesson
622 to learn, it is this:</p><blockquote><div><p>This too shall pass.</p></div></blockquote><p>I am now working for Netflix, and one of the great things about it is that
623 everyone has heard of Netflix. I can mention my job to anyone and they are
624 impressed in some way. Techies know it as one of the FAANG companies, and
625 “civilians” know it for the entertainment it produces and delivers.</p><p>When I joined Digital in 1986, at least among tech people, it was similar.
626 Everyone knew about Digital and what they had done: the creation of the
627 minicomputer, the genesis of Unix and C, the ubiquitous VT100. Many foundations
628 of the software world flowed directly and famously from Digital.</p><p>These days Digital isn’t quite yet a footnote to history, but it is more and
629 more unknown even among the most tech-involved. And the tech world carries
630 on!</p><p>My small team at Netflix has a number of young engineers, less than two years
631 out of college, and even an intern still in college. I’m sure they felt
632 incredibly excited to join a company as well-known and influential as Netflix.
633 In 39 years when they tell a story from the early days of their career will they
634 start with, “Have you heard of Netflix?” and have to adjust to the blank stares
635 they get in return?</p><p>This too shall pass.</p>
636 ]]></description>
637 </item>
638
639 <item rdf:about="https://nedbatchelder.com/blog/202505/pycon_summer_camp.html">
640 <title>PyCon summer camp</title>
641 <link>https://nedbatchelder.com/blog/202505/pycon_summer_camp.html</link>
642
643 <dc:date>2025-05-15T07:05:20-04:00</dc:date>
644 <dc:creator>Ned Batchelder</dc:creator>
645 <description><![CDATA[<p>I’m headed to PyCon today, and I’m reminded about how it feels like summer
646 camp, in mostly good ways, but also in a tricky way.</p><p>You take some time off from your “real” life, you go somewhere else, you hang
647 out with old friends and meet some new friends. You do different things than in
648 your real life, some are playful, some take real work. These are all good ways
649 it’s like summer camp.</p><p>Here’s the tricky thing to watch out for: like summer camp, you can make
650 connections to people or projects that are intense and feel like they could last
651 forever. You make friends at summer camp, or even have semi-romantic crushes on
652 people. You promise to stay in touch, you think it’s the “real thing.” When you
653 get home, you write an email or two, maybe a phone call, but it fades away. The
654 excitement of the summer is overtaken by your autumnal real life again.</p><p>PyCon can be the same way, either with people or projects. Not a romance,
655 but the exciting feeling that you want to keep doing the project you started at
656 PyCon, or be a member of some community you hung out with for those days. You
657 want to keep talking about that exciting thing with that person. These are
658 great feelings, but it’s easy to emotionally over-commit to those efforts and
659 then have it fade away once PyCon is over.</p><p>How do you know what projects are just crushes, and which are permanent
660 relationships? Maybe it doesn’t matter, and we should just get excited about
661 things.</p><p>I know I started at least one effort last year that I thought would be done
662 in a few months, but has since stalled. Now I am headed back to PyCon. Will I
663 become attached to yet more things this time? Is that bad? Should I temper my
664 enthusiasm, or is it fine to light a few fires and accept that some will peter
665 out?</p>
666 ]]></description>
667 </item>
668
669 </rdf:RDF>