[HN Gopher] Building a Shell
___________________________________________________________________
Building a Shell
Author : ingve
Score : 145 points
Date : 2026-03-17 09:52 UTC (13 hours ago)
(HTM) web link (healeycodes.com)
(TXT) w3m dump (healeycodes.com)
| zokier wrote:
| Bit of pedantry but I don't think traditional unix shell (like
| this) follows repl model; the shell is not usually doing
| _printing_ of the result of _evaluation_. Instead the printing
| happens more as a side effect of the commands.
| jermaustin1 wrote:
| I remember my first shell programming I ever did was batch in
| windows back in the 3.11/95 days.
|
| The first line was always to turn off echo, and I've always
| wondered why that was a decision for batch script. Or I'm
| misremembering. 30 years of separation makes it hard to
| remember the details.
| enoint wrote:
| Echo in that case prints command lines before executing them.
| Its analog is `set -x` rather than `echo`.
| skydhash wrote:
| It's a shell, not the whole thing. The whole thing is the
| shell+kernel+programs.
| zokier wrote:
| Even if you view the system as a whole the printing is deeply
| intertwined with the evaluation, which is very different from
| repl where eval returns a value and print prints it
| themafia wrote:
| It prints a prompt.
| zokier wrote:
| That's not what print in repl means.
| lvales wrote:
| Building a shell is a great exercise, but honestly having to deal
| with string parsing is such a bother that it robs like 2/3 of the
| joy along the way. I once built a very simple one in Go [0] as a
| learning exercise and I stopped once I started getting frustrated
| with all the corner cases.
|
| [0]
| https://github.com/lourencovales/codecrafters/blob/master/sh...
| healeycodes wrote:
| Author here, and yeah, I agree. I skipped writing a parser
| altogether and just split on whitespace and `|` so that I could
| get to the interesting bits.
|
| For side-projects, I have to ask myself if I'm writing a
| parser, or if I'm building something else; e.g. for a toy
| programming language, it's way more fun to start with an AST
| and play around, and come back to the parser if you really fall
| in love with it.
| ferguess_k wrote:
| Can say the same for control characters in terminals. I even
| think maybe it's just easier to ditch them all and use QT to
| build a "terminal" with clickable urls, something similar to
| what TempleOS does.
| chubot wrote:
| A common problem I noticed is that if you took certain courses
| in computer science, you may have a pre-conceived notion of how
| to parse programming languages, and the shell language doesn't
| quite fit that model
|
| I have seen this misconception many times
|
| In Oils, we have some pretty minor elaborations of the standard
| model, and it makes things a lot easier
|
| _How to Parse Shell Like a Programming Language_ -
| https://www.oilshell.org/blog/2019/02/07.html
|
| Everything I wrote there still holds, although that post could
| use some minor updates (and OSH is the most bash-compatible
| shell, and more POSIX-compatible than /bin/sh on Debian - e.g.
| https://pages.oils.pub/spec-compat/2025-11-02/renamed-tmp/sp...
| )
|
| ---
|
| To summarize that, I'd say that doing as much work as possible
| in the lexer, with regular languages and "lexer modes",
| drastically reduces the complexity of writing a shell parser
|
| And it's not just one parser -- shell actually has 5 to 15
| different parsers, depending on how you count
|
| I often show this file to make that point:
| https://oils.pub/release/0.37.0/pub/src-tree.wwz/_gen/_tmp/m...
|
| (linked from https://oils.pub/release/0.37.0/quality.html)
|
| Fine-grained heterogenous algebraic data types also help.
| Shells in C tend to use a homogeneous command* and word* kind
| of representation
|
| https://oils.pub/release/0.37.0/pub/src-tree.wwz/frontend/sy...
| (~700 lines of type definitions)
| cxr wrote:
| [delayed]
| mzs wrote:
| Had an assignment to build a shell in a week, how hard could it
| be? controlling terminal session leader
| job control
|
| The parser was easy in comparison.
| emersion wrote:
| Some time ago I've written an article about a particular aspect
| of shells, job control: https://emersion.fr/blog/2019/job-
| control/
| austy69 wrote:
| Fun read. Wonder if you are able to edit text in the shell, or if
| you need to implement a gap buffer to allow it?
| healeycodes wrote:
| Editing the current line works because I brought in
| https://man7.org/linux/man-pages/man3/readline.3.html towards
| the end so I could support editing, tab completion, and
| history.
|
| IIRC readline uses a `char *` internally since the length of a
| user-edited line is fairly bounded.
| austy69 wrote:
| Very cool. Currently working on the beginning of a small text
| editor so this part seemed interesting and was curious of any
| overlap. Thanks for the interesting post!
| zokier wrote:
| worth noting that you get basic line editing for "free" from
| kernels tty subsystem even if you don't use readline.
| wei03288 wrote:
| The pipe section is the part that changes how you think about
| processes. Once you've manually done the dup2 dance -- close
| write-end in parent, close read-end in child, wire them up -- it
| stops being magic and starts being obvious why `grep | sort |
| uniq` works at all. The thing that surprised me building a
| similar toy was how late in the process job control has to come:
| you can get a working pipe chain surprisingly fast, and then job
| control (SIGTSTP, tcsetpgrp, the whole mess) costs 5x more than
| everything else combined.
| chubot wrote:
| Yup, job control is a huge mess. I think Bill Joy was able to
| modify the shell, the syscall interface, and the terminal
| driver at the same time to implement the hacky mechanism of job
| control. But a few years later that kind of crosscutting change
| would have been harder
|
| One thing we learned from implementing job control in
| https://oils.pub is that the differing pipeline semantics of
| bash and zsh makes a difference
|
| In bash, the last part of the pipeline is forked (unless shopt
| -s lastpipe)
|
| In zsh, it isn't $ bash -c 'echo hi | read x;
| echo $x' # no output $ zsh -c 'echo hi
| | read x; echo $x' hi
|
| And then that affects this case: bash$ sleep
| 5 | read ^Z [1]+ Stopped sleep
| 5 | read zsh$ sleep 5 | read # job control
| doesn't apply to this case in zsh ^Zzsh: job can't be
| suspended
|
| So yeah the semantics of shell are not very well specified
| (which is one reason for OSH and YSH). I recall a bug running
| an Alpine Linux shell script where this difference matters --
| if the last part is NOT forked, then the script doesn't run
|
| I think there was almost a "double bug" -- the script relied on
| the `read` output being "lost", even though that was likely not
| the intended behavior
| chubot wrote:
| FWIW here is another piece of trivia about job control: the
| API means you can't spawn a child process "safely" in POSIX
| -- you have to trust that that the executable you're spawning
| is well-behaved (or use more advanced Linux process
| isolation)
|
| In this case it was the Zed editor spawning the zsh shell:
|
| _How to Lose Control of your Shell_ -
| https://registerspill.thorstenball.com/p/how-to-lose-
| control...
|
| zsh has a bug where it doesn't do the job control cleanup
| properly in some cases -- when fork-exec() optimizations
| happen.
|
| This can mess up the calling process. For example, here you
| could no longer kill Zed by hitting Ctrl-C, even after zsh is
| done.
|
| My comment: https://lobste.rs/s/hru0ib/how_lose_control_your_
| shell#c_dfl...
| sloum wrote:
| Yes!! This!! I wrote a shell awhile back and was pretty happy
| with it... but could _not_ get job control to work quite right.
| It was a big pain.
| hexer303 wrote:
| Unix shells are conceptually simple but hide a surprising amount
| of complexity under the hood that we take for granted. I recently
| had build my own PTY controller. There were so many edge-cases to
| deal with. It took weeks of stress testing and writing many tests
| to get it right.
| dirk94018 wrote:
| Interesting. I wanted to do toast | bash to let the AI drive the
| computer but the bash shell really got in the way. Too much
| complexity. The things that annoy humans, $ expansion, special
| characters, etc don't work for AI either. Ended up writing a
| custom shell for AI (and humans). When a tool gets in the way,
| sometimes it just time to change the tool.
| lioeters wrote:
| Link was previously posted by author:
| https://news.ycombinator.com/item?id=47398749 There are other
| good quality articles on their site, and maybe deserves the
| imaginary points.
| ratzkewatzke wrote:
| There's a very good exercise on Codecrafters
| (https://app.codecrafters.io/courses/shell/overview) to walk you
| through writing your own shell. I found it enlightening, as well
| as a good way to learn a new language.
| rrampage wrote:
| Fun read! I built a minimal Linux shell [0] in c and Zig last
| year which does not depend on libc. It was a great way to learn
| about execve, the new-ish clone3 syscall and how Linux starts a
| process. Parsing strings is the least fun part of the building
| the shell.
|
| [0]
| https://gist.github.com/rrampage/5046b60ca2d040bcffb49ee38e8...
| doe88 wrote:
| Is there a (real) shell whose code is relatively short and self
| contained and would be valuable to read? This was always
| something I wanted to do but never quite spent time to look for a
| good one to explore.
| giancarlostoro wrote:
| Although not the same... Destroy All Software has videos on
| building your own shell using Ruby. I watched it to learn and
| it was a lot of fun to watch him basically building a shell,
| I'm not really a Ruby guy, but it was easy to grasp. It's not
| free, you would need a subscription, but its worth the watch
| otherwise.
|
| https://www.destroyallsoftware.com/screencasts/catalog/shell...
| epr wrote:
| I think there's a good one if you search around for "xv6 sh.c".
| Hard to tell immediately from a google search just now since
| there are many implementations (people do it in school) and
| github's currently blocking requests from my phone.
|
| Also helpful may be running strace on your shell, then
| reviewing the output line by line to make sure you understand
| each. This is a VERY instructive exercise to do in general.
| willx86 wrote:
| https://st.suckless.org/
| vidarh wrote:
| St is a terminal, not a shell.
| tame3902 wrote:
| It depends on what you are looking for. My recommendation for
| learning "how is X done in a shell" is the OpenBSD ksh:
| https://github.com/ibara/oksh
|
| It's what they use for /bin/sh, it has everything that a
| complete shell needs (including a mechanism for providing
| command completions) and has code that is much easier to read
| than bash or zsh.
|
| Something that I also would recommend is the design document
| for the plan9 rc shell; it is a worthwhile read for anybody
| interested in shells:
| https://doc.cat-v.org/plan_9/4th_edition/papers/rc
|
| An implementation is also available if one wants to look at how
| it could be done: https://github.com/rakitzis/rc
| emmelaich wrote:
| I don't know how real you want -- those criteria are probably
| self contradictory :-)
|
| Marc Rochkind's book Avanced UNIX Programming implemented a
| basic shell, through iterations. You can see the first at e.g.
| here https://github.com/gmarler/AUPv2/blob/master/c5/sh0.c
|
| It might be a bit old too. The book is very good but again,
| quite old. There seem to be free copies of it on the net.
|
| BTW, does anyone know if Marc Rochkind is alive? His site
| basepath.com seems to be for sale :-(
| cxr wrote:
| [delayed]
| lasgawe wrote:
| Great article. There are many things every developer should do
| when starting to learn programming or when trying to improve
| their skills. This is one of them. I once built a shell-like
| programming language (not an interpreter). If anyone reading this
| wants to improve their skills, I strongly suggest building your
| own shell from scratch.
___________________________________________________________________
(page generated 2026-03-17 23:01 UTC)