[HN Gopher] Show HN: SectorLISP now fits in one sector
___________________________________________________________________
Show HN: SectorLISP now fits in one sector
Author : jart
Score : 199 points
Date : 2021-10-30 10:20 UTC (12 hours ago)
(HTM) web link (justine.lol)
(TXT) w3m dump (justine.lol)
| tomcam wrote:
| jart makes me realize that my high school counselor Stella
| Sawicki was absolutely right when she said I wasn't living up to
| my potential
| hyproxia wrote:
| How old are you? Is it too late for you to live up to your
| potential?
| metagame wrote:
| In your defense, Justine's been at this for a long time! Give
| it time and effort, and you too can do amazing technical
| things. Potential isn't a concrete thing, it's a set point you
| can move around.
|
| Throw yourself at the computer, and the computer will do
| interesting things with you, too.
|
| If it's not working, go at it from a different angle. Sometimes
| it's helpful to go back in time to come across things. I'd
| argue that 90% of cool tech phenomenon arises from studying the
| past to make something cooler in the future rather than any
| force of sheer will.
|
| A lot of Justine's projects seem to fit this mold (if I
| remember right she came across the idea for Cosmopolitan after
| finding that a legacy UNIX feature let you avoid specifying
| your shell; SectorLISP, McCarthy's metacircular, etcetera).
|
| She even cites someone whose entire deal is doing precisely
| that: Nils M. Holm.
| [deleted]
| eggy wrote:
| Yes, Justine's amazing. I have also been following and
| reading Nils M. Holm's books. I program in J/APL, and I
| picked up Nils' book on Klong, but I have not fully worked
| through it yet. It gives me a perspective on J/APL. His Lisp
| stuff is great.
| tluyben2 wrote:
| Nils his works are excellent; I always buy all that come
| out.
| eggy wrote:
| He's very prolific and the content is rich. I always feel
| I need to step up my game. People like Justine and Nils
| inspire me to keep at it!
| nils-m-holm wrote:
| Thank you all for your kind words! Now I almost want to
| write another book.
| tluyben2 wrote:
| Please do!
| nils-m-holm wrote:
| Thinking about it, but writing a book is a large effort,
| I am not getting any younger and, to be honest, it is a
| bad idea from an economic point of view. My bestsellers,
| if you can call them that, sell about 30 copies per year.
| jart wrote:
| I also wish to thank everyone here for the kind words.
| (Justine here btw.) Also Nils, awesome book! Glad I had
| the opportunity to cite it in the blog post.
| matheusmoreira wrote:
| > Sometimes it's helpful to go back in time to come across
| things.
|
| Yeah, I remember her posting about the original Unix source
| code. I've never read it but I probably should. I bet there's
| plenty to be learned.
|
| Sometimes I come across papers from the 70s and 80s too and
| they blow me away. It's amazing how much our predecessors
| achieved. Sometimes it feels like we're trying to rediscover
| or reinvent old technology.
| jart wrote:
| If that were true then modern authors would just be guys
| trying to reinvent Shakespeare. As programmers, the
| programs we write can be art as much as it can serve a
| business utility. Some things progress but some things
| never change too, and each generation finds a way to
| express those things in a new and beautiful way. While
| learning to become a better programmer, I've found it
| helpful to read and be inspired by things like the UNIX
| source code, or texts written by guys like Vannevar Bush or
| John McCarthy, just as fiction authors might find
| inspiration reading the classics.
| mikewarot wrote:
| I didn't remember an FS register... it was added on the 386, I
| wonder what would happen if this was run on an old IBM XT?
|
| I realize that something had to go in order to fit it in the size
| limit, compatibility with 8088 CPUs isn't the worst choice.
|
| [Edit] Sorry if this sounded too negative... I'm astounded that
| it's even possible to do this in a sector at all.
| adrian_b wrote:
| Intel 8086 and 8088 did not have invalid instruction
| exceptions.
|
| The opcodes that were not documented were either redundant,
| doing exactly the same thing as other documented opcodes, or
| they crashed the system.
|
| Later, in 1982, 80186 and 80286 introduced invalid opcode
| exceptions, so starting with the IBM PC/AT any program intended
| for later CPUs should abort with an error message.
|
| Most IBM PC/XT were made with Intel 8088, so the behavior of an
| 80386 program is unpredictable.
|
| Some PC/XT clones were made with 80186 or 80188, or, more
| frequently, with NEC V20 processors. All these had invalid
| opcode exceptions, so they should behave like a PC/AT.
| agumonkey wrote:
| These sorts of minimalist bare metal abstract PL articles are so
| thrilling to me. APL comes to mind.
|
| I wonder how high you can go with only 1MB say.
| bob29 wrote:
| https://www.red-lang.org/p/about.html
|
| Almost as batteries-included as python.
| diskzero wrote:
| I love Red and want to use it more. Sadly my professional
| life keeps giving me hardware that I can't run it on, so Red
| is getting marginalized. I wish I had more skill and time to
| help them out with the 32 to 64-bit transition. [1]
|
| [1] https://gitter.im/red/red?at=5e09152cd5a7f357e6a1af83
| agumonkey wrote:
| Ha, I didn't know the whole distribution fit in 1MB. Thanks
| adrian_b wrote:
| 1 MB is huge.
|
| Keep in mind that IBM PC/XT had only 640 kB, but there were
| compilers and interpreters for any language, which were
| available for it.
|
| Moreover, before IBM PC, a CP/M computer with Zilog Z80 or
| Intel 8080 had usually only 32 kB or 48 kB, but you could use
| without problems Basic interpreters, Pascal, Fortran, Cobol and
| PL/M compilers and many others.
|
| However, in order to fit in 32 kB, the compilers themselves
| were typically written using a macro-assembler, and not in a
| high-level language. The C language became popular for such
| tasks somewhat later.
| User23 wrote:
| The original Macintosh had 128k and managed to load a
| graphical OS and an app in that space. The ROM certainly
| helped, but it was quite a feat.
| akkartik wrote:
| Assuming you mean static code size, my Mu project currently
| supports an interpreted Lisp (macros, fewer parens, infix)
| implemented in a compiled memory-safe low-level language
| without _any_ C in 450KB. The "runtime library" (mostly in the
| safe low-level language) requires 120KB. Without unit tests,
| the binary sizes drop down to 200KB and 30KB respectively,
| which gives you some sense of the space devoted to tests in
| each case.
|
| As the tests hint, Mu doesn't do anything to try to reduce code
| size. It's small just by focusing on the essentials. What's
| left is written for ease of comprehension.
|
| (An example of "essentials": Mu still cannot free memory. I run
| my programs in Qemu with 2GB which seems to suffice.)
|
| More details: https://github.com/akkartik/mu
| jart wrote:
| Author here. I assume you're referring to
| https://github.com/akkartik/mu/blob/main/shell/evaluate.mu?
| It's hard for me to tell, since there's 819k lines of code in
| Mu according to find . -type f | grep -v '\\.git' | xargs wc
| -l. Does it have lambda and support metacircular evaluation?
| If not, then why do you think it's LISP-like? If you do
| nothing to reduce code size, then why do you call it Mu? (I
| assume by "Mu" you're referring to what UNICODE calls the
| MICRO SIGN.) 30kb is two orders of a magnitude larger. I
| don't see how it's relevant to SectorLISP.
| blorpleblap wrote:
| Are you really going after someone who shared a similar
| project to yours in what you seem to believe is your
| thread? What, do you think you're headed for a raise on a
| $100m valuation for this and any discussion of alternative
| small lisps weakens your hand for YC? Come on.
|
| It's pretty ironic you're here seeking validation for your
| hobby project and then tearing down someone who shared
| their _own_ hobby project with someone else and didn't even
| pit it as related to yours in any way. The person they were
| replying to was seeking context on the sizes involved and
| you leaned in on dissecting that person's entire hobby
| architecture and even its bloody name out of what, offense?
| It's not a competition for attention and multiple ideas can
| exist independently.
|
| I realize you left some plausible deniability in how you
| worded it so you can step back and say "oh, gosh, I never,"
| but the wording right now is fairly transparent in its
| dagger-like nature and clear contempt that this person
| would dare to sully your work with something _else_.
| Completely unnecessary, toxic, and a real shame because I
| respect your work. Even the condescension in your
| downthread reply is glaringly obvious and huge props to the
| other user for not rising to it.
| akkartik wrote:
| Sorry to tramp over your thread! Mu isn't too related to
| SectorLISP, I was just responding to the side-discussion
| about what we can build in under 1MB.
|
| To answer your question, evaluate.mu does support lambda (I
| call it `fn`). My estimates of size were based on `ls -l
| a.bin` after `./translate shell/*.mu`.
|
| I actually didn't really think of the micro interpretation
| of 'mu' until years after I started the project. The
| interpretation I had in mind was https://en.wikipedia.org/w
| iki/Mu_(negative)#%22Unasking%22_t...
| jart wrote:
| It that case it'd be really cool if you implemented John
| McCarthy's metacircular evaluator in Mu LISP syntax to
| demonstrate it's a LISP. Sort of like how when John
| McCarthy shared the idea for LISP one of the first things
| he had to do was show that it's able to do the same
| things as a Turing machine.
| akkartik wrote:
| No upvote needed :) but sure I'll post it here.
|
| (I haven't done this so far because I find
| metacircularity to not be very interesting. It was
| interesting when JMC proved it could be done. Mu's whole
| reason for existence is linear rather than circular
| bootstrapping. We can disagree over whether I get to call
| it a Lisp or not, but if it has the full power of macros
| I'm happy.)
| agumonkey wrote:
| He probably was just responding to my 1MB limit. Don't be
| too harsh.
| agumonkey wrote:
| the variable/register idea is neat, I always wanted some
| veneer on top of raw registries.. almost like a cpu monad
| tluyben2 wrote:
| k9 [0], an APL like, is under 160KB and includes a database in
| that.
|
| [0] https://shakti.com
| agumonkey wrote:
| I shall gather all these projects on a wiki.
| p_l wrote:
| _L_ - an embedded subset of Common Lisp used in Roomba, ran in
| 1MB of ram total afaik including compiler and support OS.
| agumonkey wrote:
| oh wow, roombas still run this subset of CL ??
|
| ps: ohh that paper was on the frontpage not long ago, I just
| didn't realize it was used in Roombas
| terafo wrote:
| 1MB is really a lot if you know what you're doing, what
| language you actually can't fit into 1MB of ram due to
| inherent specifics of language itself, not because
| interpreter/compiler doesn't optimize for such cases?
| pjmlp wrote:
| A lot, we only had 640KB to play with on most MS-DOS, Amiga and
| Atari computers, and 8bits had 64-128 KB.
| agumonkey wrote:
| hmm but were there high level languages being usable on these
| ? I remember a CLISP impl on some atari (but I think the dev
| had a ram addon).
|
| all in all, I'd really love to have or make a 1MB
| minimalistic shell with a tiny lisp/prolog/smalltalk
| guenthert wrote:
| INTERLISP was available for the Atari 800 (which max'd out
| at 48KiB RAM).
| varjag wrote:
| Most then existing HLLs were very usable in 640Kb. If you
| go down to 8-bits, while translators for many of these did
| exist the utility was rapidly diminishing. You'd
| essentially have system's native version of BASIC +
| assembler for anything practical. The rest were more of
| parlor tricks.
| abecedarius wrote:
| In that context Forth was quite practical. That was its
| heyday.
| pjmlp wrote:
| Plenty of them, here is a non-exaustive list.
|
| C, C++, Modula-2, Pascal, AMOS, Clipper, FoxPro, Turbo
| Basic, Quick Basic, Turbo Prolog, PC-Lisp, Native
| Oberon,...
| jacquesm wrote:
| I wrote a full CAD/CAM suite for lathe/mill work in a few
| hundred K, working RAM was another 500K or so...
| agumonkey wrote:
| Honestly you and people writing similarly small applications
| should gather and write a book.
|
| I don't know if I'm an alien but whenever I see frugal yet
| non trivial application my brain rejoyces.
| jacquesm wrote:
| This was back when 1 MB was considered and _amazing_ amount
| of memory. It took two years to write this, about 50K lines
| of high level code and another 10K or so of assembly. Today
| you would approach such a project in a completely different
| way, you 'd use much more time to make it look pretty and
| just that part would probably be much larger than the whole
| package that I wrote.
|
| The funny thing is that even so many years later the
| original software is still in use in some places and there
| is a whole company centered around that core that has been
| re-written a couple of times to keep it up to date and to
| expand its functionality.
|
| I highly doubt the present day version would fit in
| something that small. What's interesting to me is that that
| old stuff tended to be super productive to work with, zero
| distractions, just some clearly defined task in a clearly
| defined environment, if it worked it was bullet proof. No
| hackers, SaaS, a million connectivity options and no eye
| candy. Just that one job to be done and done as good as the
| hardware would allow you to.
| agumonkey wrote:
| But that's exactly my point. I think we forgot to aim at
| super productive, zero distraction, lightweight. And it
| would be of high educative value to see this sort of work
| again. Both on the small scale details and the pragmatic
| value.
|
| The sad part to me is that most web apps reenact the same
| functions but in a css-transition-capable DOM. But
| functionally I'm not sure you get more.
| jacquesm wrote:
| I've tried to adhere to the same principles when building
| pianojacq.com, even so the linecount is huge for such a
| limited amount of functionality.
| gavinray wrote:
| Every time I read something by Justine Tunney, I am continually
| reminded of my mediocrity.
|
| I once reached out to them personally to tell them I hope I can
| be like them when I grow up ;^)
|
| On a serious note, this is one of the coolest things I've ever
| seen.
|
| The person mentioned that helped them go from 700 -> 500 bytes,
| the said "Assembly Heisenberg" appears to be entirely obscure on
| GH too.
|
| Seems like the right set of eyes just happened to find their way
| to the project at the right time:
|
| https://github.com/agreppin
|
| I wish I was born like this.
| pmarreck wrote:
| I was already impressed by this person's work when I read about
| APE (Actually Portable Executable)... Something that should
| really be a thing IMHO
| veltas wrote:
| Not really on topic but I found myself looking around this
| website and this is easily the most interesting personal site
| I've seen on HN, and then from reading all the angry posts by the
| right people Justine also seems like the most interesting
| personality I've seen on here.
| gavinray wrote:
| Agreed - As far site content that gets posted. I also like
| rachelbythebay.
|
| But for commenters, there are lots of interesting people.
| mark_l_watson wrote:
| I also agree with you. My specialties are machine learning and
| knowledge graphs, very different than what Justine works on,
| yet I find digging into her posts fascinating.
| jart wrote:
| Justine here. I used to work on TensorBoard and I contributed
| a few hundred thousand lines of changes to TensorFlow. These
| are my hobby projects so we might not be as dissimilar as you
| think :-)
| mastax wrote:
| Was still spooky downloading and executing the same blinkenlights
| binary on Windows and Linux (WSL). Unfortunately it doesn't
| actually work in either environment: Error:
| your CPUID command does not support command 0x80000006 (AMD-style
| L2 cache information).
|
| AMD Ryzen 7 1800X
| jart wrote:
| Author here. What are you executing? It works fine on AMD Ryzen
| for me. That error message doesn't appear in the
| blinkenlights.com binary. It also doesn't perform that CPUID
| check. The only place I can find that error message online is
| for a fizzbuzz codegolf. Perhaps you're confusing sectorlisp
| with a another project? Also, Blinkenlights should work fine in
| the Windows 10 Command Prompt. So WSL isn't required but it
| does run on that too. https://justine.lol/sectorlisp/spooky.png
| mastax wrote:
| Oh, silly me. I hadn't noticed you need the `-t` flag to open
| the TUI and assumed that since the TUI didn't open that the
| error was from blinkenlights.
| pmarreck wrote:
| Does this pass any tests to prove it is a general purpose lisp?
| (I suppose, writing a lisp evaluator in it might be such a test,
| but not 100% sure)
| akkartik wrote:
| Why is this quoting the top-level lambda expression arguments?
| Unless I'm mistaken that's not how JMC did it. For example, to
| define a binding the cited resource
| http://www.paulgraham.com/rootsoflisp.html uses
| (defun null (x) (eq x '()))
|
| Replacing the _defun_ with an env binding, I believe the value
| translates to: (lambda (x) (eq x '()))
|
| I don't think this should be quoted? Even though lambda isn't
| typically assumed in the "seven primitives," it _is_ required by
| the runtime. Are your function calls implicitly dropping quotes
| when evaluating arguments?
| jart wrote:
| Author here. The quotes are indeed needed. Emacs does the same
| thing. For example: ((lambda (x)
| (message "%S" x)) (1)) ;; fails: no such
| function `1' ((lambda (x) (message "%S"
| x)) (quote (1))) ;; prints (1)
|
| The original evaluator is defined so that if the first item of
| a list, is a list whose first item is LAMBDA, then what it does
| is it calls Evlis() and Pairlis() which is sort of like a
| glorified version of Python's zip() function, to attach all the
| subsequent list items to the parameter name list.
| ((LAMBDA (ARG1 ARG2 ARG3) <YOUR PROGRAM GOES HERE>)
| (QUOTE VAL1) (QUOTE VAL2) (QUOTE VAL3))
|
| This is the "venus fly trap" that the blog post mentions. The
| process of evaluation causes the expression the clamp in on
| itself where the args get digested into an association list
| which can be referenced and then the payload is evaluated. This
| was all non-obvious to me, even after using LISP casually for
| years, since we normally don't write code that way.
|
| Now it's possible to add extra code to the evaluator so that if
| a naked lambda is encountered, i.e. it sees (LAMBDA ...) rather
| than ((LAMBDA ...) ...), it just lets it pass through, so you
| don't need the quotes. We did that originally, but deemed it
| non-essential and was removed, since that would have made it
| much harder to hit the 512-byte goal.
| akkartik wrote:
| By Emacs do you mean Elisp? Your session there is not
| surprising to me. I don't have emacs handy but what does this
| do? ((lambda (f) (f))
| (lambda() 42))
|
| You might need to throw a 'funcall in there or something, but
| my hypothesis is that this will work (return 42) without
| needing the top-level argument to be quoted. 'lambda is a
| deep special case, basically.
|
| I understand that functions always zip through their
| arguments, evaluating each one. But when you quote the args
| in the call, I tend to expect the arguments to _not_ be
| evaluated within 'evlis.
|
| I'm surprised that quoting args makes the interpreter
| smaller! That's pretty interesting. But I think this
| interpreter will have some behavior that is surprising to
| anyone used to all extant lisps for examples like this:
| ((lambda (sym) (cons sym 3)) (quote a))
|
| Here there's no binding for 'a. I expect this to return (a .
| 3) in JMC's original Lisp.
| anderskaseorg wrote:
| The lambdas are quoted because they're lambdas, not because
| they're arguments.
|
| In a more fully-featured Lisp, the code (LAMBDA () 42)
| would evaluate to a special function data type with no
| direct representation in terms of cons cells. That's a
| luxury this implementation can't afford, so its data
| representation of a function is the same as its code
| representation: a cons cell whose car is the atom LAMBDA.
| With this design, the code (LAMBDA () 42) would be self-
| evaluating: it would evaluate to the same thing as the code
| (QUOTE (LAMBDA () 42)). But since the latter syntax works,
| there's no need to waste bytes supporting the former
| syntax.
|
| In the absence of that special support, the default meaning
| of the code (LAMBDA () 42) is to call a function named
| LAMBDA, just like the code (FOO () 42) calls a function
| named FOO.
| akkartik wrote:
| I see. Kinda. The 'lambda in the first line isn't quoted.
| And the eval itself seems identical to JMC. So this is
| just the substrate level and only when evaluating args?
| Does that seem right?
| jart wrote:
| The first item in the list is evaluated, just like the
| other items in the list. For example:
| ((LAMBDA (ARG1 ARG2 ARG3) <YOUR PROGRAM GOES
| HERE>) (QUOTE VAL1) (QUOTE VAL2)
| (QUOTE VAL3))
|
| Is the same syntax as a normal function call:
| (PROGRAM (QUOTE VAL1) (QUOTE VAL2)
| (QUOTE VAL3))
|
| The only difference is that, since it's the root-level
| program, the binding of the name PROGRAM can't have
| happened yet. So it's simply inlined into the expression.
| The Apply() part of the evaluator has a special case for
| recognizing this, which causes evaluation of the first
| argument to terminate: int Apply(int
| fn, int x, int a) { int t1, si, ax;
| if (ISATOM(fn)) { switch (fn) {
| case ATOM_CAR: return Car(Car(x));
| case ATOM_CDR: return Cdr(Car(x));
| case ATOM_ATOM: return ISATOM(Car(x)) ?
| ATOM_T : NIL; case ATOM_CONS:
| return Cons(Car(x), Car(Cdr(x))); case
| ATOM_EQ: return Car(x) == Car(Cdr(x)) ?
| ATOM_T : NIL; default: //
| recurse to turn (FUNC ARG1 ARG2) into ((LAMBDA ...) ARG1
| ARG2) return Apply(Eval(fn, a), x, a);
| } } // evaluate ((LAMBDA ...) ARG1
| ARG2) if (Car(fn) == ATOM_LAMBDA) {
| t1 = Cdr(fn); si = Pairlis(Car(t1), x, a);
| ax = Car(Cdr(t1)); return Eval(ax, si);
| } return UNDEFINED; }
|
| The Eval() function which calls Apply() has already
| evaluated all the list items except for the first one,
| which is left to Apply() to figure out. The reason why
| things are this way is because, in order to save space,
| the REPL loop doesn't maintain a persistent set of global
| variables. Due to the NIL here, each expression is its
| own hermetic job: void Repl(void) {
| for (;;) { Print(Eval(Read(), NIL));
| } }
|
| If you changed that NIL to be an alist that's populated
| by things like defun / setq / etc. in the global scope,
| then the usability of the language improves dramatically.
| We didn't do it due to the singular focus on size. But at
| the same time that simply means we left fun opportunities
| for improvement to anyone wishing to hack on the codebase
| and make it their own.
| akkartik wrote:
| Ok that makes sense. Thanks!
___________________________________________________________________
(page generated 2021-10-30 23:00 UTC)