[HN Gopher] Steel - An embeddable and extensible scheme dialect ...
___________________________________________________________________
Steel - An embeddable and extensible scheme dialect built in Rust
Author : MaximilianEmel
Score : 150 points
Date : 2023-12-03 17:24 UTC (5 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| darthrupert wrote:
| This is just perfection on first glance, exactly what I've been
| looking for to augment my Rust projects that need a
| command/configuration language. And with a working repl, too,
| apparently.
|
| I hope the second and third glances will be good too.
| _nalply wrote:
| Helix Editor plans to use Steel as a plugin language [0].
|
| [0]: https://github.com/helix-editor/helix/pull/8675
| agumonkey wrote:
| emacsers looking deep from afar
| nikolay wrote:
| That's ridiculous!
| icommentedtoday wrote:
| ?
| bobbyskelton41 wrote:
| I saw your comment on the thread.
|
| A. Most WebAssembly implementations are bigger than Helix
| itself.
|
| B. WebAssembly is immature.
|
| C. Lisp is perfectly fine. Even if you don't like it, it's
| not the end of the world if you have to use it to configure
| Helix.
| zozbot234 wrote:
| It's perfectly viable as a MVP. WebAssembly plugins for Helix
| had been proposed many times but without an actual
| implementation being worked on.
| sirsuki wrote:
| When I tried to introduce s-expressions to a DSL my co-workers
| nearly lynched me. The parenthesis were so violently hated I sunk
| into a deep hole and still haven't came back out of it.
| capableweb wrote:
| It is really fun (sad) to see this, I've seen it so many times
| myself too. You show how something would be with s-expressions,
| compared to something, and all they can focus on is how many
| parenthesis there are. But when you sit down and count, they're
| the same amount as the code was when it wasn't s-expressions,
| just in different locations. And when you remove the
| parenthesis, they can kind of understand the code, kind of.
| fwip wrote:
| One option that might be suitable for a DSL, is implicit
| parens based on whitespace. A newline opens a new paren, and
| the paren closes when it reaches another line with the same
| indentation e.g: (defun factorial (x)
| (if (zerop x) 1 (* x (factorial (- x
| 1)))))
|
| could be rewritten as defun factorial (x)
| if (zerop x) 1 * x (factorial (- x 1))
| alwaysbeconsing wrote:
| Whitespace significance might be the choice even more
| controversial than Lisp parens. For myself, I appreciate
| both, but those that do not, really do not.
| nsajko wrote:
| Existing SRFI: SRFI 49 "Indentation-sensitive syntax":
|
| https://srfi.schemers.org/srfi-49/srfi-49.html
| fwip wrote:
| Thanks, I knew that there must be prior art out there. :)
| mtlmtlmtlmtl wrote:
| It really is kind of frustrating isn't it? Because anyone who's
| put any real effort into learning Lisp knows that the
| parentheses are not what's hard to understand, actually. You
| don't even have to read them, really.
|
| So when people complain loudly about parentheses in Lisp that
| just tells me they probably never made any real effort to learn
| it, and are actively opposed to trying.
|
| It's not a productive starting point for a discussion.
| alex_lav wrote:
| Counterpoint: why are people under any obligation to try?
| mtlmtlmtlmtl wrote:
| They're not. But they're also not required to flaunt their
| own ignorance as though it's a well-formed opinion. Also,
| in a professional setting, I find it a bit unprofessional
| and rude to poo poo technical ideas over trivialities.
| alex_lav wrote:
| > But they're also not required to flaunt their own
| ignorance as though it's a well-formed opinion.
|
| I don't like the parenthesis. I'm not flaunting anything.
| Is it "flaunting" to calmly and respectfully share an
| opinion? To you it's trivial, to me it's not.
|
| It seems like you're interested in creating conflict with
| people that don't like a thing that you like. Which, I
| would cast this behavior as unprofessional, tbh.
| dun44 wrote:
| Imagine someone who hates the exponentiation operator but
| wants to do mathematics nevertheless. For a living.
| alex_lav wrote:
| Interesting hyperbole.
| Munksgaard wrote:
| I don't like curly braces, but I don't let that decide
| which programming languages I use.
| alex_lav wrote:
| That's your choice!
| CooCooCaCha wrote:
| I'm not a fan of the parenthesis either, but when I
| learned about s-expressions and how lisp programs are
| also a data structures that piqued my interest and helped
| me look past them.
|
| I question people's judgement who can't look past the
| syntax when there is a very good, and interesting
| technical reason behind them.
| arnsholt wrote:
| Code as data is interesting, but mostly orthogonal to S
| expressions. For example, Prolog code has the same
| property without S expressions, and more esoterically TeX
| (which is succinctly explained to a lisper as programming
| with defmacro but not defun).
| alex_lav wrote:
| > I question people's judgement who can't look past the
| syntax when there is a very good, and interesting
| technical reason behind them
|
| The list of things that are interesting is endless,
| though. I see something I'm turned off by, I move on.
| There are plenty of valuable things to spend time on.
| adastra22 wrote:
| A-expressions are a choice, and it is fine to argue
| whether it is a good one. There are plenty of other ways
| of achieving honiconic syntax.
| adastra22 wrote:
| I've written a large application in Lisp. Hated
| parentheses before, hate them even more now. It
| needlessly obscures code.
|
| Maybe it's a personal issue, idk. But it's not out of
| ignorance nor is code readability a trivial point.
| physPop wrote:
| Have a learning mindset?
| brundolf wrote:
| I did it once in an internal UI because we needed to rapidly
| expose some functionality that could be composed/piped in
| complex ways and it would take a while to implement a normal UI
|
| Was met with skepticism, esp around the engineering/maintenance
| overhead, until I told them the parser took me a couple hours
| to write and was only a hundred lines of code or so
| oldpersonintx wrote:
| I want this to succeed!! please do not let the word "srfi" ever
| appear in the packages list...naming libraries with obscure
| numbers no one remembers was a terrible terrible idea that all
| schemes seem to perpetuate
| mattparas wrote:
| At the moment I only have 1 SRFI package, and its just to check
| against compatibility :) - my plan is to wrap the SRFI packages
| with friendly names and just include the metadata in the
| package spec so someone searching for it can find it easily.
| zelphirkalt wrote:
| It is a standard. What do you expect? But there are some or
| many Schemes that offer their own libs on top or below SRFI
| implementations, which then have readable names.
| adastra22 wrote:
| I expect people to willing not conform when nonconformance
| improves the art.
| mattparas wrote:
| This is a surprise! Steel is my project, happy to answer any
| questions anyone might have
| dataangel wrote:
| Any chance of seeing the things I miss the most from Racket in
| other langs?
|
| 1. Parameters and Syntax Parameters (Syntax parameters make
| macros more powerful)
|
| 2. Turing complete macros (not just syntax-case)
|
| 3. Typed Racket
|
| I almost used https://gamelisp.rs/ for a project but the
| nightly feature it needs broke and it's no longer maintained,
| glad to see something similar arise! You might want to consider
| adopting their choice of VecDeque as a list replacement, I
| think it makes a lot more sense than naive linked lists on
| modern machines.
| mattparas wrote:
| To answer each of these:
|
| 1. I do support parameters now, syntax parameters not yet. I
| would like to! But Racket has a pretty hefty head start on me
| so it'll take some time.
|
| 2. Right now I have syntax-rules macros, I also have defmacro
| style macros that get used internally in the kernel during
| expansion, but haven't yet opened them up for user space yet.
| Syntax case will be coming soon hopefully.
|
| 3. The odds of me being able to come up with an
| implementation to match typed racket pound for pound is
| pretty low. I have toyed with using contracts as types (where
| possible), with medium/promising success in certain
| situations. I have a soft spot for racket and have been
| modeling behavior after it, however it will take time to be
| able to create a macro system powerful enough to match it. It
| wouldn't be impossible to create an alternative syntax to
| just lower to steel after type checking, but I haven't put
| time into that.
|
| On the list type - the list in use currently is an unrolled
| linked list https://github.com/mattwparas/im-lists, which
| I've found to yield much better performance for iteration
| than the naive linked lists. When possible, the vm does some
| in place mutation on the lists as well when consing, which
| helps performance as well. I also can hot swap for a vlist,
| but at the moment have stuck with unrolled linked lists.
| mst wrote:
| Is that based on the VList paper?
| mattparas wrote:
| Yes, I don't have a link handy but I based it off the
| Phil Bagwell paper
| mst wrote:
| I do: https://trout.me.uk/lisp/vlist.pdf
| falcor84 wrote:
| Rust is gradually overcoming C and Fortran and Greenspun's tenth
| rule [0] is apparently inviolable.
|
| [0] https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule
| ghc wrote:
| Should have named it Steele since it's dragging Rust users half-
| way to Lisp :).
| bitwize wrote:
| Why would you want to do that? Lisp is dynamically typed and
| has a GC.
| Jtsummers wrote:
| It's a joke that you seem to have missed.
|
| > We were not out to win over the Lisp programmers; we were
| after the C++ programmers. We managed to drag a lot of them
| about halfway to Lisp.
|
| Guy Steele on Java
| wolverine876 wrote:
| Yes! I just posted
|
| https://news.ycombinator.com/item?id=38510054
| CooCooCaCha wrote:
| How does Steel handle garbage collection? Would it be possible to
| manually control the garbage collector? For example, a game that
| runs the GC at the end of every frame.
| mattparas wrote:
| Immutable values are reference counted, so for most code,
| things will be dropped when they exit scope. For captured
| mutable values, there is a fairly mundane mark and sweep
| collector. It is possible to manually control the garbage
| collector, however I have not optimized it for that kind of
| workload. If you were embedding Steel in a game, I don't think
| it would be explicitly necessary to tune the GC as long as you
| aren't using a ton of mutation. If you were using a lot of
| mutation and still wanted a relatively performant GC collection
| at the end of every frame, then the underlying GC
| implementation would have to be changed or swapped to a
| different one (which is not impossible - I just only have one
| GC implemented)
| mst wrote:
| You might find the Recycler algorithm interesting - nim's ORC
| collector is ARC + Recycler and seems to be working out
| rather well.
| mattparas wrote:
| Thanks for the recommendation, with some brief poking
| around it does seem promising! It looks like the algorithm
| is this? https://github.com/fitzgen/bacon-rajan-cc
|
| The link to the paper seems dead unfortunately from this
| blog post https://nim-lang.org/blog/2020/12/08/introducing-
| orc.html
|
| I could see how it works as a drop in replacement for Rc
| zozbot234 wrote:
| The bacon-rajan-cc link says it's stop-the-world. Samsara
| seems to be fully concurrent.
| mst wrote:
| The bacon-rajan-cc link has only -implemented- a stop-
| the-world version, but notes right at the top of the
| README that it -can- be concurrent, and the stop-the-
| world-only-ness is only 'Currently.'
|
| https://trout.me.uk/gc/ has two Recycler papers if you
| want more details.
| mst wrote:
| https://trout.me.uk/gc/ - see recycler-overview.pdf and
| recycler-algorithm.pdf
|
| As with /lisp/ I tend to grab my own copies of papers I
| think I'll want to refer to later in case the original
| URL vanishes in a puff of bitrot :)
| CooCooCaCha wrote:
| Thanks, I think swapping and controlling the GC would be a
| very useful feature.
|
| In the game example I gave performance is important, but
| what's also important is consistency. Interactive apps rely
| on a steady framerate so what you want to avoid is
| accumulating garbage across multiple frames, then doing a
| single large collection pass.
|
| In other words, it's better to do a bit of GC every frame
| than a bunch at once and risk stuttering.
| mattparas wrote:
| I'll make a tracking issue for it - new GC work is fun!
| I'll do some research, I have on my back log to integrate
| Steel into Bevy or some other Rust game engine, would give
| me a reason to make some fun GCs
| CooCooCaCha wrote:
| Nice! Hopefully my statements didn't come off as
| demanding. Just providing some info on that use case.
| mattparas wrote:
| Not at all! I agree it is a useful feature, I'd be
| curious how much it is necessary if not using any
| mutation, but the best way for me to find out is to try
| it out :)
| zozbot234 wrote:
| There are concurrent GC implementations for Rust, e.g.
| Samsara https://redvice.org/2023/samsara-garbage-collector/
| https://github.com/chc4/samsara that avoid blocking, except
| to a minimal extent in rare cases of contention. That fits
| pretty well with the pattern of "doing a bit of GC every
| frame".
|
| (Note that doing GC over large object graphs will
| nonetheless involve significant overhead, even with
| efficient implementations as seen here; GC is not at all a
| silver bullet, and should be avoided if at all possible.
| The actual point of GC is to enable computing over highly
| general, possibly cyclical object graphs - if that doesn't
| apply, other memory management strategies can be used
| instead.)
| wolverine876 wrote:
| I wonder where the name came from. Being HN, here is my
| nitpicking imperative:
|
| Names are important. Our inheritance is wit, self-awareness, and
| irony; names that puncture ego and power and that appeal to joy:
| C, C++, GNU, Rust, Google, Yahoo!, Vim, Git, awk, etc. Others are
| beautiful, evocative images, like Apple and Amazon. Names
| communicate our culture and ideals to each other and to the next
| generation.
|
| Careless, thoughtless names like Microsoft, IBM, etc. (ok it's
| ironic and self-deprecating, but without self-awareness or wit!),
| etc. should be hated and banned. Egotistical BS, especially
| Tolkien plagiarists who assert they are supernatural, should be
| tarred and feathered and paraded around town (with wit and
| irony).
|
| (Plenty of names fall in some middle ground.)
|
| If Steel is just a derivative of 'Rust' [edit: it is not, see the
| response below], it misses the self-awareness. Someone naming
| their development product - designed to build great structures -
| 'Rust' is engaging in a little self-deprication, joy, and self-
| awareness. Naming the derivative project 'Steel' possibly misses
| all that; there's a reason the original wasn't named Iron or
| Steel or Carbon Fiber. But maybe there's more to the name.
| pasabagi wrote:
| I don't think it's particularly careless: SBCL is short for
| 'Steel Bank Common Lisp', which is a reference to how the CMU
| university, named after Carnegie Mellon, could be respectively
| substituted with 'Steel' (Carnegie) and 'Bank' (Mellon).
| mattparas wrote:
| So here is the history:
|
| 1. Guy Steele (along with Gerald Sussman) created scheme, and
| Steel is close to Steele, just drop the e.
|
| 2. Steel is a scheme, and I observed that scheme names have a
| tendency to be named things crime related: Scheme, Racket
| (racketeering), Larceny, etc - Not a scientific analysis at
| all, but I found it funny at the time that Steel sounds like
| "steal".
|
| 3. You made the observation, Steel sounds like something that
| would be associated with Rust.
|
| Beyond that, I just liked the name. No SEO involved, and
| arguably probably should have picked something more searchable,
| but I didn't start making it with the intention of there being
| a lot of users, it started as a project for school.
| wolverine876 wrote:
| Awesome! Thank you. Someone else pointed out the Steele/Steel
| relationship and I was hopeful:
|
| https://news.ycombinator.com/item?id=38510060
| mst wrote:
| Triple pun score!
|
| (I always try and cram as many simultaneous jokes as possible
| into project names - and have a soft spot for lisp - so I
| wish you much joy and whatever level of success is most fun
| for you)
| mattparas wrote:
| Thank you - I appreciate the kind words :)
| Y_Y wrote:
| > beautiful, evocative images, like Apple
|
| Is this a joke?
| I_am_tiberius wrote:
| It sounds great but what is it exactly and what can I use it for?
| Is there an equivalent of this in other languages?
| mattparas wrote:
| Hopefully the linked README provides a general overview (I know
| I need to write some more documentation!), but Steel is an
| implementation of the scheme programming language (not entirely
| compliant yet, but aiming for R5RS and R7RS compliance). It can
| be used as a standalone language via the interpreter/repl (like
| Python or Racket), or it can be embedded inside applications,
| like Lua. There are hundreds (thousands, probably) of
| embeddable languages, each with their own flavor - see a list
| compiled here for example https://github.com/dbohdan/embedded-
| scripting-languages
|
| Use cases are generally for either configuration, scripting, or
| plugins - so scripting in games, or adding extensions to your
| text editor without having to use FFI or RPC + serializing a
| bunch of data. The advantage it has over using dynamic
| libraries (in general) is it runs in the same process, and can
| access the internal data structures directly without a lot of
| ceremony involved. The downside is that it is typically not as
| fast as native code unless a JIT is involved.
|
| Javascript is an example of an embedded scripting, where the
| browser is the host application.
| tucnak wrote:
| Is it possible to avoid rust completely when programming using
| Steel?
| mattparas wrote:
| Yep - you can write standalone steel code without interacting
| with Rust at all, just interacting with the interpreter. I've
| done the first few days of the advent of code in Steel without
| needing to touch any Rust. Now, I will say that Steel has
| gotten visibility faster than I've been able to keep up with,
| so you might find a native function missing, or something that
| you would like to use that isn't implemented yet - at which
| point you either need to implement it yourself or open up an
| issue for someone to get to :)
| schemescape wrote:
| Solving Advent of Code problems in a language you implemented
| yourself should earn you an extra good star or something.
| Nice work, and cool project!
| nerdponx wrote:
| Which Scheme is this implementing? R7RS? Any SRFIs? Is there a
| library reference document?
| mattparas wrote:
| Starting with R5RS compliance, then once that is achieved
| moving on to R7RS.
|
| I borrowed the R5RS test suite from chibi here -
| https://github.com/mattwparas/steel/blob/master/cogs/r5rs.sc...
| - only a few of these tests don't yet pass. Something like 135
| pass, 4 fail, 20 skipped since I haven't implemented the
| primitives yet.
|
| I've only tested against a few SRFIs so far, but am also
| attempting to run the R7RS benchmark suite
| https://ecraven.github.io/r7rs-benchmarks/ - So far you can
| view the progress here
| https://github.com/mattwparas/steel/tree/master/r7rs-benchma...
|
| There are some more that aren't yet checked in. I plan to get a
| document up with the exact state of compliance soon.
|
| The biggest difference right now is that, like Racket, Steel
| lists are immutable, so there I need a compatibility layer when
| running portable scheme.
| nerdponx wrote:
| Thanks! Was this originally a "just because" project, or do
| you expect it to offer some improvements or advantages over
| existing embeddable Schemes like Chibi (as you mentioned)?
| mattparas wrote:
| Originally started it as a school project, which then
| during the pandemic morphed into something to work on while
| cooped up. It is really a passion project, working on it is
| fun! There are some interesting design things I wanted to
| explore, like how to get good performance out of safe Rust,
| using unrolled linked lists or vlists instead of naive
| linked lists, using contracts, etc.
|
| Chibi is an impressive scheme implementation, and it will
| take a long time before Steel can hit the same level of
| compliance as Chibi. There is not a Chibi equivalent in
| native Rust that I am aware of. There are other embedded
| scripting languages for Rust that are pleasant - but no
| schemes of the maturity of Chibi. So in that regard, I'm
| hoping to offer a compelling scheme in native Rust to make
| integration with Rust applications relatively easy and
| painless.
|
| I also don't have a particularly strong need to be 100%
| completely compliant with the scheme specs. The plan is to
| have compatibility layers so that portable scheme code can
| be used, however there are things about scheme that I think
| Racket (for example) improved on, and I'd like to explore
| that as well as much as I can.
| perrygeo wrote:
| Fantastic project! If you're looking to embed a DSL/full
| programming language into your application, this seems right on
| point. Rhai and RustPython are two other options if you don't dig
| parentheses.
| SushiHippie wrote:
| Can someone explain these scheme and lisp languages to me? Every
| time I look at these languages, I can't grasp what you can use
| them for. And why one would use them.
|
| I always feel like I'm missing something. In the example scripts
| and code snippets, I can see that you can define functions, that
| you can use lists, mathematical operations, you can build some
| algorithms, you can print text, but it never goes further than
| that.
|
| I've only used languages like Python, Rust, C, Java, JavaScript,
| and they all have a very similar vibe, you have a std lib, which
| can interact with many things, you can build UIs, networking
| libraries and all that. And I could probably start using any
| language that is "similar" to these.
|
| But I could never use one of these scheme/lisp languages, as I
| can't really grasp them.
|
| Sorry, this comment is all over the place, because I can't really
| explain what's going on in my head when I see languages like
| this. I'd call myself a proficient programmer, but every time I
| look at these languages, it feels like as I've never seen code
| once in my lifetime.
|
| Any help or hint at what I'm missing, is appreciated.
| zozbot234 wrote:
| > you have a std lib, which can interact with many things, you
| can build UIs, networking libraries and all that
|
| The point of Scheme is to simplify the underlying definition of
| the programming language itself. Standard library facilities
| are an entirely orthogonal concern, and one that is not
| addressed by Scheme as a project. Though there is Racket, a
| Scheme-like language which does focus on having 'batteries
| included' out of the box.
| femiagbabiaka wrote:
| Basically the differences are in the concepts you'll use to
| write code. Lisps themselves are very different from each
| other, but just like the languages you're used to, many lisp
| distributions have standard libraries that can be called, and
| those building blocks can be used to build applications or
| whatever else. In this case specifically, Steel provides the
| facility to call Rust functions within a Steel program:
| https://github.com/mattwparas/steel.
|
| So, although I haven't used Steel, it looks like the advantage
| you'd get from using it is the opportunity to take advantage of
| features it provides like transducers and contracts, which are
| feature common to some other Lisps as well.
|
| So, just like choosing any other language, it boils down to a
| series of tradeoffs.
___________________________________________________________________
(page generated 2023-12-03 23:00 UTC)