[HN Gopher] On eval in dynamic languages generally and in Racket...
___________________________________________________________________
On eval in dynamic languages generally and in Racket specifically
(2011)
Author : swatson741
Score : 61 points
Date : 2025-05-28 20:44 UTC (2 days ago)
(HTM) web link (blog.racket-lang.org)
(TXT) w3m dump (blog.racket-lang.org)
| dang wrote:
| Related. Others?
|
| _On eval in dynamic languages generally and in Racket
| specifically (2011)_ -
| https://news.ycombinator.com/item?id=8098569 - July 2014 (18
| comments)
| behnamoh wrote:
| Is my understanding correct that Lisp's powerful macro system
| stems from the ability to write the eval function in Lisp itself?
| From what I gather, Lisp starts with a small set of primitives
| and special forms--seven in the original Lisp, including lambda.
| I recall Paul Graham demonstrating in one of his essays that you
| can build an eval function using just these primitives. Those
| primitives are typically implemented in a host language like C,
| but once you have an eval function in Lisp, you can extend it
| with new rules. The underlying C interpreter only sees the
| primitives, but as a programmer, you can introduce new syntax
| rules via eval. This seems like a way to understand macros, where
| you effectively add new language rules. I know Lisp macros are
| typically defined using specific keywords like defmacro, but is
| the core idea similar--extending the language by building on the
| eval function with new rules?
| samth wrote:
| No, macros and eval are quite different. You can see this for
| example in Python or JavaScript, which have eval but not
| macros.
| Y_Y wrote:
| You can make macros in Python:
| https://github.com/lihaoyi/macropy (note that that project
| was started for a class taught by Sussman)
|
| There's also a PEP to make them first-class:
| https://peps.python.org/pep-0638/
| sparkie wrote:
| That's a different meaning of _first-class_ from Strachey
| 's definition of a first-class citizen[1] - ie, one that
| can be passed as an argument, returned from a function, or
| assigned to a variable.
|
| Syntactic macros are still second-class, like Lisp macros,
| but an improvement over text-replacement style macros.
|
| For something macro- _like_ which is first-class, there are
| fexprs[2] and operatives (from Kernel[3]) - these receive
| their operands verbatim, like macros, so they don 't
| require quotation if we want to suppress evaluation.
| fexprs/Operatives can be passed around like any other value
| at runtime.
|
| [1]:https://en.wikipedia.org/wiki/First-class_citizen
|
| [2]:https://en.wikipedia.org/wiki/Fexpr
|
| [3]:https://web.cs.wpi.edu/~jshutt/kernel.html
| Y_Y wrote:
| Stratchey defined "first-class objects". This was by
| analogy with "first-class citizens" in a legal/political
| sense, since they are treated just as well as any other
| object and have no additional limitations. If we extend
| the analogy to syntax then I think it's clear enough that
| it means that it is a piece of syntax which is treated
| the same as any other and does not require special
| treatment or impose additional restrictions.
|
| Thank you for the clarification and the additional
| information, I think having macros as first-class objects
| is a cool (but separate) idea.
| matheusmoreira wrote:
| They aren't that different. Fexprs are essentially additional
| eval cases.
| sparkie wrote:
| > Is my understanding correct that Lisp's powerful macro system
| stems from the ability to write the eval function in Lisp
| itself?
|
| I wouldn't say this is the case. Nearly any language could
| implement an `eval` for itself, but obviously, it is much
| simpler in Lisps because there's little syntax and few rules.
|
| What makes Lisp macros different from say, C preprocessor
| macros, is the body of the macro is just Lisp code - so the
| "preprocessor" in this case has full access to the host
| languages facilities, which may include `eval`. The macros
| don't take textual input, but they take structured input in the
| form of S-expressions.
|
| Implementing macros is obviously simpler due to eval, because
| we need to run the evaluator on the macro body, but it's not a
| strict requirement, as macro functionality could be provided by
| the implementation and could encapsulate its own evaluator.
|
| Lisp macros are also simple due to the fact that Lisp code is
| just lists of data - you don't have to navigate a complex AST
| to walk through the code and emit particular syntax. You walk
| through the input with `car` and `cdr`, and you emit new syntax
| with `cons` (or `list`/`list*` which are derived from it).
| Macros can take code as their argument, and produce new code
| which is evaluated in place.
|
| Macros still have hygiene issues though, because they're based
| on expanding code before evaluating it, variables used in
| macros can accidentally shadow variables in the scope of the
| macro's caller. There are workarounds (gensym) to navigate
| these hygiene problems.
|
| > From what I gather, Lisp starts with a small set of
| primitives and special forms--seven in the original Lisp,
| including lambda. I recall Paul Graham demonstrating in one of
| his essays that you can build an eval function using just these
| primitives.
|
| This is largely a theoretical demonstration but not real-world
| usage. In practice, Lisps have dozens or hundreds of
| "primitives". Common Lisp in particular is a big language and
| not trivial to implement. Scheme is a bit smaller, though r6rs
| started to also grow quite large, but this approach was
| revisited in r7rs (current), which aims for a small core, with
| additional functionality being provided through SRFIs (Scheme
| requests for implementation).
|
| > Those primitives are typically implemented in a host language
| like C, but once you have an eval function in Lisp, you can
| extend it with new rules.
|
| Using Scheme as an example, some SRFIs can be implemented
| purely in Scheme, as libraries, but others require the
| implementation to provide support, which often requires writing
| C code to provide them.
|
| > This seems like a way to understand macros, where you
| effectively add new language rules. I know Lisp macros are
| typically defined using specific keywords like defmacro
|
| As you note, it's `defmacro`, or `macro`, or `syntax-rules`,
| `syntax-case`, etc, which introduce new syntax - not eval in
| particular. Some macros will use `eval` in their bodies, which
| permits control of evaluation other than the regular
| applicative form of lambdas.
|
| Macros are more than just `eval`. They're a multi-stage
| evaluation model where we first need to do some `macroexpand`
| (which will internally use `eval`), and afterwards the the
| resulting expression from the macro call is evaluated.
|
| > but is the core idea similar--extending the language by
| building on the eval function with new rules?
|
| There are some Lisps which still attempt this kind of
| minimalism.
|
| One example is Ian Piumarta's Maru[1], which support extending
| `eval` (and `apply`) with new functionality at runtime based on
| the type being evaluated. Maru basically has global maps of
| type->evaluator and type->applicator, where we can add new
| pairs to at runtime and augment the behavior of `eval`/`apply`.
|
| Kernel[2] also aims for the minimalist approach and does away
| with macros, quote and special-forms entirely, instead
| replacing them with a more general feature called an
| _operative_. The Kernel evaluator does not need to implement
| special rules for things like `lambda`, `cond`, `car`, `cdr`
| (as in Graham 's "On Lips" essay) - but it just discriminates
| two forms - operative or applicative. Obviously, some kinds of
| operative are "primitive", but there's no difference from the
| PoV of the programmer. Which set of symbols you decide to
| implement as primitive is up to the implementation. The Kernel
| report suggests a small set of primitives and demonstrates the
| remaining standard features can be implemented using only the
| functionality provided so far.
|
| [1]:https://piumarta.com/software/maru/
|
| [2]:https://web.cs.wpi.edu/~jshutt/kernel.html
| taeric wrote:
| A giant difference between eval in most languages versus lisp, is
| that eval in lisp doesn't just take in a string. Sounds somewhat
| subtle, but it can lead to some dramatic differences. And is, in
| at least a small part, why lisp people claim code is data. (I had
| a fun time exploring this a long time ago in
| https://taeric.github.io/CodeAsData.html)
| kevin_thibedeau wrote:
| Lisp was supposed to ingest M-expressions and convert them
| internally. Eval would have been more like other languages had
| that plan not been abandoned. It is not meaningfully different
| for languages that construct an AST from a string before
| evaluation.
| taeric wrote:
| I'm not sure I follow? Presuming that the M-expressions were
| also made of atoms, they would still be meaningfully
| different than taking in the string of text. Closer to the
| same, agreed, but still not the same.
| spauldo wrote:
| IIRC M-expressions broke homoiconicity, so you couldn't
| just spit sexps at eval and expect anything sensible. I
| believe that's a lot of the reason they never got
| implemented.
| RodgerTheGreat wrote:
| Many practical applications of eval() boil down to accessing
| (and perhaps manipulating) scopes in a manner which the host
| language does not normally allow. Likewise, many potential
| dangers or pitfalls of eval() relate to how it allows (or does
| not allow) dynamically-assembled code to inherit access to some
| spatial or temporal scope of variables, definitions, "unsafe"
| libraries, and so forth. The interaction of eval() with scope
| is much a much more interesting design question than whether it
| accepts an AST, a string, or something in-between, except to
| the extent that non-stringy "programs" represent some form of
| reified closure.
| taeric wrote:
| I mean, you aren't wrong. But, little bobby tables is
| essentially what happens when you build up eval statements
| using a string, no?
| RodgerTheGreat wrote:
| if and only if a piece of eval'd code has the ability to
| produce harmful (let alone observable) side effects, which
| in a functional language will in turn largely be a
| consequence of what is imported into the code's scope.
| taeric wrote:
| Strictly, this isn't true, either? If you can influence
| what is returned from the evaluation, that may be enough
| to cause bugs, no? Without needing any access to scope or
| side effects. Consider a case where you use sql injection
| style modification to change an authz query so that it
| returns indicating the current user has admin rights.
| (Notably without making any external change in doing
| this.)
|
| Granted, I fully cede that your point is largely right.
| Just feels like you are a bit too strong with the "if and
| only if" on it.
| beanjuiceII wrote:
| is (1 2 3) code or data?
| zem wrote:
| the whole point of the "code is data" mantra is that it is
| both - the list `(1 2 3)` and the lisp code that evaluates to
| said list when parsed and interpreted.
| beanjuiceII wrote:
| wouldn't the data version be '(1 2 3) ? sorry I am just
| trying to understand
| spauldo wrote:
| You're basically asking if literals are code. I imagine
| you'll get varied opinions about that.
|
| (And yes, that can be used unquoted in a few different
| contexts in Lisp, such as a special form or macro, or if
| you've managed to convince the reader that 1 is a
| function.)
| taeric wrote:
| I don't understand the question? In particular, it feels out
| of place? The common claim is that code is data. This does
| not necessarily imply that data is code.
|
| That said, amusingly, `(1 2 3)` can easily be considered code
| if you are able to evaluate permutation notation. No?
| TOGoS wrote:
| In summary: The more layers of interpretation your `eval`
| function needs to do (assumption of source language,
| tokenization/parsing rules, symbol definitions, etc), the less
| well-defined the semantics are.
|
| So on the less-well-defined end, you have something like
| JavaScript's `eval` function, in the middle you have lisp macros,
| which get an AST instead of just source code, and on the
| actually-pretty-well-defined end, you have a lambda that's
| already been parsed/compiled and just needs to be evaluated.
| programLyrique wrote:
| About how eval is used in other languages:
|
| - in Javascript:
| https://link.springer.com/chapter/10.1007/978-3-642-22655-7_...
|
| - in R: https://dl.acm.org/doi/abs/10.1145/3485502
| neilv wrote:
| I'm glad Matthew Flatt wrote this. There is a problem of newbies
| who read some academic textbook ("metacircular evaluator!") and
| then immediately try to use the eval feature when it doesn't make
| sense.
|
| (Also, there's a more general problem of newbies trying to use
| the most advanced tool they've been exposed to. Why use the
| standard `if` statement, when you can use a proprietary high-
| powered pattern-matching form, now backed by a networked
| Kubernetes cluster of deep learning and LLMs.)
|
| In the Racket community, one of my many attempts to discourage
| mistaken uses of `eval`:
|
| https://groups.google.com/g/racket-users/c/Z-IlF24RAKU/m/3h6...
|
| In the all-in-one practitioner's book I was writing, eval
| wouldn't be introduced until almost the end, in the "Dangerous
| Last Resort" part of the book. (Maybe I should've planned a
| marketing gimmick around it, as "free bonus DLC", and you need to
| do some ritual to be a Certified Certifiable Racketeer, before
| you can read the eval secret scroll.)
___________________________________________________________________
(page generated 2025-05-30 23:01 UTC)