[HN Gopher] Common Lisp Resources
___________________________________________________________________
Common Lisp Resources
Author : oumua_don17
Score : 177 points
Date : 2022-04-22 11:31 UTC (11 hours ago)
(HTM) web link (lisp-journey.gitlab.io)
(TXT) w3m dump (lisp-journey.gitlab.io)
| vindarel wrote:
| Hello, that's my website o/ I'm a normal programmer enjoying CL
| and shipping simple but useful apps in production(c). Not solo
| since a friend contributed to a web app once (without ever
| touching any Lisp before). Feel free to ask any questions :)
| math-dev wrote:
| Fun fact I learnt the other day browsing some historic posts:
|
| https://en.m.wikipedia.org/wiki/Dynamic_Analysis_and_Replann...
|
| Written in Lisp at the time and widely successful to the point it
| paid back many times over the investment DARPA put into the
| language
|
| Now the question for anyone who may know, are descendant systems
| still being written in Lisp?
| zozbot234 wrote:
| Automated planning is computationally intensive, so you'd want
| to reimplement any such system using a low-level language, with
| easier support for parallel and/or distributed compute.
| travisjungroth wrote:
| They are also cognitively intensive, so it may be worth it to
| code them in slower but more expressive language.
| nekomorphism wrote:
| What does "low level" even mean here? Lisp is second only to
| Fortran in age, so it certainly runs bare metal if that's
| what you want.
| pjmlp wrote:
| Like StarLisp.
| Jach wrote:
| I don't know about that one, but it wouldn't surprise me either
| way (still in Lisp or reimplemented/replaced by something else)
| since such things seem to be hit-and-miss and unless you do
| some digging it's probably unlikely many programmers of such
| systems check HN... But for a comparatively old government
| project in Common Lisp, there's at least one, SPIKE
| (https://www.stsci.edu/scientific-community/software/spike),
| that was developed for planning and scheduling the Hubble
| telescope (and other projects) and is still in use for the JWST
| (https://www.stsci.edu/contents/newsletters/2018-volume-35-is..
| .).
| math-dev wrote:
| Awesome, thanks for the links!
| MichaelBurge wrote:
| I've heard CL is famous for having advanced libraries and
| language features decades before other languages get them.
|
| What are some interesting libraries or language features
| developed since 2005 that haven't yet made their way to other
| languages? Not a simple "Do X" library that a C programmer could
| easily copy, but something more like CLOS, contracts, or pattern
| matching that are cross-cutting.
| hajile wrote:
| First-class functions, lambdas, and closures are mostly unheard
| of in the top 20 programming languages. Aside from JS, you're
| pretty much out of luck.
|
| Proper tail calls. Technically not part of the CL spec, but
| implemented by the best CL variants (and by scheme), they allow
| a lot of expressive power. You won't find this ANYWHERE on the
| top 20 except for JS where it is technically part of the spec,
| but Google and Mozilla refused to implement leaving just
| Safari/JSC to have 100% ES6 spec compliance 7 years after the
| spec was finished.
|
| Macros get overstated perhaps, but there are almost zero non-
| lisp languages that implement them and none of them implement
| macros ergonomically. In a language like JS, you'll often see a
| kind of DSL where they attempt to use closures and super-
| dynamic objects to create templates.
|
| I see this a lot in React where you'll have huge config objects
| passed in and then very extensive branchy and loopy code to
| turn all the bits on an off. This is a poor man's macro. In a
| lisp, you would do all this stuff at compile time with all that
| inefficient branchy stuff executed once at compile time rather
| than millions of times on each of the thousands/millions of
| machines your code is running on.
|
| The REPL has been mentioned several times, but it really is
| that good and nothing from other languages compares. People
| stress about "writing unit tests" because technically correct
| types doesn't mean your program actually works. In my time
| writing JS vs TS, I've seldom found that type errors were the
| problem aside from making the type checker happy. Most errors
| are in what the code actually does or does not do. The instant
| feedback from the entire system is far superior to unit tests
| because it can catch things that your unit tests might not be
| testing for.
|
| Dynamic, but definable types is another advantage. You start
| out without a bunch of specific types which is better for
| getting things done, but can then add in type hints either as
| documentation or as a way for the compiler to spit out fast
| code. People may be down on dynamic code, but the truth is that
| the world is dynamic and runtime type checks are far safer than
| static types where they are then thrown out and you hope for
| the best. Because of the REPL-driven development, type errors
| are almost non-existent in practice.
| wtetzner wrote:
| > First-class functions, lambdas, and closures are mostly
| unheard of in the top 20 programming languages. Aside from
| JS, you're pretty much out of luck.
|
| This was true at one point, but most popular languages have
| them now, don't they? Even C++ and Java have them now (and
| have for a while).
| hajile wrote:
| Java has a terrible hack where it is technically creating a
| singleton class and attaching a function to that class that
| is then applied behind the scenes. The Functional interface
| is a horrible, unergonomic hack.
|
| Likewise, C++ basically has function pointers (though I'd
| hardly put C/C++ in the same category). True closures
| aren't really possible without garbage collection or
| manually specifying the closure, but at that point, the
| closure is just a glorified struct and loses all its
| ergonomic advantages.
| wtetzner wrote:
| > Java has a terrible hack where it is technically
| creating a singleton class and attaching a function to
| that class that is then applied behind the scenes.
|
| I'm not sure why that's a terrible hack. Scheme is just
| creating a struct behind the scenes anyway.
|
| I actually find Java's approach to be quite ergonomic,
| because you can use lambdas anywhere you need a type that
| has exactly one virtual method. Of course, checked
| exceptions destroy the ergonomics anyway.
|
| > True closures aren't really possible without garbage
| collection or manually specifying the closure
|
| Rust's closures are not as ergonomic as e.g. Scheme's,
| but they're not bad.
| hajile wrote:
| Lambdas require a SAM functional interface to be defined
| for the thing they are going to be used for which is
| hardly easy to use or ergonomic compared to something
| like StandardML, Lisp, or even Javascript.
|
| Also on this topic, but all things "closed" over must be
| `final`. Any attempt to change the variables closed over
| by a class will immediately fail.
|
| The fact that Scheme is doing all these things behind the
| scenes is the point. The programmer isn't responsible for
| creating new structs/classes, adding all the things, and
| sending them around to the correct functions. In
| languages that require all this extra work, closures are
| seldom used because the extra code and cognitive overhead
| is almost always bigger than the advantages gained.
| wtetzner wrote:
| I'm not following. Are you somehow equating needing
| interfaces with having to manually construct classes?
|
| In any case, I don't see how needing closed over values
| to be effectively final is a problem. It's no different
| than StandardML or OCaml.
|
| I've seen lambdas used in Java all over the place, so
| they clearly have some value.
| dmux wrote:
| >First-class functions, lambdas, and closures are mostly
| unheard of in the top 20 programming languages. Aside from
| JS, you're pretty much out of luck.
|
| PHP, Perl and Swift all have first-class functions, lambdas
| and closures although I'd agree that that's not "most." This
| is probably getting too much into semantics, but aren't
| functions in JS just objects themselves? ;)
|
| >People stress about "writing unit tests" because technically
| correct types doesn't mean your program actually works. In my
| time writing JS vs TS, I've seldom found that type errors
| were the problem aside from making the type checker happy.
| Most errors are in what the code actually does or does not
| do. The instant feedback from the entire system is far
| superior to unit tests because it can catch things that your
| unit tests might not be testing for.
|
| I don't think having a "live" system like CL precludes the
| need for unit tests. Sure, it's beneficial to be able to play
| around with code right away, but unless you're planning on
| manually running through the same tests again and again, how
| do you know you've not introduced a regression at some point?
| hajile wrote:
| > PHP, Perl and Swift all have first-class functions,
| lambdas and closures although I'd agree that that's not
| "most." This is probably getting too much into semantics,
| but aren't functions in JS just objects themselves? ;)
|
| Perl does have closures, but does not have lambdas as I
| recall (only subroutines) plus, but I didn't think it was
| particularly popular today. Dart and Swift do have all
| three of these things, but they also aren't particularly
| popular outside their niche mobile domain (or outside OSX
| in the case of Swift). Python has first-class functions and
| closures, but the lambdas are nerfed into near uselessness.
|
| PHP has a Closure class, but you must manually populate
| this class with whatever things you want to save. This is
| hardly what would commonly be accepted as a closure.
|
| > I don't think having a "live" system like CL precludes
| the need for unit tests. Sure, it's beneficial to be able
| to play around with code right away, but unless you're
| planning on manually running through the same tests again
| and again, how do you know you've not introduced a
| regression at some point?
|
| You are quite correct. Unit tests are still required, but
| the coverage you get when running stuff manually is still
| usually better for the same reason that a person clicking
| buttons on your UI will find tons of issues your unit tests
| overlooked. By running your new code inside of the whole
| system, you are also integration testing on the fly. I'd
| also add that it is possible to use some of your setup
| state for unit tests later.
| mikelevins wrote:
| No, you're right; just because you can test something at
| once doesn't mean you shouldn't automate repeated tests.
|
| The thing is, though, not all tests are worth automating.
|
| I generally start work by sketching out a strawman
| representation of what I think I'm doing, then
| interrogating it interactively to find out how reality
| differs from my naive understanding. As my understanding
| improves, I refine the representation incrementally and
| interactively, and refine the interrogations right along
| with it.
|
| You can think of those interrogations as embryonic APIs and
| interactive unit tests, but most are not worth formalizing.
| They're ephemeral. The details they're interrogating are
| likely to change beyond recognition in a short time until
| discovery reveals the real shape of things.
|
| Once the real shape starts to emerge from the lump of clay,
| the representation turns into actual data structures and
| the interrogations turn into actual APIs and formal tests.
|
| I probably spend more time testing than in any other single
| activity while working in Lisp. I test pretty much every
| single expression interactively as soon as it's written,
| because it's so quick and effortless to do so.
|
| But most of those tests are not worth formalizing. Many of
| them will be executed one time and then never again. Others
| will be useful long enough to earn a temporary home in a
| source file for a while.
|
| The formalized unit and integration tests are the ones that
| have survived iteration, that are connected to structures
| and APIs that have survived and developed into something
| real.
|
| And that's the process, pretty much: rough sketch;
| interactive experimentation to discover what it really
| ought to be; formalization of discovered features
| (including tests that are worth automating).
| _dain_ wrote:
| > First-class functions, lambdas, and closures are mostly
| unheard of in the top 20 programming languages. Aside from
| JS, you're pretty much out of luck.
|
| uh, what? plenty of popular languages have all of those
| things.
| hajile wrote:
| Very few popular languages have all three and most that
| have them have them as bolted-on, unergonomic hacks that
| are generally more trouble to use than they are worth.
| deckard1 wrote:
| > First-class functions, lambdas, and closures are mostly
| unheard of in the top 20 programming languages
|
| A quick Google search tells me the "top 20" includes Python,
| JavaScript, Java, C#, C, Go, Swift, PHP possibly
| Ruby/Perl/Kotlin/etc. I can't think of a single one that
| doesn't have either first-class functions or some form of
| closures.
|
| > Proper tail calls.
|
| This is an optimization issue for most languages, and they
| have made the trade-off of keeping a stack trace vs.
| optimizing it away. Having it in the Scheme spec was
| incredibly contentious and caused many heated debates.
|
| > none of them implement macros ergonomically.
|
| I think this really depends on what you mean "ergonomically."
| That's a weasel word in this context. Because there are two
| camps. There is the CL Lisp-2 that feels they should be
| allowed to use LIST as a variable in a macro so they went and
| created a whole namespace just for functions but are
| perfectly fine with the ugly GENSYM hack. Then there is the
| Scheme camp that failed to come up with a suitable _hygienic_
| replacement for DEFMACRO for like 20 years. Macros have been
| broken in LISP world longer than most of us have been alive.
|
| > React where you'll have huge config objects passed
|
| Not sure what you're referring to here. A/B testing is a
| thing. Can't do that with macros. Hydration is also a thing,
| because of SSR. Maybe you're seeing that?
|
| > REPL-driven development, type errors are almost non-
| existent in practice
|
| ehhhh. No. And I say this as someone that generally prefers
| dynamic over static typing. You can easily have objects in
| your REPL that were the result of previous bugs and get your
| app in an inconsistent state. You either reload, or spend
| your entire life cleaning up memory and hoping you get it
| right. There is no silver bullet here, despite what others
| have commented on. Because you're talking about schema
| migrations here. Bad data structures have to be transformed
| to good data structures. Or you'll see bugs in the REPL that
| aren't actually bugs in the source. Keeping the REPL and
| source in sync is also tedious, to put it mildly.
|
| Look into Erlang does and what is required to get live code
| swapping correct. It's a headache that is only worth it in
| very niche apps (like massive telecom systems). It doesn't
| make sense doing this during development. Much saner to just
| reload and start from a fresh, consistent point. Otherwise
| you _will_ spend time hunting down bugs that do not exist
| outside your REPL.
| hajile wrote:
| > A quick Google search tells me the "top 20" includes
| Python, JavaScript, Java, C#, C, Go, Swift, PHP possibly
| Ruby/Perl/Kotlin/etc. I can't think of a single one that
| doesn't have either first-class functions or some form of
| closures.
|
| Most of them don't have all of these features together and
| they often have incredibly bad versions of those features
| that are basically unusable.
|
| > This is an optimization issue for most languages, and
| they have made the trade-off of keeping a stack trace vs.
| optimizing it away. Having it in the Scheme spec was
| incredibly contentious and caused many heated debates.
|
| The JS stack already goes away the second you do anything
| asynchronous. As this is something you do frequently, stack
| traces are already incredibly short.
|
| Meanwhile, the alternative for the use cases of PTC is
| loops and I don't hear complaints that loops don't save
| your state.
|
| It's in the spec, it should be implemented.
|
| As to Scheme, you won't find anyone today arguing that PTC
| is bad. The bigger argument would be about how full
| continuations are a performance disaster.
|
| > I think this really depends on what you mean
| "ergonomically."
|
| Infix macro systems are terrible to use compared to S-expr
| macros and I don't think this is anywhere close to
| controversial. Can you point to a counter-example?
|
| The best infix example I know of is Dylan, but not only is
| that language barely infix (its S-expr foundations are very
| apparent) and they still just aren't as easy to use.
|
| > ehhhh. No. And I say this as someone that generally
| prefers dynamic over static typing. You can easily have
| objects in your REPL that were the result of previous bugs
| and get your app in an inconsistent state. You either
| reload, or spend your entire life cleaning up memory and
| hoping you get it right. There is no silver bullet here,
| despite what others have commented on. Because you're
| talking about schema migrations here. Bad data structures
| have to be transformed to good data structures. Or you'll
| see bugs in the REPL that aren't actually bugs in the
| source. Keeping the REPL and source in sync is also
| tedious, to put it mildly.
|
| How does this entire bit have anything to do with dynamic
| vs static typing? You could have a REPL with static typing
| if you truly wanted.
|
| In any case, reloading data from the code isn't that
| difficult and most data will be held in closures which
| means you can decide exactly how far up you want to replace
| local data (and if you're going about it smartly, you have
| created a few "mocks" to work with that can be used to
| refresh data).
|
| The absolute worst case (restarting the REPL) is what other
| languages would force you to do anyway which still means
| you have a lot to gain for most development and bugfixing
| and absolutely nothing to lose.
|
| As to data is held in a database, then you have all the
| same problems in other languages except that fixing these
| issues from the REPL is generally easier because you are
| already connected and ready to start fixing things.
|
| > Look into Erlang does and what is required to get live
| code swapping correct.
|
| BEAM is an amazing platform, but a REPL for development and
| live code swapping are two different discussions. Doing
| REPL work on production systems has a very big chance of
| catastrophic damage and I wouldn't recommend it.
| pmoriarty wrote:
| Lisp syntax and simplicity is my biggest selling point for Lisp
| vs other languages (except for Scheme, which is even more
| elegant).
|
| Some people hate Lisp's heavy reliance on parenthesis and there
| have been so many languages that try to make a "Lisp without
| parenthesis", but to me they're missing half of what makes Lisp
| great.
|
| I find programming in and reasoning about Lisp way, way easier
| than other languages. There's very little syntax to stand in
| the way or remember and the verbose way Lisp is written helps
| make Lisp programs self-documenting and way easier to
| understand than other languages.
|
| This is super useful when doing a lot of language hopping or
| when coming back to the language/program after being away from
| it for a long time. Because of this I can quickly come back up
| to speed with the program and it also makes more larger and
| more complex programs more comprehensible -- that is, of
| course, if you don't use macros to turn Lisp in to some harder
| to understand language.
| sleepycatgirl wrote:
| Oh yea, definitely, With rainbow parentheses, and paredit,
| the experience is top tier
| dwringer wrote:
| Rainbow parentheses are such a great invention I can't
| understand why it isn't standard nearly everywhere.
| rmckayfleming wrote:
| I've found rainbow block-level highlighting to be even
| better: https://github.com/alphapapa/prism.el
| dwringer wrote:
| I kind of wish editors focused on having different
| highlighting modes the way some strategy games have
| different map modes. I sometimes find myself changing to
| random language modes in emacs just to see different
| highlighting schemes that focus my attention on the code
| in different ways. I also do like rainbow block
| highlighting but it's not something I want _all_ the
| time.
| bcrosby95 wrote:
| I never really bought the syntax simplicity argument, but
| what I eventually realized is it allows Lispy languages to
| implement any other paradigm or feature and it just naturally
| fits within the language.
| Jach wrote:
| Well, there are lots of "do x" libs that I don't think a C
| programmer could _easily_ copy, but hey, we 're all Turing
| complete here... As for language features, historically Lisp
| was not always first for some particular feature, but because
| of earlier design decisions that still haven't been made by
| newer languages, it can usually implement innovative features
| from other languages or PL research papers as libraries without
| having to update the language spec or an implementation itself.
| So we have immutable collections, laziness, takes on pattern
| matching, various concurrency models, more sophisticated type
| systems...
|
| One very recent library that might be of interest is "just" a
| new regex library (https://applied-langua.ge/posts/omrn-
| compiler.html) but it leverages one unique design choice Lisp
| made in that you have access to the language's compiler at
| runtime, it's not a fenced-off separate program only used to
| create a runtime executable. The compiler can (and in SBCL's
| case does) optimize and produce assembly code, SIMD
| instructions can be used too, so this regex lib is very fast
| while not being that much code.
| zozbot234 wrote:
| > but it leverages one unique design choice Lisp made in that
| you have access to the language's compiler at runtime, it's
| not a fenced-off separate program only used to create a
| runtime executable.
|
| This is best understood as having no phase separation between
| compile time and run time. Effectively, languages that are so
| designed do _everything_ at compile time, up to and including
| running the actual program.
|
| Though sometimes they can practically recover the phase
| separation by "extracting" a program that can be compiled in
| a single step and ran as a conventional binary. For example,
| a LISP program for doing math may support "extracting" a
| custom function evaluator or numerical-symbolic solver
| written in FORTRAN.
| pjmlp wrote:
| I don't want to start a language war, given that I like both
| languages anyway, you should have a look at
| https://github.com/hanickadot/compile-time-regular-
| expressio...
|
| However even counting with upcoming C++23 features, Lisp is
| much more ergonomic for this kind of stuff.
| dannyobrien wrote:
| The one that sticks out for me is condition handling,
| particularly restarts. I'm someone who reads books about
| programming more than I actually program, so I don't know how
| actually practical Lisp's ability to do extremely sophisticated
| responses to error conditions and the like (including choosing
| where to throw to programmatically at the error site) is. It
| strikes me that it might be one of those things -- like hot-
| reloading -- where there are so many factors that it's mostly
| impossible for humans to get right. But!! I've never seen it in
| any other language, and I think about it a lot.
|
| @phoe-krk's book on it,
| https://news.ycombinator.com/item?id=24867548 is very good.
| sleepycatgirl wrote:
| Might be not exactly what you ask about, but, REPL-driven
| development. (Though, to be more accurate, it is more of image
| based development, rather than REPL, which is quite big
| difference) Function does not work? Change it , re-evaluate,
| and call the function again. No need for constant restarting,
| re-compiling or so. All on-the-fly, Once you get used to it,
| it's really something else. Its simply unique, also without
| sacrificing performance (Because it is quite performant)
| atgreen wrote:
| Yes, this exactly. If you are doing k8s-based development,
| where your Lisp code is participating in k8s-hosted service
| communication, nothing beats the CL development experience,
| where you just connect Emacs to your image via SLY/SLIME and
| make live changes in the kerbernetes cluster, but your source
| files are local.
| jstx1 wrote:
| Doesn't everyone in data science do this too with
| Python/R/Julia? I don't think that it's a unique Common Lisp
| feature.
| jhgb wrote:
| Not quite. For example, languages like Python typically
| have generative class definitions -- meaning that a class
| definition always creates a new type, with no existing
| object instances being objects of that class. In
| comparison, Common Lisp has provisions for adapting old
| instances to match a new definition of an already existing
| class, including lazy updates. Taken to extreme, this
| allows for such things as large on-disk object databases
| (since presumably you don't want to convert a billion
| objects on the disk the moment you add a field to a class).
| knome wrote:
| The erlang language provides a similar facility for
| replacing code and updating object state without ever
| bringing the program to a halt.
| phoe-krk wrote:
| The main difference is that Erlang is very strict about
| the kind of programs it will run (purely functional
| green-thread-esque processes and OO systems that utilize
| message passing and code that is modified must be swapped
| in/out at strictly defined points in program lifecycle),
| whereas CL is a fully mutable environment where
| modification can happen at any time during program
| lifecycle. Much more powerful, which comes with benefits
| as much as it comes with the amount of guns to blow your
| feet off with.
| register wrote:
| In Python and Julia in most of the cases you have to
| restart ; don't know about R. In Common Lisp you can change
| a function that is running a loop while the loop is
| executing and you will see the result immediately in the
| next iteration after the change. As mentioned the closest
| thing with modern languages is JS in Chrome.
| jstx1 wrote:
| When is it useful to have a loop that executes partially
| with some code and the rest of the way with different
| code?
| phoe-krk wrote:
| First example that comes to my mind: you have a game loop
| running and you would like to change the way the scene is
| rendered without restarting the game loop altogether
| and/or losing the current state of the game.
| mikelevins wrote:
| Yep, this is definitely useful. I've done this kind of
| work. It's much faster to build an interactive world if
| you can tweak everything about it while it runs and see
| the results instantly.
|
| The same goes for some simulations I've worked on (and am
| working on). It's much faster and easier to get good
| results if you can see the effects of your changes in
| real time, as the simulation runs.
| [deleted]
| whizzter wrote:
| Variations (to various degrees) are available in plenty of
| environments.
|
| Most powerful one today by combination is actually JS inside
| Chrome, you naturally have the REPL that can do things.
|
| But REPL's are good for the first prototyping whilst later on
| you might have modules (CL might make things inspectable?)
| that try to hide things (think JS/TS modules), Chrome however
| is pretty damn good at runtime detecting more or less
| incremental updates so you can just rewrite code and re-save
| the source files and Chrome will replace the function bodies
| in-situ without any state rebuilding or reloading being
| necessary.
|
| Of more conventional languages, MSVC has had C/C++ Edit-
| Continue (as long as you didn't change type/function
| signatures) since ca 1998, it's a descendant of this that MS
| was hyping last year with Hot-Reloading for C# with body
| replacement (on top of the interactive shell("REPL") that has
| been in Visual Studio for years).
|
| Especially the conventional ones might not be as flexible as
| the Lisp ones(but handle many practical cases fairly well)
| but all dynamic "scripting" languages like JS/Python/Lua,etc
| has been able to do most repl-y things fairly well from day
| 1.
| mikelevins wrote:
| What Lisp and Smalltalk have that still sets them apart is
| that the language runtime is designed to provide
| comprehensive support for livecoding.
|
| Take, for example, Common Lisp's UPDATE-INSTANCE-FOR-
| REDEFINED-CLASS (http://clhs.lisp.se/Body/f_upda_1.htm). If
| you redefine a class in a running program, Lisp notices and
| the next time control touches an existing instance of the
| class, UPDATE-INSTANCE-FOR-REDEFINED-CLASS is automatically
| called to reinitialize the instance to reflect the new
| definition.
|
| That's not all there is to it, of course. For example, how
| does Lisp know how to reinitialize the instances for the
| new definition? The Common Lisp standard also defines a
| complete suite of features to answer that question.
|
| Smalltalk provides equivalent features.
|
| Lots of languages have repls. Some of them support hot
| reloading. These are not the same kind of thing as
| automatically noticing that a definition has changed, and
| that implies that any time you touch a previously-created
| instance of that definition the runtime needs to
| automatically catch it and provide facilities (interactive
| when needed) to dynamically fix up the data to conform to
| the new definition.
|
| Why would any language define such features? To support a
| specific style of programming, one in which the program is
| already running, it just doesn't have all the features you
| want it to have--so you tell it how to add them, while it
| runs, and test each new feature immediately, as soon as
| you've told the program how to do it.
|
| Very few languages support the comprehensive whole-system
| approach to livecoding that Common Lisp and Smalltalk do.
| These are not new features, though; they've been around for
| decades.
| LandR wrote:
| I've tried some of those, and nothing comes to close to
| Clojure / LISP repl.
|
| Just hitting a shortcut and getting the code executed using
| everything loaded into the REPL and getting inline results
| is great.
|
| It helps that Clojure only has expressions and no
| statements as well, so every form can be run and it will
| return something.
|
| Here's what I mean, this is me doing 2021-Day2 AoC puzzle
| part 1. WHen this pops up => foo
|
| Next to a line, it's because I've hit CTRL+Enter on that
| line to evaluate the form.
|
| https://www.youtube.com/watch?v=5HHLT2_a1tI
|
| (You'll need to bump to 1080 resolution otherwise its
| probably unreadable)
|
| My day job is C# and JS, I can't get anywhere close to this
| interactivity with my code. As the complexity of the code
| grows this becomes even more powerful.
| activitypea wrote:
| The Chrome console is only useful if your code doesn't get
| bundled, which is rare
| [deleted]
| hajile wrote:
| Those REPLs aren't even remotely comparable. You can TEST
| with them, but you never DEVELOP with them.
|
| If they were the same, you'd load up a repl of your entire
| codebase. Now you'd dynamically update the part you're
| working on, test it live against your current environment,
| and finally save it to move to another thing. When errors
| happen, you just drop into the debugger to find the issue.
| If you missed a call site or two, you'll probably catch it
| here. Because it's instantaneous, it's not painful. I'll
| have a REPL open pretty much all the time when messing
| around with CL.
|
| In JS (my day job), this wouldn't work at present if for no
| other reason than directly replacing a function is
| impossible. You'd create a second instance of a function,
| but all the old references from other places in the code
| would remain stuck with the old version.
|
| JS falls back to "hot reloading" which is literally
| "restart the whole JIT" which means that you have to warm
| the JIT up again and (more importantly) that all your state
| disappears.
|
| You make a change then wait a while for everything to
| recompile and run. Then you have to write/execute a bunch
| of code to get your state back to where it was. When the
| error is thrown, you now have to look through the things
| for that call site you missed, save, and reload everything
| all over again. Rinse and repeat. This heavily
| disincentivizes interactive and incremental development in
| favor of massive changes then trying everything out at
| once.
| lokedhs wrote:
| A few years back I did a presentation on this very topic.
| Some people have said they appreciated it so I'm sharing
| a link here in the hope someone find it useful:
| https://www.youtube.com/watch?v=bl8jQ2wRh6k
| dmux wrote:
| Doesn't the ability to "hot reload" in either CL or Node
| largely depend on how the application is structured? For
| example, if you were developing a web server that sat in
| an infinite loop waiting for requests to come in, how
| would you go about redefining a function referenced
| inside that loop? Wouldn't there need to be two threads
| of execution? One that was the web-server and a second
| that acted as the REPL? To redefine something in the web-
| server, the two threads would have to synchronize in
| order for the new definition to be passed off. Couldn't
| this same setup be implemented in any language that
| supports threads and some type of `eval` function?
|
| Edit: The two threads of execution I mentioned above are
| likely what the combination of SLIME + Swank are doing.
| mtlmtlmtlmtl wrote:
| I have done this exact thing in CL before. What you do is
| have your event loop function just be a loop calling
| another function, where all the event handling actually
| happens. When the event handler function is recompiled,
| the new version will run /the next time the function is
| called./
|
| If the recompiled function is currently running that
| stack frame and anything deeper won't be affected for
| that call.
|
| The only thing you can't do is redefine the loop function
| itself since it never stops running. That's why you
| factor it so there's no reason to change that function,
| just the functions it repeatedly calls.
|
| Hope that explanation made sense.
|
| Edit: And yes, the way this is implemented under the hood
| is that the repl has its own thread. You could do this
| even in C. In fact people have done this for the Linux
| kernel for example(at least for binary patches). The
| great thing in CL is the language is designed with the
| functionality in mind, and the dev tools just support it,
| so you get it for free.
| dasyatidprime wrote:
| Incidentally, this is nearly identical on top to the
| pattern Erlang uses for the same reason, where most
| processes couple a long-lived "behavior" with a callback
| module with a particular structure and a pure-data state
| value threaded through everything. When the module is
| replaced, the new functions are picked up automatically,
| and there's also a callback for adjusting the state
| information on code changes if need be.
|
| The underlying "system message requesting your process to
| ensure it picks up the code reload" part is quite
| different, though!
| aeonik wrote:
| I wonder how Clojure(script) deals with this. Do you know
| if Clojurescript restarts the entire JIT as well? Is it
| the same in Clojure with Java?
| mikelevins wrote:
| Clojure and ClojureScript don't have the comprehensive
| support for livecoding that Common Lisp and Smalltalk do.
|
| As an example, consider what happens if Common Lisp's
| UPDATE-INSTANCE-FOR-REDEFINED-CLASS, mentioned in my
| comment above, is called without a method definition for
| the redefined class in question. Lisp drops you into a
| breakloop by calling BREAK.
|
| BREAK is another feature of Common Lisp defined by the
| language standard
| (http://clhs.lisp.se/Body/f_break.htm#break). It puts you
| in a repl that has the full language and runtime
| available, but also exists inside the dynamic environment
| of the control path where BREAK was called.
|
| In this example, that means that we can see the lexical
| environment of the call to UPDATE-INSTANCE-FOR-REDEFINED-
| CLASS, along with everything that was on the stack when
| it was called. We can see the instance that needs to be
| redefined and all the definitions that are currently in
| effect. Using the features provided by the breakloop, we
| can interactively define the missing method on UPDATE-
| INSTANCE-FOR-REDEFINED-CLASS to properly reinitialize the
| instance in question, then activate a RESTART (another
| standard feature of the language) that resumes execution
| of the function, but now in a dynamic environment that
| includes the new method and any other changes we made in
| the breakloop.
|
| The original control path that landed us in the breakloop
| can now complete successfully, referencing a properly
| reinitialized instance. The method on UPDATE-INSTANCE-
| FOR-REDEFINED-CLASS is defined now, so future references
| to previously-defined instances of the redefined class
| will be transparently reinitialized and behave as if they
| always had the new definition.
|
| Clojure and ClojureScript don't have the runtime support
| to do any of this. If you redefine a class that has
| existing instances, and if those instances aren't dropped
| on the floor when your new definition is hot reloaded,
| then the old instances don't reflect the new class
| definition, and so code that relies on it is wrong. When
| a wrong access occurs, you are dropped--not into an
| interactive breakloop--but into a noninteractive printed
| stack trace.
|
| The debugging features of modern browsers give you some
| parts of what a Lisp (or Smalltalk) system does--for
| example, some tools can watch the dynamic state of
| variables and enable you to edit them. There are tools
| available for ClojureScript that give you access to some
| of these features.
|
| Even in the Clojurescript ecosystem, however, the full
| suite of livecoding tools offered by Common Lisp and
| Smalltalk (and mandated by the Common Lisp standard) is
| not available.
| aeonik wrote:
| Lisp is so cool, I wish every language could do these
| kinds of gymnastics. Thank you for sharing.
| orthecreedence wrote:
| This isn't an exact answer to your question, but I have to say
| macros.
|
| Going from CL (and scheme probably too) to macros in any other
| language is like going from a lambo to a tricycle. It's one of
| the things I miss most about CL.
|
| I also love the live-editing aspect of CL. When I was doing
| game programming with it, being able to replace functions while
| running was incredibly useful. You don't have to re-create all
| your state, you can jsut live-edit a function and boom fix a
| bug or add a new feature right there. Very good for exploratory
| programming with lots of otherwise-hard-to-replicate state.
| reddit_clone wrote:
| Well put. I agree on both points.
|
| CL has a very advanced TDD. You actually develop by executing
| snippets in REPL and pull them all together in a function
| (already tested!).
| eadmund wrote:
| Coalton comes to mind: https://github.com/coalton-lang/coalton
|
| It adds static typing to Common Lisp, while (I believe) still
| allowing one to escape to the dynamic world when needed. Sure,
| there are statically-typed languages, but I don't know of
| another dynamic language in which something like Coalton has
| been done.
| math-dev wrote:
| Its not very smart to add static typing to Lisp, goes against
| the strengths of the language. At that point just program in
| another language, there's a lot of benefit of static typing
| that IDEs can take advantage of but for anyone half decent at
| Lisp, they wouldn't feel any need for static typing (outside
| of compiler optimisations, which is a different thing).
|
| Now in terms of interesting things, check out European Lisp
| Symposium https://european-lisp-symposium.org/
|
| The talks from this year were highly entertaining
|
| The other really cool thing in CL IMO is Allegro Cache - in
| memory datastore that is a joy to use
| Tabular-Iceberg wrote:
| Did they record the talks this year?
| math-dev wrote:
| I think so, try asking on reddit or IRC if you still
| can't find them via some google fu
| medo-bear wrote:
| > Its not very smart to add static typing to Lisp [...]
| (outside of compiler optimisations, which is a different
| thing)
|
| i think this is precisely the point of coalton. cl allows
| quite handy compiler optimizations and having the option to
| drop into a native statically typed environment just gives
| more bang to your lisp ;)
|
| coalton is not lispy, sure, but neither is the loop macro,
| yet it is probably the most powerful looping mechanism in
| any language :)
| math-dev wrote:
| Those already exist and have been used by many.
|
| Coalton is a set of macros to make it easier for to write
| with the safety rails of static typing.
|
| Now if I write in Swift, for example, static typing is
| great, because the IDE can give a lot of hints during
| development. Non lisp languages have quite idiosyncratic
| syntaxes for various constructs, so static typing really
| does help a lot there. In large code bases in those
| languages it also helps a lot because they do not have
| the power of Lisp macros to reduce syntactical complexity
| via a Domain Specific Language.
|
| Now with Common Lisp, with a powerful generic functions
| and object system, with runtime dispatch and
| introspection that far exceeds other languages, plus with
| the abstraction power of macros, trying to add static
| typing guard rails simply doesn't make any sense. I
| rather program in TypeScript if I am so far removed from
| the core essence of CL development.
|
| Each language has its strengths and the dynamic nature of
| CL is one of its biggest strengths (with the lack of a
| GUI and interop with other tools being its largest
| deficiencies today). Static typing in Lisp is like trying
| to fit a square plug in a round hole. If somebody wants
| so much guard rails, other languages are much more
| suitable because they have been designed (quite
| successfully if I may add) in that way.
|
| TLDR; As per On Lisp by PG, lisp code should be built in
| layers over each other and hence static typing is
| unnecessary. Furthermore generic dispatch provides
| similar features to some extent (although i recommend
| combining with closer mop and catching edge cases etc)
| medo-bear wrote:
| > Static typing in Lisp is like trying to fit a square
| plug in a round hole. If somebody wants so much guard
| rails, other languages are much more suitable because
| they have been designed (quite successfully if I may add)
| in that way.
|
| i belive the author of the coalton package is not very
| concerned with IDEs or putting up safety nets for
| developers. i think he uses coalton for making optimizing
| compilers. think of it as cl+ml not as cl+ts
|
| EDIT: typos
| Kototama wrote:
| > lisp code should be built in layers over each other and
| hence static typing is unnecessary
|
| It could help for refactoring though.
| math-dev wrote:
| You bring a good point about the loop macro (which I do
| like too). Maybe you are right. My personal feeling is
| trying to do static typing will lead to less elegant code
| (I say less elegant, because any code that works and is
| fast enough is good enough - style doesn't matter that
| much, esp if readability is somewhat maintained) and has
| more far reaching implications on a code base than the
| loop macro which can be isolated to a specific section.
|
| But I get your point too, for those who crave static
| typing, then at least there's something for them to try.
| Jach wrote:
| Java's type system is just powerful enough that sometimes
| you can approach a problem in a way that resembles a
| different paradigm than normal, i.e. you solve it with
| the type system itself much more so than what's expressed
| by particular procedural or object-oriented or
| declarative code along the way. Supposedly Haskell
| encourages this approach as its primary one (and the
| approach sometimes gets called the "functional
| programming paradigm" but that I think is a misnomer as
| FP is primarily about immutability and higher order
| functions, not static types), to the same extent Java
| otherwise encourages a Java-style-OOP approach to most
| things, but I'm too unfamiliar with Haskell to make this
| claim with high confidence.
|
| Still, different paradigms are useful, and one of the
| best things about Lisp is you aren't forced into one, and
| there's no threat of one being taken away from you.
| Coalton to me seems nice for people who like what it
| provides, it's nice that it exists, even though I don't
| see myself using it anytime soon since I don't personally
| enjoy working in that paradigm. There are two other non-
| type-proofs-paradigm benefits of static typing, but I
| don't really miss them in CL. The first is related to
| automatic refactoring tools, but Lisp has features to
| ease refactoring to compensate, and text-search methods
| for renaming aren't _that_ bad (and even in Java
| necessary if you want to make sure you haven 't missed
| reflection calls, though even then you can miss some) and
| anyway one can read the second edition of the Refactoring
| book that uses JavaScript if one needs convincing that
| refactoring can be done just fine even in a lower quality
| dynamic lang. The second is related to trivial
| conveniences like compile-time typo protection on symbol
| names, interface mismatches (like swapped function call
| argument order -- but failing to that indicates I have an
| unintuitive interface, and CL has many tools to help me
| make it better the simplest being keyword arguments), and
| faster assembly code. But SBCL's handling of the standard
| CL type system is good enough to get all of those things
| to a pleasing degree.
| math-dev wrote:
| Great comment, agree a lot with this
| mikelevins wrote:
| I'd add one other feature of Haskell- and ML-style type
| systems that I like: they provide clear, succinct,
| convenient ways to precisely describe data structures.
|
| Common Lisp is far and away my favorite surviving
| programming language, but I like Haskell and ML fairly
| well. I mean, when I've been paid to work in them, I
| definitely missed working in Lisp, but still, working in
| Haskell and ML is not torture. I liked it quite a bit
| better than working in some other languages.
|
| One of the things I liked was the ability to succinctly
| express in some detail exactly what I intend some data
| structure to be.
|
| I've even used Haskell as a data-description language to
| work out some data structure that I then implemented in
| Lisp.
|
| I keep intending to do something with Coalton for this
| reason alone.
| nemoniac wrote:
| > coalton is not lispy, sure, but neither is the loop
| macro, yet it is probably the most powerful looping
| mechanism in any language :)
|
| I would argue that iterate is more powerful as well as
| being more lispy in the sense that it's extensible and
| macro-programmable.
|
| https://iterate.common-lisp.dev/
| martinflack wrote:
| I like the lispiness and features of ITERATE but it
| breaks source location for debugging errors in SBCL with
| SLIME. Instead the ITER form is indicated, which dampens
| my enthusiasm for it somewhat.
| (defpackage :test (:use :cl :iterate))
| (in-package :test) (defun test-iter ()
| (declare (optimize (debug 3) (safety 3) (speed 0)))
| (iter:iter ;;; <-- highlights whole ITER form
| (sleep 2) (error "test"))) (defun
| test-loop () (declare (optimize (debug 3) (safety
| 3) (speed 0))) (loop :do (sleep 2)
| :do (error "test"))) ;;; <-- highlights ERROR form
| Jtsummers wrote:
| Iterate also breaks the count function. Or did the last
| time I tried to use it.
| pjmlp wrote:
| Thanks for sharing, nice to see SisCog still at it.
| rscho wrote:
| Racket. With typed racket.
| newtwilly wrote:
| People are talking about REPL development, and they are right,
| but one of the best things that you can do with CL that you
| can't do with other REPLs, such as Clojure, is... at least
| using Slime in Emacs, probably other editors too... when
| there's an error/exception or whatever, you drop right into the
| stack at the error location. You can see all the values of the
| variables. You can evaluate expressions. Travel up the stack
| and do the same things. That is SO powerful, and can save so
| much time debugging simple errors.
| mikelevins wrote:
| You can also edit the values and change other definitions
| while you're roaming the stack.
|
| Smalltalk can do it, too.
| lisper wrote:
| I don't know about "since 2005" but there are two features in
| the original CL spec which I find indispensable but which still
| have not made it into any other languages (AFAIK):
|
| 1. Generic functions
|
| 2. Being able to redefine a class and have existing instances
| of that class be automatically updated to conform to the new
| definition
| bmitc wrote:
| What is a generic function?
| dotancohen wrote:
| When would you ever legitimately use #2?
|
| If e.g. an API wants to support monkey patching that's a
| decision that can be made. But thinking that any e.g. plugin
| or addin could could redefine extant class instances sounds
| like a nightmare, and a security issue just waiting to
| happen.
| lisper wrote:
| _Any_ code modification is a potential security issue.
| There is nothing special about dynamic class redefinition
| in this regard.
|
| I use it for development and deployment. I can deploy new
| code without having to take my application down. In fact,
| not only do all my existing instances get updated, but I
| also use an ORM [1] that automatically updates my database
| tables too.
|
| [1] https://github.com/rongarret/ergolib/blob/master/layer1
| /sql....
| mikelevins wrote:
| I use Common Lisp in my day job, and I generally build
| software by livecoding, so I use this and other interactive
| programming support on a daily basis. Seems "legitimate" to
| me. I don't want to have to stop my work-in-progress
| program running and lose all of the incremental state I've
| built just because I redefined a class or something.
| dotancohen wrote:
| So it's a feature that you would use during development,
| but not in the final (ha, ha, that never happens) state
| of the application?
| p_l wrote:
| Shinmera's Trial game engine, used in Kandria (which you
| can find on Steam), uses bits of the same machinery to
| implement the resourc system inside the game.
|
| IIRC it has separate classes for "unloaded entity" and
| classes for various types of resources, and the protocol
| (think interface) for them includes the generic function
| that loads them into memory. The implementation of said
| function for unloaded objects uses CHANGE-CLASS method
| redefine the instance in place - meaning all direct
| memory pointers to the class are still valid and you
| don't need to use extra layer of indirection.
| mikelevins wrote:
| I can agree that the most common uses are for development
| in progress, but I wouldn't go so far as to say I
| wouldn't use those facilities in a fully worked-out
| program. A sibling comment offers one reasonable example.
| I've done similar things before and probably will do them
| again.
| moonchild wrote:
| In longrunning/high-availability applications, this sort of
| thing is _necessary_ in order to avoid restarting and
| losing uptime or state. E.G. per nasa:
|
| > The Remote Agent software, running on a custom port of
| Harlequin Common Lisp, flew aboard Deep Space 1 (DS1), the
| first mission of NASA's New Millennium program. Remote
| Agent controlled DS1 for two days in May of 1999. During
| that time we were able to debug and fix a race condition
| that had not shown up during ground testing. (Debugging a
| program running on a $100M piece of hardware that is 100
| million miles away is an interesting experience. Having a
| read-eval-print loop running on the spacecraft proved
| invaluable in finding and fixing the problem
|
| (from https://flownet.com/gat/jpl-lisp.html)
| dotancohen wrote:
| Interesting, thank you!
| CraigJPerry wrote:
| In VSCode when my cursor is inside an s-exp, i can press
| Alt+Shift+E and it instantaneously shows the result of that
| call inline in the editor.
|
| If i change the code at all, i just press the same key combo
| and see the new result. It remembers all the state of what i've
| done up to that point too so i don't need to mess around
| setting up the call with data to feed in for example.
|
| The closest i can find to replicating this is the various hot-
| reload implementations in other languages but they all incur
| some kind of delay (naturally, because they're all doing a lot
| more work behind the scenes) which ruins the use of this
| feature, it stops you using it to evaluate your thought process
| in realtime. They also don't render the results inline - mostly
| because the syntax of JS/TS and other languages doesn't really
| lend itself to this kind of usage.
| darthrupert wrote:
| Jupyter notebooks manage to replicate a somewhat similar
| experience.
| pjmlp wrote:
| It is a nice approximation, but fails short of the whole
| experience.
|
| Imagine that your shell was like those notebooks, and not
| only you can interact with the whole OS, you can even
| change existing notebooks dynamically by reloading code
| that they depend on, e.g. a displayed image gets a new
| effect added to it.
| CraigJPerry wrote:
| Hmmm, somewhat but it's missing the ergonomics to the point
| that it can't be used the same way.
|
| The best analogue i can come up with is a sql session
| connected to a live database.
| greymalik wrote:
| What extensions are you using for this?
| CraigJPerry wrote:
| https://github.com/nobody-famous/alive
| kkfx wrote:
| In Emacs you can execute sexps anywhere with C-x C-e (eval-
| last-sexp) it's common in all classic environment treat code
| as a live thing, text as a live thing. A feature lost in most
| modern software.
| daniel-cussen wrote:
| Oh I'd love to share it, I just can't. Well I made one that did
| make its way to Fortran by being independently developed, the
| Lisp version was in 2011 and the Fortran I think...2017. The
| brute forcer in AI-Feynman.
|
| https://github.com/SJ001/AI-Feynman/blob/master/aifeynman/sy...
|
| I talk about it in my portfolio, it's Guesser 1.0 and Guesser
| 2.0.
|
| https://github.com/daniel-cussen/portfolio/blob/master/portf...
___________________________________________________________________
(page generated 2022-04-22 23:01 UTC)