[HN Gopher] Why your REPL experience sucks
___________________________________________________________________
Why your REPL experience sucks
Author : tosh
Score : 77 points
Date : 2022-11-30 13:12 UTC (9 hours ago)
(HTM) web link (srasu.srht.site)
(TXT) w3m dump (srasu.srht.site)
| nikanj wrote:
| We keep on making fun of php, but it had this problem solved in
| the 90s. Hit F5 in the browser, get the latest version of your
| software. No waiting 30s for a rebuild, no stale versions in
| caches, no headaches.
| iLemming wrote:
| > Hit F5 in the browser, get the latest version of your
| software.
|
| What about the state though? If I'm building a sophisticated
| app, let's say an e-commerce thing, imagine having to develop a
| shopping cart experience, where a user has to make multiple
| choices before getting to the shopping cart view. Every time
| you hit F5, you'd have to click a bunch of buttons, fill out
| fields, etc. Or you're gonna have to write a script that takes
| you there and restores the state, which is still time-consuming
| and annoying to have to do. With Clojurescript, I can change
| things in the code, eval it and the change would be reflected
| in the app immediately, without having to reload the entire
| app.
|
| > it had this problem solved in the 90s
|
| We weren't building many complex web apps back in the 90s, so
| the problems associated with hot-reloading weren't so apparent
| back then.
| cess11 wrote:
| You could put some state in the session to keep it persistent
| across page reloads and get a result similar to continuations
| or however you're keeping that state on the JVM.
| cess11 wrote:
| These days we also have psysh.org which is a pretty sweet
| interactive PHP shell.
|
| Doesn't have the sophisticated halting and rewinding and
| redefining that Common Lisp and other image based REPL:s do,
| but it allows for very rapid development in a similar way.
| freedomben wrote:
| Modern PHP can be like the REPL experience though depending on
| situation. Half the time the new code loads, the other half the
| time it serves from cache (opcode cache?). I'm not a PHP person
| so I'm probably doing something dumb, but I've found that
| inserting a `systemctl restart php-fpm.service nginx.server`
| into my test workflow avoids that problem.
| tonetheman wrote:
| PHP was and still is great. Make a change and hit refresh.
| Simple and easy.
| cutler wrote:
| The Clojure repl allows you to eval changes within a live
| application. Try that in PHP and I don't think you'll get
| very far.
| cess11 wrote:
| What do you mean by "live application"? Is a PHP
| application under nginx "live" or not?
|
| If it is "live", changing a file will immediately be
| reflected in the application on the next "eval", i.e.
| require or execution by the interpreter.
| klibertp wrote:
| Nope. That's not what the GP means. I'm not sure how it
| works currently with PHP, but what was meant was a long-
| running process that can change its behavior/be updated
| by evaluating new code without ever being restarted.
| Think of it like this: you have a page that never stops
| rendering, ie. you send some HTML but instead of
| finishing processing, you go into an infinite loop.
| "Live" system, in that situation, would still be
| updatable - you could change the body of the loop and
| make it produce another bit of HTML, without reloading
| the page.
|
| Again, I'm sure modern PHP was already optimized far from
| the initial "execute all the code on each request" model,
| but unless there's a long-running process that can be
| updated in memory without restart (and I _don 't_ mean
| doing `exec` on itself, that's cheating!) - it's not
| "live" by this definition.
|
| It's not that clear-cut a distinction, either. For
| example Elixir and Phoenix are definitely live, in that
| there's a process that responds to requests on one hand,
| and can be updated without restarting it on the other -
| but page reload is still required in many cases, even if
| triggered via websocket.
|
| This idea of liveness predates both PHP and the Web by 2
| decades at least and comes from Lisp and Smalltalk. It's
| been incorporated by Erlang and later Elixir, and by
| Clojure. Other than that, some embedded scripting
| languages support this kind of development, for example
| Awesome WM is scripted in Lua, and you can evaluate
| arbitrary Lua code while it's running. Emacs is another
| system that is meant to be used this way.
|
| Good to read: https://gbracha.blogspot.com/2020/01/the-
| build-is-always-bro...
| mm007emko wrote:
| I upvoted your comment but I only partially agree. PHP is
| great in some ways and totally atrocious in other ways. It
| put bread on my table in the 2000s but I am not looking back.
| The language is horribly designed. Writing AJAX calls before
| Google Chrome was even released (let alone its V8 Engine
| which made JavaScript fast) was no fun either. Any non-
| trivial web app we developed was slow (everything had to
| start from scratch with each request) and flooded with a
| couple of layers of caching which often didn't get
| invalidated properly.
|
| Now I am a purely backend&desktop developer, I leave web
| frontends to people who are willing to live in the JavaScript
| world where if no big framework is released every 5 minutes
| you have an impression that the Earth stopped spinning. I
| learnt the basics of React.js the other year and it's a far
| cry from what we had 20 years ago. ClojureScript+React even
| with its difficulties must be an ivory tower of modern web
| frontend development. I want to remember the good parts of
| PHP, frameworks Nette and Symfony with nostalgia. But I never
| want to re-live it again.
|
| I mean Request/Response websites with a bit of AJAX still
| work today, I even develop one as a side project since I am
| part-time involved in academia (neural networks & nature-
| inspired algorithms simulator, in Common Lisp with cl-who,
| Parenscript and a couple of custom components based on HTML5
| Canvas). But modern development? When you need to keep up
| with the latest trends? No way it can work.
| agumonkey wrote:
| php exchanged restart time for debugging time
| PaulHoule wrote:
| Workspace-based programming environments have always struggled.
| When I use Jupyter I rerun the whole book pretty frequently. If
| the book takes 30 seconds to run it is a big annoyance but saves
| time relative to the randomly irreproducible problems I see other
| people have. If the book takes 3 hours to run it is a different
| story.
| nerdponx wrote:
| Fixing this situation is supposed to be the promise of
| "reactive" notebooks like Julia's Pluto.
| PaulHoule wrote:
| That's one answer to the problem, but not the only answer or
| necessarily a complete answer... That is, people forget that
| the really special thing about spreadsheets is not the grid
| organization but the hidden graph organization of
| computations.
|
| I would point to this as a product that was ahead of its time
| and still seems without peer
|
| https://en.wikipedia.org/wiki/TK_Solver
|
| Another challenge Jupyter has is the conflict between "a way
| to present a report" and "a way to write a script that
| generates a report" and everything in between.
| eigenspace wrote:
| > That's one answer to the problem, but not the only answer
| or necessarily a complete answer... That is, people forget
| that the really special thing about spreadsheets is not the
| grid organization but the hidden graph organization of
| computations.
|
| Uh did you mean this the other way around or something?
| Reactive notebooks like Pluto _only_ address the hidden
| graph organization of computations, they don 't address
| grid organization at all.
| all2 wrote:
| The idea of "book" and the linear visual in a "book" limit the
| interaction of the user.
|
| What if, in addition to the "book", you had a visual
| representation of the environment? Further, what if you had a
| function application visualization?
|
| Say you start with with open(your_csv_here)
| as my_csv: ... some actions
|
| In the function application viz you'd see something like
| your_csv_here | open as |
| my_csv
|
| In the environment visualization you'd see two variables
| your_csv_here my_csv
|
| Ideally, you'd be able to grab either of those in the "book"'s
| linear representation of code blocks using auto-complete.
|
| ---
|
| When I've played with data sets in Pandas DFs, for example,
| having some awareness of my environment and the available
| states of data -- and how that data has been mutated -- would
| be extremely helpful.
|
| ---
|
| I've taken the view that a Jupyter notebook or similar
| environment represents a filter for one or more streams of
| data. Ideally, you need to be able to quickly visualize the
| execution environment and previous actions/filters applied to
| your data streams. I'll admit, this does nothing for working
| with logic (except that you'd be able to inspect inputs and
| outputs pretty quickly).
| cpursley wrote:
| I've only used a few REPLs so far but Ruby and Elixir are great
| (and I assume inspired by Lisp). I cut my teeth developing with
| REPLs and actually struggle a lot with trying to use language
| environments without them. Like, how can I just experiment and
| run a little one-of bit of code and data before starting the
| actual implementation? Or call and play around with the function
| I just wrote?
| onetom wrote:
| Have a look at Fred0verflow's videos, for example:
|
| * https://www.youtube.com/watch?v=St0_jsPnGAw - analyze some SO
| HTTP API responses *
| https://www.youtube.com/watch?v=TaazvSJvBaw - transducers from
| the ground up
|
| I think they are great, crystal-clear examples of REPL-based
| development.
| cpursley wrote:
| Oh, I think you misunderstood. I'm literally in the Elixir
| repl all day every day and do know how I'd don't know how I'd
| manage without one.
| TacticalCoder wrote:
| I really only ever encountered this issue when working with ring
| routes, as in TFA. Love the macro at the end, may start to use it
| because why not.
|
| Now I want to say this to Clojure beginners: launching Emacs with
| a very complicated, non optimized, init (3400 lines of custom
| elisp code I wrote over the years) takes 1.4s on my system. Now
| launching the app, with two REPLs (one for Clojure on the back-
| end, one for ClojureScript on the front-end I'm testing) and
| about 30 dependencies (including figwheel, which is kinda heavy)
| takes...
|
| 16 seconds.
|
| So, from complete scratch, launching Emacs, opening a Clojure
| source code file, (which btw also triggers the launch of the
| Clojure LSP server) and then launching the webapp server, two
| REPLs and loading the main webapp page in a browser takes less
| than 18 seconds. Well, ok, it's not from complete scratch as
| source files that haven't been modified do not need to be
| recompiled, I'll grant that.
|
| It's a 2019 AMD 3700X with 32 GB of RAM and a NVMe M.2 PCIe 3.0
| x4 lanes SSD. Not a bad machine but not last gen either (I'm
| probably swapping it for a 7700X one of these days).
|
| I'm pretty sure the startup time is going to go significantly
| down when I'll switch to the 7700X.
|
| So even though it's great to not have to restart the app/REPLs it
| _should not_ be a problem to restart the whole thing.
|
| If your REPLs take two minutes to start, I'd argue there's an
| issue and it's time to fix that first. Move from _lein_ to
| _deps.edn_ (which launches one JVM less, which really helps as
| Clojure startup times on the JVM are slow), buy a faster dev
| machine. Buy a PCIe SSD. Use chrome /chromium instead of Firefox
| for testing your webapp (in my experience chrome is simply way
| faster for anything JavaScript and, oh boy, does ClojureScript
| generate lots of JS). Etc.
|
| As a sidenote I'd say this specific "ring route not updated" is a
| variation of the age old issue of cache invalidation.
|
| So basically: I love working at the REPL and keep my REPL(s)
| opened for a very long time, hot reloading everything (including
| ClojureScript on the client-side) without needing to restart
| everything _but_ I don 't lose sleep over the fact that I do
| sometimes relaunch everything for it takes not even 20 seconds.
| runevault wrote:
| I used to be a big REPL head, especially in my time with Clojure.
| But in recent years since most of my programming was c# I got
| away from thinking in REPLs. However these days I'm doing
| personal work in F# so I have access to the REPL again, I just
| keep not thinking to take advantage of it. I guess this is my
| reminder to use ALL the tools a language/environment gives me.
| cutler wrote:
| The kind of repl the author refers to is strictly Lisp-based.
| Your F# repl wouldn't be any different from a Python or Ruby
| repl and falls well short of the Lisp/Clojure repl experience
| which is based on the code-as-data feature of Lisps.
| runevault wrote:
| I'd have to check and see, but I know at least some of the
| features overlap (for example f# lets you send a single
| function/etc to the repl and update the existing definition.
| However to the specific point he brought up, I'm not sure if
| anything like the var capture issue comes into play.
| uptownfunk wrote:
| R Studio repl experience one of a kind, have never found anything
| that replicates it perfectly.
| mm007emko wrote:
| I always preferred Matlab to R Studio, TBH.
| markeibes wrote:
| Whenver I use a REPL I'm annoyed I didn't write it as a test, so
| I get proper import completion and generally tooling support.
| Finally I usually want to reuse the code or have a regression.
| capableweb wrote:
| I think you're thinking about a different type of "REPL" than
| what the author is referring to. What you're thinking about is
| more of a "Programming Shell", a separate window/tab/pane where
| you enter code and when you're happy, you copy-paste that into
| your source file.
|
| What the author is talking about is a REPL running in the
| background while connected to your editor. So most of the time,
| you're entering code directly into your file, while evaluating
| snippets in the REPL but you never leave the editor itself.
|
| With that approach, nothing is stopping you from using the REPL
| to write the tests themselves. In fact, that's what I tend to
| get the most value from, writing unit tests together with a
| background REPL, interactively building up the test case until
| I'm happy, then committing the unit test that has been written
| in the process.
| jwr wrote:
| A related question is why would you want to reload the entire
| file/namespace every time you make a change?
|
| Coming from a Common Lisp background, I am used to evaluating
| single forms/functions. I would sometimes re-evaluate the entire
| namespace, but that would be relatively rare.
|
| I switched to Clojure many years ago and followed the same
| workflow. Then ClojureScript and tools like figwheel came along
| and I was somewhat surprised to find that people admire the
| "auto-reload" functionality so much: save a ClojureScript file
| and all of it gets re-evaluated in the browser interpreter. I
| found this to be inefficient, somewhat annoying, and a big step
| back from evaluating exactly what you wanted to.
|
| My guess is that lots of those people never worked with a REPL
| where you eval single forms from the editor, so this auto-reload
| on each save seemed much better. But it isn't! And it creates its
| own problems, like the one the OP solves in the article.
| onetom wrote:
| I have a shortcut, which reloads all modified files, then re-
| evaluates the previously evaluated expression, within the
| context of the same namespace it was evaluated in previously.
|
| This way I can be in any file, modify any `def` of `defn` and
| see its effect on the expression I'm working on.
|
| After modified something, I might jump to its unit test or jump
| to some definition of some adjacent vars, where i see a point
| which I want to probe (with a function like this: `(defn ? [x]
| (pprint x) x)`).
|
| Then I have 2 modified files, and my cursor is in a 3rd file
| and the expression I'm iterating on is in a 4th; doesn't
| matter, I can just keep hitting the same shortcut (Cmd-Ctrl-
| Enter) to see the effects of my changes on the "expression-
| under-test".
|
| It's an extremely convenient workflow, which is also easy to
| teach to ppl!
|
| Downside is that you can't have namespaces, which have side-
| effects WHEN they are loaded.
|
| To deal with that problem, I'm using `redelay`, `rmap`, `meta-
| merge` & `defonce`, eg: (ns service
| (:require [meta-merge.core :refer [meta-merge]]
| [gini.rmap :as rmap :refer [rmap ref] :rename {ref $}]
| [redelay.core :as redelay]) (def kit
| (merge {:cfg {:db/uri "..."}} (rmap {:db
| (connect (:db/uri ($ :cfg)))}))) (defonce service-
| ref (redelay/state :start (rmap/valuate!
| (meta-merge service/kit {:cfg
| {:param 'override} :some
| 'special-component})))) (defn service [] (deref
| service-ref)) (comment (.close service-
| ref) (-> (service) (some-operation x y z))
| )
|
| More info on this topic is here:
| https://functionalbytes.nl/clojure/rmap/2020/07/04/rmap-2-up...
| electroly wrote:
| This feels a little too strong to me. DrRacket has as its
| fundamental UI flow a combination of whole-reloaded-file and a
| REPL, and obviously the creators of Racket are intimately
| familiar with Lisp REPLs.
|
| One problem being addressed is the discrepancy between REPL-
| based development and non-image-based languages: you have to
| get your code back out into a source file somehow, without
| forgetting anything. In image-based languages like Smalltalk
| and APL, you just save your environment after modifying it
| interactively; there are no separate source files and the
| problem does not exist. But in file-based languages, you're
| stuck using the clipboard to copy code back and forth between
| the REPL and the source file. It can sometimes be easiest to
| write the code in the source file, then reevaluate it in the
| REPL environment. If that process is automatic instead of
| highlighting specific forms and pressing a button, it can
| sometimes be a slight productivity boost and less error-prone
| because you won't forget to reevaluate some change you made in
| the source.
| Jtsummers wrote:
| > In image-based languages like Smalltalk and APL, you just
| save your environment after modifying it interactively; there
| are no separate source files and the problem does not exist.
|
| My experience with APL was not based on using the image
| (longterm), but with Smalltalk, there is a source file paired
| with the image (you can see this with both Squeak and Pharo).
| And Iceberg has been around for years allowing you to select
| specific portions to export to or import from git repos as
| source files. It was preceded by other version control
| systems that let you export to and import from source files.
| electroly wrote:
| Definitely. Dyalog APL has support for source files, too,
| but I didn't want to overcomplicate things. The way that it
| saves environments (i.e. to a binary file or to a source
| file) is tangential to the point: that it has the ability
| to save the environment at all. REPLs go best with
| languages that can save their live environments, so that
| you don't have to worry about getting your work back out of
| the REPL environment. If your language can't save its
| environment, you need some other way to improve the REPL-
| to-source workflow.
| mumblemumble wrote:
| It's historically been pretty janky, though, and an
| acknowledged problem within the Smalltalk community. I
| think Pharo perfected their solution only within the past
| 10 years.
| greggman3 wrote:
| A system I used you'd edit the source and press some key-
| combination that injected the current function only from the
| editor to the running app
| aidenn0 wrote:
| FWIW, I just use C-c C-k with CL because it's easier. The
| standards lawyer in me also wants to point out that
| implementations are free to inline function calls that are in
| the same file as the function definition, but I'm not aware of
| any implementation that takes advantage of this, so that's
| rather academic.
| TylerE wrote:
| No friction is better than friction.
|
| Having to manually reload parts of a file is friction, and
| introduces possible heisenbugs... e.g. A,B, C were changed, but
| you forgot to reload A.
|
| There were good reasons for doing it that way when we had
| 10000x less computing power, but these days? Nah..
| pletnes wrote:
| Having spent tons of time in jupyter notebooks, I can tell
| you one thing - loading data and training ML models is not
| free. Keeping a running system/repl/notebook has immense
| value.
| simongray wrote:
| Hot reloading is useful when you're making something related to
| a React-based UI, which you likely are close to 99% of the time
| in ClojureScript. I seriously doubt it has anything to do with
| a lack of REPL experience. I use a traditional editor-connected
| REPL 100% of the time in Clojure and 0% of the time in
| ClojureScript. I think most of us doing full-stack Clojure work
| like that.
| jwr wrote:
| How is it useful? I've been developing a React app for the
| last 8 years or so, and I'd much rather eval single functions
| rather than reload everything. I feel much better in clj and
| cljc files, where I am not forced to do that.
| TacticalCoder wrote:
| > A related question is why would you want to reload the entire
| file/namespace every time you make a change?
|
| Wait... What change does it make to evaluate a single function
| or to evaluate that single function plus re-evaluate all the
| other function in the same Clojure source file? It's not slow.
| It doesn't break anything?
|
| Regarding figwheel, which I find very convenient (it reloads
| not just the modifications in _.cljs_ files but also in HTML
| and CSS files AFAICT), where is the problem and would you
| reproduce what figwheel does without figwheel?
|
| Most of my Clojure and ClojureScript source file have zero
| state, zero variable. And from what I can tell many Clojure
| projects are like that. So what change does it make if I define
| (or redefine) a function from a REPL or from the source file
| and have figwheel re-evaluate the whole source file?
|
| Heck... I got "confused" at some point: if I modify a _.clj_
| file I need to eval what I modified (or the whole file), to
| have my REPL "synched" with the file. While with figwheel I
| don't need to eval but _save_ the file for figwheel to update
| the front-end 's JavaScript in realtime.
|
| So what I did is this: I modified my shortcuts that does "eval
| buffer" to do "save + eval buffer" and modified my shortcut
| that does "save" to also do "save + eval buffer".
|
| That way everything works in the same manner and I don't need
| to remember which one does what.
|
| If I modify a _.cljc_ source file (common to both Clojure and
| ClojureScript) and either eval the file or save the file, both
| REPLs see the new definitions _and_ figwheel also hot reloads.
| I really like that.
|
| And, yet, I still have my two REPLs at all times and I can
| directly eval stuff at the REPLs if I want to.
|
| > My guess is that lots of those people never worked with a
| REPL where you eval single forms from the editor, so this auto-
| reload on each save seemed much better.
|
| OK but for ClojureScript running on the front-end, how would I
| then go, without figwheel's hot reload, to make a modification
| and have the (transpiled) JavaScript be updated immediately?
|
| > And it creates its own problems, like the one the OP solves
| in the article.
|
| But the problem of ring routes not being updated have nothing
| to do with auto-reload? I may be wrong but wouldn't the problem
| be exactly the same if I were to re-eval the _router_ var from
| the REPL (I 'll try it and see later on)?
|
| Anyway I'd say that, when worse comes to worse and you screwed
| your application's state / components lifestyle, a full restart
| one every blue moon ain't a biggie.
| capableweb wrote:
| > I switched to Clojure many years ago and followed the same
| workflow. Then ClojureScript and tools like figwheel came along
| and I was somewhat surprised to find that people admire the
| "auto-reload" functionality so much: save a ClojureScript file
| and all of it gets re-evaluated in the browser interpreter. I
| found this to be inefficient, somewhat annoying, and a big step
| back from evaluating exactly what you wanted to.
|
| It seems to be the default in some tooling in the ecosystem,
| but you can turn that off and use the "regular" Clojure way in
| ClojureScript environment too. That's what I usually tend to
| do.
|
| > And it creates its own problems, like the one the OP solves
| in the article.
|
| What OP solves in the article doesn't seem to be about
| reloading an entire namespace, AFAIK. It's certainly not common
| in the Clojure ecosystem to reload your entire namespace
| instead of just the function you're working on.
| suskeyhose wrote:
| So I am no stranger to CL, but the problem solved in the
| article isn't actually one that's solved by evaluating less
| than the whole file at once, and in fact I didn't tell anyone
| to re-evaluate the whole file at all in this article.
|
| The critical idea here is it applies to long-running functions,
| like a loop, a server, or other things. These, when handed
| function objects, will not reflect new behavior if you re-
| evaluate a single function.
|
| The article solves for this by explaining to intermediate
| programmers that there is a difference between using function
| objects and using vars, something that Common Lisp programmers
| learn too by using #' to retrieve function objects and passing
| symbols and using symbol-function to look up the associated
| function as needed.
| dmux wrote:
| >The critical idea here is it applies to long-running
| functions, like a loop, a server, or other things. These,
| when handed function objects, will not reflect new behavior
| if you re-evaluate a single function.
|
| Do you mind going into more details about what you mean here?
| In my experience, recursive loops will pick up a new function
| definition but non-recursive loops will only pick up the
| definition of a function upon entry.
| aidenn0 wrote:
| _Leaving wrong explanation for posterity and to keep thread
| understandable. See everything after [edit] for corrected
| explantion_
|
| This is an important distinction between (funcall #'foo ...) and
| (funcall 'foo ...) in Common Lisp as well. #'foo evaluates to the
| function named by the symbol foo, while 'foo evaluates to the
| symbol foo itself. So if you redefine the function named by the
| symbol foo any already compiled forms like (funcall #'foo) will
| still use the old version but (funcall 'foo) will use the new
| version.
|
| [edit]
|
| The above explanation is wrong; whenever #'foo is _evaluated_ it
| will resolve to the current function slot for the symbol foo.
| Since the form (funcall # 'foo) evaluates #'foo each time, it
| will naturally be affected by changes to the function-slot for
| foo. If you bind a variable to #'foo (like e.g. a route in a
| routing library as TFA does for clojure) then the evaluation
| happens when you bind the variable. See my reply to throwaway1x31
| for a full example showing the difference.
| throwaway1x31 wrote:
| In sbcl those two work the same.
| juki wrote:
| They don't. What aidenn0 said applies to all CL
| implementations, although with a small correction that it's
| not "any already compiled forms", but rather any
| variables/objects that already hold the value of `#'foo` that
| will keep pointing to the old definition. Compiled code
| calling `(funcall #'foo ...)` will get the new definition
| normally (unless `foo` is inlined).
| aidenn0 wrote:
| Derp, of course it does because (funcall #'foo) evaluates
| "#'foo" every time it runs. You need to bind (or assign) the
| value for it to actually show the difference:
| (defun bar () 3) (defvar *the-fn* #'bar)
| (defvar *the-sym* 'bar) (defun foo ()
| (funcall *the-fn*)) (defun baz ()
| (funcall *the-sym*))
|
| Then at the REPL: CL-USER> (values (baz)
| (foo)) 3 3 CL-USER> (defun bar () 2)
| WARNING: redefining COMMON-LISP-USER::BAR in DEFUN
| BAR CL-USER> (values (baz) (foo)) 2 3
___________________________________________________________________
(page generated 2022-11-30 23:02 UTC)