[HN Gopher] On Repl-Driven Programming
___________________________________________________________________
On Repl-Driven Programming
Author : todsacerdoti
Score : 167 points
Date : 2021-01-03 07:35 UTC (15 hours ago)
(HTM) web link (mikelevins.github.io)
(TXT) w3m dump (mikelevins.github.io)
| FraserGreenlee wrote:
| I don't see how this is different from using the pdb Python
| module? You can catch exceptions and use to write functions.
| gumby wrote:
| Back in the early 80s a friend described this approach as
| "programming by successive approximation." I was stung by that,
| but I saw his point.
|
| Nevertheless I felt it was wrong. The approach of designing a
| data structure, making a few functions to manipulate it,
| designing another, making sure they work together, etc isn't
| really that different in lisp from C++. The loop is tighter when
| you can interactively test.
|
| Either way you have to think ahead.
| avl999 wrote:
| Is there a modern non-Smalltalk, non-Lisp (or its derivatives)
| non functional imperative programming language that supports a
| "real repl" that the author is talking about? I don't know if
| that kind of environment is for me but I'd love to give it a
| spin.
|
| I have never found the REPL (the Python/Ruby REPL which is not a
| "real repl" according to this article) to be that useful beyond
| quickly playing with API of a library with sample data and
| getting a feel of it and its return types. So this version of
| repl described in the article does sound interesting.
| tlarkworthy wrote:
| https://news.ycombinator.com/item?id=25620899 (JavaScript)
| dnautics wrote:
| Bash?
| eikenberry wrote:
| Shells in general are a great example IMO. They are what I'd
| maybe call repl-first environments, in that they are used
| primarily as interactive repls but can be used to write
| programs.
| CraigJPerry wrote:
| I'm thinking a SQL console mostly qualifies per the article's
| idea of a repl. The break loop concept isn't perfect there
| though.
|
| I've always liked the python repl, breaking out a django repl
| and being able to manipulate the db via the api always felt
| like a super power to me when coupled with the expressiveness
| of basic python but last year i started dabbling with Clojure.
| My first "real" repl driven development experience.
|
| 2 things stood out for me:
|
| 1. You end up with some genuinely useful executable
| documentation and it just makes sense to me to preserve some of
| what i did in the repl while developing. I saw this first here:
| https://github.com/stuartsierra/component/blob/master/dev/ex...
|
| 2. Integration with your editor is essential - having a
| keystroke to send a form to the repl or evaluate in-line is a
| real productivity boost. It's like writing code while an
| intelligent debugger is permanently live. You don't need to
| wait for a compiler run then parse the feedback, it's just
| instant in-line runtime feedback. "Paredit" is really sweet. I
| used calva in vs code for this and tbh it's as attractive as
| the language in some ways. I have no idea how to meaningfully
| apply those concepts to something like python or javascript
| though. They just don't lend themselves to that kind of
| structural editing.
| moocowtruck wrote:
| have you ever tried something like https://gtoolkit.com/
| cutler wrote:
| There's Hy (hylang.org) for Python which comes with a real
| repl.
| keymone wrote:
| Author describes a nice feature, but that's not what makes repl-
| driven development tick for me. Repl really shines when editor
| integrates with it - send a form to evaluate, instantly get
| result back into the editor, eval a form and replace it with the
| value received, define and redefine the environment on the fly,
| get instant feedback if your code works without having to drop
| into shell and run some test command, micro-tests and usage
| examples that you can actually run inside editor to see how they
| work.
|
| You can get similar integration with ruby and python, but there's
| way more friction because that kind of development is not the
| blessed way and so nobody bothers to make it work.
| qznc wrote:
| > If it now works correctly, then congratulations; you found the
| problem!
|
| This notion if "correct" is "finished the current example as
| intended". Running a suite of unit tests would give me more
| confidence.
|
| Even that is only for programming in the small and that is
| comparatively easy to programming in the large. I don't see how a
| REPL would help there.
|
| That means a REPL makes easy things easier and hard things are
| unaffected. Does not sound like a competitive advantage to me.
| coldtea wrote:
| > _Even that is only for programming in the small and that is
| comparatively easy to programming in the large. I don't see how
| a REPL would help there._
|
| Because the right way of programming in the large is
| programming in the small and combining things (e.g. functions,
| components, classes, whatever, etc.), not building some huge
| monolithic monstrocity (this is orthogonal to monoliths vs
| microservices btw).
|
| Plus, this "programming in the large", when it gets to, say,
| 10.000.000 lines of code will still have bugs and behavior you
| need to check/search for/and fix in individual parts, and this
| "restarting/dynamic change" REPL will still be a great tool
| here.
| twobitshifter wrote:
| For Ruby users, pry gets you most of the features discussed in
| this post. You can get the error break loop, and drop into the
| live environment with binding.pry. Code can be added and
| expressions adjusted at this point. It's very powerful and the
| only "debugger" I've ever needed for Ruby.
| mikelevins wrote:
| I've used it a good bit. When I'm using it, I miss the tools I
| have in Smalltalk and Lisp environments.
|
| When I use Smalltalk or Lisp, I don't miss pry or the other
| tools I have when working with Ruby.
|
| I do like Ruby, though. I probably like it better than any
| other language that isn't made of s-expressions. (Well, I like
| ML a lot, too.) But the whole time I'm working with it it just
| makes me want to write another Ruby implementation, but this
| time on top of a proper interactive runtime.
|
| There is one, actually: MagLev, which is built on a Smalltalk
| runtime. As far as I can tell, though, it's dormant.
| civilized wrote:
| How is this different from, say, programming in Python and being
| forced to drop into a debugger whenever there is an error? The
| debugger also lets you see everything in the stack, modify
| definitions, etc.
|
| MATLAB and R both have options to drop you into a debugger if
| there is an error. Not sure if Python's debugger can do this, but
| other than that, this style of programming is perfectly doable
| outside of Lisp and Smalltalk.
| mikelevins wrote:
| So go ahead and set up that working situation and try this:
|
| Define a function foo() in one module that calls a function
| bar() in another one, but don't define bar().
|
| Now call foo().
|
| Now, in the debugger that you break into, give a definition for
| bar(), and resume the execution of foo(). Don't quit the
| program and restart it; that's cheating!
|
| If that works fine, please tell me what tools you're using,
| because they'll materially improve my tools for some work I
| need to do.
|
| Also, try this:
|
| Define some classes and make a bunch of instances of them.
| Write a few methods on the classes. Start your program running
| with code that uses those classes and methods.
|
| Now change one or more of the class definitions. Don't stop and
| reload the program! Again, that's cheating.
|
| When you land in the debugger (you do land in an interactive
| debugger, right?) use it to inspect the stack and find out
| which classes and methods are on the stack and what's
| responsible for the breakage. Ask the Python runtime to show
| you all the functions, types, and values that are on the stack,
| and when you find the likely culprit, ask it to take you to the
| source for the version that's currently on the stack.
|
| Now, while you're still suspended in the call stack, redefine
| the offending classes and methods and tell Python to use the
| new ones. Now resume the broken computation. If you hit another
| problem, use the same tools to find and fix that one, too.
|
| After you've done all that, please let me know what your Python
| toolchain is so that I can use it, too.
|
| Also, if that's all perfectly doable in Matlab and R, that's
| good to know. Let me know about those tools, too. I haven't had
| much occasion to use them, but that might have to change if
| they work that way, too.
| dm319 wrote:
| This is what makes R (and Julia I think) so special (where
| everything evaluates to an S-expression). Having REPL for
| exploratory data analysis is especially useful.
| alknemeyer wrote:
| For what it's worth, IPython's "autoreload" magic and Julia's
| "Revise" package get you _closer_ to what the author describes.
| In both cases, one can define functions/classes/structs in a
| file/module, load it in a repl, create objects etc, modify the
| file and have the changes propagate through to live objects
| joshlk wrote:
| You can also add breakpoints which automatically occurs when
| exceptions are raised. Which is similar to the breakloop
| functionality mentioned in the article.
| anentropic wrote:
| I find ipython + ipdb super useful in this regard
|
| But AFAIK neither provide the feature described in the
| article where you can continue running after an unhandled
| exception.
| jmuhlich wrote:
| True, but running the %debug magic command in IPython
| immediately after an unhandled exception drops you into the
| debugger at the point of the raised exception, complete
| with the full interpreter state including the stack. It
| doesn't permit edit-and-continue but you can still evaluate
| expressions (including calling any function) to explore
| what went wrong without having to recreate the situation
| inside an explicit pdb session.
| djm_ wrote:
| Yes, running pytest with the --pdb flag will drop you into
| the debugger on an unhandled exception which gets a
| comparable workflow but it's not quite the same as, a)
| writing tests to file first is not repl-driven development,
| and b) you generally have to think about doing it first.
|
| In an ideal repl-driven world you could write the test in the
| repl entirely and commit it to disk once you're ready.
| globular-toast wrote:
| Closer but still so far. As soon as you have more than one
| module in Python it completely falls apart, despite any
| autoreload magic. It just isn't the same at all as REPL based
| programming. In Lisps you can write your entire program with an
| instance and a REPL running the entire time. No restarting the
| instance required. It works because it's just how Lisp works. A
| REPL is a trivial thing you can write yourself that runs in the
| same instance.
|
| Whenever I think about it I can't quite put my finger on
| exactly what Python would need to make it the same. It's
| something to do with the way imports and namespaces work,
| though.
| sedachv wrote:
| > Whenever I think about it I can't quite put my finger on
| exactly what Python would need to make it the same. It's
| something to do with the way imports and namespaces work,
| though.
|
| Python has a fundamentally broken module system, and Python 3
| did nothing to fix it:
|
| https://nedbatchelder.com/blog/201908/why_your_mock_doesnt_w.
| ..
|
| https://docs.python.org/dev/library/importlib.html#importlib.
| ..
| miccah wrote:
| I admit I have not experienced true REPL driven development as
| the author defines it, but I have created an easier way for
| "development with a REPL" in vim. I love it because it's like a
| looser, faster Jupyter notebook, and it works with any language
| that has a REPL.
|
| It works by sending highlighted text to vim's terminal buffer. I
| wrote a short blog post with a demo and the vimscript code here:
|
| https://mcastorina.github.io/posts/vim-repl-driven-developme...
| miccah wrote:
| I should have read more comments before posting, but it seems I
| reinvented an inferior wheel:
|
| https://github.com/jpalardy/vim-slime
| magv wrote:
| For those of us that don't use Lisp (or Emacs+SLIME), and are
| stuck with Python/Julia/Lua/etc and Vim may I give a practical
| recommendation? The Vim-Slime plugin [1] is a half-decent way of
| making interactive development work. With a bit of tuning you can
| make it start a terminal emulator window inside Vim, and have it
| send the current paragraph of your source code into it with a
| press of a button.
|
| While not as great as working with a proper Repl-oriented
| language (as the article explains), this is still so much more
| pleasant than having to re-run the whole program every time you
| change a function.
|
| [1] https://github.com/jpalardy/vim-slime
| alpaca128 wrote:
| > With a bit of tuning you can make it start a terminal
| emulator window inside Vim, and have it send the current
| paragraph of your source code into it with a press of a button.
|
| Defining keybindings for splitting the window, opening a
| terminal buffer and copying a paragraph does not really require
| a plugin. That's one of the features I like most in Vim, its
| ability to just map any input to another sequence of keypresses
| can replace whole scripts and plugins provided you're fine with
| readability comparable to regular expressions.
| dm319 wrote:
| And nvim-R for vim/nvim and R.
|
| [1] https://github.com/jalvesaq/Nvim-R
| pjmlp wrote:
| Without caring about VIM, this would be my options,
|
| Python IDE tooling on Microsoft stack.
|
| https://docs.microsoft.com/en-us/visualstudio/python/python-...
|
| Juno IDE for Julia
|
| https://junolab.org/
| nikhizzle wrote:
| I also recommend jupyterlab for a similar experience. I was a
| hardcore vim terminal guy for 17 years, but now I won't be
| going back.
|
| The ability to interactively develop functions in almost any
| language inside the same environment had been revolutionary for
| my productivity given my short attention span.
| alpaca128 wrote:
| If you're using Jupyter within the browser you can have both
| via the FireNVim plugin in Firefox. It connects to NeoVim in
| the background and lets you edit any multi-line text field
| with 100% of your personal Vim configuration, the only
| limitation is that the color scheme cannot be changed.
| Jugurtha wrote:
| Would you mind sharing what you're doing with JLab? Do you
| use it in machine learning or are you doing something
| completely different?
| GianFabien wrote:
| AFAIK both Lisp and Smalltalk environments are image based. That
| is, all source code and all objects are stored in memory. You
| need to save that image in order to continue from where you left
| off. Snapshots are also used to allow rollback to previous "good"
| states.
| pjmlp wrote:
| Dart, Java and .NET have a kind of edit-and-continue without
| images.
|
| It is all a matter of tooling.
| andybak wrote:
| > It is all a matter of tooling.
|
| I think the point of the article is that tooling can only get
| you so far. The abstraction they attempt to provide is leaky
| if the support for this style of development isn't firmly
| designed into the entire system.
| pjmlp wrote:
| Yeah, but in that regard developer culture is what matters.
|
| I have seen very few people actually using these kind of
| workflows back when Smalltalk and Lisp were more relevant
| (I used Smalltalk/V back then).
|
| It is like using gdb, many don't go beyond step, next,
| print, run, breakpoint and discover how powerful it
| actually is (same applies to other debuggers).
| mikelevins wrote:
| I am indeed claiming that tooling only gets you so far,
| if you don't have good support designed into the runtime.
|
| I take you at your word that you didn't use the sorts of
| workflows I've described, but I did, and I still do, to
| the extent that I can, and it was common place among, for
| example, the programmers working on the bauhaus OS or the
| ones working on the SK8 development environment.
|
| We kept images around as a matter of course for various
| purposes, for example.
| phoe-krk wrote:
| _> That is, all source code and all objects are stored in
| memory._
|
| Common Lisp usually stores code in form of normal source files
| that are then possible to load into a clean-slate Lisp image.
| It's possible to dump images and restore them, but it's not the
| norm of working with CL.
|
| Unlike in Smalltalk, I currently know of no tools that allow
| one to easily edit source code of functions/methods that have
| already been compiled into the system, and therefore the
| dependency on the filesystem source files is heavy and
| immediate.
|
| AFAIK dumping images in CL is mostly used for shortening load
| times by preloading code and data, and for application
| delivery, but not for live editing support.
| brainbag wrote:
| It's not compiled, but Ruby's Pry REPL allows you to edit and
| update code files during live execution. I use it all the
| time for adding breakpoints and live fixing issues. It's very
| productive for problem solving.
| lispm wrote:
| > Unlike in Smalltalk, I currently know of no tools that
| allow one to easily edit source code of functions/methods
| that have already been compiled into the system
|
| Lots of Common Lisp systems support that in some way. Those
| record the location of the source code together with the
| machine code. These locations are stored in the image.
|
| For example the standard CL function ED takes a function name
| and in many Lisp systems this will edit the function in some
| implementation specific way. In Emacs one uses M-. to get the
| definition of a function. In those implementation it will ask
| the Lisp for the location of that function source code.
|
| The one system that worked similar to Smalltalk is Interlisp.
| See https://interlisp.org . It recently has been open sourced
| and is in the process of making it more accessible. It's a
| glimpse into an alternative world of computing from the past.
| phoe-krk wrote:
| I know that many implementations record source location,
| but the source itself is still on the filesystem, not in
| memory, therefore making source locations useless if we
| move the filesystem out of the equation.
|
| I also know that many implementations store the forms in
| FUNCTION-LAMBDA-EXPRESSION, but as I said, I know of no
| _easy_ way to edit these in-memory forms either. Interlisp
| has had a structure editor to work with those, but that
| utility was not ported back into the CL workflow. I hope it
| eventually will be, with Interlisp being open sourced now.
| lispm wrote:
| > therefore making source locations useless
|
| Just make source available on the file system. Mount it.
| Copy it.
|
| That's why Common Lisp has logical pathnames. Typically
| one uses logical pathnames in source locations. When I
| set up my application on a machine, I define a
| translation table to point to the source code.
|
| One can also just use standard pathnames and mount the
| source code to a standard path. That's how one also has
| set up Lisp systems in clusters. Every machine mounts the
| source in a standard path (or a logical path) and a
| client system will then have access to that source code.
| The image still has the recorded locations.
|
| Btw., even Smalltalk stores its source code not in the
| image. The Smalltalk source code is stored OUTSIDE of the
| image in the file system and it has to have the files
| available and needs to know the location of these source
| files.
|
| See: https://squeak.org/downloads/ The
| Squeak/Smalltalk programming system consists of three
| parts: * a virtual machine for your
| platform, * both image and changes files of a
| particular version, and * a sources file for the
| particular image file.
|
| The source and changes files contain the source code of
| the running Smalltalk. An edit of source code will then
| lead to an entry in the changes file.
|
| Smalltalk usually can decompile byte code, so one can
| edit code which has no corresponding source code, with
| some loss of original source information.
| milansuk wrote:
| REPL is very useful, but few months ago I realize[0] that setting
| a break-point is not easy as people think, because the program
| can visit the line through multiple paths(function can be a call
| from different places), so you can spend a lot of time
| debugging/finding the right path to the line, which means that
| you probably have to set more break-points to get the program to
| the right place faster.
|
| [0] https://skyalt.com/blog/repl.html
| aidenn0 wrote:
| In lisp I do something like: (when (in-state-i-
| care-about) (break))
|
| Many non-lisp debuggers have conditional breakpoints, though
| they often have a larger performance overhead than the above.
| khalilravanna wrote:
| What are people's thoughts on how using vs not using a REPL help
| or hinder one's _thinking_ as a programmer. Specifically I mean
| if you have no REPL and maybe a long compile time you're forced
| to put a little more thinking and planning in up front if you
| don't want to waste your time. You might be more meticulous in
| catching bugs. Whereas with a REPL you can throw stuff at the
| wall. If something breaks you can just tweak variables and
| structure in real time till it starts working.
|
| That's just one characterization I came up with off-hand and it
| may be inaccurate. I'm mostly trying to paint a picture where
| tools affect the thinking of the programmer.
|
| What are people's thoughts on this? Does using a REPL or not
| using a REPL change your thinking and how so?
| mikelevins wrote:
| I think it's a valid observation. Circumstances that make you
| think things through ahead of time compel you to learn things
| and gain skills that you would not otherwise learn and gain.
|
| The other way around is true, too, of course.
|
| But the intensely-interactive style of programming-by-teaching
| is the less common approach. In pretty much every thread I've
| been part of about the topic there have been commentators
| who've said they just weren't even aware of it as an option,
| who had no idea that full-featured repl-driven environments had
| ever existed or were a possibility.
|
| What are the odds that such an obscure approach to programming
| has already attracted all the programmers who would benefit
| from it? Long odds, I'm guessing.
|
| So my guess is that it's worthwhile to point out the option for
| the sake of people who would benefit from it but don't know
| they have the option.
|
| For my benefit too, of course. I want more people to know about
| it, because that increases the chances that demand will rise.
| Rising demand increases the chances of greater investment in
| those kinds of tools, and reduces the chances of their
| extinction.
| Jtsummers wrote:
| > Specifically I mean if you have no REPL and maybe a long
| compile time you're forced to put a little more thinking and
| planning in up front if you don't want to waste your time
|
| Amusingly, to me, this is something people describe as the
| difference between having to submit punch cards to (or schedule
| a job on) a mainframe versus having a compiler on your own
| machine. That having such quick access to the compiler would
| lead to the "throw stuff at the wall" approach.
|
| IME, faster feedback loops _do_ increase the "throw stuff at
| the wall" approach, but for those of us who still (mostly) sit
| back and think, it's an enabler and not (just) a crutch. I can
| get in the flow much better in a language like Lisp and stay
| there. If, for instance, there's some confusion in my mind
| about how a function or data structure works, in C++ it takes
| me longer (more of a constant factor longer rather than orders
| of magnitude longer) to write something and test it out
| (assuming documentation doesn't clear it up for me). But while
| the time to explore it isn't _huge_ , I've been pulled out of
| my focus for longer. With Lisp, I can test it in seconds and
| get right back to whatever I was doing.
|
| But also, when I don't have a REPL, I break my programs into
| smaller programs (libraries/modules) which can be composed into
| the larger program I want. This lets me, partially, recreate
| the REPL experience. Using a test runner (a proper one or an ad
| hoc one) and a set of small CLI apps that let me use the
| smaller modules directly I get something approaching the
| feedback speed of the REPL. With fewer dependencies for any
| part under test (or as a CLI app), I get much faster
| compilation speeds vs needing to recompile a much larger
| program.
| blunte wrote:
| This article helps us understand what real repls enable that
| interactive language shells do not.
|
| What would be very helpful next would be a video comparison of
| writing a program that is just complex enough to not be trivial
| or too artificial, first using the common approach, and next
| using repl-driven approach.
| globular-toast wrote:
| Why not just learn a Lisp yourself? You'll get far more out of
| it than watching a video. The old adage is still true: learning
| a Lisp makes you a better programmer.
|
| Your choices are:
|
| * Common Lisp: install emacs, SBCL and set up SLIME,
|
| * Clojure: install emacs, Clojure and set up CIDER,
|
| * Scheme: install emacs, racket (or guile) and set up Geiser,
|
| * Emacs Lisp: install emacs.
|
| You'll notice that they all involve emacs. There are probably
| other ways but you want something that is quite tightly
| coupled, which all of those emacs packages provide. REPL driven
| programming is better served by an editor like emacs rather
| than vim. You want something where you can quickly write/edit
| code in one buffer and send it to the REPL with a couple of
| keystrokes. The lower the "cost" of this operation the better;
| it should be as easy as typing (as it becomes that frequent of
| an operation).
|
| To help you choose: Common Lisp and Clojure are the most
| practical. They are both general-purpose languages with a
| wealth of libraries available. Common Lisp, as the older
| language, has far more literature available including some of
| the best programming books ever written. Scheme is the most
| beautiful language and has one of the best textbooks ever made:
| _Structure and Interpretation of Computer Programs_ (SICP).
| Emacs Lisp is the most fun, practical but also the most quirky.
| Luckily, learning any Lisp will put you in a better position to
| learn any other Lisp.
|
| Emacs Lisp is fun because it presents the most exciting part of
| REPL based programming: hacking a live, running system. Most of
| the time you run a lisp instance just for the purpose of
| development, but if your program is working and doing
| something, then why not hack on it while it's running? There's
| no difference between running a REPL in a "development"
| instance and a live, production instance. When you hack on
| emacs you are doing just that: hacking a live, running
| instance. It's not often you get to use a tool to hack on the
| tool itself while it's running.
|
| Whatever you choose, good luck on your journey. I truly envy
| those who have yet to experience the beauty of Lisp
| programming. It changes you forever.
| blunte wrote:
| Nice reply. In my case, I'm already an effective amateur at
| Clojure, but I only use the repl for testing ideas and
| debugging.
|
| In the same way there are guides that teach TDD, I would
| enjoy seeing a RDD (repl-driven ...) tutorial.
| simongray wrote:
| In Clojure the "Rich Comment Blocks" are a very common way
| to use the REPL: https://betweentwoparens.com/rich-comment-
| blocks
| tincholio wrote:
| You could have a look at this video by Sean Corfield:
| https://www.youtube.com/watch?v=UFY2rd05W2g It gives you a
| good idea of what the REPL workflow is like. Sean will give
| a more in-depth talk about this next week, and it will be
| available in the London Clojurians channel afterwards, if
| you're interested.
| coldtea wrote:
| > _Why not just learn a Lisp yourself? You 'll get far more
| out of it than watching a video._
|
| Because one is a 30 minute endeavor and the other is a
| weeks/years endeavor.
| ashtonkem wrote:
| Having used Clojure and Common Lisp professionally, I think
| advocates for REPL driven programming universally overstate the
| utility of a first rate REPL.
|
| Yes, it's nicer than Python's. Yes, it's convenient. No, I never
| ended up doing my development in the REPL _first_. Why? Because
| editing mistakes in a REPL usually sucks, because a REPL is not a
| text editor.
|
| What I ended up using heavily was REPL to file integration, which
| gave me the ability to write a function normally, evaluate it in
| the attached REPL session, and then play around with it in the
| REPL. This is far short of the "REPL driven development" that's
| commonly discussed, and frankly something that's probably
| possible with the Python REPL if they wanted to.
|
| Editing data and functions in the REPL is a neat trick, but it's
| a double edged sword, because a REPL can crash, and it provides
| incredibly rudimentary support for diffing current state and
| migrating changes back to permanent files. I would never start
| with anything more than a trivial "how do I manipulate this list"
| in the REPL for that reason. Oh, and we had a lot of issues with
| REPLs getting into a bad state with Clojure due to multi methods
| and protocols; if you're doing your primary work in the REPL,
| then having to restart it due to it becoming unstable really
| sucks.
| blandflakes wrote:
| I 100% agree with you and another comment:
| https://news.ycombinator.com/item?id=25622990
|
| I spent years in Clojure and found that writing code in the
| REPL just... isn't fun. This has been true for every
| interactive system I've ever used (including... the shell).
|
| The shell as a REPL for a succinct language is nice for
| interactive workflows, but for trying to build less ephemeral
| pieces of code, the editor is the first class citizen I care
| about. Sending code to a REPL to be interacted with? Cool, and
| I _think_ this is what a lot of people mean when they say REPL-
| driven.
|
| I'd imagine convenience around that in a language or its
| tooling would be what convinces me to do more REPL-style
| things.
| mikelevins wrote:
| As I said elsewhere, when I say "repl-driven programming", I do
| not mean "programming in a repl window", or "programming at a
| command shell". I'm talking about the runtime's read-eval-print
| loop, not the UI's repl window.
|
| When I'm working, there is little or no migrating of code from
| the repl buffer to a file because it's already all in a file. I
| almost always work in a file. I write snippets in a file, send
| them to the repl with a keystroke, and build up the world
| incrementally as I discover what it needs to be. As the
| contents of the file get larger and more complicated, I move
| things around and organize them. It's a conversation _with_ the
| repl, not an editing session _in_ the repl _window_.
|
| I don't consider any Clojure tools I know of to constitute a
| proper repl-driven environment, precisely because the language
| and runtime lack support for the kinds of programming and
| debugging that I've taken for granted for decades. If I can't
| inspect and edit and redefine _everything_ in the runtime, it
| 's not the full, proper set of tools.
|
| I've written a good bit of Clojure code, and I'll happily do it
| again if I need to do something that isn't convenient in, say,
| Common Lisp, but I consider it an acceptable alternative to
| things like Haskell and Scala and F# and Swift, not an
| attractive alternative to Lisp and Smalltalk.
|
| I do occasionally need to restart a Lisp environment because of
| some gnarly breakage I've committed, but it's pretty rare.
| Moreover, killing and restarting my favorite lisps takes about
| --wait, let me check--okay, a second and a half to kill a live
| app in staging and have it back up, fully-functioning.
| chris_j wrote:
| > What I ended up using heavily was REPL to file integration,
| which gave me the ability to write a function normally,
| evaluate it in the attached REPL session, and then play around
| with it in the REPL
|
| When I do REPL driven development in Clojure, that's exactly
| what I do and that's what other folks that I know mean by REPL
| driven development (in Clojure): define types, functions and
| variables in a source file, often in a (comment) form, eval
| them one by one, and copy the code out of the (comment) form
| when it's stable enough. I wouldn't type code directly into the
| REPL; that's not a pleasant experience in Clojure - but might
| be pleasant in Common Lisp or Smalltalk for all I know. The
| process that I and others use in Clojure most definitely
| differs from the process that the OP describes and I lack
| enough familiarity with Common Lisp to know if the process is
| more pleasant in that language. I assume it is and I must make
| time to learn Common Lisp properly some day.
| tlarkworthy wrote:
| I think the best modern example is ObservableHQ [1]. I took a
| screen shot yesterday when I found myself interleaving runtime,
| TDD, IDE, debugger and partial recompilation in one window. It
| blew my mind.
|
| I have never experienced such a productive programming
| environment.
| https://twitter.com/tomlarkworthy/status/1345321532650905601...
|
| For there I can change variable at runtime. Change the
| implementation and have the tests auto run. Correct the tests,
| set a breakpoint in the test or implementation with "debugger;".
|
| I think it might be more productive than smalltalk because of the
| spreadsheet-like reactive recomputation. Plus it supports real
| markup as inline documentation.
|
| [1] https://observablehq.com/@tomlarkworthy/rate-estimation
| mikelevins wrote:
| Thanks for pointing it out. The site went on my reading list.
| oumua_don17 wrote:
| Writing a ray tracer in Common Lisp is an excellent video that
| helps one understand how live editing aka REPL driven programming
| works with a Lisp such as CL that supports it from the ground up.
|
| [1] https://www.youtube.com/watch?v=N1oMRw04W3E
|
| edit: fix typo
| adamkl wrote:
| For anyone who would like to see what REPL-driven programming
| looks like in practice, take a look at this video here:
|
| https://vimeo.com/230220635
|
| The author uses the Clojure REPL to walk through the process of
| developing some non-trivial functionality (calling an API and
| parsing the results).
|
| It's a good intro to what it looks like to interactively build a
| program while it's running.
|
| Personally, I've found that having such a tight feedback loop
| makes development a lot more enjoyable.
| irjoe wrote:
| Interesting. I do the same thing in Elixir where I'll attach an
| iex session to a Phoenix application so I can interrogate
| modules and APIs as I'm building them out.
|
| I'm slightly disappointed that it's already something I do day
| to day. I had hoped that the power of the REPL wasn't
| overstated.
| adamkl wrote:
| I think just because someone is able to do something similar
| in another language doesn't mean that the power of a fully
| integrated REPL is overstated.
|
| Most developers are using languages where this sort of thing
| isn't possible, and for them, experiencing a REPL driven
| development flow can be an eye opening experience (even if
| it's just to add it to their tool box along side more common
| approaches like attaching debuggers and using TDD to shorten
| the development feedback loop).
|
| I don't know enough about Elixir to understand how your
| approach is the same/different than using something like a
| REPL with Clojure, but I did come across a pretty interesting
| discussion on the topic:
|
| https://elixirforum.com/t/what-do-you-all-think-of-
| clojures-...
|
| TL;DR - you can accomplish something similar with Elixir, but
| the underlying technical details are different.
| rozap wrote:
| It's nice to be able to do this in production for debugging.
| It's super powerful and one of the my favorite features of
| erlang.
| phissenschaft wrote:
| My concept of a "REPL" is mostly defined by emacs. You would have
| a buffer with a code file, with an active jupyter kernel with the
| correct dependencies loaded in it. Then one would send any active
| region with `C-c C-c` and get timely feedback.
|
| With this mode https://github.com/nnicandro/emacs-jupyter one can
| connect to a jupyter kernel running locally or remote (would
| mostly prefer SSH port forwarding or kubectl port-forward the
| remote jupyter server). It makes life so much easier to interact
| with cloud environment (e.g. spark).
| crabmusket wrote:
| This reminds me of Unison's "codebase manager":
| https://www.unisonweb.org/docs/tour#-the-big-technical-idea
|
| It's not exactly a repl, but it shares similarities to the idea
| of interacting with your source code itself via software, not
| just by typing bytes into a file.
| globular-toast wrote:
| As someone who has used CL and Clojure to write non-trivial code
| using REPL driven programming it makes me mad that the word REPL
| has been hijacked to mean "interactive shell".
|
| I'm thankful for this article shedding light on what it really
| is. It's sad that most programmers haven't experienced.
| lispm wrote:
| The examples from Mikel are just not possible in a default
| Clojure setup. It does not have break loops and it does not
| have a dynamic object system (by design).
| tluyben2 wrote:
| Ah that answers my question I guess. Thanks
| globular-toast wrote:
| Yeah, true. I would still consider Clojure a "proper" REPL
| driven language, though.
| tluyben2 wrote:
| I have some experience with clojure but besides using the repl
| as 'interactive shell' I never tried it further (for some
| reason I always thought it was a crippled lisp even though I
| did some fair amount of work in it); does it offer the full
| experience? I should try it again as that's something that's
| actually used in real life (and has a good client side cljs
| experience).
| irjoe wrote:
| Is there a simple way to get code I write in a lisp REPL back
| into my editor? That's the part missing for me and why I usually
| only use interactive shells (REPL or otherwise) for testing APIs
| or small pieces of code.
|
| I can't imagine writing a program in its entirety in a REPL.
| gumby wrote:
| The standard technique is to run the loop inside the editor.
| This technique dates back to the 1970s.
| tarboreus wrote:
| You write it in the editor and send statements to the REPL.
| You're not just sitting looking at a command line. The REPL is
| a conversation with running state, but the conversation doesn't
| have to take place only through a single blinking terminal
| window. In proper REPL-driven development, the editor is fully
| integrated with the running session, and you can evaluate
| either in an attached terminal session or through your editor.
| Or by attaching whatever other tools to the running session.
| mikelevins wrote:
| Yes, exactly. It's perhaps my fault for not making this more
| explicit in the essay, but the "repl" in "repl-driven
| programming" does not mean the repl _window_.
| lebuffon wrote:
| Nobody has mentioned low-level programming in a REPL. Not as
| common to be sure. Forth works this way and even has REPL
| Assembly Language. It's been there for over 40 years.
|
| Testing Forth and ASM code snippets in the REPL before committing
| to them helps eliminate those nasty assumptions about what
| "should" work.
| mikelevins wrote:
| Agreed, FORTH is repl-driven in the sense I mean. The main
| difference from Lisp and Smalltalk systems is that FORTH
| environments are, generally speaking, more spartan.
|
| In the late 1980s I had a group of friends at Apple that
| included Smalltalk, Lisp, and FORTH programmers. We certainly
| found plenty of things to admire and attempt to steal from one
| another, and everyone accepted the basic goodness of building
| systems by engaging in conversation with them as they run.
|
| Lisp and Smalltalk cross-pollenated each other more than each
| did with FORTH, but maybe Slava Pestov's Factor is a glimpse of
| what you get if FORTH is more in the mix.
| tluyben2 wrote:
| I think it would help as well to list languages that have various
| levels of 'real repl' implementations. Wonder what modern
| (systems you can 'make money with') there are that support this.
| I know common lisp + smalltalk and worked with both and really
| liked them for these reasons. I miss this functionality all the
| time as it was far more efficient (to me!) than modern debuggers.
| rightbyte wrote:
| Isn't Python and Matlab a 'real' REPL? They are money makers.
| mettamage wrote:
| The article specifically stated that Python does not have a
| real Read-Eval-Print-Loop mechanism like Lisp or Smalltalk.
| For example, the Python REPL doesn't go into a breakloop
| mechanism for undefined functions.
| mikelevins wrote:
| It's not about whether they make money. It's about whether
| you can do absolutely everything needed to build your program
| interactively by talking to it while it's while it's running.
| coldtea wrote:
| The article describes specific capabilities (full
| program/system dynamic redefinition) afforded by a real or
| full REPL.
|
| Python doesn't have that, and it's mentioned explicitly as
| not having that.
|
| It's not about "existing in the real world" or "real enough
| to make money with".
| mvc wrote:
| Definitions matter. A repl is a read, eval, print, loop.
|
| That means the code you enter at the prompt is converted from a
| String into literal data that can be evaluated (read). The data
| is then evaluated to produce a result (eval), and the result is
| then printed (print).
|
| If that's not what your REPL is doing then it's not a repl.
|
| That is not what's going on in e.g. python or ruby "repls". There
| is no "read" step here converting the text of the program to
| data. The program text is simply passed to an "eval" function
| that produces the result directly.
| TazeTSchnitzel wrote:
| This seems extremely pedantic. Python and Ruby will "read" the
| string inside the "eval" function.
| mikelevins wrote:
| Other people are making the same point I'm about to make, but
| I'm going to try to clarify it anyway because, y'know,
| besides being a programmer, I'm also a technical writer, and
| I just have to scratch that itch.
|
| Common Lisp source code (and the source code of its immediate
| ancestors) is not made of text strings. It's made of
| S-expressions, which are made of cons cells, symbols,
| numbers, and so on.
|
| A text file of "Lisp source code" does not actually contain
| Lisp source code. It contains a text-based serialization of
| Lisp source code. Other serializations are possible (and
| there are things you can do in a Common Lisp repl to see some
| of them).
|
| The "read" in "read-eval-print" means "deserialize the text
| into the source data that it's meant to represent".
|
| This point is not trivial pedantry because the full power of
| the Lisp language is available to the read process, and can
| be brought to bear on how reading is done and what happens
| when you do it. Compilers for other languages certainly do
| read text strings and convert them into tree structures and
| so forth, but the difference is that those data structures
| are private to the compiler; the data structures that Lisp
| reads into are standard parts of Lisp's public API, as are
| the read function, the compile function, the eval function,
| the print function, and so on. It's all on the table for you
| to work with.
|
| The same is true of the disposition of the s-expressions
| produced by the read process; you have an opportunity to
| bring the whole of the Lisp language to bear on those
| s-expressions before they are ever passed to (compile or)
| eval. Then, once again, what eval produces is S-expressions,
| and those, not strings, are passed to the print function. You
| once again have the opportunity to intervene in the process
| that produces the text serialization.
|
| It so happens that I've spent the past six months working on
| an AI machine-control system written in Common Lisp, and
| every one of these capabilities was an important part of the
| work we were doing.
| lispm wrote:
| In Lisp it makes a difference, because source code is data
| (other than text) and one can also compute source code in the
| REPL.
|
| For example we can write a macro in Lisp and play around with
| it giving it code as data and see the result as code as data.
| CL-USER 1 > (defmacro while (condition &body body)
| `(tagbody start (if (not
| ,condition) (go end)) ,@body
| (go start) end))
| WHILE CL-USER 2 > (setf a 1) 1
| CL-USER 3 > '(while (< a 4) (print a) (incf a))
| (WHILE (< A 4) (PRINT A) (INCF A)) CL-USER 4 >
| (macroexpand-1 *) (TAGBODY START (IF (NOT (< A 4))
| (GO END)) (PRINT A) (INCF A) (GO START) END) T
| CL-USER 5 > (pprint *) (TAGBODY START
| (IF (NOT (< A 4)) (GO END)) (PRINT A)
| (INCF A) (GO START) END)
| CL-USER 6 > (eval ***) 1 2 3
| NIL
| kitd wrote:
| All very nice, but in Python/Ruby/etc, the source code
| isn't data. So your point is only relevant to Lisp, not
| REPLs generally.
| lispm wrote:
| That's why it is not a REPL (Read Eval Print Loop), but a
| ReadString, Parse, Compile, Execute, Loop.
| [deleted]
| mikewarot wrote:
| I had no idea this was possible now, and to find it was
| commonplace for decades makes it even more amazing, and worrying
| what else I've missed. Thanks for sharing this here!
| pjmlp wrote:
| Be prepared to be amazed with what Xerox PARC and others were
| doing, while Bell Labs was busy pushing for UNIX.
|
| "Eric Bier Demonstrates Cedar"
|
| https://www.youtube.com/watch?v=z_dt7NG38V4
|
| "Emulating a Xerox Star (8010) Information System Running the
| Xerox Development Environment (XDE) 5.0"
|
| https://www.youtube.com/watch?v=HP4hRUEIuxo
|
| "Documents as User Interfaces Video Demo"
|
| https://www.youtube.com/watch?v=0-_zVkrWCOk
|
| "SYMBOLICS S-PACKAGES 3D GRAPHICS AND ANIMATION DEMO"
|
| https://www.youtube.com/watch?v=gV5obrYaogU
|
| "Alto System Project: Dan Ingalls demonstrates Smalltalk"
|
| https://www.youtube.com/watch?v=uknEhXyZgsg
|
| "Action!, the worlds first dynamic interface builder - 1988"
| (Interface Builder percursor, written in Lisp)
|
| https://vimeo.com/62618532
|
| "The Interlisp programming environment"
|
| http://larry.masinter.net/interlisp-ieee.pdf
|
| And the cherry, how Lucid used Lisp ideas for their Energize
| C++ IDE, including an image based format for AST storage
|
| https://www.youtube.com/watch?v=pQQTScuApWk
|
| https://www.dreamsongs.com/Cadillac.html
| ncfausti wrote:
| For people who use Clojure (or another REPL-capable language),
| would you say that this is the main reason why you use it?
|
| Is it common to write Clojure or other REPL-capable languages in
| a more "traditional" manner (like how one would write say, Java
| or C)?
| tincholio wrote:
| I'm not sure I'd say the REPL is "the main" reason I use
| Clojure (there are many reasons, really), but it is an
| important part of the experience.
|
| I don't know of anyone who is experienced in Clojure and works
| in a "write-compile-test" way like you'd do in C or Java. While
| it's certainly feasible, it's not how you're supposed to do it.
| psykotic wrote:
| He's right to emphasize the properties of the systems that
| support interactive development with a long-running image. In
| Common Lisp, the difference between defvar and defparameter is a
| simple example. Traditional Smalltalk systems only supported
| image-based development. But I always preferred the moderate
| approach exemplified by most Lisp systems where the source code
| isn't overly entangled with the image state.
|
| As a long-time but now lapsed Lisper, I never understood some
| people's elevation of the REPL. A command-line REPL is a poor
| man's development environment. If you're doing interactive
| development in Lisp, you'll be far more productive if you use a
| normal buffer with eval-defun, eval-buffer and friends. When
| debugging, you'll primarily want auto-updating watch expressions
| and object inspectors. And when you do have good use for a REPL,
| you'll still want it to exist within a proper buffer with
| persistent history, inline object inspection, etc, instead of the
| low-effort rlwrap terminal experience which usually passes for a
| REPL.
|
| All of this holds doubly true for Smalltalk environments.
| mumblemumble wrote:
| Why not all of it?
|
| Long ago, I was doing Objective-C development with an embedded
| F-script REPL, hot code swapping, and XCode's built-in
| debugger. It was very nearly as nice as Smalltalk in many ways,
| only without the whole IBD situation.
| moocowtruck wrote:
| that doesn't sound nearly as nice as smalltalk
| lispm wrote:
| The mindset is slightly different. From a buffer one can
| interact with the system and it may show things inline.
|
| The REPL, often called a 'Listener' in Lisp is an explicit
| dialog with the system, where the dialog is visible and parts
| of the dialog can be reused and inspected. Also the dialog can
| be suspended temporarily and we interact with the running code
| itself in break loops until we resume the original dialog in
| some way.
|
| This usually is the horror for people wanting predictable code
| - where in an interactive Lisp, one can change code
| interactively at runtime.
| mikelevins wrote:
| You misunderstand me. By "repl-drive programming" I do not mean
| programming that is fixated on the "command-line repl". On the
| contrary, what I'm talking about has much more in common with
| what you describe as "far more productive". It's not about a
| particular shell or window or buffer; it's about a runtime
| environment that is designed to support writing software by
| building and changing it as it runs.
|
| By "repl", I do not mean the window or the buffer or the shell
| program. I mean the loop of: read an expression, evaluate it,
| and present the results, in the context of a runtime that is
| designed to comprehensively support it.
|
| Moreover, you can have all of the tools that you listed and
| still not be working with a properly-designed repl-driven
| environment. For example, I built a 3D interactive environment
| on the JVM that worked quite well--it launched into the 3D
| environment ns no more than a second or two, and could build
| whole procedurally-generated scenes in a few seconds. It
| supported networked multiuser interactions. I could start it
| from a repl and dynamically alter scenes and objects and their
| behavior in the environment by talking to them.
|
| It still wasn't a proper repl-driven environment because the
| underlying runtime could not correctly handle dynamic
| redefinition of classes and methods. That meant that if I
| decided that I needed to change a representation or something,
| I had to kill the environment and rebuild it. It meant that
| there was always a gratuitous barrier that I might run into at
| any moment. It meant that some abitrary set of things I was
| working on was always on the other side of that barrier.
|
| Contrast that to working with, for example, SK8, where I could
| redefine absolutely everything in the environment (including,
| for example, the system-level procedures used to draw window
| frames) without ever restarting the code that was under
| development.
___________________________________________________________________
(page generated 2021-01-03 23:01 UTC)