[HN Gopher] Interactive Code Cells
___________________________________________________________________
Interactive Code Cells
Author : nalgeon
Score : 141 points
Date : 2023-12-18 11:00 UTC (12 hours ago)
(HTM) web link (antonz.org)
(TXT) w3m dump (antonz.org)
| samsquire wrote:
| I have dreamed about an extremely rich document which actually
| presents the internal state of the runtime and compiler. So all
| the abstract syntax tree and control flow graph, SSA and data
| structures are presented in different sections of the document.
| Have animations of data flow too.
|
| It could double as REPL and you can reach into any part of the
| compiler pipeline and write your own transformations.
|
| Does anybody remember the documentation for Backbone? The
| "annotated source" documentation was side by side the code, it
| was really helpful.
|
| https://backbonejs.org/docs/backbone.html (This is the annotated
| source) Imagine if it was "alive"?
|
| I have practically no exposure to Bazel and Buck but these
| advanced build systems have advanced refresh logic which is feels
| it is an advanced caching.
|
| These systems and Makefiles essentially do refresh
| logic/regeneration from changed sources. Maybe it could be like
| Greenspun's 10th rule, every system eventually builds its own
| caching system and refresh logic and dirty-rechecking. (See
| React's virtual DOM)
|
| [1]: https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule
|
| There's also crossovers with Knuth's literate programming here.
| And spreadsheets.
| remexre wrote:
| You might be interested in CodeProber:
| https://www.youtube.com/watch?v=lkTJ4VL0xtY
| GaryNumanVevo wrote:
| Pluto.jl [1] is an interesting project around creating
| reproducible notebooks. MIT's Intro to Computational Thinking
| uses it for displaying interactive code cells [2].
|
| They're reactive cells too, so if you modify an variable or
| function, subsequent cells will automatically update. Something
| that Jupyter lacks currently.
|
| [1] - https://plutojl.org/
|
| [2] - https://computationalthinking.mit.edu/Fall23/
| akshayka wrote:
| I develop a similar project for Python, called marimo [1] [2]:
| marimo is an open-source reactive notebook for Python with
| built-in UI elements.
|
| Our users use marimo to do traditional notebook work faster and
| with more confidence (computational experiments, data
| exploration), and also to build pipelines and apps: marimo
| notebooks can be run as Python scripts and seamlessly converted
| to apps.
|
| We've designed marimo with devX in mind (pure Python file
| format, git-friendly, black formatting built-in, GitHub Copilot
| built-in, modularity coming soon ...).
|
| [1] - https://github.com/marimo-team/marimo
|
| [2] - https://docs.marimo.io/
| bwanab wrote:
| Not only that - if you save the content of a Pluto notebook,
| you've got a julia program that will run in any context that
| julia programs run.
| michelpp wrote:
| Can Pluto do the same Postgres example given? I googled for it
| but didn't see anything immediately stand out.
| adamhp wrote:
| I ran the insert statement before the create statement and it
| appeared to work.
| nalgeon wrote:
| Exactly. Because of this:
|
| > The last thing I want as a reader is code examples that fail
| or behave oddly because they were run out of order. Or because
| I didn't run some cells. Or because I changed a cell and didn't
| re-run it.
|
| > Codapi code cells, unlike Jupyter's, have no hidden state.
| Instead, they execute the whole chain of dependencies as needed
| to ensure that the reader gets a consistent result.
| pimlottc wrote:
| It's confusing because while it does run the previous cells as
| needed, it does not update the "output" of those cells on the
| page. So it's not very clear what the dependencies are and what
| steps were actually executed.
| revenga99 wrote:
| This is amazing! Would be cool to add some data visualization
| examples. Either some python or a js library.
| tpoacher wrote:
| This is nice. Reminds me of the old Iodide notebooks; I'm so sad
| those lost to jupyter, in my mind they were so much nicer...
| nalgeon wrote:
| Yeah, one of the many abandoned Mozilla projects.
| nocsi wrote:
| I prefer functional programming with Livebook[1] for this type of
| thing. Once you run a cell, it can be published right into a web
| component as well.
|
| [1] - https://livebook.dev
| Beefin wrote:
| are these typically just data queries? how do you get past
| arbitrary code vulnerabilities?
|
| this is something i'm exploring today with: https://nux.ai/
|
| which has a similar experience (run code within the browser)
| nalgeon wrote:
| Any code, really. Runs in (highly restricted) Docker
| containers.
| cxr wrote:
| > Behind the scenes, codapi-snippet calls a codapi server (either
| a cloud or self-hosted instance) so it can run any programming
| language, database or software you've configured.
|
| I'd rather have a flavor of notebooks that work by referencing
| other notebooks comprising human- and machine-readable
| specifications that describe those other languages' syntax and
| semantics and how their execution models work. Look at the way
| IETF RFCs are crosslinked, for example. With your browser pointed
| at one of these documents, it traverses all the linked documents
| and resolves a given cell's results based its working knowledge
| of the language in that cell. (If you really want to, you can
| point your server to the same corpus and have it cache some of
| the processing in order to relieve pressure re compute
| requirements for the browser, but in principle it should work the
| same way.)
| zekenie wrote:
| this is so interesting! does it memoize dependencies? if i have
| several cells that all depend on a central one, will the central
| one only run once? what about state? can you declare a variable
| in one cell then use it in another?
| nalgeon wrote:
| Let's go through these one by one :)
|
| > does it memoize dependencies?
|
| Not sure what do you mean by memoization here, but there is no
| stored state at all (see the original post for the reasoning
| behind this).
|
| > if i have several cells that all depend on a central one,
| will the central one only run once?
|
| Sure!
|
| > can you declare a variable in one cell then use it in
| another?
|
| You can, as long as it's in global scope.
| zekenie wrote:
| what i'm really wondering is: could you build a stateful repl
| with this?
| tkiolp4 wrote:
| Off topic: does any other here have trouble with queries like
| this? Rank the employees according to their
| salaries in each department
|
| My SQL is "good" I think, but to be honest in my day to day I
| usually don't deal with aggregation nor windowed queries (my BI
| colleagues are masters are those, though )
| nalgeon wrote:
| You'd be surprised how useful they are once you've grokked
| them. I have a 4-part interactive tutorial on SQL window
| functions, if you are interested:
|
| https://antonz.org/sql-window-functions-ranking
| enoch2090 wrote:
| Nice concept and demonstration! Would argue though that it is the
| states in Jupyter notebooks that make it work so well for
| prototyping, and that should be its main purpose always. Using
| dependencies might break that.
|
| As for the documentation purposes it should be the
| programmers/authors' duty to ensure that the code works out by
| executing all cells in order once.
| jbverschoor wrote:
| Love that it has transactions/states between blocks. It's super
| annoying if you override a variable / change an object in Jupyter
| and try to re-run it.
| skybrian wrote:
| The main problem I see is relying on a server. Running sandboxes
| server-side takes up resources even when idle (the user went to
| lunch, or left it running overnight). Google can do it with
| Colab, but this is expensive to provide to the general public, so
| often the sandbox gets killed after a timeout.
|
| Observable does a similar thing, but in the browser, so the cells
| run JavaScript. I suppose a WASM implementation could provide
| more languages?
|
| Or it might be better done using a VS Code plugin or something
| like that. (Thry already exist.) But that means casual readers
| won't be able to play with it.
| nalgeon wrote:
| In this case, running sandboxes does not consume resources when
| idle. There is no state, and the containers exist only for the
| duration of the "run" operation.
| skybrian wrote:
| There's a database to pass from one cell to the next, which
| seems rather stateful to me?
|
| If no state is cached and running a cell automatically reruns
| all its dependencies from scratch, that's the equivalent of a
| clean rebuild when using a makefile. It might be too slow to
| use interactively if you're doing anything heavy. It's how
| continuous builds work, though.
| michelpp wrote:
| What piques my interest in this tool is that I haven't found a
| good "notebook" like solution for Postgres yet, I've tried using
| SQL magic functions in Jupiter to generate some documentation for
| pgsodium[1] but I'm not entirely pleased with the results,
| there's too much Python still showing through.
|
| Would this tool work as a general purpose documentation generator
| for a Postgres extension?
|
| [1]
| https://michelp.github.io/pgsodium/Public_Key_Cryptography.h...
| nalgeon wrote:
| I think it would! Are you interested in trying it out? I can
| probably prepare a simple example and open an issue in the
| pgsodium repo to discuss the rest.
| daemonk wrote:
| While I get that states is frustrating for share-ability and
| communicating your work, it is extremely useful for prototyping.
| Perhaps there just needs to be an explicit toggle for stateful vs
| non-stateful for notebooks
| TheIronYuppie wrote:
| This gets close to something really important, but I'd like to
| forward something that I wrote previously - the SAME project[1]
|
| The difference/compliment here is that after you finish writing
| these cells, i think there's a translation into a backend service
| for execution at scale.
|
| I'd love someone to pick this up and run with it in collaboration
| with tools like this!
|
| [1] https://github.com/SAME-Project/same-project
| mwenge wrote:
| In the same vein: https://mwenge.github.io/blog/io10.html
| lbeckman314 wrote:
| I love these kinds of interactive code/sandbox projects! The
| author's example page [1] gives some more background on the
| design of Codapi as well [2]. Excited to try this out!
|
| Another similar project is Runno which runs client-side in the
| browser [3].
|
| [1] https://antonz.org/code-examples/
|
| [2] https://codapi.org/
|
| [3] https://runno.dev/
___________________________________________________________________
(page generated 2023-12-18 23:01 UTC)