[HN Gopher] A Python-first data lakehouse
___________________________________________________________________
A Python-first data lakehouse
Author : akshayka
Score : 89 points
Date : 2025-06-17 22:22 UTC (3 days ago)
(HTM) web link (www.bauplanlabs.com)
(TXT) w3m dump (www.bauplanlabs.com)
| flakiness wrote:
| There have been so many "better notebook" implementations over
| the years that I cannot catch up. What are the promising one? Is
| this "marimo" one of them or rather a newcomer?
| simonw wrote:
| Marimo is very impressive. It's effectively a cross between
| Jupyter and https://observablehq.com/ - it adds "reactivity",
| which solves the issue where Jupyter cells can be run in any
| order which can make the behavior of a notebook unpredictable,
| whereas in Marimo (and Observable) updating a cell
| automatically triggers other dependent cells to re-execute,
| similar to a spreadsheet.
|
| Marimo is pretty new (first release January 2025) but has a
| high rate of improvement. It's particularly good for
| WebAssembly stuff - that's been one of their key features
| almost from the start.
|
| My notes on it so far are here:
| https://simonwillison.net/tags/marimo/
| lvl155 wrote:
| I think it's safe to say Observable's inability to properly
| price their services made people look elsewhere. Their new
| offering is interesting but also ridiculously priced.
| ayhanfuat wrote:
| I was also wondering their pricing because Canvas seemed so
| cool at first. Now that I've seen your comment I checked
| and $900/month (includes 10 users) is indeed very high. I
| guess they are primarily targeting big enterprises.
| akshayka wrote:
| Thanks Simon for the kind words!
|
| For those new to marimo, we have affordances for working with
| expensive (ML/AI/pyspark) notebooks too, including lazy
| execution that gives you guarantees on state without running
| automatically.
|
| One small note: marimo was actually first launched publicly
| (on HN) in January 2024 [1]. Our first open-source release
| was in 2023 (a quiet soft launch). And we've been in
| development since 2022, in close consultation with Stanford
| scientists. We're used pretty broadly today :)
|
| [1] https://news.ycombinator.com/item?id=38971966
| Peritract wrote:
| > it adds "reactivity", which solves the issue where Jupyter
| cells can be run in any order
|
| This is one of the key _features_ of Jupyter to me; it
| encourages quick experimentation.
| sodality2 wrote:
| Once you get to a certain complexity of notebooks, I find
| it only serves to complicate my mental model to
| "experiment" out of order. It makes me far more likely to
| forget to "commit" an ordering change.
| abdullahkhalids wrote:
| Jupyter notebooks do store the execution order of the
| cells. Just enforce a pre-commit or pre-merge hook that
| doesn't allow adding notebooks that have out-of-order
| cells.
| akshayka wrote:
| marimo still allows you to run cells one at a time (and has
| many built-in UI elements for very rapid experimentation).
| But the distinction is that in marimo, running a cell runs
| the subtree rooted at it (or if you have enabled lazy
| execution, marks its descendants as stale), keeping code
| and outputs consistent while also facilitating very rapid
| experimentation. The subtree is determined by statically
| parsing code into a dependency graph on cells.
| theLiminator wrote:
| I personally really like marimo. It's very easy to use and for
| data analysis type tasks it seems to work a lot better than
| jupyter in most cases.
| cantdutchthis wrote:
| marimo is open source and uses a reactive model which makes it
| fun to mix/match widgets with Python code. It even supports
| gamepads if you wanted to go nuts!
|
| https://youtu.be/4fXLB5_F2rg?si=jeUj77Cte3TkQ1j-
|
| disclaimer: I work for marimo and I made that video, but the
| gamepad support is awesome and really shows the flexibility
| Snakes3727 wrote:
| One of the most critical aspects a Lakehouse is protecting data
| for security and compliance reasons and this article completely
| just glosses over it which makes me really uncomfortable.
| jtagliabuetooso wrote:
| Thanks for the feedback. Bauplan actually features a few
| innovative points in this area, and full Pythonic at that: Git
| for Data (https://docs.bauplanlabs.com/en/latest/concepts/git_f
| or_data...) to sandbox any data change, tag it for compliance
| and make it querable; full code and data auditability in one
| command (AFAIK, the only platform offering this), as every
| change is automatically versioned and tagged with the exact run
| and code that produced it (https://docs.bauplanlabs.com/en/late
| st/concepts/git_for_data...).
|
| Our sandbox with public data is free for you to try, or just
| reach out and ask any question!
| zelphirkalt wrote:
| When I first quickly glanced at this heading, I read
| "Leakhouse" instead of "Lakehouse" :D And then I saw your
| comment...
| jtagliabuetooso wrote:
| Hey, founder of Bauplan here. Happy to field any questions or
| thoughts. Yes, marimo is great, and it's the only way to work
| within a real Python ecosystem for production use cases shipping
| proper code.
| waffletower wrote:
| Rolling a notebook out to a service rapidly is an attractive idea
| -- but, as mentioned, has security implications -- I can add that
| there are also a host of monitoring implications as well --
| service quality & continuity, model quality etc.
| jtagliabuetooso wrote:
| You mean on the data side? Data access in the example (and in
| real-world) is mediated by production-grade Iceberg compatible
| catalog, sandboxed changes, and full auditability trail (https:
| //docs.bauplanlabs.com/en/latest/concepts/git_for_data...). Or
| do you mean something else?
| waffletower wrote:
| I don't think python is always the best suited language for
| managing models and agents, but it certainly is the most popular
| and has the largest choice of related libraries. "Python first"
| or "pythonic" invites skepticism from me.
| davistreybig wrote:
| Huge fan of Marimo - fixes so many of the annoying problems w/
| notebooks
| blooalien wrote:
| I find Marimo best for when you're trying to build something
| "app-like"; an interactive tool to perform a specific task. I
| find Jupyter lab more appropriate for random experimentation
| and exploration, and documenting your learnings. Each
| absolutely has it's place in the toolbox, and does it's thing
| well, but for me at least, there's not much overlap between the
| two other than the cell-based notebook-like similarity. That
| similarity works well for me when migrating from exploration
| mode to app design mode. The familiar interface makes it easy
| for me to take ideas from Jupyter into Marimo to build out a
| proper application.
| marcoalopez wrote:
| This is exactly my impression.
| akshayka wrote:
| Thanks for the kind words. Many of our users have switched
| entirely from Jupyter to marimo for experimentation
| (including the scientists at Stanford's SLAC alongside whom
| marimo was originally designed).
|
| I have spent a lot of time in Jupyter notebooks for
| experimentation and research in a past life, and marimo's
| reactivity, built-in affordances for working with data (table
| viewer, database connections, and other interactive
| elements), lazy execution, and persistent caching make me far
| more productive when working with data, regardless of whether
| I am making an app-like thing.
|
| But as the original developer of marimo I am obviously biased
| :) Thanks for using marimo!
| debarshri wrote:
| If you click on See certifications in Security section[1]. It
| resolves to an empty section.
|
| [1]
| https://security.bauplanlabs.com/#resources-b2152df0-4179-48...
| markhahn wrote:
| I am strangely unmoved by some new SaaS which is not open-source
| and self-hostable.
___________________________________________________________________
(page generated 2025-06-20 23:00 UTC)