[HN Gopher] Electric Clojure - A signals DSL for full-stack web UI
___________________________________________________________________
Electric Clojure - A signals DSL for full-stack web UI
Author : Borkdude
Score : 271 points
Date : 2023-02-13 09:49 UTC (13 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| patates wrote:
| > There is no client/server dichotomy from the programmer's
| perspective
|
| But on the first example, I see a lot of e/client and e/server
| calls. Am I misunderstanding this? If so, how is this different
| from meteor.js?
| popinman322 wrote:
| Seems like e/client and e/server are for when you'd like to
| force sections of code to run on one side or the other.
| Otherwise the compiler decides what lives where?
| kimi wrote:
| Exactly. And when one side needs the other, it makes magic
| happen.
| patates wrote:
| Well then 2 humble suggestions:
|
| - Would be better to show an example where magic would
| happen
|
| - Selectable text instead of a screenshot would have been
| nicer
| kimi wrote:
| Magic does happen: you are rendering a table on the
| client, pause to fetch data on the server, and display
| it. And it looks like a PHP page written in Clojure.
| gfodor wrote:
| This looks cool. If I think one step ahead, this reduces enough
| incidental complexity its the kind of thing that, when combined
| with AI, could produce the holy grail of an AI writing most of
| the code for a full CRUD app.
| dustingetz wrote:
| yup! For a sneak peak -
| https://hyperfiddle.notion.site/Hyperfiddle-progress-report-...
| tr14 wrote:
| I respect an engineers that can code Clojure. Nice job.
| eduction wrote:
| "The Electric compiler performs deep graph analysis of your
| unified frontend/backend program to automatically determine the
| optimal network cut, and then compile it into separate client and
| server target programs that cooperate and anticipate each other's
| needs."
|
| This make it sound like client and server segments are determined
| on the fly based on prevailing conditions, but then the sample
| code has forms wrapped with e/client and e/server. What do you
| mean when you say you determine "the optimal network cut" and how
| do you reconcile this with the expressly marked client and server
| code?
|
| By the way I've been following this project for a while and am
| very excited about it, kudos on getting to this point.
| dustingetz wrote:
| Great question. A trivial example, expr (e/client (inc
| (e/server (e/client x)))) should simplify to just (p/client
| (inc x)) and eliminate the pointless (and perf-damaging) server
| round trip.
|
| It gets interesting when you introduce lambdas, revealing that
| this optimization is non-local - you don't know if the transfer
| is superfluous until you know where the function is called
| from, and consider the caller and callee together.
|
| The good news is that the DAG (Circuit really) contains
| everything there is to know about the data flow, so _insert
| compilers research here_ and poof, insanely fast app.
| den1k wrote:
| I've been building with electric for several months now and it's
| hands down incredible! Why? I think most importantly because it
| allows for functional composition across the network
| (client/server-distributed systems more generally) while also
| ensuring payloads are as small as possible (only what functions
| in each env access) which translates to FAST.
|
| IMO this is a paradigm shift similar to GC. No one wants to
| manage their garbage. No one wants to manage the network. It's
| grunt work. Let the compiler take care of it.
|
| I've naively rewritten a re-frame (react/redux in js) app that
| started getting prohibitively slow due to a bunch of
| deserialization on the client in electric and the performance is
| outstanding! TBH I'm taking a break from additional UI work on
| the app to do research because I realized that with electric I
| can build something much more amazing than I thought possible
| before.
|
| Caveat: the future will not be served on a silver platter. it's
| early days and there are some quirks
| yenda wrote:
| Could you share which kind of quirks you've identified?
| den1k wrote:
| Small bugs overall, some of which the team already addressed.
| One Clojure specific that is outstanding is that dynamic
| variables are not rebound between electric and Clojure
| functions. Also, the compiler currently does not optimize the
| callstack so the whole app stream-renders in and out which
| has a cascading effect. None of these are deal breakers for
| me, they're being worked on and I'm glad the community can
| now help ship fixes and improvements.
| dustingetz wrote:
| (founder here) Thank you Dennis! All, broadly, to date
| we've focused on achieving correct program semantics over
| syntax/perf/ux/dx. We're rotating onto these issues now,
| since we've reached a point where, after 2 years of
| hardcore dogfooding and 8 months of private beta, we're
| comfortable that brave Clojurists like Dennis will be
| successful and have an acceptable development experience.
| (Especially when compared with the benefit of strong
| composition at the system level - functional programming
| over entire systems!)
| beders wrote:
| Can't wait to try this.
|
| Congratulations on the release! A great showcase for the
| flexibility of Lisp.
|
| Others invent whole programming languages around a paradigm,
| while Lisp just marches on with s-expressions. Well done!
| samsquire wrote:
| This is similar to my idea "structured logging 2 system"
|
| The idea is that we write what should be done and it is scheduled
| onto different systems and communication is inserted.
|
| https://github.com/samsquire/structured-logging-2-system
|
| I really like the idea of being capable of switching where
| something runs on the server or the client by changing a
| boundary.
| yladiz wrote:
| Not to be a downer, but while I think in theory this sounds
| interesting, and treating frontend and backend differences as a
| DAG that is compiled for the different environments seems cool,
| I'm skeptical that it would actually be productive in a large
| application.
|
| One thing I'd be particularly concerned about is debugging. One
| of the things that's on the roadmap is "developer experience",
| and I don't see any particular debug docs or helpers, so it tells
| me that if there is a bug somewhere debugging it is going to be a
| pain, even more so because I won't easily be able to see whether
| the problem is my client or server code as they live together and
| are compiled into separate things. As a critique the fact that
| developer experience is on the roadmap rather than thought about
| from the beginning is a little off-putting, but maybe that's not
| what was meant here.
|
| Another thing that gives me hesitation is that I can't see how I
| would extend the setup to support offline capable applications.
| This is increasingly becoming a big thing for web+desktop apps,
| does this easily support that?
|
| Lastly, the docs seem to point to use a homegrown DOM library,
| rather than one of the major players. How easily would this work
| with a library like Reagent, Helix, or UIx?
| den1k wrote:
| I've been building with this for 2+ months. Debugging is fine
| (not great but when is it great? Especially in Clojure). It
| prints stacktraces in each env. Haven't found debugging harder
| than traditional client/server dev. But complexity has
| decreased thanks to it so less bugs.
| dustingetz wrote:
| Re. abstraction weight - You're right, it's true that the
| program is less inspectable than before, and that it adds more
| distance between programmer and metal. We're aware of the issue
| and have already begun - we landed async stack traces a while
| ago (which works about the same as in React.js).
|
| And yet. It's also true that HLL (high level languages) like C,
| Java, Python all faced the same issue. The benefit is a huge
| gain in terms of ability to reason about ever larger programs.
| Java gave us garbage collection. Not everyone liked that; C++
| developers want to manage their own memory. But at what cost?
|
| Our thinking is: first get the semantics right. Then polish the
| experience (today we are here). Then add observability tools.
| Then make it fast. And if we've screwed up the semantics, then
| the project will fail for the reasons you describe.
|
| The saving grace is we're seeing 10x LOC reduction (18k to 2k)
| in rebuilding Electric's sister project, Hyperfiddle (a
| spreadsheet like tool for robust UI development), as well as
| massive gains in performance.
|
| Re. offline-first - we think the model can be extended to that
| but this is a blog post for another day
| dustingetz wrote:
| One quick sketch of offline-first (there are many other
| ways):
|
| imagine a database like DataScript (already OSS and built in
| Clojure), move key components like the query engine into
| Electric Clojure, now the "DAG slice" can cut right through
| the query engine and it's internal data structures, such that
| the indexes are partially on the client, and the final stages
| of query filtering are on the client. A network-transparent
| database! The bulk of the work, and sensitive data, stays on
| the server; but the last mile user data querying happens on
| the client. Electric can transparently move portions of the
| underlying database indexes to the client for local offline
| query.
|
| 0 LOC cost to userland! No loss of relational query when
| offline! No performance cost of putting the entire query
| engine in the browser (a low compute environment)!
| dustingetz wrote:
| (Founder here; I'll address the debugging concern in a separate
| comment)
|
| Here's a demo of React.js interop via Reagent, the wrapper is
| 25 LOC:
| https://gist.github.com/dustingetz/9854d23037b55bfab3845539f...
|
| Our DOM module is only 300 LOC - it's bare metal DOM point
| writes + Electric (reactive language) + macros for JSX-y
| syntax. When the programming language itself is reactive, DOM
| rendering falls out for free.
|
| Mechanically, Electric is comparable to Solid.js except the
| reactive engine is general purpose, not coupled to DOM
| rendering, which is a special case of incremental view
| maintenance.
|
| Electric's reactive engine is
| https://github.com/leonoel/missionary ; Electric can be seen as
| a Clojure to Missionary compiler (if you ignore the
| distribution stuff - which btw is optional). Missionary is
| cross-platform functional effect and streaming system that
| today is tested on JVM, Node and Browser platforms. Missionary
| was created by Leo Noel, who is also the architect of Electric.
| yenda wrote:
| Regarding offline I tested the photon-starter-app repo a bit
| and it looks like it already supports disconnections but
| without optimistic updates:
|
| Here's what I did - killed the network - added a bunch of items
| to the todo list, which disappeared from the input field but
| didn't appear in the list - switched network back on, items
| appeared in the list
|
| I'm curious to see how adding optimistic updates to the mix
| would work
| dustingetz wrote:
| (founder here) Should be straightforward; as you discovered,
| everything is properly backpressured by the compiler
| (automatically!) so no state is lost.
| scary-size wrote:
| Agreeing on the debugging concerns. Adding another layer of
| abstraction on top of everything else isn't going to help
| anyone. Creating an abstraction that fits so well that you
| don't have to worry about the layer underneath is extremely
| hard. That might not show during initial development, but will
| come up in the maintenance phase for sure.
|
| A bit unsure where the observation of more offline capabilities
| comes from. I have the impression that things are moving in the
| opposite direction: Everything needs to be online.
| ElectricalUnion wrote:
| Everything needs to be online, but "online" those days
| probably means cutting off constantly between 5GHz and 2.4GHz
| wifi, with bad ISP DNS and IPv6 routing in the way.
| ldh0011 wrote:
| I think they mean apps that don't just stop working when
| you're offline, not apps that are always offline.
| yladiz wrote:
| Yes, that's what I meant. I didn't mean an app that's never
| online (or connected to a server), since that doesn't
| really make sense in the context of this kind of framework,
| I meant something that is resilient to being offline (aka
| "offline capable").
| yenda wrote:
| based on the todomvc example, I can say that it is
| resilient to being offline, in the sense that whatever
| you do while temporarily offline is resumed when you are
| back online. there's no optimistic updates so it looks
| like the items you add while offline are disappearing,
| but they end up in the list as soon as you are back
| online.
| mst wrote:
| > As a critique the fact that developer experience is on the
| roadmap rather than thought about from the beginning is a
| little off-putting
|
| Implementing it is on the roadmap but that's no evidence that
| it hasn't been thought about extensively and for something like
| this you really -have- to think things out up front (it's more
| like a math proof than a CRUD app in a lot of ways) or you're
| going to find yourself with holes in both feet and a codebase
| you have to throw away and restart from scratch.
|
| From what I can see, they've been laser focused on getting the
| core to be correct by design, which will have required
| repeatedly gutting parts of it as conceptual mistakes are
| shaken out, so if you -wrote- the DX code as you were going
| rather than simply keeping in mind that the codebase needs to
| be amenable to it -being- written (and periodically thinking
| through how you'd do so as a sort of mental dry run) I'd expect
| the DX parts to have to be thrown away entirely, repeatedly,
| which is a motivation sink and in the best case scenario
| would've substantially slowed down getting the core right.
|
| But yeah, the key thing is "not yet implemented" and "hasn't
| been thought about at all" are very different things and I'd
| strongly suspect that my read of what was meant is more likely
| to be the right one given the nature of the project.
|
| (please take the above as "an explanation of why I suspect
| their actions are fine and their chosen trade-offs make sense"
| rather than "let me tell you why you're wrong" because you do
| have a point that if you don't keep DX in mind at all in the
| early stages the end result tend to be unhappy-making, and,
| well, let's just say I speak from repeated experience of doing
| that that to myself ;)
| college_physics wrote:
| Practicalities aside (and there might be many issues and even be
| show-stoppers), the approach is a breath of fresh air in what
| appears like a stagnating swamp. How to orchestrate overall
| client/server computations is obviously a vital importance if not
| _the_ most important design choice in the modern tech landscape.
| Yet while strong opinions abound, frameworks that help us to
| think more concretely and rationally about the tradeoffs and
| break the redundancy of infinite alternative possibilities are
| not common.
|
| If I understand the idea correctly effectively here you have a
| platform where alternative choices are simple rearrangements of
| code.
| kimi wrote:
| This is major - potentially killing off a thousand tons of
| boilerplate.
|
| Not sure if it is too late, because Elixir's LiveView seem to
| have gotten important because it works around the same issue.
|
| But it's time to push the wooden stake through the heart of the
| SPA (and the phone app too, but that's for another day).
| incrudible wrote:
| You can already kill the boilerplate with code generation and
| metaprogramming. Once you have done that, you figure out why
| that is not always a good idea and how to best deal with the
| exceptions that _your particular use case_ brings to the table.
| kimi wrote:
| I'm thinking more of all the cruft that goes to handle state
| on the client and keep it synced to the server in your React
| app. Redux and the gazillions of other packages which names
| sound like warthogs having breakfast.
| incrudible wrote:
| The same argument applies. I think Redux is a pointless
| indirection, but you can use metaprogramming to make it
| more palatable, at the cost of another indirection.
| i_cannot_hack wrote:
| I don't know much about this kind of architecture, but wouldn't
| it be problematic from an infosec standpoint? It seems like it
| would be very easy to accidentally leak information from the
| server to the client in a subtle manner if the developer makes a
| mistake somewhere. And in ways which would not be obvious from
| reading the code unless you carefully consider the implications
| of each e/server and e/client with respect to the mentioned graph
| analysis?
|
| Are there any mitigations in place to prevent this, except the
| obvious "be careful and don't write bad code"? Or have I
| misunderstood things completely and it's a non-issue?
| prettyStandard wrote:
| I tried to get hands-on experience with this hyper fiddle stuff
| a few months ago but I think it was still in closed beta.
|
| I would imagine you can inspect the network traffic. I would
| hope it would be readable so that you could have a good idea of
| what's going on and errors would be more obvious.
| moondowner wrote:
| I see a sample todo app in their repo, so yes, one can run it
| and check out how it works
| https://github.com/hyperfiddle/electric-starter-app
| den1k wrote:
| Code is precompiled so you could add all your safeguards during
| compilation. Should not be too hard to enforce fo only
| precompiled code to run instead of exposing eval. However, I
| don't believe this is implemented at this stage.
| chii wrote:
| Accidental code execution vuln might not be the only issue
| though. How do you know that state which is being checked and
| verified on server is not bypassed if the application is
| written without client/server divide? How do you know the
| data isn't being passed back to the client which should've
| always stayed on the server?
| den1k wrote:
| There is a client/server divide as you can see in the code
| examples. But I think the issue you're describing is a
| permissions problem, not a question of whether the compiler
| manages the network for you. There are traditional
| client/server apps with wide open permissions that allow
| all kinds of queries to pass through to a prod DB. Then
| again there are others that are heavily permissioned. Those
| are implementation details. If you _can_ control which code
| can run then you can write that code to _only_ run with
| certain constraints.
| dustingetz wrote:
| (founder here) You're right that accidents can happen, and if
| we need to build in safeguards to prevent accidents (i.e. tag a
| value as server-only) - that should be trivial. To be clear,
| your backend still exists and has the same security model as a
| REST Backend-for-Frontend. The client/server markers denote at
| compile time where a computation (function) must occur. A scope
| binding like `password` will only transfer if the programmer
| asked it to by using it from a client region. That would be
| like putting passwords in a REST payload - it's a programming
| bug and should be caught in review. But you're right, providing
| safeguards is a great idea.
| i_cannot_hack wrote:
| I agree it would be user error, but it seems to be a
| particularly easy mistake to make compared to the REST
| example, where you explicitly create REST endpoint to send a
| value to the client.
|
| If you accidentally move filtering of a list of users from
| server-scope to client-scope during refactoring, everything
| will still work just as you expect it to and you'll be none
| the wiser - but suddenly every user has access to all the
| user data in the list. There's not many other frameworks I'm
| aware of where moving an operation to an adjacent row or
| forgetting to change a word suddenly (and silently) exposes
| the data to the client - except perhaps PHP, which does not
| have a good reputation when it comes to security to say the
| least. Altough to be fair adlpz mentioned something similar
| with Next.js in another thread.
|
| Tagging values as server-only mostly seems like another thing
| that could be easily forgotten. Personally I would feel more
| comfortable if values intended to be sent from the server to
| client had to be explicitly tagged as mutual, with an
| optional compilation flag that enforces this, such that the
| intent to share it with the client must always be stated in
| plain text during creation.
| adlpz wrote:
| Just as a data point, this is also true about very much
| noeadays "vanilla" stacks like Next.js.
|
| It _seems_ like you 're doing SSR but not exactly.
|
| You return the wrong prop in a getServerSideProps and boom,
| your stripe keys are dumped in a nice JSON for all to see.
| reilly3000 wrote:
| Yikes!!!
|
| Is anyone aware of automated testing/scanning for secret
| leakage via js framework magic? If not, how would you
| implement one?
| dustingetz wrote:
| Founder here - woke up to this. Here is some context for HN:
|
| Demo videos: https://hyperfiddle.notion.site/Electric-Clojure-
| progress-De...
|
| Prior HN discussions addressing many of the FAQs:
| https://news.ycombinator.com/item?id=28630209
| https://news.ycombinator.com/item?id=31217448
|
| Some notes on the network layer:
| https://www.reddit.com/r/Clojure/comments/vizdcc/hyperfiddle...
|
| I'll try to address some FAQs in another post
| ordinaryradical wrote:
| This seems similar in approach to Lamdera for Elm. Just
| curious, did you take any inspiration from that project or
| arrive at this solution independently?
| dustingetz wrote:
| (founder here) We got here from first principles by thinking
| deeply about pure functional programming. A lot of what is
| happening in pure FP ecosystems is "http servers but with
| monads" and we think that misses the point.
| nih0 wrote:
| Is this similar too Java/Vaadin, that also allows calling of
| backend database repos and the like from the frontend ?
| den1k wrote:
| Yes, literally anything on the backend.
| la-ruby wrote:
| lately I've been doing https://github.com/hotwired/turbo-
| rails/blob/main/app/models...
___________________________________________________________________
(page generated 2023-02-13 23:01 UTC)