[HN Gopher] The Jank Language: LLVM Hosted Clojure
___________________________________________________________________
The Jank Language: LLVM Hosted Clojure
Author : bobnamob
Score : 286 points
Date : 2022-08-17 07:11 UTC (15 hours ago)
(HTM) web link (jank-lang.org)
(TXT) w3m dump (jank-lang.org)
| synthc wrote:
| Really cool! The repl is great for development, but static
| binaries are great for deployment, this may give us both.
| chrisweekly wrote:
| Yes, naming things is hard, but, "jank"? Yikes.
|
| "Jank" describes undesirable, problematic, glitchy UX. I
| genuinely thought that the title was denigrating Clojure on LLVM.
| Maybe it's intended to be funny or ironic, and if it catches on
| it won't matter, but it sure seems like a footgun.
| Jeaye wrote:
| It is self-deprecating, yeah. But hopefully it's memorable!
| chrisweekly wrote:
| ... which is a shame, bc the linked landing page does a great
| job of explaining what jank-lang is, and it looks really cool.
| jshaqaw wrote:
| Love this. As a Lisper I'm always intrigued by Clojure but as
| someone with no knowLedge of or desire to learn to JVM I stayed
| away since I figured eventually I'd hit JVM questions/issues I
| frankly didn't feel like dealing with.
|
| Is it really 100 percent compatible with Clojure or is that just
| aspirational.
|
| Also, is it the case that lots of libraries/workflows outside the
| core language in Clojure assume Java integration? I always
| thought a big Clojure selling point was the awesomeness of Lisp
| while having full access to the libraries of Java.
| capableweb wrote:
| > Is it really 100 percent compatible with Clojure or is that
| just aspirational.
|
| Aspirational. The project is really early still, and in the
| middle of a grand rewrite so basically it doesn't build at the
| moment, see https://github.com/jeaye/jank/issues/7
|
| Current progress seems to be outlined here: https://jank-
| lang.org/progress/
| vaylian wrote:
| The official Clojure (JVM) project has an official sister
| project: ClojureScript, which is designed to be compiled to
| JavaScript (and maybe Webassembly in the future?). There is a
| lot of code that runs with both languages and therefore I don't
| think you should worry too much about using Jank with existing
| Clojure/ClojureScript libraries.
| mjul wrote:
| Clojure is a language without a platform so all the Clojure
| dialects have the different semantics of the underlying
| platform, be it the JVM or JavaScript.
|
| For example ClojureScript has all the "wat" semantics from
| JavaScript primitive types.
|
| You can build libraries that try to hide the differences but
| the leaky abstraction is inherently still there.
| z9znz wrote:
| In my not so extensive Clojure experience, there's almost
| nothing you need to know about JVM to do real things with
| Clojure. If you're building something big/complex enough that
| you need to know something about the JVM, then you probably
| already discovered what you needed along the way.
|
| Basic Java interop is pretty simple (and concise!). But unless
| you're specifically needing a Java library, you won't even do
| this much. You _can_, but it tends to be more for performance
| optimization in my experience.
|
| In other words, don't be scared of JVM. I would say it's kind
| of like Elixir/Erlang. You can do real things with Elixir
| without knowing hardly anything about Erlang. Of course, later
| on you can do more awesome things if you know how and when to
| leverage the Erlang VM. Same probably goes for Clojure/JVM.
| ithrow wrote:
| The only reason Clojure is viable for production use is because
| it relies on the really good JVM GCs. How will Jank try to solve
| the the GC problem?
| abc_lisper wrote:
| How is that. I thought GC pressure was less of a issue after
| transducers came into being. Are there benchmark/measurements
| on memory churn?
| eatonphil wrote:
| Python, Ruby, Go all seem to do ok with much worse GCs than
| Java.
| reitzensteinm wrote:
| The difference is Clojure's persistent data structures. You
| don't add an element to an array, you create a new copy of
| the array with the element attached, and the old array is
| accessible.
|
| Internally this is done with structural sharing, so it's way
| faster than copy on write, but it's still potentially
| allocating a half dozen objects for what would usually be a
| trivial operation that does not involve allocation.
|
| On the plus side, Clojure in theory has some pretty nice
| guarantees that older objects can't point to newer objects.
| From memory Haskell takes advantage of these guarantees to
| accelerate its version of eden. There may be a way to get a
| fast GC that's not nearly as complex as the full JVM GC.
| eatonphil wrote:
| Does Clojure really compete closely in overall performance
| with other immutable-first languages like OCaml/Standard
| ML?
| eatonphil wrote:
| This is all a bit of a distraction though.
|
| > The only reason Clojure is viable for production use is
| because it relies on the really good JVM GCs. How will
| Jank try to solve the the GC problem?
|
| I think this original statement is wrong because people
| are fine with poor performance (see Python, Ruby) if the
| language (ecosystem) is effective. And Clojure is
| effective (IMO) moreso because of the JVM ecosystem not
| the JVM's performance.
| reitzensteinm wrote:
| I think you're underestimating how slow a naive
| implementation would be. The only Clojure implementations
| that have gained traction are built on top of both great
| JITs and GCs.
| slowmovintarget wrote:
| The question is answered in Clojurian Slack:
|
| "Reference counting is the current design. This is something
| I'll be digging more into, but I really don't think a GC would
| be ideal."
| [deleted]
| drunkenmagician wrote:
| 1+ hoping this project gets some momentum. I'd love to see a
| 'native' Clojure, something that operated more like SBCL would be
| amazing.
| simongray wrote:
| FYI but you can already natively compile Clojure code using
| GraalVM Native Image, provided it doesn't rely too much on
| reflection AFAIK.
| fredrikholm wrote:
| Right? Clojure has so many good things going for it already
| that a step towards CL (native, image-based) would be the
| dream.
| brabel wrote:
| Why not use something like
| https://github.com/ruricolist/cloture (a Clojure "adapter"
| that runs on Common Lisp) if you want that?
| dunefox wrote:
| That seems very abandoned.
| vindarel wrote:
| How so? Last commit: June of this year. State: "very
| early (pre-alpha) stages, but it has progressed far
| enough to load clojure.test, allowing the test suite to
| actually be written in Clojure." and the author is still
| a very active lisper. So, pretty nice for an experiment,
| I guess.
| dunefox wrote:
| Yes, that commit changed the readme. The commit before
| that was in January of 2021, the one before that in March
| 2020. Doesn't look very active to me.
| Georgelemental wrote:
| The name seems unfortunate... unless it is meant in jest?
| perrygeo wrote:
| See also babashka, a native Clojure implementation for scripting.
| https://github.com/babashka/babashka
| cube2222 wrote:
| An important difference is that afaik Babashka is an
| interpreter, so will be fairly slow. It's fast at _starting_.
|
| While this is LLVM JIT, so will compile the Clojure code to
| native code just-in-time, and then run the native code.
| joshlemer wrote:
| It also is not the entire Clojure language as far as I
| understand, some features are not supported because it's
| using the "Small Clojure Interpreter"
| synthc wrote:
| babashka is great for scripting, but it is basically is a
| Clojure interpreter written in AOT compiled Graal-compatible
| Clojure.
|
| Because is is AOT compiled is starts fast, but its much slower
| than JVM-Clojure for long running tasks.
|
| A natively compilable Clojure would be great for other use
| cases, especially if it can be made more memory-efficient
| (Clojure is a memory hog)
| vindarel wrote:
| CLASP? CL on LLVM.
|
| https://github.com/clasp-developers/clasp/
|
| > Clasp is a new Common Lisp implementation that seamlessly
| interoperates with C++ libraries and programs using LLVM for
| compilation to native code. This allows Clasp to take advantage
| of a vast array of preexisting libraries and programs, such as
| out of the scientific computing ecosystem. Embedding them in a
| Common Lisp environment allows you to make use of rapid
| prototyping, incremental development, and other capabilities that
| make it a powerful language.
| bumblebritches5 wrote:
| tankfeeder wrote:
| PicoLisp on PicoLisp on LLVM-IR.
|
| https://github.com/picolisp/pil21
| cube2222 wrote:
| This looks so cool!
|
| It does look like it has some technical similarities to Julia,
| with the LLVM JIT and such. It'd be great to see some benchmarks.
|
| Also see the GitHub[0], which looks like the author put a ton of
| work into this so far. Very impressive.
|
| That said, looking at the progress page[1], this is still far
| from achieving 100% feature parity. I'll be following it closely
| with my fingers crossed though.
|
| If you want to support this, it looks like the author is also
| accepting GitHub sponsorships[2].
|
| [0]: https://github.com/jeaye/jank
|
| [1]: https://jank-lang.org/progress/
|
| [2]: https://github.com/sponsors/jeaye
| e12e wrote:
| Hm.. Given:
|
| > Where jank differs from Clojure is that its host is C++ on top
| of an LLVM-based JIT.
|
| So this is similar in kind to pypy? (or maybe hy on pypy..).
| matrix12 wrote:
| And Pixie ran on Pypy. Hy just uses what you have.
| panick21_ wrote:
| No, PyPy is a Jit-ing the interpreter.
| dangoor wrote:
| Unless I'm reading it wrong, it sounds from the license[1] like
| programs made with Jank that you want to distribute must have
| their source available. If the intention is that distributions of
| the _Jank compiler itself_ are the only things that need source
| available, I think it'd be good to spell that out.
|
| [1]: https://github.com/jeaye/jank/blob/main/LICENSE
| Jeaye wrote:
| This applies to the compiler itself, in the same way that GCC
| is under the GPL but that doesn't mean all programs you compile
| with it are also under the GPL.
| terinjokes wrote:
| I'm curious if there is a reason you went with (what I
| believe is) a derivative of the Sleepycat license, instead of
| one of the more familiar licenses?
|
| Your comment here is at odds with my understanding of how the
| Sleepycat license works and how Berkeley DB is licensed.
| dangoor wrote:
| Thanks! That seems totally reasonable to me.
|
| I do feel like this needs to be explicit in the license,
| especially if jank includes a runtime that is distributed
| along with your programs.
|
| Looks like a great project to follow!
| drekipus wrote:
| I love the idea of this. I remember hearing about how
| "technically even llvm could host clojure " and I've been looking
| forward to it ever since.
|
| However the issue is that one of clojure's strengths is the JVM,
| so I'm interested in how this plays out
| runetech wrote:
| Amazing! Can't wait to try it ;-)
| vegancap wrote:
| Exciting idea, I love Clojure, but every time I tried Clojure, I
| found myself butting heads with the Java ecosystem, and I always
| wished it had its own ecosystem and interpreter/compiler. I will
| caveat that that's just because I've never spent long in the Java
| ecosystem, so it's not a criticism of the current Clojure + Java
| ecosystem, more my own lack of experience.
| barrenko wrote:
| Kinda same experience, but I do have some experience with Java
| so imho Clojure makes it easier overall.
|
| Not sure interop with C/C++ would be somehow easier.
| kimi wrote:
| The big issue here is that (most) Clojure libraries that
| interact with the rest of the world do wrap some high-quality
| Java libraries (JDBC, web servers, etc) so Jank would start
| afresh here. CLJS has the same issue, of course, while Graal
| doesn't.
| [deleted]
| jasau wrote:
| Interesting project! I have been wondering why there isn't a LISP
| on top of LLVM yet (except Common Lisp port, Clasp). Good that
| someone started to work on this.
| mark_l_watson wrote:
| Jank looks like a project to check in on its status a few times
| a year, exciting possibilities.
|
| For your thinking about Clasp and alternatives for Common Lisp:
| I don't think there is a gap to be filled with Common Lisp
| deployment. For free, build SBCL from source enabling heap
| saves compression. Then creating SBCL Common Lisp apps produces
| a fairly small app size, easy to build and distribute. Or, pay
| for something like LispWorks and creating compact standalone
| applications is perhaps even easier.
| eatonphil wrote:
| ECL and Chicken Scheme both build with whatever C compiler
| you're using.
| exitb wrote:
| It's extremely interesting, but it has a to be noted that, even
| though the main page claims 100% compatibility with Clojure, the
| status page lowers the claim down to 42%, while the GitHub page
| states you can't even build it.
| ImprobableTruth wrote:
| >while the GitHub page states you can't even build it
|
| It states that it's not _very buildable_ i.e. that the build
| process is very fragile. Building it with nix definitely works,
| I just tried it.
| capableweb wrote:
| It might build, but can you actually execute something?
| According to https://github.com/jeaye/jank/issues/7, you
| won't be able to compile `(if true 1 0)` at the moment even.
| Jeaye wrote:
| Everything on the landing page talks about jank's goals and
| aspirations. 100% Clojure compatibility is a big one. The
| progress page starts with this disclaimer:
|
| > jank is under heavy development. It's safest to assume that
| any feature advertised is partially developed or in the
| planning stages. There is no sales pitch here; just a lot of
| work and some big plans. All development happens on Github, so
| watch the repo there!
| masukomi wrote:
| The landing page explicitly says "jank is 100% compatible
| with Clojure."
|
| not "jank aims to be..." No, "jank IS"
|
| this is... ridiculously uncool and misleading.
| Jeaye wrote:
| Thanks for the feedback.
|
| The usage of "jank" here refers to its design; the concept
| of jank. This is different from its current implementation.
| jank is designed to be many things, as the landing page
| shows, and the implementation is catching up with that.
| nocman wrote:
| No matter how you spin it, this makes the landing page
| very misleading.
|
| Nobody cares about the "design" of a language when there
| is no implementation that fully realizes the design.
| Otherwise, anyone can _say_ "my language is 100%
| compatible with clojure, but oh, yeah, the clojure
| compatibility part isn't fully implemented yet".
|
| If your landing page said "jank's goal is to be 100%
| compatible with clojure", or "jank is _designed_ to be
| 100% compatible with clojure (when fully implemented) "
| that would be fine. Right now the statement is just
| false, because right now, the language _isn 't_ 100%
| compatible with clojure.
|
| I really think you should change the wording -- and I say
| that as someone genuinely interested in the project. I
| hope you achieve 100% clojure compatibility, that would
| be really cool. With the way things are worded now the
| first impression you are giving people is that the
| language implementation is way more complete than it is,
| which I believe will be a big negative for you in the
| long run.
| ordinaryradical wrote:
| Can you explain the type checking a bit more? Is this
| occurring within the JIT for compilation efficacy or is the
| goal here that you would get gradual type inference across a
| codebase facilitated by some kind of LSP?
| Jeaye wrote:
| Type checking gets wonky in the REPL world, since things
| are changing a lot. During JIT usage (i.e. REPL usage or
| running a program right from source), type checking happens
| for each new form, taking into account what it knows of the
| types so far. In this context, some things, such as
| redefining vars, don't retroactively type check all
| previous usages of the var. So that sort of analysis is
| disabled.
|
| During AOT compilation, though, jank does whole program
| analysis and uses the type information to box/unbox,
| monomorphize, compute at compile-time, etc. I'll be
| publishing documentation on each of these designs (the type
| system, the JIT, the interop support) on the website as
| things move forward. Stay tuned. :)
| moomin wrote:
| Honestly I don't think compatibility with "pure Clojure" is
| that useful. Almost everything has a couple of native calls
| hidden away.
| Jeaye wrote:
| Most libraries I tend to use are generally pure Clojure,
| with some reader conditionals at the edges. jank will also
| support reader conditionals, so those libraries can grow to
| support C++ interop where needed.
| widdershins wrote:
| I'll be curious to see what the performance comparisons of
| typical Clojure programs will be. Persistent data structures tend
| to benefit a lot from highly optimised garbage collectors like
| those packaged with the JVM. And Clojure's highly dynamic style
| probably benefits a lot from JIT techniques too.
|
| Still, it's a fascinating project, which I will watch closely.
| Jeaye wrote:
| jank is currently using immer for persistent data structures:
| https://github.com/arximboldi/immer
|
| Very much inspired by Clojure, with a lot of time put into
| benchmarking and profiling.
|
| It's too early for me to share numbers on jank vs Clojure
| itself, especially due to the rewrite, but my earlier jank
| versions were definitely competitive with AOT compiled Clojure
| jars.
| whizzter wrote:
| This is a very very early project but mostly I think it'll be
| interesting to see how well they manage to utilize type
| annotations and how well the Clojure ecosystem provides this?
| (ie, is general Clojure code mostly untyped or is there enough
| for compilers to deduce most actual types?)
|
| Why would this be important? Well because they are relying on
| LLVM for optimization, and while that's good if you know type
| info (via annotations or lower tiered profiling), there's a lot
| of code to start using this and those optimizations can be
| fragile.
|
| If you read the articles posted here in the past year on the
| Erlang JIT they mirrored this sentiment, earlier "advanced" JIT
| attempts built on classic static language techniques like LLVM
| prefers failed because they didn't have enough type
| information, whilst the "simple" JIT that mainly removed
| overheads and concerned more with local types was showing good
| promise without too much engineering headache.
| askonomm wrote:
| I've been dreaming of something like this for a long time. While
| you can also make native executables with Clojure, using GraalVM,
| they tend to weigh at minimum around 10MB, and setting up the
| whole build system is also something I never want to deal with,
| so this is very awesome.
| substation13 wrote:
| Is 10mb really a hurdle these days? Remember that since that
| 10mb includes loads of support infra (such as GC) the user code
| will be smaller too.
| ducktective wrote:
| Yes, when the same functionality could be achieved in like
| 200KiB, 10 MiB is too much. It adds up and suddenly you may
| have to bundle a whole browser with your ap...oh
| mike_hearn wrote:
| Bear in mind that this 10mb is not a fixed multiplication
| factor. Like, doubling you code size doesn't double your
| binary size with native-image because that base contains a
| lot of stuff that's generic. Not just GC but stuff like
| support for timezones.
|
| It also contains a cached heap (in fact most of it is
| cached heap) which is one reason these binaries start as
| fast as C programs do. Especially important for Clojure
| where the runtime and libraries are written in such a way
| as to do lots of unnecessary work at startup. Native images
| can cache all that so you get rid of the startup overhead
| of Clojure itself.
| burjui wrote:
| That begs the question: if Clojure is so inefficient,
| that a program written in it needs 10 MB of some "cached
| heap" just to start as fast as a C program, and it does a
| lot of unnecessary work at startup, why use Clojure and
| this Jank (great name lol) in the first place? We have
| Rust, after all. I mean, creating an optimizing native
| compiler for any language is not an easy task, there must
| be some seriously powerful stuff in Clojure to warrant
| such an undertaking. Or just madness.
| huahaiy wrote:
| Rust and Clojure address completely different domains of
| problems.
|
| Rust is competing with C++ for writing low level systems,
| whereas Clojure is designed for writing business
| applications.
|
| Low level systems tend to have stable and well understood
| requirements, whereas the same cannot be said for
| business applications. Efficiency is not a primary
| concern of business applications, but flexibility is. The
| efficiency of Clojure is often "good enough" for business
| application, and there is no peer in flexibility. This
| explains the many fintech companies using Clojure, as
| well as many other B2B companies.
| ithrow wrote:
| It's like they have not seen how much engineering
| resources and talent have been put into V8 and JVM GCs,
| and it's only because of these state of the art GCs (and
| JITs) that Clojure is bearable and practical for day to
| day use.
|
| I'm almost sure the author would drop the project as soon
| as he realizes and lives the frustration of not being
| able to come anywhere near close to the performance of
| Clojure in the JVM.
| huahaiy wrote:
| That's indeed a valid concern. Keeps my finger crossed.
| cardanome wrote:
| If they could pull off easy integration with the C and C++ world
| that would be an absolute killer feature.
|
| They are basically trying to make my new dream language.
|
| The only thing I am worried about is compile time. It seems many
| projects using LLVM have trouble with that. See Julia with its
| LLVM JIT, though they did manage to improve it somewhat. I guess
| it is simply to early to tell, lets hope for the best.
| Jeaye wrote:
| Compile times will be slower than Clojure's, for sure. That
| also means run-time speed will be affected, due to JIT
| compilation time. This will not affect AOT compiled run-time
| speed, though. I don't have numbers yet, but you're right that
| compiling C++ with LLVM could be much faster. :)
| stingraycharles wrote:
| I'd say it can't possibly be much worse than with GraalVM, but
| maybe my expectations are too high.
|
| I'm very interested in runtime performance as well, though, and
| how it compares to vanilla JVM-based Clojure.
| eigenspace wrote:
| LLVM is not a particularly slow compiler (though it is
| progressively getting slower). There are faster compilers out
| there, but it's fine.
|
| The reason julia has so much compiler latency is not that we
| have a slow compiler, it's that we compile a LOT of code.
|
| Julia is very very (runtime) performance oriented, and so we do
| everything we can to speed up execution time. This means that
| we have very aggressive code specialization, inlining, and
| unrolling heuristics, and people take heavy advantage of them.
| It's a very normal thing in julia to try and write very high
| level generic code that performs like low level optimized code
| by taking advantage of versions code generation techniques.
|
| In addition to all of that, we end up _recompiling_ a lot of
| code that was already compiled before because we have very
| strict heuristics to make sure that when a method body is
| altered, all downstream dependants of that method get
| invalidated and must be recompiled, and sometimes the
| heuristics overeager.
|
| Julia compile times have been dropping rapidly over the past
| couple years mostly because people have put a lot of work into
| avoiding invalidations, and being smarter about how compile-
| time code generation is approached. Almost none of that
| improvement has to do with LLVM, it all happens well before we
| hand things off to LLVM.
___________________________________________________________________
(page generated 2022-08-17 23:02 UTC)