[HN Gopher] Tilde, My LLVM Alternative
___________________________________________________________________
Tilde, My LLVM Alternative
Author : davikr
Score : 314 points
Date : 2025-01-21 17:33 UTC (3 days ago)
(HTM) web link (yasserarg.com)
(TXT) w3m dump (yasserarg.com)
| Rochus wrote:
| Cool. The author has set himself a huge task if he wants to build
| something like LLVM. An alternative would be to participate in a
| project with similar goals that is already quite progressed, such
| as QBE or Eigen (https://github.com/EigenCompilerSuite/) ; both
| so far lack of optimizers. I consider Eigen very attractive
| because it supports much more targets and includes assemblers and
| linkers for all targets. I see the advantage in having a C
| implementation; Eigen is unfortunately developed in C++17, but I
| managed to backport the parts I'm using to a moderate C++11
| subset (https://github.com/rochus-keller/Eigen). There are
| different front-ends available, two C compilers among them. And -
| as mentioned - an optimizer would be great.
|
| EDIT: just found this podcast where the author gives more
| informations about the project goals and history (at least the
| beginning of the podcast is interesting):
| https://www.youtube.com/watch?v=f2khyLEc-Hw
| wffurr wrote:
| What's unfortunate about C++17? It has some nice features that
| build on C++11's safety and ergonomic improvements.
| Rochus wrote:
| You need a large, modern C++ compiler and standard library,
| which are not available for most older systems, and you're
| inviting an excess of dependencies because not all compilers
| support all parts of the newer C++ standards (in the same
| way), and require a lot more resources and newer versions of
| APIs and libraries, which further limits their usability on
| older systems. Furthermore, C89 and C++98 are much easier to
| bootstrap than a colossus like LLVM and Clang. The few "nice
| features" are perhaps enticing, but the costs they incur are
| disproportionate.
| o11c wrote:
| For reference, GCC 4.7 (released March 2012) was the last
| build of GCC that is written in C, and supports almost all of
| the C++11 language (4.8.1, written in C++, finished the last
| bits) and a fair amount of the library.
|
| If you have to work on a system that hasn't been updated
| since 2014 or so (since it's fair enough to avoid the .0
| releases), getting support for later C++ standards is
| significantly more complicated.
| s3graham wrote:
| This looks pretty cool. I've been looking at all the "small"
| backends recently. It's so much nicer to work with one of them
| than trying to wrangle LLVM.
|
| QBE, MIR, & IR (php's) are all worth a look too.
|
| Personally I've settled on IR for now because it seemed to match
| my needs the most closely. It's actively developed, has aarch64
| in addition to x64 (looks like TB has just started that?), does
| x64 Windows ABI, and seems to generate decent code quickly.
| mungaihaha wrote:
| > I believe it's (LLVM) far too slow at compiling and far too big
| to be fixed from the inside
|
| What are you doing to make sure Tilde does not end up like this?
| fermigier wrote:
| "You either die a hero or live long enough to see yourself
| become the villain".
| Ygg2 wrote:
| You either reinvent the wheel but square or live long enough
| to make it a circle.
| snowfarthing wrote:
| I just had a random thought: perhaps it would be a good idea to
| have a project that doesn't do optimizations, but just focuses
| on fast compiling.
|
| Then again, I now can't help but wonder if LLVM (or even GCC)
| would be fast, if you just turned off all the optimizations ...
|
| (Of course, at this point, I can't help but think "you don't
| need to worry about the speed of compilation" in things like
| Common Lisp or Smalltalk, because everything is compiled
| incrementally and immediately, so you don't have to wait for
| the entire project to compile before you could test something
| ...)
| negate32 wrote:
| One of the big things which makes LLVM very slow is the
| abundance of passes, I believe I last counted 75 for an
| unoptimized function? My solution for this is writing more
| combined passes, due to the SoN design I'm combining a lot of
| things which are traditionally separate passes. For instance,
| my equivalent to "SimplifyCFG" "GVNPass", "InstCombine",
| "EarlyCSEPass" and "JumpThreadingPass" is one combined peephole
| solver which runs faster than all of these passes separately.
| This is for two main reasons:
|
| * Less cache churn, I'm doing more work per cacheline loaded in
| (rather than rescanning the same function over and over again).
|
| * Combining mutually beneficial optimizations can lead to less
| phase ordering problems and a better solve (this is why SCCP is
| better than DCE and constant prop separately).
|
| In a few years when TB is mature, I'd wager I'll have maybe
| 10-20 real passes for the "-O2 competitive" optimizer pipeline
| because in practice there's no need to have so many passes.
| melodyogonna wrote:
| Chris Lattner seems to have also created an alternative for LLVM
| - https://mlir.llvm.org/
|
| Because of how the architecture works, LLVM is one of the
| backends, but it doesn't have to be. Very interesting project,
| you could do a lot more IR processing before descending to LLVM
| (if you use that), that way you could give LLVM a lot less to do.
|
| Chris has said LLVM is fast at what it is designed to do - lower
| IR to machine code. However, because of how convoluted it can
| get, and the difficulty involved in getting information from some
| language-specific MIR to LLVM, languages are forced to generate
| tons upon tons of IR so as to capture every possible detail. Then
| LLVM is asked to clean up and optimize this IR.
|
| One thing to look out for is the problem of either losing
| language-specific information when moving from MIR to Low-level
| IR (be it Tilde or LLVM) or generating too much information, most
| of it useless.
| _flux wrote:
| I wonder if one solution would be have tighter integration
| between the layers, so the backend could ask for some IR to be
| generated? Basically starting from the program entrypoints.
| This way the frontend wouldn't need to generate all the
| possible code up-front.
|
| Mind you, I've never written a compiler after that Uni course
| and touched LLVM IR a long time ago
| melodyogonna wrote:
| That is how MLIR works. Basically you have multiple levels of
| IR, you can optimize each level until you get to the last
| level.
|
| It also has the advantage of being able to parallelize passes
| zellyn wrote:
| I wonder if this question can attract any MLIR people to answer
| my question:
|
| From Chris Lattner's descriptions of LLVM vs MLIR in various
| podcasts, it seems like LLVM is often used as a backend for
| MLIR, but only because so much work has been put into
| optimizing in LLVM. It also seems like MLIR is strictly a
| superset of LLVM in terms of capabilities.
|
| Here's my question: It seems inevitable that people will
| eventually port all LLVM's smarts directly into MLIR, and
| remove the need to shift between the two. Is that right?
| jcranmer wrote:
| MLIR is less a specific IR and more a generic framework for
| expressing your own custom IR (which is itself composed of
| many pluggable IRs)--except these IRs are renamed "dialects."
| One of the MLIR dialects is the LLVM dialect, which can be
| lowered to LLVM IR.
|
| In the future, it's plausible that dialects for hardware ISAs
| would be added to MLIR, and thus it would be plausible to
| completely bypass the LLVM IR layer for optimizations. But
| the final codegen layer of LLVM IR is not a simple thing (I
| mean, LLVM itself has three different versions of it), and
| the fact that GlobalISel hasn't taken over SelectionDAG after
| even a decade of effort should be a sign of how difficult it
| is to actually replicate that layer in another system to the
| point of replacing existing work.
| CalChris wrote:
| Earlier compilers were a pipeline of specialized IRs. I
| used to think that MLIR was an acknowledgment that this
| specialization was necessary. Ok, it is necessary. But
| MLIR's real contribution is, as you say, a _generic
| framework_ for expressing your own custom IR.
| superlopuh wrote:
| I know of a few projects looking in that direction, each
| optimising for different things, and none getting near the
| capability of LLVM, which is going to take some time. I spoke
| with some of the core MLIR developers about this, and they're
| generally open to the notion, but it's going to take a lot of
| volunteer effort to get there, and it's not clear who the
| sherpa will be, especially given the major sponsors of the
| LLVM project aren't in a particular hurry. If you're
| interested in this feel free to look our paper up in a week
| or two, we've had a bit of trouble uploading it to arxiv but
| should be ready soon.
|
| https://2025.cgo.org/details/cgo-2025-papers/39/A-Multi-
| Leve...
|
| Here's a quick pres from the last dev meeting on how this can
| be leveraged to compile NNs to a RISC-V-based accelerator
| core: https://www.youtube.com/watch?v=RSTjn_wA16A&t=1s
| fooker wrote:
| All the important bits of MLIR are closed source and there's
| no indication that'll change anytime soon.
|
| The big players have their own frontend, dialects, and mostly
| use LLVM backends. There's very little common usable
| infrastructure that is upstreamed. Some of the upstreamed
| bits are missing large pieces.
| almostgotcaught wrote:
| > There's very little common usable infrastructure that is
| upstreamed
|
| Hmm I wonder what all that stuff is then that's under
| mlir/lib?
|
| Like what are you even talking about? First of all there
| are literally three upstream frontends (flang, ClangIR, and
| torch-mlir). Most people use PyTorch as a frontend (some
| people use Jax or Triton). Secondly, downstream users
| having their own dialects... is basically the whole point
| of MLIR. Core dialects like linalg, tensor, memref, arith
| are absolutely generically useful. In addition many (not
| all) MLIR-based production quality compilers are fully open
| source (IREE, Triton) even if wholly developed at a for-
| profit.
| marssaxman wrote:
| What important bits are those? I can't imagine what you
| have in mind here; my current job and my previous job have
| both revolved around MLIR-based compilers, and it has never
| seemed to me that there is anything missing. I wonder if
| you might be expecting MLIR to do a job it's not really
| meant for.
| marssaxman wrote:
| They solve different problems. MLIR is not a backend, but a
| toolkit for defining intermediate representations
| ("dialects") and the passes which optimize them or lower from
| one to another. You can use MLIR to organize everything your
| compiler does between its front-end and its back-end; MLIR
| doesn't care where the IR comes from or what you ultimately
| do with it.
|
| MLIR does include a couple dozen built-in dialects for common
| tasks - there's an "scf" dialect which defines loops and
| conditionals, for example, and a "func" dialect with function
| call and return ops - and it so happens that one of these
| built-in dialects offers a 1:1 representation of the
| operations and types found in LLVM IR.
|
| If you choose to structure your compiler so that all of your
| other dialects are ultimately lowered into this MLIR-LLVM
| dialect, you can then pass your MLIR-LLVM output through a
| converter function to get actual LLVM-IR, which you can then
| provide to LLVM in exchange for machine code; but that is the
| extent of the interaction between the two projects.
| mshockwave wrote:
| > Here's my question: It seems inevitable that people will
| eventually port all LLVM's smarts directly into MLIR, and
| remove the need to shift between the two. Is that right?
|
| Theoretically, yes -- taking years if not decades for sure.
| And set aside (middle-end) optimizations, I think people
| often forgot another big part of LLVM that is (much) more
| difficult to port: code generation. Again, it's not
| impossible to port the entire codegen pipeline, it just takes
| lots of time and you need to try really hard to justify the
| advantage of moving over to MLIR, which at least needs to
| show that codegen with MLIR brings X% of performance
| improvement.
| ksec wrote:
| >I'm calling it Tilde (or TB for tilde backend) and the reasons
| are pretty simple, i believe it's far too slow at compiling and
| far too big to be fixed from the inside. It's been 20 years and
| cruft has built up, time for a "redo".
|
| That put a smile on my face because I remember that was how LLVM
| was born out of frustration with GCC.
|
| I dont know how the modern GCC and LLVM compares, I remember LLVM
| was fast but resulting binary were not as optimised, once those
| optimisation added it became slower. While LLVM was a wake up
| call to modernise GCC and make it faster. In the end competition
| made _both_ a lot better.
|
| I believe some industry ( Gaming ) used to swear by VS Studio /
| MS Compiler / Intel Compiler or languages that depends / prefer
| the Borland ( What ever they are called now ) compiler. Its been
| very long since I last looked I am wondering if those two are
| still used or have we all merged mostly into LLVM / GCC?
| pjmlp wrote:
| It is pretty much Visual Studion on Windows and XBox, Nintendo
| and Sony have clang forks.
|
| Embarcadero owns Borland, unfortunely stuff like C++ Builder
| doesn't seem to get much people outside big corps wanting to
| use it, which is a shame given its RAD capabilities and GUI
| design tooling for C++.
|
| Also has a standard ABI between Delphi and C++ Builder, which
| allows to similar development workflows that .NET offered later
| with C#/VB alongside Managed C++ extensions (later replaced by
| C++/CLI).
| bbatha wrote:
| Intel still has a decent use for compiling math heavy code for
| intel processors -- so it gets a decent amount of use in HPC
| applications. It has some of the best vectorization passes but
| they only work with actual intel cpus. So it's starting to get
| less traction as AMD takes the performance crown and as
| vectorized math moves to the gpu.
| CUViper wrote:
| Intel's compilers are now based on LLVM too.
| willvarfar wrote:
| Back 20 or more years ago I used to do a lot of rec math
| competition programming and found that the metrowerks c++
| compiler made massively faster programs than gcc, vsstudio,
| intel and everything else I tried then.
|
| This seemed to be simply down to variable alignment; the
| programs took more memory but ran much faster, particularly
| multi-core (which was still high end then).
|
| And this was on x86 where metrowerks weren't really competing,
| and was probably accidental. But the programs it compiled were
| fast.
|
| I'd be surprised if anyone even knew that metrowerks had a c++
| compiler on x86 on windows. At the time metrowerks were on the
| tail end of their domination of mac compilers from before mac
| ran on x86.
| dan_hawkins wrote:
| I was using Metrowerks C++ compiler suite to develop code for
| Dragonball (68000) embedded system 22 years ago!
| bruce343434 wrote:
| Memory access patterns are everything. Memory delay is almost
| always the bottleneck anyway. I have the feeling that more
| and more this is becoming common knowledge, and techniques
| like "struct of arrays" are becoming more wide spread and
| talked about.
| whimsicalism wrote:
| msvc is very much in third but it is one of the three that i
| think of when i think of c++ compilers
| cfiggers wrote:
| Tsoding explored this project on a recent stream:
| https://youtu.be/aKk_r9ZwXQw?si=dvZAZkOX3xd7yjTw
| nurettin wrote:
| I got tsoding fatigue after youtube started suggesting him on
| an hourly basis. He's on ignore.
| sylware wrote:
| Again, somebody who comes to the realization something is
| seriously wrong with ultra-complex languages in the SDK (c++ and
| similar).
|
| In other words, since this alternative LLVM is coded in plain and
| simple C, it is shielded against those who are still not seeing
| that computer languages with an ultra complex syntax are not the
| right way to go if if want sane software.
|
| You also have QBE, which with cproc will give you ~70% of latest
| gcc speed (in my benchmarks on AMD zen2 x86_64).
| pjmlp wrote:
| Cough, C23 and C2y roadmp.
| Rochus wrote:
| We are fortunately free to ignore "C23 and C2y" and stick
| with C89 (with the common extensions) or C99.
| pjmlp wrote:
| Applies to any programming language, feel free to use
| C++ARM for example.
|
| Until there is that special library that doesn't care about
| this target group of developers that want to stay in the
| past.
| uecker wrote:
| What are the specific things you do not like about C23 or
| the C2y road map (whatever this is, it is more random
| walk)? I have my own list of course, but overall I still
| have some hope that C2y does not turn out to be a total
| disaster.
| Rochus wrote:
| I've spent too little time with the recent
| standards/draft to give a specific answer. But I have a
| general attitude: C89 with extensions or C99 were just
| perfect for almost any purpuse; newer standards may well
| correct minor inadequacies or integrate things, that used
| to be implemented by proven libraries, directly into the
| language; but the price for these relatively minor
| improvements is high; people who write supposedly
| reusable code in the newer standards effectively force
| all older projects to switch to the newer standard; the
| costs of this are rarely justifiable. And there is C11
| which made mandatory parts of the C99 standard optional,
| thus breaking backwards compatibility.
| uecker wrote:
| It is great to see some new C tooling emerge. I will likely
| make my own C FE public some time but it now uses some toy
| backend which needs be replaced...
| fuhsnn wrote:
| There is no shame in simple codegen if it is correct and
| unsurprising!
| kibwen wrote:
| C itself is an ultra-complex language. I do not understand the
| mindset of the "C is simple" crowd. Is it nostalgia or
| romanticism for the past? If we want to devise a truly simple
| language, we need to start by realizing that C is just the
| Javascript of its day: hacked together in a weekend by someone
| who wished they were using a different language and then
| accidentally catapulted into the future by platform effects.
| Rochus wrote:
| What aspects do you consider "ultra-complex"? I agree that it
| has a very strange syntax, many features of it being unknown
| to most people; but besides that, it's as easy as Pascal,
| isn't it?
| pjmlp wrote:
| Pascal doesn't have half as UB as C, or possibilities to
| memory corruption.
|
| Pascal here meaning compilers people actually use, not ISO
| Pascal from 1976, people love their C extensions after all.
|
| A for C's simplicity, one just needs to organise a pub
| quiz, using ISO, and key extensions as source of
| inspiration.
| Rochus wrote:
| When refering to Pascal, I mean something like Turbo, Vax
| or Apple Pascal, i.e. the version used at the height of
| popularity. Original Pascal has much less degrees of
| freedom. And I have no reason to assume that Turbo or
| Apple Pascal have less possibilities for memory
| corruption, or are better specified.
| pjmlp wrote:
| Starts by having proper strings and array types with
| bounds checking instead of pointers, followed by memory
| allocation with types instead of sizeof math, less
| scenarios for implicit conversions, reference parameters
| reducing the use cases where an invalid pointer might be
| used instead.
| uecker wrote:
| UB has nothing to do with complexity. In any case, from
| about 87 UB in the core language, we eliminated 15 in the
| last meeting, and already have concrete proposals for 10
| more. C2Y will likely not have any trivial UB and
| hopefully also optional safety modes that eliminate the
| others.
| pjmlp wrote:
| It certainly has, as proven by recent talk at BlueHat
| 2024, on Windows kernel refactorings, as not everyone is
| knowledgeable of ISO C minutia and how optimisers take
| advantage of it, and still think they know better than
| analysers.
| uecker wrote:
| Maybe you can explain this better. People not knowing
| about footguns is also not the same complexity, it is
| just having footguns and people not knowing about them.
| pjmlp wrote:
| It is a matter of wording, doesn't change the trap is on
| the path.
|
| Nonetheless, it is good to see efforts to reduce UB on
| the standard.
|
| My complaints apply to C++ as well, naturally.
| uecker wrote:
| While it has its fair share of quirks it is certainly not an
| "ultra-complex" language.
| kibwen wrote:
| The sleight of hand here is that by leaving so many things
| undefined, unspecified, and implementation-defined, C gets
| to foist complexity off on the implementations and then act
| as though it's absolved of blame when things go off the
| rails. The fact that what felt like half of all traffic on
| Usenet and IRC in the 90s was comprised of people language-
| lawyering over what is and is not valid C disqualifies it
| from being considered a simple language. It's as though
| someone designed a language where the spec is the single
| sentence "the program does what the user intends it to do"
| and then held this up pinnacle of simplicity. C has an
| entire Wikipedia article about how needlessly difficult it
| is to parse: https://en.m.wikipedia.org/wiki/Lexer_hack . C
| has an entire website for helping people interpret its type
| gibberish: https://cdecl.org/ . C's string handling is
| famously broken. C's formatting machinery is Turing
| complete! Coercions out the wazoo. Switch fallthrough
| having exactly the wrong default. The fact that Duff's
| Device works at all. Delegating all abstraction to an
| infamously error-prone textual macro language. You could
| spend an entire career learning this language and still
| find exciting new ways to blow your leg off. If C is our
| bar for simplicity, it explains a lot about the state of
| our profession.
| uecker wrote:
| I write C every day and wrote C compiler. I know all all
| the issues it has very well, but it still a relatively
| simple language. It also not that difficult to parse. The
| lexer hack is interesting, because you can almost get
| away with using a context-free lexer, but you need this
| one hack. But this is not really a problem. People
| failing to read c declarations is also not the same thing
| as complexity, although I would agree that those are
| weird. Null-terminated strings have safety issues, but
| this is also not the same thing as complexity.
| uecker wrote:
| I would argue that the problem of C is something else: It
| does not provide enough functionality out of the box. So
| instead of using some safe abstraction, people open-code
| their string manipulation or buffer management. This is
| then cumbersome and error prone, leading to complexity of
| the solution and safety issues which would could easily
| be avoided.
| sylware wrote:
| I prefer a simple computer language with many real-life
| alternative compilers, and on the long run. The code will
| be hardened, where appropriate, because it is not cheap,
| over time.
|
| And we must not forget that, 100% "safe" high level code
| should never be trusted to be compiled into 100% safe
| machine code.
| sylware wrote:
| And you are right, C syntax is already too complex: integer
| promotion should go away like implicit casts, 1 loop
| statement is sufficient, should have had only sized primitive
| types, etc, etc. Just need a few new inline keywords for
| modern hardware architecture programming (atomics, barriers,
| endianness).
|
| C99+ is just the less worse compromise.
| muth02446 wrote:
| Shameless plug for another similar system: http://cwerg.org Less
| ambitious with a focus on (measurable) simplicity.
| Rochus wrote:
| Cwerg looks interesting indeed. I had it on my radar for some
| time. Especially its focus on simplicity and independence (e.g.
| that it can directly generate ELF executables) are attractive.
| From my humble point of view, both Python 3 and C++17 are a bit
| unfortunate as implementation languages. I can understand that
| the author didn't want to use C, but C++98 would have resolved
| this issue with less build and dependency complexity than
| C++17. Last year, I intensively evaluated backends and
| eventually settled on https://github.com/EigenCompilerSuite/,
| which supports a large number of targets, has a very powerful
| IR code generator and also comes with its own linkers. Also
| Eigen is unfortunately written in C++17, but I managed to port
| all relevant parts to a very moderate C++11 subset (even C++98
| seems feasible in future).
|
| Am I wrong to assume that Cwerg doesn't support x86, or is this
| just assumed by "X86-64"?
| muth02446 wrote:
| Ultimately, the plan for Cwerg is to be self-hosting, so C++
| is just another stepping stone. I am curious about the issues
| with C++17 (vs say C++11) though.
|
| About using C++:
|
| Cwerg is NOT going "all in" on C++ and tries to use as little
| STL as possible. There are some warts here and there that
| C++17 fixes and those are used by Cwerg - nothing major.
| There is also a lightweight C wrapper for Cwerg Backend.
|
| About not using C:
|
| I do not get the fetishizing of C. String handling is just
| atrocious, no concept of a span, no namespaces, poorer type
| system, etc. Cwerg is actually trying to fix these.
|
| If Cwerg was written in C instead of C++, a lot of the
| constructs would become MACRO-magic.
|
| About Backends:
|
| Currently supported are: 64 bit ARM, 32 bit ARM (no thumb),
| 64 bit x86 There are no plans to support 32 bit x86
| Rochus wrote:
| > _I am curious about the issues with C++17 (vs say C++11)
| though._
|
| It's about dependability and bootstrapping. GCC 4.7 was the
| last version implemented in C, and it supports C++98/03 and
| a subset of C++11.
|
| > _There are some warts here and there that C++17 fixes
| [..] nothing major_
|
| But it's C++17 and thus requires many more bootstrap cycles
| until we have a compiler. I think a backend which only
| supports a subset of the features of a C++17 compiler
| should not depend on C++17, otherwise its usefulness is
| restricted by the availability of such a compiler.
|
| > _I do not get the fetishizing of C. String handling is
| just atrocious..._
|
| C is just a pretty primitive high-level language (with a
| very strange syntax) a compiler of which exists on almost
| any system. The next complexity step is C++98 (or the
| subset supported by cfront), which solves your issues and
| is even good enough to build something as complex as Qt and
| KDE.
|
| > _There are no plans to support 32 bit x86_
|
| Ok, thanks. The support for ARM32 already enables many use
| cases.
| muth02446 wrote:
| I can commiserate. I did some bootstrapping of gcc 10
| years ago and it was the most miserable experience ever.
| You make a change somewhere. Kick off "make" and 20 min
| later you get some bizarre error in some artifact that is
| hard to find, generated by a build system that is
| impossible to trace.
|
| A self-hosting Cwerg will hopefully be much easier to
| bootstrap because of its size. But until then, why do you
| need the (continuous) bootstrapping. You can use a cached
| version of the bootstrapped C++ compiler or cross
| compile.
| Rochus wrote:
| I didn't express a requirement for Cwerg, but just tried
| to explain why I prefer to implement a compiler in C++98
| than C++17.
| shipp02 wrote:
| Given how long it takes for compilers to mature, by the time
| it's ready I didn't think anyone will care about x86.
| Similarly for c++11 vs 17. No?
| Rochus wrote:
| Well, for all applications which don't require to addres
| more than 4 GB memory, a 64 bit machine is overkill. This
| especially applies to embedded systems (which make up the
| majority of all systems). This is unlikely to change for
| the next fifty years.
| mtlynch wrote:
| I saw Yasser present this at Handmade Seattle in 2023.[0] He
| explained that when he started working on Tilde, he didn't have
| any special knowledge or interest in compilers. But he was
| reading discussions in the Handmade forums, and one of the most
| popular requests was for an alternative to LLVM, so he thought,
| "Sure, I'll do that."
|
| [0] https://handmadecities.com/media/seattle-2023/tb/
| bjourne wrote:
| Good stuff. Hope they succeed. LLVM support for jit:ed and gc:ed
| languages is pretty weak so a competitor that addresses those and
| other shortcomings would be welcome.
| pjmlp wrote:
| GraalVM, PyPy, naturally the more the merrier.
| fguerraz wrote:
| Looking at the commit history inspires some real confidence!
|
| https://github.com/RealNeGate/Cuik/commits/master/
| wild_pointer wrote:
| chicken (+558998, -997)
| jamil7 wrote:
| Cursed. I had a coworker once would commit diffs like that
| but always with the message "Cleanup". The git history was
| littered with "Cleanup" commits that actually hid all kinds
| of stuff in them. If you pulled them up on it (or anything
| else) they went into defensive meltdown mode, so everyone on
| the team just accepted it and moved on.
| artemonster wrote:
| went to write exactly that. Ambitions are great and I dont want
| to be dissuasive, but monumental tasks require monumental
| effort and monumental effort requires monumental care. That
| implies good discipline and certain "beauty" standards that
| also apply to commit messages. Bad sign :)
| KolmogorovComp wrote:
| Not really. In the initial phase of a project there is
| usually so much churn than enforcing proper commit messages
| is not worth it, until the dust settle down.
| kccqzy wrote:
| I am deeply suspicious of anyone who doesn't bother or who
| is unable to explain this churn. For the right kind of
| people, this is an excellent opportunity to reflect: why is
| there churn? Why did the dust not settle down? Why was the
| initial approach wrong and reworked into a new approach?
|
| I can understand this if you are coding for a corporate.
| But if it's your own project, you should care about it
| enough to write good commit messages.
| torstenvl wrote:
| Is your objection to the inevitable fact that
| requirements churn early on (regardless whether you're
| doing agile or waterfall)?
|
| Or is your objection that solo devs code up prototypes
| and toy with ideas in live code instead of just in their
| mental VM in grooming sessions?
|
| Or is your objection that you don't think early
| prototypes and demos should be available in the source
| tree?
| kccqzy wrote:
| None of the above. My objection is the lack of
| explanation.
|
| Churn is okay. Prototypes are okay. Toying with ideas is
| okay. They should all be in the source tree. But I would
| want an explanation for the benefit of future readers,
| including the future author. Earlier in my life I have
| more than once run blame on a piece of code to find
| myself writing a line of code where the commit message
| does not explain it adequately. These days it's much
| rarer because I ask myself to write good commit messages.
| Furthermore the act of writing a commit message is also
| soothing and a nice break from writing for computers.
|
| Explain how requirements have changed. Explain how the
| prototype didn't work and led to a rewrite. Explain why
| some idea that was being toyed with turned out to be bad.
|
| Notice that the above are explanations. They do not come
| with any implied actions. "Why is there churn" is a good
| question to answer but "how do we avoid churn in the
| future" is absolutely not. We all know churn is
| inevitable.
| kunley wrote:
| How much code could be written and debugged during the
| time when someone b*tches about damn commit messages
| written by a generous bloke on his own time
| apocalypses wrote:
| I massively disagree. It would have taken the author
| approximately 1 minute to write the following high quality
| hack-n-slash commit message:
|
| ``` Big rewrites
|
| * Rewrote X
|
| * Deleted Y
|
| * Refactored Z ```
|
| Done
| fooker wrote:
| Different people work differently.
|
| Spending a minute writing commit messages while
| prototyping something will break my flow and derail
| whatever I'm doing.
| jandrewrogers wrote:
| Many times it is "threw everything out and started over"
| because the fundamental design and architecture was
| flawed. Some things have no incremental fix.
| pveierland wrote:
| Eh, when you're hacking away as a solo developer on something
| big and new I don't think this matters at all. In my current
| project I did about 200x commits marked "wip" before having
| enough structure and stability to bother with proper commit
| messages. Whatever lets you be productive until more structure
| is helpful.
| pkal wrote:
| Perhaps, but I still think it is lazy. A very nice counter
| example of someone with high commit standards can be seen in
| this repository:
| https://github.com/rmyorston/pdpmake/commits/master/
| jasonjmcghee wrote:
| Another example is ghostty
| jandrewrogers wrote:
| The code base may go through several almost total rewrites
| before it stabilizes, especially for non-trivial systems
| that are performance sensitive. Changes to the code may be
| intrinsically non-modular depending on the type of
| software. This prior history can be enormous yet have no
| value, essentially pure noise.
|
| The efficient alternative, which I've seen used a few times
| in these cases, is to retcon a high-quality fake history
| into the source tree after the design has stabilized. This
| has proven to be far more useful to other engineers than
| the true history in cases like this.
|
| Incremental commits are nice but not all types of software
| development lends itself to that, especially early in the
| development process. I've seen multiple cases where trying
| to force tidy incremental commit histories early in the
| process produced significantly worse outcomes than they
| needed to be.
| muke101 wrote:
| If you're going to rewrite LLVM, you should avoid just trying to
| 'do it again but less bloated', because that'll end up where LLVM
| is now once you've added enough features and optimisation to be
| competitive.
|
| Rewriting LLVM gives you the opportunity to rethink some of its
| main problems. Of those I think two big ones include Tablegen and
| peephole optimisations.
|
| The backend code for LLVM is awful, and tablegen only partially
| addresses the problem. Most LLVM code for defining instruction
| opcodes amounts to multiple huge switch statements that stuff
| every opcode into them, its disgusting. This code is begging for
| a more elegant solution, I think a functional approach would
| solve a lot of the problems.
|
| The peephole optimisation in the InstCombime pass is a huge
| collection of handwritten rules that's been accumulated over
| time. You probably don't want to try and redo this yourself but
| it will also be a big barrier to achieving competitive
| optimisation. You could try and solve the problem by using a
| superoprimisation approach from the beginning. Look into the
| Souper paper which automatically generates peepholes for LLVM:
| (https://github.com/google/souper,
| https://arxiv.org/pdf/1711.04422.pdf).
|
| Lastly as I hate C++ I have to throw in an obligatory suggestion
| to rewrite using Rust :p
| jcranmer wrote:
| > The backend code for LLVM is awful, and tablegen only
| partially addresses the problem. Most LLVM code for defining
| instruction opcodes amounts to multiple huge switch statements
| that stuff every opcode into them, its disgusting. This code is
| begging for a more elegant solution, I think a functional
| approach would solve a lot of the problems.
|
| So one of the main problems you run into is that your elegant
| solution only works about 60-80% of the time. The rest of the
| time, you end up falling back onto near-unmaintainable,
| horribly inelegant kludges that end up having to exist because
| gee, real architectures are full of inelegant kludges in the
| first place.
|
| Recently, I've been working on a decompiler, and I started out
| with going for a nice, elegant solution that tries as hard as
| possible to avoid the nasty pile of switch statements. And this
| is easy mode--I'm not supporting any ugly ISA extensions, I'm
| only targeting ancient, simple hardware! And still I ran into
| the limitations of the elegant solution, and had to introduce
| ugly kludges to make it work.
|
| The saving grace is that I plan to rip out all of this manual
| work with a fully automatically-generated solution. Except
| that's only feasible in a decompiler, since the design of that
| solution starts by completely ignoring compatibility with
| assembly (ISAs turn out to be simpler if you think of them as
| "what do these bytes do" rather than "what does this
| instruction do")... and I'm worried that it's going to end up
| with inelegant kludges because the problem space more or less
| mandates it.
|
| > You could try and solve the problem by using a
| superoprimisation approach from the beginning. Look into the
| Souper paper which automatically generates peepholes for LLVM:
|
| One of the problems that Souper ran into is that LLVM IR is too
| abstract for superoptimization to be viable. Rather than the
| promise of an automatic peephole optimizer, it's instead
| morphed more into "here's some suggestions for possible
| peepholes". You need a really accurate cost model for
| superoptimization to work well, and since LLVM IR gets shoved
| through instruction selection and instruction scheduling, the
| link between LLVM instructions and actual instructions is just
| too tenuous to build the kind of cost model a superoptimizer
| needs (even if LLVM does have a very good cost model for the
| actual machine instructions!).
| fuhsnn wrote:
| >So one of the main problems you run into is that your
| elegant solution only works about 60-80% of the time. The
| rest of the time, you end up falling back onto near-
| unmaintainable, horribly inelegant kludges that end up having
| to exist
|
| This is generally true, though for small compiler backends
| they have the luxury to straight up refuse to support such
| use cases. Take QBE and Cranelift for example, the former
| lacks x87 support [1], the latter doesn't support varargs[2];
| which means either of them support the full x86-64 ABI for
| C99.
|
| [1]https://github.com/michaelforney/cproc?tab=readme-ov-
| file#wh...
|
| [2]https://github.com/bytecodealliance/wasmtime/issues/1030
| muth02446 wrote:
| I think you are generally correct but the two examples you
| gave "triggered" me ;-)
|
| What damaged would there be if gcc or LLVM did decide to
| not support x87 anymore. It is not much different from
| dropping an ISA like IA64. You can still use the older
| compilers if you need to.
|
| Similarly, what is varargs used for? Pretty much only for C
| and its unfortunate printf, scanf stdlib calls. If a
| backend decides not support C, all this headache goes away.
| The problem is, of course, that the first thing every new
| backend designer does is to write a C frontend.
| jcranmer wrote:
| > What damaged would there be if gcc or LLVM did decide
| to not support x87 anymore.
|
| For starters, you'd break every program using long double
| on x86.
|
| And as far as "complexities of the x86 ISA" goes, x87
| isn't really that high on the list. I mean, MMX is
| definitely more complex (and LLVM recently ripped out
| support for that). But even more complex than either of
| those would be anything touching AVX, AVX-512, or now
| AVX-10 stuff, and all the fun you get trying to build
| your systems to handle encoding the VEX or EVEX prefixes.
| o11c wrote:
| "Everything should be as simple as it can be but not
| simpler!" --Roger Sessions, loosely after Albert Einstein
| elvircrn wrote:
| No benchmarks yet?
| Ygg2 wrote:
| Does anyone else experience site disappearing on scroll?
| muizelaar wrote:
| I thought the sea-of-nodes choice was interesting.
|
| V8 has been moving away from sea-of-nodes. Here's a video where
| Ben Titzer is talking about V8's reasons for moving away from
| sea-of-nodes: https://www.youtube.com/watch?v=Vu372dnk2Ak&t=184s.
| Yasser, the author of Tilde, is is also in the video.
| o11c wrote:
| TL;DW version: sea of nodes requires a scheduling pass, which
| was taking 20% of their compilation time. But it sounds like
| there's a lot of legacy baggage, so ...
| IshKebab wrote:
| I dunno if "twice as fast as Clang" is very impressive. How fast
| is it compared to Clang 1.0?
|
| Also starting a new project like this in C is an interesting
| choice.
| oguz-ismail wrote:
| > interesting choice
|
| No serious alternative
| IshKebab wrote:
| Rust? Zig?
| twic wrote:
| Compilation is high-level work, so you could do it in any
| high-level language.
| einpoklum wrote:
| I'm not familiar with a lot of the acronyms and catch-phrases
| already in the first part of the article... let me try to make a
| bit of sense of this: IR = Intermediate
| Representation SSA = Single Static Assignment CFG =
| Control-Flow Graph (not Context-Free Grammar)
|
| And "sea of nodes" is this:
| https://en.wikipedia.org/wiki/Sea_of_nodes ... IIANM, that means
| that instead of assuming a global sequence of all program (SSA)
| instructions, which respects the dependecies - you only have a
| graph with the partial order defined by the dependencies, i.e.
| individual instructions are nodes that "float" in the sea.
| e4m2 wrote:
| https://github.com/RealNeGate/Cuik/blob/5c6f6ef9bfa983eb358a...
| orliesaurus wrote:
| The maintainer said that LLVM has 10M lines of code making it too
| hard to improve, so he's building its own. That sounds weird to
| me: but good luck I guess?
| coolThingsFirst wrote:
| Is it just me or I find it difficult to believe that 19 year olds
| can implement the LLVM alternative?
| uecker wrote:
| I don't find this difficult to believe.
| coolThingsFirst wrote:
| Seems far fetched but ok
| uecker wrote:
| A bit. But Linus Torvalds was not much older when he wrote
| Linux.
| mbrubeck wrote:
| Chris Lattner was 21 or 22 when he created LLVM.
| astrange wrote:
| Compilers aren't that hard. You'll miss out on user
| requirements by not knowing about them, but getting older isn't
| a good way to solve that; instead getting more people to work
| on your project is.
|
| (But you probably need to be older to be a good project
| manager.)
| quesera wrote:
| Never underestimate adolescents who are not distracted by
| ordinary teen drama.
|
| They exist, and have truly enviable amounts of time for
| projects.
|
| Also, don't _overestimate_ compilers (or kernels). They can be
| much simpler than they might seem! The difficult /tedious parts
| are optimizations(!) and broad compatibility with bizarre real-
| world stuff.
| negate32 wrote:
| It's a game of knowledge and I have time, if you think I don't
| know what I'm doing please just call it out so I can politely
| disagree :P
| ziofill wrote:
| Wow, my brain read "my LLM alternative" and I was genuinely
| confused for a while when reading the blog post :facepalm:
| mshockwave wrote:
| I'm definitely happy to see this happening. But I would like to
| point out two ingredients that constitute LLVM's success beyond
| academic merits: License and modularity. I'm not a lawyer so
| can't say much about the first one, all I can say is that I
| believe license is one of the main reasons Apple switched to LLVM
| decades ago. Modularity, on the other hand, is one of the most
| crucial features of LLVM and something GCC struggles to catch up
| even nowadays. I really hope Tilde can adopt the modularity
| philosophy, provide building blocks rather than just tools
| pjmlp wrote:
| Had it not been for GPL 3, or the resistance to have GCC being
| more modular, and Apple, followed by Google, would not have
| sponsored it.
|
| The idea behind LLVM isn't new per se, there have been other
| similar tools in the past, e.g. Amsterdam Compiler Toolkit.
| astrange wrote:
| LLVM is not very modular though. For instance there's no
| backwards/forwards compatibility in the IR.
| mshockwave wrote:
| modular in terms of using only some of the LLVM libraries
| without the need to pull the entire compiler into your
| project. In fact, many of the LLVM libraries have absolutely
| nothing to do with LLVM IR and have zero dependency on it.
| For instance, LLVMObject and LLVMDebugInfoDWARF. You can use
| those libraries to build useful tools, like your own objdump
| or just use it to read debug info.
| Night_Thastus wrote:
| I'm confused, is this some kind of re-post? I saw this exact same
| post with the exact same comments on it awhile ago. Could have
| been more than a couple of weeks. Very strange.
| jsnell wrote:
| It was posted three days ago, and got re-upped by the mods via
| the the second chance pool[0].
|
| [0] https://news.ycombinator.com/pool
| laweijfmvo wrote:
| > It's been 20 years and cruft has built up, time for a "redo".
|
| Ah.. is this one of those "I rewrote it and it's better" things,
| but when people inevitably discover issues that "cruft" was
| handling the author will blame the user?
| rc_mob wrote:
| What a strangely pessimistic and negative comment.
| snowfarthing wrote:
| I think this is more a problem with the nature of technology
| in general.
|
| If we want simple and fast, we can do that, but sometimes it
| doesn't cover the corner cases that the slow and complicated
| stuff does -- and as you fix those things, the "simple and
| fast" becomes "complicated and slow".
|
| But, as others have observed about GCC vs LLVM (with LLVM
| having had a similar life cycle), the added competition
| forced GCC to step up their game, and both projects have
| benefited from that competition -- even if, as time goes on,
| they get more and more similar to what each can do.
|
| I think all our efforts suffer from the effects of the Second
| Law of Thermodynamics: "You can't win. You can't break even.
| And it's the only game in town."
| tester756 wrote:
| You want to create LLVM alternative and you write it in C?
|
| I'm saying this as someone who uses LLVM daily and wishes that it
| was written in anything else than C/CPP,
|
| those languages bring so many cons that it is unreal.
|
| Slow compilation, mediocre tooling (cmake), terrible error
| messages, etc, etc.
|
| What's the point of starting with tech debt?
| triilman wrote:
| I not understand about IR or compiler backend but I know another
| LLVM alternative like QBE. https://c9x.me/compile/
___________________________________________________________________
(page generated 2025-01-24 23:00 UTC)