[HN Gopher] Compiling C++ with the Clang API
___________________________________________________________________
Compiling C++ with the Clang API
Author : ingve
Score : 91 points
Date : 2025-03-09 11:51 UTC (1 days ago)
(HTM) web link (maskray.me)
(TXT) w3m dump (maskray.me)
| fooblaster wrote:
| This is pretty cool. What are the use cases?
| wdpk wrote:
| compile c++ code at runtime for instance. Lots of use cases,
| most obvious one specializing/instantiating dense computational
| kernels on values only known at runtime... but so many more
| things would be possible if the compiler was just a reusable
| library.
| ska wrote:
| This is something Common Lisp got really right.
| yuserx wrote:
| ~22MB download for a full IDE
| https://www.kvraudio.com/forum/viewtopic.php?t=560622
| menaerus wrote:
| REPL and JIT come to mind.
| secondcoming wrote:
| I've wanted something like this to compile XML business rules
| to native code on the fly.
|
| I would have a script that converted the XML to C and then turn
| that into machine code that I'd load directly from the .text
| section of the binary (shellcode style).
|
| Turns out this is error prone because compilers can emit things
| like jump tables into the .rodata section, so you need that
| too. It's easier to just create a shared library.
| high_na_euv wrote:
| XML business rules? Is it xml-oriented-programming again?
| They were doing it in Java 25 years ago
| mshockwave wrote:
| I think this is exactly how Zig compiler does under the hood
| for C/C++ sources. So I guess you can do a similar thing for
| your own programming languages to support interop.
| t43562 wrote:
| What I always wanted from Clang and was not smart enough to
| understand was how to use it to get an AST or similar so that I
| could rewrite code.
|
| I was working on a big project with old code that needed a
| radical overhaul to make it ready for new boost and new C++
| versions. It seemed a huge task to do this accurately by hand. I
| also thought about combining this with some static analysis of a
| trivial kind - looking for behaviours that are not wrong in C++
| overall but were wrong in the context of that project.
|
| I know there are several tools for clang that do rewrites and the
| unfortunate problem was that clang at the time wasn't able to
| build that code for some reasons that I cannot remember now. The
| rewriting tools that were available were not very sophisticated
| from my memory and made it difficult to do the kinds of changes
| that I thought would be needed.
|
| So I'm not so interested in being able to _compile_ but being
| able to parse and then walk the parse tree in some very easy
| language like python.
|
| I think the answer now might be "libclang":
| https://pypi.org/project/libclang/
| menaerus wrote:
| libclang has been with us for ~15 years already.
|
| > I'm not so interested in being able to compile but being able
| to parse
|
| Chances are that you're not going to be able to walk the AST
| without getting errors unless the (lib)clang can cleanly parse
| the code. There are at least two prerequisites for that: (1)
| marrying your build system with the (lib)clang so that you pass
| the exact include directories, include paths, build flags, etc.
| for any given translation unit, and (2) ability to build the
| code with clang. If you use CMake then it's going to be a
| little bit easier. If not, I recommend first switching to it.
| You will also have to be somewhat creative about making it
| possible to correctly parse the code in the header files - a
| header file fundamentally isn't a translation-unit.
|
| Back in the days, I wrote some of the utilities built on top of
| the libclang and the process was very hairy. Although I pulled
| it off, for the reasons above it required a lot of unsexy work
| exactly around the integration of (CMake) build system,
| patching the codebase itself, and working around the design
| limitations imposed by the language itself. Occasionally,
| libclang wouldn't offer me enough control over AST so I also
| had to work around those limitations by patching the libclang
| code itself. libtooling seemingly gives more control but I have
| never tried it.
|
| If you still want to try it, I think your best bet is to find
| something similar that already exists and build upon it.
| yaantc wrote:
| castxml (https://github.com/CastXML/CastXML) may be what you
| want. It uses the Clang front-end to output an XML
| representation of a C or C++ parse tree. It is then possible to
| turn this into what you want. I've used it and seen it used to
| generate code to do endianess conversion of structures from
| headers, or RPC code generation for example.
|
| It can be used from Python through pygccxml
| (https://github.com/CastXML/pygccxml). The name comes from a
| previous instance, gccxml, based on the GCC front-end.
|
| Both castxml and pygccxml are packaged in Debian and Ubuntu.
| dataflow wrote:
| If all you want is _pure_ syntax with no semantics, check out
| tree sitter. You can get it past macros by pre-processing via
| alternative implementations in some cases.
|
| But most likely you want _some_ semantics. The "one weird
| trick" you can pull here is that if all you want is syntactic
| manipulation, you mostly don't have to care about semantics and
| codegen, or anything past that (like linking or embedding
| resources) -- and you don't really have to care about most of
| the compiler flags, either. That takes out a fair bit of
| complexity. Moreover you can do the migration to Clang file-by-
| file, and you can do that with #ifndef __clang__ wherever
| you're certain the code is unaffected by your migration (which
| you can detect via compiling the file).
|
| And on top of that you can use the most relaxed warnings
| possible in Clang - think MSVC compatibility flags and
| disabling all warnings and such.
|
| I'm not suggesting it's trivial, but I've done it before, and
| it was not as daunting as I first imagined it. If you haven't
| tried these already, I would definitely give it a shot for a
| few files, and see how long it takes you on average per file.
| inetknght wrote:
| > _I was working on a big project with old code that needed a
| radical overhaul to make it ready for new boost and new C++
| versions._
|
| You might find Kristen Shaker's CppCon 2023 talk to be
| intriguing.
|
| https://www.youtube.com/watch?v=torqlZnu9Ag
|
| > _I know there are several tools for clang that do rewrites
| and the unfortunate problem was that clang at the time wasn 't
| able to build that code for some reasons that I cannot remember
| now._
|
| Oh. Well that's unfortunate. You're probably going to have a
| bad time if _no_ version of clang is able to compile (eg,
| libclang probably won 't help). But GCC has something similar:
|
| https://stackoverflow.com/questions/15800230/how-can-i-dump-...
|
| Alas, I'm not familiar with either tool. I would find it
| interesting to read a long-form blog post from you with more
| information and what you end up doing!
| esbranson wrote:
| The Roslyn APIs are the .NET equivalent:
| https://learn.microsoft.com/en-us/dotnet/csharp/roslyn-sdk/c...
| HdS84 wrote:
| MS best idea was to create Source Generators. Absurdly powerful
| for some problems.
| pjmlp wrote:
| They still miss something like good old T4 templates though.
| tway223 wrote:
| I have been wondering if someone could improve golang's cgo infra
| using clang like what zig is doing..
___________________________________________________________________
(page generated 2025-03-10 23:01 UTC)