https://maskray.me/blog/2025-03-09-compiling-c++-with-clang-api

MaskRay

Home Archives Feeds TIL Presentations [github] [twitter]  
[                    ]
2025-03-09

Compiling C++ with the Clang API

This post describes how to compile a single C++ source file to an
object file with the Clang API. Here is the code. It behaves like a
simplified clang executable that handles -c and -S.

1 cat > main.cc <<eof

1   #include <clang/CodeGen/CodeGenAction.h> // EmitObjAction
2   #include <clang/Driver/Compilation.h>
3   #include <clang/Driver/Driver.h>
4   #include <clang/Frontend/CompilerInstance.h>
5   #include <clang/Frontend/FrontendOptions.h>
6   #include <llvm/Config/llvm-config.h>   // LLVM_VERSION_MAJOR
7   #include <llvm/Support/TargetSelect.h> // LLVMInitialize*
8   #include <llvm/Support/VirtualFileSystem.h>
9   
10  using namespace clang;
11  
12  constexpr llvm::StringRef kTargetTriple = "x86_64-unknown-linux-gnu";
13  
14  namespace {
15  struct DiagsSaver : DiagnosticConsumer {
16    std::string message;
17    llvm::raw_string_ostream os{message};
18  
19    void HandleDiagnostic(DiagnosticsEngine::Level diagLevel, const Diagnostic &info) override {
20      DiagnosticConsumer::HandleDiagnostic(diagLevel, info);
21      const char *level;
22      switch (diagLevel) {
23      default:
24        return;
25      case DiagnosticsEngine::Note:
26        level = "note";
27        break;
28      case DiagnosticsEngine::Warning:
29        level = "warning";
30        break;
31      case DiagnosticsEngine::Error:
32      case DiagnosticsEngine::Fatal:
33        level = "error";
34        break;
35      }
36  
37      llvm::SmallString<256> msg;
38      info.FormatDiagnostic(msg);
39      auto &sm = info.getSourceManager();
40      auto loc = info.getLocation();
41      auto fileLoc = sm.getFileLoc(loc);
42      os << sm.getFilename(fileLoc) << ':' << sm.getSpellingLineNumber(fileLoc)
43         << ':' << sm.getSpellingColumnNumber(fileLoc) << ": " << level << ": "
44         << msg << '\n';
45      if (loc.isMacroID()) {
46        loc = sm.getSpellingLoc(loc);
47        os << sm.getFilename(loc) << ':' << sm.getSpellingLineNumber(loc) << ':'
48           << sm.getSpellingColumnNumber(loc) << ": note: expanded from macro\n";
49      }
50    }
51  };
52  }
53  
54  static std::pair<bool, std::string> compile(int argc, char *argv[]) {
55    auto fs = llvm::vfs::getRealFileSystem();
56    DiagsSaver dc;
57    std::vector<const char *> args{"clang"};
58    args.insert(args.end(), argv + 1, argv + argc);
59    auto diags = CompilerInstance::createDiagnostics(
60  #if LLVM_VERSION_MAJOR >= 20
61        *fs,
62  #endif
63        new DiagnosticOptions, &dc, false);
64    driver::Driver d(args[0], kTargetTriple, *diags, "cc", fs);
65    d.setCheckInputsExist(false);
66    std::unique_ptr<driver::Compilation> comp(d.BuildCompilation(args));
67    const auto &jobs = comp->getJobs();
68    if (jobs.size() != 1)
69      return {false, "only support one job"};
70    const llvm::opt::ArgStringList &ccArgs = jobs.begin()->getArguments();
71  
72    auto invoc = std::make_unique<CompilerInvocation>();
73    CompilerInvocation::CreateFromArgs(*invoc, ccArgs, *diags);
74    auto ci = std::make_unique<CompilerInstance>();
75    ci->setInvocation(std::move(invoc));
76    ci->createDiagnostics(*fs, &dc, false);
77    // Disable CompilerInstance::printDiagnosticStats, which might display "2 warnings generated."
78    ci->getDiagnostics().getDiagnosticOptions().ShowCarets = false;
79    ci->createFileManager(fs);
80    ci->createSourceManager(ci->getFileManager());
81  
82    LLVMInitializeX86AsmParser();
83    LLVMInitializeX86AsmPrinter();
84    LLVMInitializeX86Target();
85    LLVMInitializeX86TargetInfo();
86    LLVMInitializeX86TargetMC();
87  
88    switch (ci->getFrontendOpts().ProgramAction) {
89    case frontend::ActionKind::EmitObj: {
90      EmitObjAction action;
91      ci->ExecuteAction(action);
92    } break;
93    case frontend::ActionKind::EmitAssembly: {
94      EmitAssemblyAction action;
95      ci->ExecuteAction(action);
96    } break;
97    default:
98      return {false, "unhandled action"};
99    }
100   return {true, std::move(dc.message)};
101 }
102 
103 int main(int argc, char *argv[]) {
104   auto [ok, err] = compile(argc, argv);
105   llvm::errs() << err;
106 }

1 eof

Building the code with CMake

Let's write a CMakeLists.txt that links against the needed Clang and
LLVM libraries.

1  cat > CMakeLists.txt <<eof
2  project(cc)
3  cmake_minimum_required(VERSION 3.16)
4  find_package(LLVM REQUIRED CONFIG)
5  find_package(Clang REQUIRED CONFIG)
6  
7  include_directories(${LLVM_INCLUDE_DIRS} ${CLANG_INCLUDE_DIRS})
8  add_executable(cc main.cc)
9  
10 if(NOT LLVM_ENABLE_RTTI)
11   target_compile_options(cc PRIVATE -fno-rtti)
12 endif()
13 
14 if(CLANG_LINK_CLANG_DYLIB)
15   target_link_libraries(cc PRIVATE clang-cpp)
16 else()
17   target_link_libraries(cc PRIVATE
18     clangAST
19     clangBasic
20     clangCodeGen
21     clangDriver
22     clangFrontend
23     clangLex
24     clangParse
25     clangSema
26   )
27 endif()
28 
29 if(LLVM_LINK_LLVM_DYLIB)
30   target_link_libraries(cc PRIVATE LLVM)
31 else()
32   target_link_libraries(cc PRIVATE LLVMOption LLVMSupport LLVMTarget
33     LLVMX86AsmParser LLVMX86CodeGen LLVMX86Desc LLVMX86Info)
34 endif()
35 eof

We need an LLVM and Clang installation that provides both lib/cmake/
llvm/LLVMConfig.cmake and lib/cmake/clang/ClangConfig.cmake. You can
grab these from system packages (dev versions may be required) or
build LLVM yourself-I'll skip the detailed steps here. For a DIY
build, use:

1 # cmake ... -DLLVM_ENABLE_PROJECTS='clang'
2 
3 ninja -C out/stable clang-cmake-exports clang

No install step is needed. Next, create a build directory with the
CMake configuration above:

1 cmake -S. -Bout/debug -G Ninja -DCMAKE_BUILD_TYPE=Debug -DCMAKE_CXX_COMPILER=$HOME/Stable/bin/clang++ -DCMAKE_PREFIX_PATH="$HOME/llvm/out/stable"
2 ninja -C out/debug

I've set a prebuilt Clang as CMAKE_CXX_COMPILER-just a habit of mine.
llvm-project isn't guaranteed to build warning-free with GCC, since
GCC -Wall -Wextra has many false positives and LLVM developers avoid
cluttering the codebase.

1 % echo 'void f() {}' > a.cc
2 % out/debug/cc -S a.cc && head -n 5 a.s
3         .file   "a.cc"
4         .text
5         .globl  _Z1fv                           # -- Begin function _Z1fv
6         .p2align        4
7         .type   _Z1fv,@function
8 % out/debug/cc -c a.cc && ls a.o
9 a.o: ELF 64-bit LSB relocatable, x86-64, version 1 (SYSV), not stripped

Anonymous files

The input source file and the output ELF file are stored in the
filesystem. We could create a temporary file and delete it with a
RAII class llvm::FileRemover:

1 std::error_code ec = llvm::sys::fs::createTemporaryFile("clang", "cc", fdIn, tempPath);
2 llvm::raw_fd_stream osIn(fdIn, /*ShouldClose=*/true);
3 llvm::FileRemover remover(tempPath);

On Linux, we could utilzie memfd_create to create a file in RAM with
a volatile backing storage.

1  int fdIn = memfd_create("input", MFD_CLOEXEC);
2  if (fdIn < 0)
3    return {"", "failed to create input memfd"};
4  int fdOut = memfd_create("output", MFD_CLOEXEC);
5  if (fdOut < 0) {
6    close(fdIn);
7    return {"", "failed to create output memfd"};
8  }
9  
10 std::string pathIn = "/proc/self/fd/" + std::to_string(fdIn);
11 std::string pathOut = "/proc/self/fd/" + std::to_string(fdOut);
12 
13 // clang -c -xc++ /proc/self/fd/3 -o /proc/self/fd/4

LLVMInitialize*

To generate x86 code, we need a few LLVM X86 libraries defined by
llvm/lib/Target/X86/**/CMakeLists.txt files.

1 LLVMInitializeX86AsmPrinter();
2 LLVMInitializeX86Target();
3 LLVMInitializeX86TargetInfo();
4 LLVMInitializeX86TargetMC();

If inline assembly is used, we will also need the AsmParser library:

1 LLVMInitializeX86AsmParser();

We could also call LLVMInitializeAll* functions instead, which
initialize all supported targets (build-time LLVM_TARGETS_TO_BUILD).

Here are some notes about the LLVMX86 libraries:

  * LLVMX86Info: llvm/lib/Target/X86/TargetInfo/
  * LLVMX86Desc: llvm/lib/Target/X86/MCTargetDesc/ (depends on
    LLVMX86Info)
  * LLVMX86AsmParser: llvm/lib/Target/X86/AsmParser (depends on
    LLVMX86Info and LLVMX86Desc)
  * LLVMX86CodeGen: llvm/lib/Target/X86/ (depends on LLVMX86Info and
    LLVMX86Desc)

EmitAssembly and EmitObj

The code supports two frontend actions, EmitAssembly (-S) and EmitObj
(-c).

You could also utilize the API in clang/include/clang/FrontendTool/
Utils.h, but that would pull in another library clangFrontendTool
(different from clangFrontend).

Diagnostics

The diagnostics system is quite complex. We have DiagnosticConsumer,
DiagnosticsEngine, and DiagnosticOptions.

1 DiagnosticsEngine
2 +- DiagnosticIDs (defines diagnostics)
3 +- SourceManager (provides locations)
4 +- DiagnosticOptions (configures output)
5 +- DiagnosticConsumer (handles output)
6    +- Diagnostic (individual message)

We define a simple DiagnosticConsumer that handles notes, warnings,
errors, and fatal errors. When macro expansion comes into play, we
report two key locations:

  * The physical location (fileLoc), where the expanded token
    triggers an issue-matching Clang's error line, and
  * The spelling location within the macro's replacement list
    (sm.getSpellingLoc(loc)).

Although Clang also highlights intermediate locations for chained
expansions, our simple approach offers a solid approximation.

1  % cat a.h
2  #define FOO(x) x + 1
3  % cat a.cc
4  #include "a.h"
5  #define BAR FOO
6  void f() {
7    int y = BAR("abc");
8  }
9  % out/debug/cc -c -Wall a.cc
10 a.cc:4:11: warning: adding 'int' to a string does not append to the string
11 ./a.h:1:18: note: expanded from macro
12 a.cc:4:11: note: use array indexing to silence this warning
13 ./a.h:1:18: note: expanded from macro
14 a.cc:4:7: error: cannot initialize a variable of type 'int' with an rvalue of type 'const char *'
15 % clang -c -Wall a.cc
16 a.cc:4:11: warning: adding 'int' to a string does not append to the string [-Wstring-plus-int]
17     4 |   int y = BAR("abc");
18       |           ^~~~~~~~~~
19 a.cc:2:13: note: expanded from macro 'BAR'
20     2 | #define BAR FOO
21       |             ^
22 ./a.h:1:18: note: expanded from macro 'FOO'
23     1 | #define FOO(x) x + 1
24       |                ~~^~~
25 a.cc:4:11: note: use array indexing to silence this warning
26 a.cc:2:13: note: expanded from macro 'BAR'
27     2 | #define BAR FOO
28       |             ^
29 ./a.h:1:18: note: expanded from macro 'FOO'
30     1 | #define FOO(x) x + 1
31       |                  ^
32 a.cc:4:7: error: cannot initialize a variable of type 'int' with an rvalue of type 'const char *'
33     4 |   int y = BAR("abc");
34       |       ^   ~~~~~~~~~~
35 1 warning and 1 error generated.

We call a convenience function CompilerInstance::ExecuteAction, which
wraps lower-level API like BeginSource, Execute, and EndSource.
However, it will print 1 warning and 1 error generated. unless we set
ShowCarets to false.

clang::createInvocation

clang::createInvocation, renamed from createInvocationFromCommandLine
in 2022, combines clang::Driver::BuildCompilation and
clang::CompilerInvocation::CreateFromArgs. While it saves a few lines
for certain tasks, it lacks the flexibility we need for our specific
use cases.

Share

  * clang
  * llvm

Older
Migrating comments to giscus

Popular

Tag Cloud

adc ai9 algorithm arm asc assebmly assembler assembly automaton
awesome bctf binary binutils bmc build system c c++ ccls cgc chroot
clang clang-format codinsanity coffee script compiler compression
computer security contest cpp csv ctf data structure debug defcon
desktop docker elf emacs email emoji emscripten event expect ext4
fdpic feeds firmware floating point forensics fp freebsd game gcc gdb
gentoo github glibc graph graph drawing gtk hacker culture hackerrank
hanoi haskell hpc image inotify ipsec irc isc j javascript josephus
problem jq kernel kythe ld leetcode libunwind linker linux lld lldb
llvm lsp m68k makefile math maze mirror ml musl mutt n-body neovim
network nginx nim nlp node.js noip notmuch npm ocaml offlineimap oi
oj openwrt parallel parser generator perl powerpc presentation puzzle
python qq radare2 regex regular expression reverse engineering review
riscv router rtld ruby ructfe s390x sanitizer scheme search security
shell ssh stringology student festival puzzle suffix array suffix
automaton summary suricata telegram telegramircd terminal tls
traversal tree trendmicro udev unicode unix usb vim vpn vte wargame
web analytics webqqircd website wechat wechatircd window manager
windows x86 xbindkeys xmonad xz yanshi

Blogroll

  * BYVoid
  * fqj1994
  * ppwwyyxx

(c) 2025 MaskRay
Powered by Hexo
Home Archives Feeds TIL Presentations