https://yeet.cx/blog/bpf-from-scratch-in-rust/

 
github
home discover blog | Light Dark
BPF From Scratch In Rust.
Posted on 2025-04-03

Author: Julian Goldstein

We're not saying yeet is the Dr. Dre of observability... but it's been
in the lab with a pen and a pad trying to get the kernel off its
back.

Not just at the level of "how do I write a tracing program," but "how
do I build an entire dynamic runtime on top of the Linux kernel's
deepest, weirdest, most misunderstood subsystem -- without losing our
minds (or yours)."

If you're just here to play around, we even built a sandbox where you
can run real BPF programs (safely!) without touching your local
machine.

For those who have never heard of BPF, you can think of it as a
virtual machine you can program to change the behavior of the Linux
kernel in a way that is guaranteed not to crash it! Linux statically
proves the safety and termination of any given program before
executing it in kernel space using a complex process called
verification.

Ken & Dennis

The more we push the limits of what BPF can do, the more we find
ourselves diving beneath the abstractions  -- unraveling low-level
internals, deciphering verifier riddles, and occasionally losing our
grip on reality... all so you don't have to.

So today, we're going to do something a little ridiculous, a little
beautiful, and (honestly) kind of empowering:

We're going to write a BPF program in Rust. From scratch.

 No macros.

 No frameworks.

 No hidden helpers.

 No getting roundhouse kicked in the face by the verifier
(Rex-Kwon-Do style, of course).

Rex Kwon Do

Setting up the Rust build toolchain

Let's start by creating a new Rust project:

cargo new bpf-from-scratch

Since the Rust toolchain uses LLVM as its backend to produce
byte-code, it should be perfectly capable of producing linking BPF
binaries that yeetd can run. We just have to supply the proper
configuration.

Here's the minimal .cargo/config.toml you'll need:

[build]
target = "bpfel-unknown-none"
rustflags = [
  "-C", "debuginfo=2",
  "-C", "link-arg=--btf",
  "-C", "panic=abort"
]

[unstable]
build-std = ["core"]

This tells cargo to build our Rust code for Linux's BPF virtual
machine without the standard library with debug info and BTF support
-- just what yeetd expects.

Writing the world's tiniest BPF program (in Rust)

Time to touch the void. We're going to write a real, working BPF
program in pure Rust.

No C anywhere. Just #![no_std], a few #[link_section]s, and direct
calls into the kernel -- with some inline BPF assembly to finish it
off.

This is the smallest thing we can give the verifier that still makes
the kernel go:

"Yeah, alright. You can run that."

This program is rather simple: It hooks into a tracepoint
(sys_enter_nanosleep), increments a counter, and uses
bpf_trace_printk() to print a message to the trace pipe.

Short, structured, and slightly cursed.

In the project's src/main.rs file we write:

#![no_std]
#![no_main]
#![feature(asm_experimental_arch)]
use core::arch::asm;
use core::panic::PanicInfo;

static mut COUNTER: u64 = 0;
static FMT: [u8; 16] = *b"hello kernel %d\0";

#[link_section = "license"]
#[no_mangle]
#[used]
static LICENSE: [u8; 12] = *b"Dual BSD/GPL";

#[link_section = "tracepoint/syscalls/sys_enter_nanosleep"]
#[no_mangle]
pub unsafe fn test(_ctx: *mut u8) -> i32 {
    asm!(
        r#"
        r1 = {0}
        r4 = *(u32 *)(r1 + 0)
        r4 += 1
        *(u32 *)(r1 + 0) = r4
        r1 = {1}
        r2 = 16
        r3 = r4
        call 6
    "#,
    in(reg) &raw const COUNTER as *const u64,
    in(reg) &FMT as *const u8,
    );

    0
}

#[panic_handler]
fn panic(_info: &PanicInfo) -> ! {
    loop {}
}

We build with:

cargo +nightly build --release

[DEL:[?][?] Note: At the time of this writing the bpfel-unknown-none
target is only available on nightly:DEL]

[Edit: This actually does work on stable. Thank you to niklaskorz for
pointing out the correction here.]

To run this. You need to install yeet which can be done via:

curl -fsSL https://yeet.cx | sh

This script installs both the yeet CLI and the yeet system daemon /
dynamic runtime yeetd

Finally, to get yeetd to recognize and run your freshly built BPF
program, you'll need to add a YEET file to the root of your project:

[info]
name = "bpf-from-scratch"
object = "target/bpfel-unknown-none/release/bpf-from-scratch"

This file is analogous to a systemd service file -- it's a simple
descriptor that tells yeetd where your BPF program is located on the
file system and where you can supply configuration on how you want
yeetd to dynamically link and load the BPF object into the kernel.

We can register this newly created BPF object with the daemon by
running the following command inside of the root of your rust
project.

sudo yeet add .

Followed by a start command:

sudo yeet start bpf-from-scratch

You can verify it started correctly by running:

sudo yeet ls

To see the output we can write:

sudo yeet trace

The result should be:

tailscaled-714     [001] ...21  2166.021924: bpf_trace_printk: hello kernel 29252
tailscaled-714     [001] ...21  2166.021999: bpf_trace_printk: hello kernel 29253
tailscaled-714     [001] ...21  2166.022033: bpf_trace_printk: hello kernel 29254
tailscaled-714     [001] ...21  2166.022114: bpf_trace_printk: hello kernel 29255
tailscaled-714     [001] ...21  2166.022189: bpf_trace_printk: hello kernel 29256
tailscaled-714     [001] ...21  2166.022264: bpf_trace_printk: hello kernel 29257
tailscaled-714     [001] ...21  2166.022338: bpf_trace_printk: hello kernel 29258
tailscaled-714     [001] ...21  2166.022412: bpf_trace_printk: hello kernel 29259
tailscaled-714     [001] ...21  2166.022486: bpf_trace_printk: hello kernel 29260
tailscaled-714     [001] ...21  2166.022580: bpf_trace_printk: hello kernel 29261
tailscaled-714     [001] ...21  2166.022714: bpf_trace_printk: hello kernel 29262
tailscaled-714     [001] ...21  2166.022928: bpf_trace_printk: hello kernel 29263
tailscaled-714     [001] ...21  2166.023302: bpf_trace_printk: hello kernel 29264
tailscaled-714     [001] ...21  2166.023997: bpf_trace_printk: hello kernel 29265
tailscaled-714     [001] ...21  2166.025337: bpf_trace_printk: hello kernel 29266
tailscaled-714     [001] ...21  2166.027952: bpf_trace_printk: hello kernel 29267
tailscaled-714     [001] ...21  2166.033131: bpf_trace_printk: hello kernel 29268
tailscaled-714     [001] ...21  2166.041890: bpf_trace_printk: hello kernel 29269
tailscaled-714     [001] ...21  2166.041965: bpf_trace_printk: hello kernel 29270
tailscaled-714     [001] ...21  2166.042218: bpf_trace_printk: hello kernel 29271
tailscaled-714     [001] ...21  2166.042293: bpf_trace_printk: hello kernel 29272
tailscaled-714     [001] ...21  2166.049350: bpf_trace_printk: hello kernel 29273
tailscaled-714     [001] ...21  2166.049425: bpf_trace_printk: hello kernel 29274
tailscaled-714     [001] ...21  2166.064448: bpf_trace_printk: hello kernel 29275
tailscaled-714     [001] ...21  2166.064864: bpf_trace_printk: hello kernel 29276
tailscaled-714     [001] ...21  2166.071629: bpf_trace_printk: hello kernel 29277
tailscaled-714     [001] ...21  2166.083817: bpf_trace_printk: hello kernel 29278
tailscaled-714     [001] ...21  2166.084231: bpf_trace_printk: hello kernel 29279
tailscaled-714     [001] ...21  2166.093196: bpf_trace_printk: hello kernel 29280
tailscaled-714     [001] ...21  2166.105598: bpf_trace_printk: hello kernel 29281
tailscaled-714     [001] ...21  2166.106027: bpf_trace_printk: hello kernel 29282
tailscaled-714     [001] ...21  2166.108765: bpf_trace_printk: hello kernel 29283
tailscaled-714     [001] ...21  2166.109107: bpf_trace_printk: hello kernel 29284

You can see on this system in particular tailscaled is triggering
most of the activity on sys_enter_nanosleep

Finally to stop it all you need to do is:

sudo yeet stop bpf-from-scratch

You can verify it stopped correctly by running:

sudo yeet ls

[?] Stripping It Down: What You Actually Need

Okay, that's the entire program -- but let's be honest:

A lot of it is just scaffolding to keep the compiler and the kernel
from throwing a shoe at you.

Who throws a shoe

So let's separate the interesting parts (what makes this a BPF
program) from the uninteresting ones (what makes it compile without
exploding).

 The Interesting Bits

This part is what makes the whole thing tick:

#[link_section = "tracepoint/syscalls/sys_enter_nanosleep"]
#[no_mangle]
pub unsafe fn test(_ctx: *mut u8) -> i32 {
    asm!(
        r#"
        r1 = {0}
        r4 = *(u32 *)(r1 + 0)
        r4 += 1
        *(u32 *)(r1 + 0) = r4
        r1 = {1}
        r2 = 16
        r3 = r4
        call 6
    "#,
    in(reg) &raw const COUNTER as *const u64,
    in(reg) &FMT as *const u8,
    );

    0
}

  * The #[link_section] specifies on which of Linux's thousands of
    tracepoints yeetd should attach this code.
  * test is the actual entry point of the program.
  * asm! : Wraps raw BPF bytecode, hand-written and beamed directly
    into verifier hell.

 Black Magic

Let's walk through the asm! block. It's small, but it packs a lot --
and every line is doing something very specific:

From a high level, you can think of the asm! block as being split
into 2 sections:

asm!(
   [I. Incrementing the counter by 1]
   [II. Printing the counter's value to the kernel's trace pipe]
)

I. Incrementing the counter by 1

In the first section we simply load the counter's value from memory,
increment it and store it back.

r1 = {memory address of COUNTER}   // r1 points to our static counter
r4 = *(u32 *)(r1 + 0)              // load the value of the counter into r4
r4 += 1                            // increment it
*(u32 *)(r1 + 0) = r4              // store it back into memory

II. Printing the counter's value to the kernel's trace pipe

In BPF, helper functions like bpf_trace_printk() are invoked using a
simple register-based calling convention:

  * Functions can take up to 5 arguments
  * Arguments must be loaded in order into r1 to r5
  * The return value is passed back in r0
  * Calls are made using the call instruction, followed by a helper
    ID number, which traps back into the kernel so it can service
    your request similar to a system call for regular programs.

In our case, we're calling bpf_trace_printk() (helper ID 6), which
has the signature:

int bpf_trace_printk(const char *fmt, int fmt_size, ...);

So in our program, we set things up like this:

r1 = {pointer to format string} // r1 = pointer to "hello kernel %d\0"
r2 = 16                         // r2 = length of the format string
r3 = r4                         // r3 = the counter value
call 6                          // call bpf_trace_printk(fmt, fmt_size, ...)

Conclusion

You just wrote a working BPF program in raw Rust and loaded it into
the kernel with yeet -- no macros, no C, no training wheels.

 Hooks into the kernel

 Passes the verifier

 Prints live output to the trace pipe

It's low-level, verifier-safe magic... And you built it entirely from
scratch, in Rust!

This is the kind of register allocation bars we spit every day while
building yeet -- not because we want you to suffer... but because we
know how to go from zero to Dre on the verifier all so you can just...
yeet start and get answers.

Dr. Dre

Wanna see how far down the BPF rabbit hole goes? Check out our
sandbox

You can also see what packages we have available via the yeet package
manager which is our ever-growing package index of pre-made BPF
packages.

Welcome to ring-zero. Welcome to yeet

Knibb High Football Rules