[HN Gopher] Show HN: bpfquery - experimenting with compiling SQL...
___________________________________________________________________
Show HN: bpfquery - experimenting with compiling SQL to bpf(trace)
Hello! The last few weeks I've been experimenting with compiling
sql queries to bpftrace programs and then working with the results.
bpfquery.com is the result of that, source available at
https://github.com/zmaril/bpfquery. It's a very minimal sql to
bpftrace compiler that lets you explore what's going on with your
systems. It implements queries, expressions, and
filters/wheres/predicates, and has a streaming pivot table
interface built on https://perspective.finos.org. I am still
figuring out how to do windows, aggregations and joins though, but
the pivot table interface actually lets you get surprisingly far. I
hope you enjoy it!
Author : knuckleheads
Score : 47 points
Date : 2024-08-27 14:55 UTC (3 days ago)
(HTM) web link (bpfquery.com)
(TXT) w3m dump (bpfquery.com)
| password4321 wrote:
| This seems like it will be awesome, kind of like WMI but
| ridiculously more capable. Don't be discouraged that it didn't
| catch on it's first time on HN!
|
| I'm on Firefox Mobile on Android and the first time I visited the
| page the 3rd/data section just kept refreshing continuously
| rather than showing results (maybe showed results the first load,
| but then refreshes of just rows with a dash?). On the second
| visit to the page the 2nd/C translation section failed to load.
| knuckleheads wrote:
| Thank you very much for your very kind comment!!
|
| And I will endeavor to fix those bugs. I put the web interface
| on over the last few days and when it works it is great, but
| often times, it does not work. Thank you for looking at it!
| password4321 wrote:
| Maybe you got a second chance bump back onto the front page,
| nice!
|
| It looks like the issues I reported earlier have been
| resolved, though the second time it took a while to load
| (hopefully due to all the traffic!).
| knuckleheads wrote:
| Awesome and yeah I had to put some limiters in place to
| make it work with the visitors. It's not a performant
| website yet, got some multiplexing that needs to happen I
| think first.
| 3r7j6qzi9jvnve wrote:
| > --I ran ctags across the linux kernel source code, pulled out
| all the signatures and then reference the signature when
| compiling the query, so you don't have to do a bunch of casts in
| the query.
|
| note you don't need the casts if you use kfuncs instead, which
| also let you reference arguments by name (from
| https://github.com/bpftrace/bpftrace/blob/master/man/adoc/bp...
| ): kfunc:tcp_connect { if
| (args->sk->__sk_common.skc_daddr == (uint32)pton("127.0.0.1"))
| ... }
|
| With that said, kfuncs don't work (yet?) on aarch64, so this is
| great for me -- I'll definitely give it a try next time I need
| it.
|
| (EDIT: formatting)
| knuckleheads wrote:
| oh sick! thank you for letting me know. I will look into using
| this more for sure. I am curious, do you know if there is a way
| to access the arguments for kfuncs ahead of time? does the
| kernel provide this information somehow at runtime, or is it
| all still implicit?
|
| Also, you might also enjoy https://github.com/zmaril/hancock
| which is some of the code I've been using to run ctags across
| the versions of the kernel.
|
| (edit: I checked bpftrace -l and saw that it does have the
| arguments and type structs ahead of time, which is absolutely
| perfect, thank you very much! Super helpful.)
| evgpbfhnr wrote:
| Try this: bpftrace -lv kfunc:*
|
| (or any specific function instead of wildcard) will get you
| all kfuncs and their respective arguments.
|
| output snippet:
| kfunc:vmlinux:tcp_conn_request struct
| request_sock_ops * rsk_ops const struct
| tcp_request_sock_ops * af_ops struct sock * sk
| struct sk_buff * skb int retval
| kfunc:vmlinux:tcp_connect struct sock * sk
| int retval kfunc:vmlinux:tcp_create_openreq_child
| const struct sock * sk struct request_sock * req
| struct sk_buff * skb struct sock * retval
|
| I'm not sure how to get this info without bpftrace itself -
| bpftool might have the info available somehow?
| knuckleheads wrote:
| That's awesome! Very helpful, thank you. I had been holding
| off on parsing that output and combining it with vmlinux.h,
| but that just jumped up way higher priority. Thanks again!
| westurner wrote:
| RIL about how the ebpf verifier attempts to prevent infinite
| loops given rule ordering and rewriting transformations.
|
| There are many open query planners; maybe most are hardly
| reusable.
|
| There's a wasm-bpf; and also duckdb-wasm, sqlite in WASM with
| replication and synchronization, datasette-lite, JupyterLite
|
| wasm-bpf: https://github.com/eunomia-bpf/wasm-bpf#how-it-works
|
| Does this make databases faster or more efficient? Is there
| process or query isolation?
___________________________________________________________________
(page generated 2024-08-30 23:01 UTC)