[HN Gopher] OpenTelemetry profiles enters public alpha
       ___________________________________________________________________
        
       OpenTelemetry profiles enters public alpha
        
       Author : tanelpoder
       Score  : 133 points
       Date   : 2026-03-26 16:14 UTC (6 hours ago)
        
 (HTM) web link (opentelemetry.io)
 (TXT) w3m dump (opentelemetry.io)
        
       | secondcoming wrote:
       | > Continuously capturing low-overhead performance profiles in
       | production
       | 
       | It suprises me that anything designed by the OTel community could
       | ever meet 'low-overhead' expectations.
        
         | phillipcarter wrote:
         | Anything to actually add?
        
         | felixge wrote:
         | OTel Profiling SIG maintainer here: I understand your concern,
         | but we've tried our best to make things efficient across the
         | protocol and all involved components.
         | 
         | Please let us know if you find any issues with what we are
         | shipping right now.
        
         | tanelpoder wrote:
         | The reference implementation of the profiler [1] was originally
         | built by the Optimyze team that Elastic then acquired (and
         | donated to OTEL). That team is very good at what they do. For
         | example, they invented the .eh_frame walking technique to get
         | stack traces from binaries without frame pointers enabled.
         | 
         | Some of the OGs from that team later founded Zymtrace [2] and
         | they're doing the same for profiling what happens _inside_ GPUs
         | now!
         | 
         | [1] https://github.com/open-telemetry/opentelemetry-ebpf-
         | profile...
         | 
         | [2] https://zymtrace.com/article/zero-friction-gpu-profiler/
        
           | rnrn wrote:
           | > For example, they invented the .eh_frame walking technique
           | to get stack traces from binaries without frame pointers
           | enabled.
           | 
           | This is not an accurate summary of what they developed.
           | 
           | Using .eh_frame to unwind stacks without frame pointers is
           | not novel - it is exactly what it is for and perf has had an
           | implementation doing it since ~2010. The problem is the
           | kernel support for this was repeatedly rejected so the kernel
           | samples kilobytes of stack and then userspace does the unwind
           | 
           | What they developed is an implementation of unwinding from an
           | eBPF program running in the kernel using data from eh_frame.
        
             | tanelpoder wrote:
             | True, I should have been more specific about the context:
             | 
             | Their invention is about pushing down the .eh_frame walking
             | to kernel space, so you don't need to ship large chunks of
             | stack memory to userspace for post-processing. And eBPF
             | code is the executor of that "pushed down" .eh_frame
             | walking.
             | 
             | The GitHub page mentions a patent on this too:
             | https://patents.google.com/patent/US11604718B1/en
        
       | genthree wrote:
       | Relatedly: Has anyone profiled the performance and reliability
       | characteristics of rsyslogd (Linux and FreeBSD distributed
       | syslogger, maybe other platforms too) in its mode where it's
       | shipping logs to a central node? I've configured and used it with
       | relatively small (high single digit nodes, bursts of activity to
       | a million or two requests per minute or so) set-ups but have
       | wondered if there's a reason it's not a more common solution for
       | distributed logging and tracing (yes it doesn't solve the UI
       | problem for those, but it does solve collecting your logs)
       | 
       | Like... has anyone done a Jepsen-like stress test on rsyslogd and
       | shared the results? I've half-assedly looked before and not been
       | able to find anything.
        
         | nesarkvechnep wrote:
         | People don't care about syslog. 98% of my colleagues haven't
         | heard of it.
        
           | malux85 wrote:
           | You are drawing a global conclusion from a tiny sample!
        
         | jbaiter wrote:
         | We're doing this with a few dozen GiBs of logs a day (rsylog ->
         | central rsylog -> elasticsearch). It works reliably, but the
         | config is an absolute nightmare, documentation is a mixed bag
         | and troubleshooting often involves deep dives into the C code.
         | We're planning to migrate to Alloy+Loki.
        
       | ollien wrote:
       | Very excited for this. We've used the Elixir version of this at
       | $WORK a handful of times and have found it exceptionally useful.
        
       | SEJeff wrote:
       | I wonder how this compares to grafana pyroscope, which is really
       | good for this sort of thing and already quite mature:
       | 
       | https://grafana.com/oss/pyroscope/
       | 
       | https://github.com/grafana/pyroscope
        
         | ollien wrote:
         | As far as I'm aware, Pyroscope itself is not a profiler, but a
         | place you can send/query profiles. OpenTelemtry is releasing a
         | profiler, so they don't compare. One can be used with the
         | other.
        
         | sciurus wrote:
         | You can send profiles collected by opentelemetry to pyroscope.
         | 
         | https://grafana.com/docs/pyroscope/latest/configure-client/o...
        
       ___________________________________________________________________
       (page generated 2026-03-26 23:00 UTC)