[HN Gopher] Introducing Parca - Open Source Continuous Profiling
___________________________________________________________________
Introducing Parca - Open Source Continuous Profiling
Author : kakkoyun
Score : 41 points
Date : 2021-10-08 16:02 UTC (6 hours ago)
(HTM) web link (www.polarsignals.com)
(TXT) w3m dump (www.polarsignals.com)
| brancz wrote:
| Hey! Founder of Polar Signals here! We are super excited to be
| announcing Parca[1] an open source continuous profiling storage
| and query engine as well as an eBPF based profiler. Oh, and also
| we got funded! Polar Signals was founded almost to the day a year
| ago, and we launched a private beta for our product earlier this
| year[2]. We are long time Prometheus, Kubernetes, Thanos, and
| Cortex maintainers, which are all very performance sensitive so
| we wished we had had access to continuous profiling tools when we
| first started working on those projects.
|
| What is continuous profiling? Instead of capturing profiling data
| on-demand and as a manual process, continuous profiling always
| captures profiling data, and stores it to be queried over time to
| aggregate and compare.
|
| Using our private beta we explored further what our users want
| most from continuous profiling, and we identified it to be:
|
| * Cost saving: We discovered all companies to have low hanging
| fruit that can easily be optimized away, often saving easily
| 20-30% in resources. The only problem is companies don't know
| because they don't measure. * Optimizing individual interactions:
| Latencies of interactions have a proven correlation between
| converting a customer or not. It's incredibly hard to obtain
| profiling data at the right time, but if it's always captured
| it's just a matter of querying it. * Understanding change:
| Performance engineering tends to be very sensitive and small
| changes can have a big impact, therefore the ability to deeply
| understand what changed from a point in time or a newly rolled
| out version of a software is invaluable for optimizing but also
| in order to understand incidents better.
|
| Parca features a fully from scratch designed storage which
| implements multiple optimizations that are unique to profiling
| data, specifically our sparse XOR encoding for uint64 series is
| what allows us to store this data at raw resolution. More
| detailed information can be found in the storage docs[3]. And
| with the newly open sourced Parca Agent[4], we can capture CPU
| profiling data with practically unnoticeable overhead for the
| profiled processes, it automatically discovers all containers in
| a Kubernetes cluster and starts profiling them with zero
| instrumentation needed!
|
| This is only the beginning of the project, but we are excited to
| work with the community to make this project more and more
| useful!
|
| We will be hanging out in the comments section today, so please
| leave your feedback, criticisms, or just say hi!
|
| [1] https://parca.dev/ [2]
| https://www.polarsignals.com/blog/posts/2021/02/09/announcin...
| [3] https://www.parca.dev/docs/storage [4]
| https://www.parca.dev/docs/parca-agent-design
| newman314 wrote:
| How does this compare to Prodfiler? My understanding is that
| prodfiler uses ebpf under the hood too.
| brancz wrote:
| Thanks, that's a great question! The eBPF part is quite
| similar. One difference is that we strongly believe the
| component needs to be open source for security reasons, since
| anything that uses eBPF essentially needs root. Aside from
| open sourcing it, we also took various other security
| precautions[1].
|
| A few feature differences to prodfiler:
|
| * Slice and dice queries by labels, not just process name or
| functions.
|
| * Apart from merging and comparing profiles we also support
| querying by their total value as time series to identify cpu
| or memory spikes and investigate them.
|
| * The storage supports any pprof formatted profile, so we
| support any type of profiling (like heap profiling) as long
| as it's in the right format.
|
| [1] https://www.parca.dev/docs/parca-agent-security
|
| /edit: formatting
| diego wrote:
| FYI, La Parca is the grim reaper in Spanish. Not saying you
| should change the name, just be aware that for some people
| "death" is the most obvious connotation of the name.
| brancz wrote:
| Thanks! Yeah, naming is hard. I think it did come up when we
| were searching for a name. For clarification, the name is a
| play on the Program for Arctic Regional Climate Assessment
| (PARCA), where "ice core profiling" was performed to study
| climate change.
| jzelinskie wrote:
| Howdy team!
|
| What do you consider to be your core differentiator over other
| profiling projects like Pyroscope.io, Datadog, and Google's
| StackDriver?
| brancz wrote:
| Great question! This new storage allowed us to implement a
| really powerful label-based data model, much like labeling and
| selection of time-series in Prometheus[1]. This is incredibly
| powerful when slicing and dicing data to find exactly what
| you're looking for. Many of our early users switched just
| because of this.
|
| Also, the eBPF based profiler we just released allows profiling
| targets with low overhead and best of all: no instrumentation
| needed. It automatically discovers targets to profile, which
| currently supports Kubernetes and systemd. So all you need to
| do to profile your entire infrastructure is to deploy the
| agent.
|
| Lastly we strongly believe in open standards, so the profiler
| produces pprof formatted profiles, and any pprof compatible
| profile can be written to Polar Signals / Parca. We also have
| exporting any query to a pprof formatted profile implemented
| but not exposed to the UI yet.
|
| The storage we specifically designed for all of this to work at
| scale and low latency and cost is also very interesting!
|
| Happy to dive more into details or talk about more things.
|
| [1] https://prometheus.io/
___________________________________________________________________
(page generated 2021-10-08 23:01 UTC)