[HN Gopher] Observability vs. monitoring debate: An irreverent view
___________________________________________________________________
Observability vs. monitoring debate: An irreverent view
Author : simskij
Score : 29 points
Date : 2021-11-26 08:48 UTC (3 days ago)
(HTM) web link (ubuntu.com)
(TXT) w3m dump (ubuntu.com)
| thinkingkong wrote:
| Both concepts are related but separate. I'd argue that monitoring
| is the act of alerting or taking action when some metric is out
| of bounds with normal values. Those values are based on absolute
| thresholds, or some changing thresholds based on historical
| values. Observability is the ability to see what's happening
| across and between multiple systems so that any issues raised by
| monitoring can be diagnosed quickly. "I know theres a problem vs
| I know which part of the system its happening in"
| ittan wrote:
| Not to be rude, Monitoring is observing problems and then
| responding to the alarm in realtime.
|
| You cannot compare the two. Yet we do in certain circumstances
| and that is a loss in understanding, which I am sad about.
| mmanciop wrote:
| I very much agree: observability is a property of a system,
| like transparency is for glass. Monitoring is what you need to
| do with a lot of systems to make them observable.
| csours wrote:
| I see it as a tiered approach - Detailed Process Logs saved to a
| db, Audits, Regular logs streamed to your favorite log dump,
| Monitoring, Health Checks, Metrics, and Usage Statistics are all
| part of Observability.
|
| If it is a critical process, use detailed process logs.
|
| If it is an extremely critical process add Auditing.
| mmanciop wrote:
| Yep, pretty much. There is plenty different types of telemetry
| one can use, and which ones are important for one application
| depends both on the nature of that application, and who
| operates them.
| vmarsy wrote:
| Observability isn't just a rebranding of Monitoring, it's
| Monitoring + making it the most actionnable possible via
| standardization.
|
| Specifically, how to make the sum of all monitored "pillars" more
| useful than each of them individually.
|
| 3 major pillars being:
|
| - Metrics (whether application or higher-level of the stack, like
| OS)
|
| - Logs (whether structured or unstructured)
|
| - Traces
|
| Observability is these major pillars and how to easily "jump"
| from one to another to very quickly identify the root cause of an
| issue. I.e. go Metrics <-> Logs, Logs <-> Traces, or Metrics <->
| Traces,
|
| For instance, with good Metrics, one can easily figure out & get
| alerts when there is a large spike of 500 errors. But when
| Metrics & Logs can work together, one can easily see the
| exception from stack trace that are emitted with those 500
| errors.
|
| Similarly, with good Metrics, one can easily figure out that the
| frontend service latency p90 has increased by 5x. But with
| Metrics & Traces working together(for instance via Exemplar[1]),
| one can look at a bunch of the traces that have a very high
| latency, and identify the upstream service responsible for this
| increase.
|
| With Monitoring only, you could get a nice Metrics solution in
| place, with fancy alerting rules, but all it was good at is
| informing you "Something bad is currently happening". With a good
| "Observability" setup, you should also be able to change it to
| "Something bad is currently happening and the root cause is right
| here."
|
| [1] https://grafana.com/docs/grafana/latest/basics/exemplars/
| idoco wrote:
| I see OpenTelemetry as an application of the same idea pushed by
| Google via the k8s revolution.
|
| Create a great vendor-agnostic open source tech. Get everyone
| riled up about the dangers of vendor-locking solutions. Use the
| new tech to carve yourself a piece of the market from the current
| incumbent.
|
| It is pretty great and all, but sometimes it is easier to build
| your app with a simple vendor-locked tech than a super generic
| agnostic technology.
| pm90 wrote:
| I agree with this.
|
| It's kinda important to understand who all this is meant for.
| If you're a lean startup just use the best/cheapest/quickest
| tool regardless of vendor lock in. It's when you get to a
| certain scale that vendor agnosticism becomes a real concern,
| but by then you probably have enough resources to hire folks
| that will rebuild your stack.
| pm90 wrote:
| There's certainly elements of hype to the term since it's new and
| a lot of startups and projects are trying to get the biggest
| mindshare.
|
| Monitoring to me is exclusively about metrics and alerts. Metrics
| are really useful but they often don't give you the whole
| context, and might sometimes be misleading. Eg you see a spike in
| cpu usage for a service; you probably just autoscaler and call it
| a day, and that's the end of it. Having metrics is SO much better
| than not having metrics though, it delivered insights that were
| just not possible before.
|
| Observability to me is the next iteration of this process of
| understanding system behavior. Metrics are limiting, so maybe you
| look at logs. Well, they suffer from some of the same issues, so
| you try profiling, you try tracing. Ultimately the goal is to
| explore tools that allow developers to quickly get a truthful
| understanding of how their systems really work, and use that
| knowledge to improve their systems.
| goodpoint wrote:
| "Observability" existed 20 years ago and it consisted in
| application metrics, OS metrics, log analysis and correlation,
| SIEM.
|
| It was called monitoring.
| simskij wrote:
| which i think is the point of the article as well. :)
| mmanciop wrote:
| It very much is :-)
___________________________________________________________________
(page generated 2021-11-29 23:01 UTC)