[HN Gopher] OpenObserve: Observability platform for logs, metric...
___________________________________________________________________
OpenObserve: Observability platform for logs, metrics, traces,
analytics
Author : thunderbong
Score : 75 points
Date : 2024-10-23 18:10 UTC (4 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| ewuhic wrote:
| How does it compare to Grafana suite?
| prabhatsharma wrote:
| You should read this - https://openobserve.ai/blog/openobserve-
| vs-grafana
| viraptor wrote:
| That's an interesting take for OpenObserve, but for me in an
| ops role it misses the compatibility. I appreciate that with
| Grafana I can choose a specific backend and configure it as
| required in my environment. I know other systems can reach
| the same database because they have known APIs. I also know I
| can move to another store with the same front-end if the
| owners pull an Elastic/Redis.
|
| OO integration may save me a couple of days of setup, but
| long-term it's a dangerously limiting idea / lock-in.
| prabhatsharma wrote:
| You are in the same danger with Grafana (Front end,
| Elasic/Redis fate and lock in) as you are with OpenObserve.
| No difference there.
| 1oooqooq wrote:
| just forwarding the hosts logs from journald to a openobserve
| running in a vm on that host requires using an agent that will
| cause you more reasons to have to observe a log than anything
| else.
|
| And syslog/syslogng are also problematic to ingest.
| prabhatsharma wrote:
| OpenObserve is built for centralized logging - Not really for
| installing it on every linux host. If that is your use case, I
| would recommend you to look for other tools.
| MrDarcy wrote:
| Another one for the sso wall of shame - https://sso.tax/
| prabhatsharma wrote:
| You should read this - https://openobserve.ai/blog/sso-tax
| Eridrus wrote:
| I think you don't understand the core argument re the SSO
| Tax, which is that security is a positive sum good, which is
| why it should not be the feature used for price
| discrimination.
|
| Not all products have other good features to use for price
| discrimination, so I have some sympathy for vendors here, but
| I think it often indicates laziness in thinking about what
| they can use to do the necessary price discrimination.
| prabhatsharma wrote:
| I do understand it's super important for security, and I
| want large companies who have ample money and spend a lot
| on security to pay me as well for it. If you are running
| OpenObserve in your basement or are a small startup you get
| it for free in OpenObserve and stay secure.
| osigurdson wrote:
| How should companies monetize products? Maybe people should
| just go back to full commercial models, not sure.
| prabhatsharma wrote:
| Most people who talk about SSO Tax don't really care for it's
| values but rather want free stuff. I have had conversations
| with multi-billion dollar companies who would avoid paying a
| single dollar to support open source companies and bring SSO
| tax into conversation.
|
| On our part OpenObserve offers free SSO on our cloud service
| to anyone and Free SSO for anyone using enterprise version if
| they ingest under 200 GB/Day (6 TB/Month).
| MajimasEyepatch wrote:
| The problem with not offering SSO on lower tiers is that it
| can make it hard to test into a new service. I might want
| to try out a new tool with one team for a couple of months
| and see how it goes before recommending adoption for a
| broader group. I don't want to have to sign a year-long
| six-figure contract just to try something out. Sometimes
| you can work out a trial period with the sales team, but
| that's not always easy, and it puts a strict ticking clock
| on things that doesn't work in all situations.
| yourapostasy wrote:
| _> ...conversations with multi-billion dollar companies..._
|
| Slight clarification here. Might not apply in
| OpenObservability's case but might help others on their
| journey to enterprise sales with their projects.
|
| Those are typically conversations with managers holding $X
| purchasing authority, typically like $500K for a US
| director'ish level, within multi-billion dollar companies.
| These managers usually aren't averse to spending on open
| source projects. They're averse to cutting a check not tied
| to a support contract with responsive, polite, helpful
| support with published support policies at 0300h local time
| on a break-fix line with 75 other people from other support
| teams in the company watching. A surprising number of open
| source projects won't offer that guarantee, and instead
| only offer the option to "donate" with vague promises of
| priority support. More projects are getting better at this
| more recently, but it takes a surprising amount of red tape
| to onboard as a vendor into these organizations, and a lot
| of open source teams don't have the appetite for putting up
| with that.
|
| Until kind of recently, the conversation switching to the
| SSO Tax is really about accessing that level of guaranteed
| support delivery.
| prabhatsharma wrote:
| Thanks @yourapostasy . Agree with you for the most part.
|
| Not all managers are averse to paying, but many are. I
| have had discussions with Director/Sr. Director and VP
| level folks in these companies. I have been paid and I
| have been denied.
|
| Our biggest customer is a fortune 10 company and we are
| able to offer the kind of support that they need. It
| indeed takes a lot to provide that kind of support,
| though, and would be difficult for most small open source
| projects to do.
| terminalbraid wrote:
| That's the company's problem.
|
| They're entitled to their business model. They're not
| entitled to it working. They're not entitled to someone
| figuring out a business model for them if people don't like
| it.
| Groxx wrote:
| SSO often costs quite a lot to maintain, given how widely
| varied the systems are. Seems reasonable to charge for an
| optional high-complexity and high-maintenance-burden feature.
| terminalbraid wrote:
| Are you referring to the development cost or just the "keep
| SSO wired to other orgs in for our cloud product"?
| Development-wise, SSO standards don't change much and aren't
| terribly difficult to get up an running if you stick to oauth
| and saml.
|
| By not offering that in a self-hosted open source version
| where the maintenance is delegated to the user turns this to
| a naked cash grab.
| nhumrich wrote:
| How is this on the SSO tax wall of shame? They support it SSO
| on their free tier.
| hijinks wrote:
| does anyone use this?
|
| I'm really starting to get sick of companies that claim they
| operate at petabyte at scale and find you need to spend 400k a
| month to support that scale.
| prabhatsharma wrote:
| Thousands of active deployments globally.
|
| How many open source log systems work at PB scale given any
| number of resources? Also FWIW, OpenObserve can ingest data at
| 28 MB/Sec/Core (We are working on optimizing it even more) and
| ingesting 1 PB of data would cost just $435 based on on-demand
| prices (AWS m7g family).
| terminalbraid wrote:
| That doesn't answer the question of _who_? A (rightly)
| cynical reading of what you posted could just be "thousands
| of active deployments" you did for yourself to prove
| benchmarks.
| prabhatsharma wrote:
| Machines I would use for benchmarking would go down after
| some time and won't be active.
| djbusby wrote:
| Still didn't answer the "who" part.
| prabhatsharma wrote:
| We will publish many names on our website soon.
| Veserv wrote:
| Why is it only 28 MB/core-second?
|
| Is that production rate, inbound bandwidth, rate to
| persistence, rate to processed, or rate to display?
| prabhatsharma wrote:
| Compute power is required to process and store the incoming
| data.
|
| It's not "only 28 MB/Sec/Core". Try doing same with
| Splunk/Elasticsearch - You won't go past 5 MB/Sec/Core
| (Typically it will be lower) on their best day.
| Veserv wrote:
| To what state?
|
| Suppose I have 28 GB of trace data in memory on a machine
| and then I fire that off. What do I have after 1000
| seconds?
|
| Do I just have a file of 28 GB of raw trace?
|
| Do I have 28 GB of raw trace in memory ready to be
| indexed?
|
| Do I have a data structure in memory ready to be
| searched?
|
| Do I have the full trace information rendered on my
| screen (or a aggregated visualization derived after
| processing all the data)?
|
| If it is the first, that would be ridiculously slow. If
| it is one of the latter ones, then it would depend on
| what querying operations are fast.
|
| 28 MB/core-second makes no sense without the context of
| what you can do quickly after the "processing" is done.
| prabhatsharma wrote:
| Too much to give all details in an HN thread. To simplify
| the conversation, Data will be persisted and usable for
| individual searches and aggregations. I would welcome you
| to our slack workspace for any further questions you may
| have - https://short.openobserve.ai/community
| niux wrote:
| Had a pretty bad experience with this. The web app is very buggy
| and frustrating.
| prabhatsharma wrote:
| What bugs? Care to file a GitHub issue?
| wiradikusuma wrote:
| How is it compared to Signoz?
| mdaniel wrote:
| AGPLv3 versus MIT Expat + open core for one thing
|
| https://github.com/openobserve/openobserve/blob/v0.12.1/LICE...
|
| https://github.com/SigNoz/signoz/blob/v0.56.0/LICENSE
| prabhatsharma wrote:
| Thanks @thunderbong for the post.
|
| We have spent over 2 years building OpenObserve into a simple,
| highly usable and efficient observability tool. You could run it
| using a single binary that provides all the functionality of
| logs, metrics, traces, front end monitoring, dashboards (18
| different chart types), alerts and pipelines.
|
| OpenObserve is being used by startups, mid tier enterprises and
| fortune 100 companies. There are thousands of active
| installations of OpenObserve globally.
|
| Folks have replaced Elasticsearch, Splunk, Graylog, Datadog ,
| Newrelic and more for OpenObserve.
|
| Comment from a user -
|
| We moved from 5 node OpenSearch cluster to single node
| OpenObserve and measured using our actual everyday queries, which
| are reasonably complex queries (1 to 5 conditions applied) over
| our real logging data. We see that typically they complete in
| about the same time. OpenObserve costs us 10 times less though
| (instances + storage)
|
| Also, we are currently working on replacing one of the world's
| largest splunk installations.
|
| p.s. I am one of the maintainers of OpenObserve. Feel free to ask
| questions. I will be happy to answer them. You can also visit our
| slack workspace at https://short.openobserve.ai/community for
| discussions.
| ethanwillis wrote:
| How does the storage cost for 3 nodes stay exactly the same as
| for 1 node for openobserve?
| prabhatsharma wrote:
| By using object storage (Think s3 and similar) and not
| replicating data for HA (Not needed if using s3) which is done
| by legacy systems like Elasticsearch and Splunk.
| gclawes wrote:
| No SSO in open-source, pass. I'll stick w/ Grafana
| prabhatsharma wrote:
| You should read - https://openobserve.ai/blog/sso-tax and
| https://openobserve.ai/blog/openobserve-vs-grafana
| NewJazz wrote:
| I've read articles like that time and time again. Doesn't
| change my requirements.
| prabhatsharma wrote:
| God bless you my friend. Thanks for the comment.
| apitman wrote:
| I'm curious what are your requirements? Their SSO tax
| appears to be structured in such a way that only large
| enterprises would have to pay.
| Volundr wrote:
| How so? This chart [1] has SAML under enterprise with no
| price tag other than "get in touch"
|
| [1] https://openobserve.ai/pricing
| apitman wrote:
| I'm going off the article linked above:
| https://openobserve.ai/blog/sso-tax
| Volundr wrote:
| Fair enough. I couldn't find information of self-hosting
| enterprise on their site, but did in the GitHub
| repository [1] and 200GB is indeed a lot. At the same
| time it's also a non-starter for me. I'm not going to
| install "enterprise" anything where I'm going to start
| depending on it, and one day the price will go up to ???.
|
| [1]
| https://github.com/openobserve/openobserve?tab=readme-ov-
| fil...
| Volundr wrote:
| > enterprise SSO solutions like Okta are not free for users
| and cost a lot of money for organizations to implement and
| use.
|
| There are free and open source solutions like Keycloak and
| Zitadel. I don't dispute they are less common than Okta and
| Entra, but the definitely exist and are deployed in the real
| world. My workplace (state government) uses Keycloak for
| example.
|
| Another thing that the article doesn't really touch is that
| SSO is locking a security best practice, important for an
| organization of any size, behind a paywall. With SSO, when
| someone leaves the organization you can disable their
| singular account and be confident they are locked out of your
| shared folders, gitlab, jira, etc, etc, rather than having to
| manually track down and disable each one, with a high
| likelihood of missing something. This is important for an
| organization of any size > 1 from a bootstrapped startup all
| the way to fortune 500. Hiding it behind higher cost makes it
| more likely that an org will try to do without and have a
| security breach as a result.
|
| I also take issue with:
|
| > Developing and maintaining SSO solutions requires
| significant investment in research, development, and
| infrastructure.
|
| Having done it myself, this is overstated. No feature is free
| but implementing a SAML or OAuth flow is not THAT much work,
| nor does it represent a huge amount of ongoing maintenance.
|
| I actually don't mind the SSO tax too much in cases where
| it's the differentiator between free or open source vs paid.
| I find it far more egregious when it's a product that already
| has a cost and SAML auth jacks up the price 2-10x. I don't
| think the blog post is a particularly good discussion of the
| tradeoffs though.
| cshark007 wrote:
| used this on self-hosted and ingested 150GiB daily and absolutely
| no issues, fancy UI and more buttons are not needed if you get
| the value from ingestion speed.
| max_streese wrote:
| How are you folks related to Anguilla? Couldn't really find
| anything AI specific about OpenObserve so I am guessing the
| domain is for other reasons?
| _fat_santa wrote:
| Funny I was actually shopping around for an logging platform
| yesterday and ended up going with Grafana. The thing that sold it
| (for now) is their generous free tier, though if I had to self-
| host (which I intend to do eventually), this seems like a much
| easier thing to self host (and I really really like that it's
| just one binary)
| __turbobrew__ wrote:
| How does this better than grafana? Loki -> logs Tempo -> traces
| Prometheus/mimir -> metrics
|
| I know personally of several companies which use this stack with
| tens of thousands of nodes. Everything runs on object storage
| which means it is easy to scale up, and you can move between
| storage providers as long as they implement the S3 API. The
| grafana ecosystem is very sticky as well. If I download some
| random helm chart it most likely will come with a grafana
| dashboard which I can easily import and instantly have dashboards
| and alerts for the helm chart.
| __turbobrew__ wrote:
| I see you address this in a blog post: "You will also hear of
| LGTM stack (Loki, Grafana, Tempo, Mimir) which is a pretty good
| stack for observability. Each of these components are separate
| open source tools built by grafana labs."
|
| Not very convincing why I wouldn't go for the LGTM stack which
| has been proven to be effective.
| prabhatsharma wrote:
| By all means, if LGTM works for you stay with it.
|
| For those looking at much more simplicity, and much higher
| performance OpenObserve is the way to go. Many folks have
| moved from Loki to OpenObserve due to performance issues with
| Loki. Many have moved from LGTM stack completely to
| OpenObserve. Many have chosen to use Grafana as a front end
| for OpenObserve too.
|
| Take a look at how easy it can be to build dashboards in
| OpenObserve.
|
| It takes time for community and ecosystem to build for great
| products. Grafana started in 2014. OpenObserve started in
| 2022.
___________________________________________________________________
(page generated 2024-10-23 23:01 UTC)