[HN Gopher] Datadog acquires Quickwit
___________________________________________________________________
Datadog acquires Quickwit
Author : lrx
Score : 83 points
Date : 2025-01-09 17:49 UTC (1 days ago)
(HTM) web link (quickwit.io)
(TXT) w3m dump (quickwit.io)
| bk146 wrote:
| Curious what Datadog is going to build with this tech. Believe
| this company pitching themselves as OSS competitor to Datadog a
| few months ago.
| okbro wrote:
| Based on the [press
| release](https://www.datadoghq.com/blog/datadog-acquires-
| quickwit/) it looks like they're hinting towards offering a
| 'self-hosted' model for customers that can't use pure SaaS
| solutions due to regulations:
|
| > Organizations in financial services, insurance, healthcare,
| and other regulated industries must meet stringent data
| residency, privacy, and regulatory requirements while
| maintaining full visibility into their systems. This becomes
| challenging when logs need to remain at rest in customers'
| environments or specific regions, hindering teams' ability to
| attain seamless observability and insight. To help our
| customers meet these requirements without sacrificing
| visibility or introducing multiple logging tools, we are
| pleased to announce that Quickwit--a popular open source
| distributed search engine--is joining Datadog.
| everfrustrated wrote:
| >it looks like they're hinting towards offering a 'self-
| hosted' model
|
| That makes sense. Datadog has been pure SaaS the whole time,
| which is unusual. Buying a good db engine like Quickwit would
| be a smart head-start into the on-prem segment which is a
| natural expansion opportunity.
|
| I've previously made the prediction that Datadog is the new
| Cisco - can expect lots of acquisitions to be made going
| forward.
| spamizbad wrote:
| We switched from Datadog to Grafana (do not recommend unless
| they got you over a barrel on pricing and you need to escape)
| and one nice thing Grafana gives you is the ability to self-
| host for local development so you can even run integration
| tests against your observability... an edge case need but if
| you need it you're glad it has it.
| samjewell wrote:
| I work at Grafana. Can you say more about what specifically
| you don't recommend?
| winrid wrote:
| I could only imagine they want to replace their existing infra
| with this as a potential cost saving measure.
| andrewstuart2 wrote:
| And then I'm sure they'll pass the savings onto customers
| given the current crazy high prices.
| politelemon wrote:
| A larger bill for enterprise customers.
| everfrustrated wrote:
| Pretty clear they want it to keep a moat on their side. Can't
| see Datadog continuing investing in this - it's a pretty direct
| competitor.
|
| What happened to Vector the last opensource they bought? Are
| they still hired?
| slt2021 wrote:
| arguably they acquired them so they dont build anything further
| with this tech, as it threatens renewals of their existing
| enterprise contracts
| Hixon10 wrote:
| It's a bit sad that many modern databases were recently acquired.
| They had the potential to bring a lot of innovations.
|
| 1. https://www.warpstream.com/
|
| 2. https://www.orioledb.com/
|
| 3. https://quickwit.io/
| oliverrice wrote:
| (disclaimer: supabase employee)
|
| OrioleDB continues to be a fully open source and liberally
| licensed. We're working with the OrioleDB team to provide an
| initial distribution channel so they can focus on the storage
| engine vs hosting + providing lots of user feedback/bug
| reports. Our shared goal is to advance OrioleDB until it
| becomes the go-to storage engine for Postgres, both on Supabase
| and everywhere else.
|
| Happy to hear any concerns you have
| chrisweekly wrote:
| Please forgive and help remedy my ignorance: it's a coherent
| goal to want OrioleDB to be the go-to storage engine for
| Postgres, on Supabase?
| oliverrice wrote:
| I don't want to hijack Datadogs+Quickwit's post comment
| section with unrelated promotional-looking info. Quick
| summary below but if you have any other questions pls tag
| olirice in a Supabase GH discussion.
|
| The OrioleDB storage engine for postgres is a drop-in
| replacement for the default heap method. Its takes
| advantage of modern hardware (e.g. SSDs) and cloud
| infrastructure. The most basic benefit is that throughput
| at scale is > 5x higher than heap [1], but it also is
| architected for a bunch of other cool stuff [2]. copy-on-
| write unblocks branching. row-level-WAL enables an S3
| backend and scale-to-zero compute. The combination of those
| two makes it a suitable target for multi-master.
|
| So yes, given that it could greatly improve performance on
| the platform, it is a goal to release in Supabase's primary
| image once everything is buttoned up. Note that an OrioleDB
| release doesn't take away any of your existing options. Its
| implemented as an extension so users would be able to
| optionally create all heap tables, all orioledb tables, or
| a mix of both.
|
| [1] https://www.orioledb.com/blog/orioledb-beta7-benchmarks
|
| [2] https://www.orioledb.com/docs
| satvikpendem wrote:
| Makes sense, perhaps the previous commenter thought
| OrioleDB was itself a database rather than an
| implementation detail alternative to current databases.
| That's what I thought before I went to their site.
| gnabgib wrote:
| Related:
|
| _Binance built a 100PB log service with Quickwit_ (228 points, 6
| months ago, 195 comments)
| https://news.ycombinator.com/item?id=40935701
|
| _Show HN: Quickwit - OSS Alternative to Elasticsearch, Splunk,
| Datadog_ (145 points, 1 year ago, 51 comments)
| https://news.ycombinator.com/item?id=38902042
| slt2021 wrote:
| * note to myself: build a successful product that threatens
| large slow moving legacy provider and get quickly acqui-hired
| aseipp wrote:
| Well, it looks like Quickwit _was_ going to add an Enterprise
| license as of earlier this year (PR #5529), which I had been
| keeping eyes on, but this announcement says they 're instead
| going to relicense as Apache 2.0 so the "community can continue
| on":
|
| > We will be focused on building a new product with Datadog, and
| to ensure our open-source community can continue, we will soon
| release a major update of both Quickwit with a relicense to
| Apache License 2.0 and tantivy.
|
| So, it looks like we'll get a more liberally licensed Quickwit,
| but reading between the lines suggests development of it is might
| otherwise be winding down? It has been pretty nice and stable in
| my experience, so I can't really complain much. But I was really
| looking forward to what else it could bring.
|
| Congrats to the team, in any case!
| mindcrash wrote:
| "So, it looks like we'll get a more liberally licensed
| Quickwit, but reading between the lines suggests development of
| it is might otherwise be winding down?"
|
| They will stop fulltime day-to-day effort in it themselves,
| probably because they have been relocated to writing a similar
| service but closed and integrated in DD, but it seems they want
| to opensource the current product with a OSI compliant license
| in the hopes that the community picks up the tab.
|
| I think that's a nice trade. Could have been much worse.
|
| By the way, also note that DD is not a total stranger in the
| OSS space. They actually opensourced their observability
| pipeline tooling for general use as Vector, which is a rock
| solid product. - https://vector.dev/
| everfrustrated wrote:
| They bought Vector - it was always opensource
| Dylan1312 wrote:
| Vector was already OSS when they acquired the company that
| created it, timber.
|
| https://www.datadoghq.com/blog/datadog-acquires-timber-
| techn...
| psanford wrote:
| Happy that Quickwit is going to Apache 2. Sad that the team won't
| be working on it anymore.
|
| I loved that I was able to setup Quickwit on AWS lambda and have
| a good cloud based search engine for $0.01 / month.
| francoismassot wrote:
| Co-founder of Quickwit here. Seeing our acquisition by Datadog on
| the HN front page feels like a truly full-circle moment.
|
| HN has been interwoven with Quickwit's journey from the very
| beginning. Looking back, it's striking to see how our progress is
| literally chronicled in our HN front-page posts:
|
| - Searching the web for under $1000/month [0]
|
| - A Rust optimization story [1]
|
| - Decentralized cluster membership in Rust [2]
|
| - Filtering a vector with SIMD instructions (AVX-2 and AVX-512)
| [3]
|
| - Efficient indexing with Quickwit Rust actor framework [4]
|
| - A compressed indexable bitset [5]
|
| - Show HN: Quickwit - OSS Alternative to Elasticsearch, Splunk,
| Datadog [6]
|
| - Quickwit 0.8: Indexing and Search at Petabyte Scale [7]
|
| - Tantivy - full-text search engine library inspired by Apache
| Lucene [8]
|
| - Binance built a 100PB log service with Quickwit [9]
|
| - Datadog acquires Quickwit [10]
|
| Each of these front-page appearances was a milestone for us. We
| put our hearts into writing those engineering articles, hoping to
| contribute something valuable to our community.
|
| I'm convinced HN played a key role in Quickwit's success by
| providing visibility, positive feedback, critical comments, and
| leads that contacted us directly after a front-page post. This
| community's authenticity and passion for technology are
| unparalleled. And we're incredibly grateful for this.
|
| Thank you all :)
|
| [0] https://news.ycombinator.com/item?id=27074481
|
| [1] https://news.ycombinator.com/item?id=28955461
|
| [2] https://news.ycombinator.com/item?id=31190586
|
| [3] https://news.ycombinator.com/item?id=32674040
|
| [4] https://news.ycombinator.com/item?id=35785421
|
| [5] https://news.ycombinator.com/item?id=36519467
|
| [6] https://news.ycombinator.com/item?id=38902042
|
| [7] https://news.ycombinator.com/item?id=39756367
|
| [8] https://news.ycombinator.com/item?id=40492834
|
| [9] https://news.ycombinator.com/item?id=40935701
|
| [10] https://news.ycombinator.com/item?id=42648043
| nextaccountic wrote:
| I think you forgot to add the links
|
| Anyway tantivy is great! I love pg_search
| https://www.paradedb.com/blog/introducing_search (which appears
| to be built by another company, but on top of tantivy, which is
| a great feature of open source)
|
| Now, I am worried about development being stalled after this
| acquisition. How does further developing tantivy in the open
| helps Datadog's bottom line?
| jhgg wrote:
| Hoping this leads to datadog launching a logging solution that
| does not cost an arm and a leg at scale.
| iandanforth wrote:
| Honestly this is my _only_ criticism of datadog. The product is
| great but good lord that pricing.
| maxwellg wrote:
| > Mezmo recently put in production Quickwit to serve thousands of
| customers and petabytes of logs, drastically reducing
| infrastructure cost and complexity while delivering the same user
| experience.
|
| I can't imagine they feel great about Quickwit getting bought by
| a competitor after that.
| PittleyDunkin wrote:
| It's a risk you deal with giving back to open source.
|
| The good part for the rest of us is it's a signal that there's
| likely some appetite for a fork if Datadog screws the pooch.
| liminal wrote:
| We just moved from Elastic to Quickwit. Sigh. What other open
| source, object storage backed logging databases do people
| recommend?
| scop wrote:
| I hate Datadog. We use their name as an epithet at our company
| for how not to sell/market. Their selling tactics circa 2015-2018
| completely burned us out. Endless calls and emails. The icing on
| the cake was an AWS reInvent presentation on Lambda right when
| lambda was first announced. We were pumped to get in on lambda
| early. Got the whole crew to attend the talk. Turned out to be a
| rudimentary copy of a Barr "lambda up and running" blog wrapped
| in a stand up comedy routine hawked by a Datadog employee who
| made sure to tell us he was a Datadog employee. Get us all drunk
| and happy and think Datadog is cool.
|
| Genuine question: has the company changed enough in the interim
| to deserve a second look?
___________________________________________________________________
(page generated 2025-01-10 23:00 UTC)