[HN Gopher] 4Bn rows/sec query benchmark: ClickHouse vs. QuestDB...
___________________________________________________________________
4Bn rows/sec query benchmark: ClickHouse vs. QuestDB vs. Timescale
Author : bluestreak
Score : 12 points
Date : 2022-06-01 17:56 UTC (5 hours ago)
(HTM) web link (questdb.io)
(TXT) w3m dump (questdb.io)
| bluestreak wrote:
| Last year we released QuestDB 6.0 and achieved an ingestion rate
| of 1.4 million rows per second (per server). We compared those
| results to popular open source databases [1] and explained how we
| dealt with out of order ingestion under the hood while keeping
| the underlying storage model read-friendly. Since then, we
| focused our efforts on making queries faster, in particular
| filter queries with WHERE clauses. To do so, we once again
| decided to make things from scratch and built a JIT (Just-in-
| Time) compiler for SQL filters, with tons of low-level
| optimisations such as SIMD. We then parallelized the query
| execution to improve the execution time even further. In this
| blog post, we first look at some benchmarks against Clickhouse
| and TimescaleDB, before digging deeper in how this all works
| within QuestDB's storage model. Once again, we use the Time
| Series Benchmark Suite (TSBS) [2], developed by TimescaleDB,: it
| is an open source and reproducible benchmark.
|
| We'd love to get your feedback!
|
| [1]:https://news.ycombinator.com/item?id=27411307
|
| [2]:https://github.com/timescale/tsbs
| ericb wrote:
| It is definitely useful to be able to consume a lot of data
| quickly, especially high-cardinality data. Inevitably, an
| infinite flood of data will eventually consume any finite space
| limitations. I'm wondering what QuestDB's story for data
| aggregation and cleanup looks like?
| bluestreak wrote:
| Aggregation is also optimised quite a bit via SIMD and map-
| reduce. They are as fast as the "where" predicates. Multiple
| field keyed aggregation is not as optimal yet. I would also
| suggest our demo site (free and fully open) to see how queries
| that you use work.
|
| Cleanup is semi manual for now. Time partitions can be removed
| or detached via SQL. We're working on automating that.
| ysleepy wrote:
| This looks cool, I've been looking at time series DBs lately and
| mostly landed with timescale because of the ability to have
| complete freedom querying the dataset with postgres kitchen sink.
|
| The post here really focuses on one query and that is weirdly
| without a time sort. Would similar queries be also fast? - What
| about a join, aggregates, lag()-over, subqueries, unions, etc
| queries
| nhourcard wrote:
| You're right, this post focuses on filter queries with the
| WHERE clause. Aggregates are also pretty fast, a live demo is
| currently available with three datasets, one of them being more
| than 1.6 billion rows: https://demo.questdb.io/ On this you can
| also perform a join, in particular ASOF joins (where timestamps
| do not exactly match).
|
| NB: The launch of this demo was done some time ago on HN:
| https://news.ycombinator.com/item?id=23616878
___________________________________________________________________
(page generated 2022-06-01 23:03 UTC)