[HN Gopher] Show HN: QuestDB with Python, Pandas and SQL in a Ju...
___________________________________________________________________
Show HN: QuestDB with Python, Pandas and SQL in a Jupyter notebook
- no install
Author : bluestreak
Score : 34 points
Date : 2023-02-21 16:30 UTC (6 hours ago)
(HTM) web link (play.questdb.io)
(TXT) w3m dump (play.questdb.io)
| amunra__ wrote:
| Hi, I'm Adam Cimarosti, one of the core engineers at QuestDB.
|
| We built play.questdb.io to make it easy for anyone to try our
| database. No installation.
|
| There's a Jupyter Lab notebook, data, sample code, queries and
| graphs.
|
| We'd love to hear what you think.
| Cilvic wrote:
| Reading your pitch here, i'd love to have a vague idea what
| questdb is and why I should care.
| nwsm wrote:
| I take it you did not visit the link?
| amunra__ wrote:
| Most databases store the latest state of something. We don't.
| We ingest events. After all, life is a function of time :-)
| The whole world ticks and we take those ticks and store them.
| If part of your application tracks anything happening over
| time (trades, ocean pollution levels, ships moving, rocket
| simulation metrics.. or whatever else then it makes sense to
| store those events in a time series database. What we
| provide, primarily, is two basic pieces of functionality: (1)
| Taking in lots of events FAST. Our ingestion rate is high
| (and we also integrate with things like Kafka, Pandas -- see
| the notebook, etc). Each of our time series tables (we
| support regular ones too) comes with a special timestamp
| column. (2) Specialized SQL to make sense of data that's
| changed over time, such as grouping and resampling by time
| and more. Take a look at our docs for things like SAMPLE BY,
| LATEST ON, ASOF JOIN, LT JOIN and more. On disk, we also
| guarantee that all records are sorted by time and this gives
| us great query performance for these time-based types of
| queries.
|
| PS. We're also wire-compatible with PostgreSQL.
| Wonnk13 wrote:
| So I guess it would be fair to say you compete with
| Timescale and Clickhouse as a timeseries database?
| nhourcard wrote:
| yes correct - although Clickhouse is more of an OLAP
| database. Timescale is built on top of Postgres, while
| QuestDB is built from scratch with Postgres wire
| compatibility. You can run benchmarks on
| https://github.com/timescale/tsbs
| jmholla wrote:
| I was once in the market for time series databases, but all
| I could find required down sampling of older data. I don't
| know if this has changed, and to be fair I haven't been
| looking for quite some time, but does yours allow for
| keeping data with the captured precision in perpetuity (or
| until my hard drive fills up)? My guess from the way you
| describe your approach is yes, but I wanted to check.
| amunra__ wrote:
| Yes. We're pretty good with large volumes of data.
|
| Eventually all local drives fill up though.
|
| When ingesting data we partition data by time. By default
| we partition by day. This give you the flexibility to
| detach partitions, store them somewhere slower and
| cheaper with more capacity for longer term storage and
| reattach them later if need be.
|
| Built on top of our open source primary product, we also
| have a cloud variant of QuestDB which runs on AWS. One of
| the things that we're building there is cold storage. It
| will automate this process onto S3 such that if a query
| ever needs to access this older data it will re-enstate
| it automatically for you with no admin overhead.
___________________________________________________________________
(page generated 2023-02-21 23:01 UTC)