[HN Gopher] InfluxDB 3 Open Source Now in Public Alpha Under MIT...
___________________________________________________________________
InfluxDB 3 Open Source Now in Public Alpha Under MIT/Apache 2
License
Author : otoolep
Score : 12 points
Date : 2025-01-14 20:09 UTC (2 hours ago)
(HTM) web link (www.influxdata.com)
(TXT) w3m dump (www.influxdata.com)
| pauldix wrote:
| Post author, cofounder and creator of InfluxDB here. Happy to
| answer questions in this thread.
|
| I'm guessing there will be questions about the 72 hour limit.
| There are two things we're looking at:
|
| First, we're considering giving a free tier for at home and
| hobbyist usage of Enterprise, which doesn't have this limitation.
| So this would be kind of like what Tailscale does giving a free
| usage plan for their commercial software.
|
| Second, for Core, the open source build, we're working on an
| update that will let it query any 72 hour window of historical
| data. Right now it doesn't evict data, it all still exists on
| disk or object storage as Parquet files, but we remove the
| metadata information from RAM to keep things optimized for the
| most recent 72 hours.
|
| When the update is done, you'll be able to write and query for
| any period of time. But an individual query will be limited to a
| 72 hour time range. This is a service protection mechanism
| because of how the data is organized.
|
| A file gets created for every 10 minute block of time for each
| table. So 72 hours is 432 files, which is a lot of GET requests
| to S3 for a single query. We don't want to increase the range
| because of that. Multiple queries combining a longer range, or
| accessing the data from third-party clients is all still
| possible.
|
| In Enterprise, our commercial product, we have a compactor that
| collapses these files into larger time blocks that also creates
| an index that the query engine can use.
|
| Doing it this way was a deliberate choice so that we could have a
| permissively license open source project separate from the
| commercial product. If we put the compactor into the open, we'd
| have to put it under a source available license to limit usage so
| that we can still sell the database.
|
| Our hope is that there's still an audience of users that will
| find Core useful on its own, even without any commercial
| relationship with us. It's not a full historical TSDB, but it's
| not intended to be. It's meant to be a recent data engine that
| can collect, process, monitor, ship, and store data paired with a
| fast analytical query engine against the recent buffer (or
| recently persisted buffer).
|
| Happy to answer any followup questions about this or the release
| generally.
| mendocinox wrote:
| I haven't used influxdb in a project yet, but I'm a fan of its
| capabilities!
|
| The core-enterprise dichotomy seems more or less the same as
| what scylla had until recently. Does influxdata have different
| considerations from scylla that will allow influxdb to remain
| open source in the long term?
| pauldix wrote:
| We're open core and have been since 2016. We've deliberately
| limited the scope of what the open source project is supposed
| to do. It should be great at this use case of collecting
| processing, storing, and querying recently buffered data.
|
| The commercial offering is the historical time series DB
| along with a bunch of other features around high
| availability, read replication, fine grained security, and
| the compaction engine which enables longer range queries and
| row level deletes.
|
| I think Scylla had most of their DB in the open and then a
| small slice of Enterprise functionality (although I'm not
| super familiar with their product line).
|
| Ideally, we'd have many open source users and even our
| commercial customers would use the open source in addition to
| the commercial offering.
|
| But ultimately, it's about finding a sustainable business
| model that keeps more software coming. We have a preference
| for permissive open source over source available. In my view,
| we may as well create freemium rather than source available.
|
| With this version of InfluxDB, we've been able to invest
| heavily into Apache projects that lie at the core of it:
| Arrow, DataFusion, Parquet, and the object store crate, which
| we developed and donated to the ASF.
|
| We'd like to continue that work because we think that a
| highly performant, modular, vectorized query engine (i.e.
| DataFusion) should be a free commodity that's widely
| available and widely contributed to.
| JSTrading wrote:
| Excellent! Keep up the great work.
___________________________________________________________________
(page generated 2025-01-14 23:01 UTC)