[HN Gopher] Show HN: Hydra - Open-Source Columnar Postgres
___________________________________________________________________
Show HN: Hydra - Open-Source Columnar Postgres
hi hn, hydra ceo here hydra is an open-source extension that adds
columnar tables to Postgres for efficient analytical reporting.
With Hydra, you can analyze billions of rows instantly without
changing code. demo video (5 min): https://youtu.be/1yzxgb0Oyrw
github repo: https://github.com/hydradatabase/hydra For 1.0 GA
release, aggregate queries are over *60% faster* than Hydra beta
due to aggregate vectorization. Spatial indexes (gin, gist, spgist,
and rum indexes) and pg_hint_plan are now enabled for performance
optimization. postgres is great, but aggregates can take minutes
to hours to return results on large data sets. long-running
analytical queries hog database resources and degrade performance.
use hydra to run much faster analytics on postgres without changing
code. for testing, try the hydra free tier to create a column
postgres instance on the cloud. https://dashboard.hydra.so/signup
Author : coatue
Score : 80 points
Date : 2023-09-19 16:05 UTC (6 hours ago)
(HTM) web link (www.hydra.so)
(TXT) w3m dump (www.hydra.so)
| I_am_tiberius wrote:
| I have 2 questions.
|
| 1. Is this optimized for constantly adding and removing rows to
| the columnar table?
|
| 2. Is this supported by Microsoft Azure Flexible Server for
| Postgres?
| gregwebs wrote:
| > Whenever possible, design data coming into the data warehouse
| as append-only. Hydra's columnar store only supports inserts.
| If you need to update or delete data, you will need to use row
| (heap) tables.
| jerrysievert wrote:
| that is not the case at this point. updates, deletes, and
| vacuuming are all available.
| jerrysievert wrote:
| I can answer number 1:
|
| updates and deletes are available, as well as the ability to
| compact the table.
| mlenhard wrote:
| Congrats on the 1.0 Release, big milestone.
|
| I'm personally really excited about all of the recent tooling for
| postgres aggregates. Definitely a pain point for a lot of
| developers and its easy to fall in trap where things work fine in
| the beginning and then query times explode as requirements change
| and the dataset grows. Nice to not have to spin up another DB in
| order to solve the problem as well.
| MuffinFlavored wrote:
| > I'm personally really excited about all of the recent tooling
| for postgres aggregates. Definitely a pain point for a lot of
| developers
|
| Could you give a few examples of what you are speaking of?
| garysahota93 wrote:
| I've been using Hydra for the last ~2 months & genuinely love it.
| The team is really talented & it's so great to see the progress
| they've been making. Congrats on the 1.0 GA release! Huge step!
| coatue wrote:
| Thank you!
| pella wrote:
| Congratulations!
|
| Please also add this info :
|
| #1. to the pgsql-announce list:
| https://www.postgresql.org/search/?m=1&ln=pgsql-announce&q=h...
| "Your search for hydra returned no hits."
|
| #2. to the https://planet.postgresql.org/
| adultSwim wrote:
| Watch out. There used to be another Hydra project, a data
| repository with rich linked metadata, that changed its name after
| legal threat over trademark from Hydra Corporation. Now it's
| called Hyku, https://hyku.samvera.org/
|
| I hope you choose to defend your name.
| iepathos wrote:
| Nice tool, only unfortunate name, consider changing it. Already
| very well know security tool named hydra
| https://github.com/vanhauser-thc/thc-hydra been around since
| 2001. Then facebook went ahead and named their config tool hydra
| https://github.com/facebookresearch/hydra on top of it. Like we
| get it, hydra popular mythology but we could use more original
| naming for tools
| cultofmetatron wrote:
| yea _acropolis_ would be a better name given that its columns
| are famous
| coatue wrote:
| I was thinking X.com - is it available?
| hamoid wrote:
| Or https://github.com/hydra-synth/hydra (Livecoding networked
| visuals in the browser, since 2017)
| robertlagrant wrote:
| Let's hope Ory never uses it! Oh no[0].
|
| [0] https://www.ory.sh/hydra
| coatue wrote:
| Should have mentioned, if you want to chat about open source,
| analytics, or meet some of the Hydra team swing by our event in
| SF this Thursday: https://partiful.com/e/gowvDVdnNcBLKUzfGOPv
| giovannibonetti wrote:
| > For 1.0 GA release
|
| You may want to check that box in the README, assuming it is
| already done.
| coatue wrote:
| Great catch - updating, please hold
| cjonas wrote:
| What's the workflow for leveraging this extension in real-time
| for an existing database?
|
| Say I wanted to use this to create a high performance
| "aggregation" API of my existing "write heavy" tables.
|
| Is there a way to keep a `heap` & `columnar` table in sync?
|
| (relative Postgres noob here)
| jerrysievert wrote:
| there are a couple of ways to do it, and none of them that I'm
| able to think of are great - maybe some others will be able to
| answer better than I am, but ...
|
| if the data is append-only, an insert trigger could work. if it
| gets updated and deleted, then insert, update, and delete
| triggers could be added. of course if the table is very active,
| this could get bad, fast.
|
| alternately, you can do an insert every hour or so, like insert
| into table_columnar where created_at > DATE_TRUNC('hour',
| created_at)
|
| or, even truncate the columnar table daily and re-insert all of
| the data.
|
| likely none of these is the _best_ solution, but they could
| help you find what might be the best solution for you.
|
| alternately, if the query patterns work well, you can simply
| convert the table to columnar, but that's not a panacea.
___________________________________________________________________
(page generated 2023-09-19 23:01 UTC)