[HN Gopher] Show HN: Houseplant - Database Migrations for ClickH...
       ___________________________________________________________________
        
       Show HN: Houseplant - Database Migrations for ClickHouse
        
       Author : vortex_ape
       Score  : 26 points
       Date   : 2024-12-27 18:11 UTC (4 hours ago)
        
 (HTM) web link (houseplant.readthedocs.io)
 (TXT) w3m dump (houseplant.readthedocs.io)
        
       | christiangenco wrote:
       | I'm new to ClickHouse. Why is this necessary? Isn't there a
       | native way to do ClickHouse migrations?
        
         | benpacker wrote:
         | You can connect and run SQL, but there's no built in support
         | for versioning, testing, and rolling back DDL.
        
         | owenthejumper wrote:
         | Migrations is a confusing word in the context. It's a modern
         | word for 'configuration management for database schemas'.
        
       | akdor1154 wrote:
       | On one hand i see SQL-in-yaml and instantly nope outta here.
       | 
       | On the other, I've definitely had trouble with getting an
       | existing migrations tool to work with Clickhouse.. currently
       | using golang-migrate but it's not painless. I think Atlas might
       | have the right idea but I'm hesitant to lock my company into
       | commercial tooling. Sqlmesh may also kind of do what's needed,
       | but its focus is more on transforming tables of data, managing
       | your ddl happens as a happy accident along the way.
        
       | caust1c wrote:
       | In my experience, my pain hasn't been from managing the migration
       | files themselves.
       | 
       | My pain comes from needing to modify the schema in such a way it
       | requires a data migration, reprocessing data to some degree, and
       | managing the migration of that data in a sane way.
       | 
       | You can't just run a simple `INSERT INTO table SELECT * FROM
       | old_table`, or anything like that because if the data is large,
       | it takes forever and a failure in the middle could be fairly
       | difficult to recover from.
       | 
       | So what I do is I split the migration into time-based chunks,
       | because nearly every table has a time component that is immutable
       | after writing, but I really want a migration tool that can figure
       | out what that column is, what those chunks are, and incrementally
       | apply a data migration in batches so that if one batch fails I
       | can go in there to investigate and know exactly how much progress
       | on the data migration has been made.
        
       ___________________________________________________________________
       (page generated 2024-12-27 23:00 UTC)