[HN Gopher] Show HN: Turn CSV Files into SQL Statements for Quic...
       ___________________________________________________________________
        
       Show HN: Turn CSV Files into SQL Statements for Quick Database
       Transfers
        
       This package lets you unload a CSV from a warehouse, turn that CSV
       into SQL statements creating a temp table and inserting data with a
       CLI command, and copy those SQL statements into your query editor
       so you can start using the data in a different warehouse. Useful
       for when you need to join data together that's stored in two
       separate data warehouses (e.g. finance and product data).
        
       Author : ryanwaldorf
       Score  : 9 points
       Date   : 2024-05-28 12:26 UTC (10 hours ago)
        
 (HTM) web link (github.com)
 (TXT) w3m dump (github.com)
        
       | nbbaier wrote:
       | This is neat! Correct me if I'm wrong, but don't a lot of systems
       | already have the ability to ingest CSV data? Why this
       | intermediate step?
        
         | bdcravens wrote:
         | Perhaps ease in troubleshooting? I know dealing with line
         | endings, mystery encodings, etc can make CSV ingestion tedious.
        
       | bdcravens wrote:
       | For small datasets, this is good, but for anything significant,
       | using the database's native bulk load will be more efficient than
       | INSERT statements.
        
         | retox wrote:
         | I recently had to transfer a relatively small amount of data,
         | ~2GB between MS SQL databases hosted on the same server. It was
         | a consolidation of the data from 10 total DBs, merging the data
         | from 5 of them into a different schema on the other 5. Think 5
         | DBs for current data and 5 for legacy data, where the outcome
         | was the original 5 current data DBs now contained the current
         | and legacy data.
         | 
         | There was a requirement that the migration didn't blow up the
         | transaction logs on the target DBs. I thought it would be a
         | simple affair but after much testing I had to write a tool to
         | generate a list of all the tables in each source DB, generate
         | format dumping bcp commands for all the tables, then bcp
         | commands to dump the data for each table, then SQL statements
         | to bulk import the data using both files. The MS docs were
         | woefully unhelpful.
        
           | bormaj wrote:
           | Isn't there a query option to disable/configure log writes
           | for cases like this?
        
       ___________________________________________________________________
       (page generated 2024-05-28 23:03 UTC)