[HN Gopher] Amazon Redshift Re-Invented
       ___________________________________________________________________
        
       Amazon Redshift Re-Invented
        
       Author : belter
       Score  : 47 points
       Date   : 2022-05-20 15:32 UTC (3 days ago)
        
 (HTM) web link (www.amazon.science)
 (TXT) w3m dump (www.amazon.science)
        
       | belter wrote:
       | PDF:
       | https://assets.amazon.science/93/e0/a347021a4c6fbbccd5a05658...
        
       | mountainriver wrote:
       | Amazon is losing their edge, the toxic company culture is pushing
       | away good engineers and we are seeing more and more 3rd party
       | companies begin to dominate over spaces amazon previously did
       | e.g. snowflake
       | 
       | I welcome this
        
       | coderintherye wrote:
       | Does anyone choose Redshift these days, asides from those
       | completely tied into AWS? Feels like Redshift is playing catch up
       | but moving so slow it will never catch up.
       | 
       | As another commenter noted, Redshift in my experience was an
       | operational hassle.
       | 
       | Snowflake and BigQuery just work.
       | 
       | Why choose Redshift at this point?
        
       | websap wrote:
       | How does this compare to offerings from Snowflake and Databricks?
        
         | oxfordmale wrote:
         | In the last two years I have used both Redshift (AQUA) and
         | Snowflake, and I prefer Snowflake by a mile. Snowflake is just
         | a lot easier to use, scales better and has a much better
         | permission model.
        
       | yveezy wrote:
       | I've worked with Redshift for about 5-ish years and BigQuery for
       | about a year. IMO BigQuery wins hands down.
       | 
       | From an operational perspective we've had almost 0 issues with
       | BQ, whereas with Redshift we had to constantly keep giving it
       | TLC. Right from creating users /schemas to WLM tuning,
       | structuring files as Parquet for Spectrum access, understanding
       | why and how spectrum performs in different scenarios, etc.
       | everything was a chore. All this redshift specific specialization
       | I was learning was not really contributing to the product in a
       | meaningful way.
       | 
       | Switched to Bq, a year ago, and it's been mostly self driving.
       | the only thing we had to tend to was the slots and bit of a
       | learning curve for the org about partitioning keys (there is a
       | setting in BQ that fails your query IF partition key is not
       | specified)
       | 
       | Having switched to BQ it's really hard for me to imagine going
       | back to Redshift. It almost feels antiquated.
        
         | dehrmann wrote:
         | I briefly worked on Redshift and have used Athena/Presto and
         | Bigquery. Redshift felt like an architectural middle ground.
         | Presto can query almost anything, and Bigquery requires storing
         | data in Bigquery, but like Presto, you don't have to pay for
         | inactive compute use. Redshift's scaling story is more
         | complicated, and paying for inactive compute wasn't ideal. It
         | sounds like it might have improved, but you're still
         | essentially building Bigquery at that point. There might be
         | some use cases that need a fast, columnar store that's already
         | online, so queries take 3s, not 10s with Bigquery.
         | 
         | I generally prefer Bigquery, and between it and Bigtable, I
         | actually prefer GCP over AWS because their offerings for hard-
         | to-do things are really good. I'd honestly pick GCP just for
         | those two products.
        
           | beckingz wrote:
           | Bigquery alone is the reason I recommend GCP.
           | 
           | That said, both AWS and GCP have some real rough edges.
        
           | contol-m wrote:
           | I work on BigQuery. All of these are great points: just
           | wanted to point out that BigQuery can federate into external
           | data sources as well: e.g. files on cloud storage and
           | BigTable. Relevant feature is BigLake:
           | https://cloud.google.com/bigquery/docs/biglake-intro
        
         | TuringNYC wrote:
         | Our most shocking discovery on Redshift was that primary key
         | constraints are not honored. (Not sure why they even have PK
         | identifiers given this, it just adds more confusion.)
        
       | halotrope wrote:
       | I am glad there is some innovation happening. If you compare
       | redshift to BigQuery it feels quite archaic in the current
       | production version. I understand that Google commands a metric
       | ton of engineering excellence, yet the difference should not be
       | so extreme especially when you consider AWS still being the king
       | of cloud. BigQuery is downright magic.
       | 
       | I would love to see more competition in this space as having
       | large amounts of data with Google always makes me feel uneasy for
       | all kinds of reasons.
        
       | advisedwang wrote:
       | Sounds like BigQuery!
        
       ___________________________________________________________________
       (page generated 2022-05-23 23:00 UTC)