[HN Gopher] Supabase Storage v2: Image Resizing and Smart CDN
___________________________________________________________________
Supabase Storage v2: Image Resizing and Smart CDN
Author : samwillis
Score : 97 points
Date : 2022-12-13 14:12 UTC (8 hours ago)
(HTM) web link (supabase.com)
(TXT) w3m dump (supabase.com)
| kiwicopple wrote:
| hey hn, supabase ceo here
|
| hopefully it's obvious from the title what we're shipping today.
| "Supabase Storage" is for larger assets - it's a wrapper around
| s3, where the buckets/folders are mapped into your postgres
| database so that you can write access rules using Row Level
| Security.
|
| This release adds image resizing. It looks like this:
| supabase.storage.from('bucket').getPublicUrl('image.jpg', {
| transform: { width: 500, height: 600,
| }, })
|
| In tandem added some webhooks/events to the storage server to
| enable some smarter cache-busting. We're using Cloudflare for
| caching on our hosted platform, but the addition of webhooks
| should provide the primitives to use any CDN/service.
|
| The Storage API is 100% open source and can be hosted as a
| standalone server (with Postgres+PostgREST):
| https://github.com/supabase/storage-api
| no_wizard wrote:
| Any plans to switch to R2? It's cheaper than S3 and would run
| in network with Cloudflare, allowing Supabase to take advantage
| of the CDN more optimally over time
| kiwicopple wrote:
| It's one of the storage providers we'll investigate, now that
| R2 is GA.
|
| Our Storage server is hosted on AWS, so it wouldn't
| immediately save money (at least for egress) but we'll also
| investigate moving the servers to another cloud provider.
| skrtskrt wrote:
| Could this eventually be pluggable with bring-your-own
| s3-compatible provider? Would love this in front of my
| DigitalOcean object storage
| kiwicopple wrote:
| yes definitely. It should be easy to add s3-compatible
| providers (which are numerous, since s3 seems to be the
| defacto file storage API now).
|
| We haven't added "BYO" functionality to our hosted
| Platform because, frankly, we already spend a lot of time
| debugging various quirks and network issues - even for
| infra which we provide. Adding an extra level of
| opaqueness at our current team size would be very
| stressful from a support-perspective. That said, we're
| designing/developing so that it's possible and we'll
| continue to assess as the team grows.
| braindead_in wrote:
| Great work. Congrats!
|
| I am looking to build a file management app using Supabase +
| NextJS (e.g. Dropbox clone). Can you please point me the right
| way. Thanks!
| kiwicopple wrote:
| I'm not sure if I could describe the full implementation of a
| Dropbox clone in a HN comment, but our Storage Docs[0] should
| give you an idea of most of the details that you need.
|
| If you prefer videos, I see there are a few tutorials on
| YouTube when you search "Supabase Storage"
|
| [0] Storage Docs: https://supabase.com/docs/guides/storage
| danpalmer wrote:
| > it's a wrapper around s3
|
| > We're using Cloudflare for caching
|
| It sounds like Supabase is fairly explicitly targeting being
| the glue between a bunch of lower level infrastructure, that
| may be in different cloud services, and providing one platform
| with a better UX. Is this fair to say?
|
| If so this is an interesting strategy to be so open about this.
| Other platforms-as-a-service don't seem to like to acknowledge
| the services they use under the hood. I can see this being both
| a positive (brand recognition, trust), and a negative ("we
| could just build this ourselves and have more choice").
|
| Is this an intentional strategy? Can you talk any more about
| how you're thinking about it?
| kiwicopple wrote:
| > _explicitly targeting being the glue between a bunch of
| lower level infrastructure_
|
| It might be more precise to say that we provide the
| primitives that a developer needs to host Supabase on any
| cloud provider. Becoming the "glue" appears to be a byproduct
| of this approach
|
| For example: we use s3 right now, but we've built the storage
| server so that it can be extended with other storage
| providers (in fact, it has a "local disk" provider for
| developing locally).
|
| We use Cloudflare right now for the CDN, but as mentioned in
| the blog post the Webhooks we built into this release are so
| that we could migrate to any CDN service without locking
| ourselves into any CDN-specific cache invalidation.
|
| Portability is something we think about a lot (although
| harder to solve this problem at a CDN-level).
|
| > _Other platforms-as-a-service don 't seem to like to
| acknowledge the services they use under the hood Is this an
| intentional strategy? Can you talk any more about how you're
| thinking about it?_
|
| There's no strategy behind it. We are trying to build a great
| developer platform, and at our current stage these platforms
| provide a better service for some features than we could
| ourselves. There's no shame in that from our POV, and these
| services deserve the credit for delivering something useful.
| I doubt we'll ever get to a "build our own datacenter" stage
| - we'll probably just continue to focus on the application
| level (with a Database-centric view of the world)
| no_wizard wrote:
| I'm perhaps overly bullish on your platform so I say never
| say never!
|
| Transparently, I hope you guys are hiring developers again
| soon, I had look into becoming a technical engineer on the
| sales side but we couldn't make the numbers work.
|
| I really can't wait to see your guys next round of
| innovations!
| chrisweekly wrote:
| Great point. Maybe 2y ago I asked Vercel salespeople about
| their infr and learned it was virtually all AWS. (Technically
| "multi-cloud" bc they used GCP in some minor way.)
| techn00 wrote:
| When will the storage dashboard be open source?
| kiwicopple wrote:
| Just caught up with the FE team on this one - it's pending in
| this PR: https://github.com/supabase/supabase/pull/10810
|
| One of the engineers is testing this now and it should get
| merged within the hour. The build for amd64 takes about
| 6mins, and about 4 hours for arm64. I'll update here when
| it's merged (edit: it was actually merged earlier today, I
| originally linked to the wrong PR)
|
| For more context, the Storage UI was merged a couple of weeks
| ago but we simultaneously rolled out a new login screen for
| our Dashboard (for SSO) which then got rolled out to the
| Docker image. We've added a feature flag. Sorry for the
| delays - the lead up to launch week has everyone a bit
| stretched.
| eclipxe wrote:
| What a great comment. I applaud the transparency and depth.
| Nice.
| inian wrote:
| Supabase Engineer here
|
| There were a lot of primitives we built to ship image resizing -
| Storage events exposed via webhooks, rate limiting, a queue on
| top of Postgres, a smart CDN cache, object metadata endpoints,
| etc. These are already available if you are self hosting Supabase
| Storage, so that you can integrate your own CDN, listen to
| storage events, etc. Over the next few months, we will be working
| on exposing these on the Supabase platform too.
| babelfish wrote:
| Is image resizing built on top of Cloudflare too?
| https://developers.cloudflare.com/images/image-resizing/
| inian wrote:
| We use imgproxy[1] for doing the resizing. We utilize open
| source services for the entire stack, so that you can self-
| host Supabase.
|
| [1] https://github.com/imgproxy/imgproxy
| d1l wrote:
| nginx already comes with:
| http://nginx.org/en/docs/http/ngx_http_image_filter_module.h...
| kiwicopple wrote:
| We assessed a few resizing tools - imgproxy, Imaginary,
| Sharp, etc. We went with imgproxy mostly because of it's
| feature-completeness (for example, it supports AVIF and it
| looks like nginx does not)
| LukeLambert wrote:
| In nearly two decades developing for the web, I've never found
| a good reason to enlarge an image server-side, so thank you for
| anticipating that footgun. So many of your competitors get it
| wrong.
|
| However, it's unclear from the docs [0] if one of the most
| common use cases is supported: requesting an image at a
| specific aspect ratio while not exceeding a maximum size.
|
| For instance, I want a 1600x900px hero image at the top of my
| blog posts, but my source image is only 1200px wide. I request:
| transform: { width: 1600, height: 900,
| resize: 'cover' }
|
| Will the returned image be 675px tall, maintaining the 16x9
| aspect ratio I want?
|
| A few more notes:
|
| The blog post states "default mode maintains aspect ratio and
| the resize mode is fill", but the docs say that cover is the
| default mode.
|
| The docs say, "If only one [width or height] parameter is
| specified, the image will be resized and cropped, maintaining
| the aspect ratio." I believe that with only one parameter, you
| could either resize or crop, but not both. I hope it's resize.
|
| For cropping modes, is it always from the center? Are there
| plans to support other "gravity" modes?
|
| [0] https://supabase.com/docs/guides/storage/image-
| transformatio...
| kiwicopple wrote:
| > _The blog post states "default mode maintains aspect ratio
| and the resize mode is fill", but the docs say that cover is
| the default mode._
|
| The default resize mode is cover and the blog post had a
| typo. Fixed now, thanks!
|
| > _resize or crop_
|
| We crop by default since the default mode is cover.
|
| > _gravity_
|
| we will be adding crop with gravity soon.. currently the crop
| is always from the center
|
| > _requesting an image at a specific aspect ratio while not
| exceeding a maximum size._
|
| I'll get back to you on this one - Inian had to get some
| sleep, it's very late (early) for him
| LukeLambert wrote:
| Thank you for following up. IMHO, object-fit is not the
| right model for server-side image manipulation, since its
| behavior is only specified when both width and height are
| supplied. I've also never seen a real world use case for
| stretching, squeezing, padding, or enlarging images on the
| server. That should be done on the client. I mentioned some
| of the same issues on the Cloudflare forum when they
| announced an image resizing product based on object-fit.
|
| I believe all resizing operations a user might
| realistically want can be achieved with three parameters:
| width, height, and crop. Additional x and y parameters
| (floats between 0.0 and 1.0) could be used for setting the
| focal point of the crop box.
|
| Examples: source.jpg?width=400
|
| Iff source width is greater than 400px, scale down to 400px
| wide. source.jpg?height=300
|
| Iff source height is greater than 300px, scale down to
| 300px high. source.jpg?width=400&height=300
|
| Iff source width is greater than 400px or source height is
| greater than 300px, scale down to fit within a 400px by
| 300px box.
| source.jpg?width=400&height=300&crop=true
|
| Crop source to a 4x3 aspect ratio. Iff result is larger
| than 400px by 300px, scale down.
| swyx wrote:
| actually, image upsizing is pretty much required for all of
| the AI Art generators. was even a big feature in SD 2.0. i'd
| love to have it as part of this API but itd probably cost too
| much
| niklasd wrote:
| Great job! What are the implications on resizing regarding
| storage egress & download size? Can I expect both to be smaller
| when resizing an image?
| inian wrote:
| Yes! If you resize your images correctly, your users use
| lesser data to load assets from Storage. A side benefit is
| you will be paying us lesser in egress fees.
___________________________________________________________________
(page generated 2022-12-13 23:01 UTC)