[HN Gopher] You can now use WebGPU in Cloudflare Workers
___________________________________________________________________
You can now use WebGPU in Cloudflare Workers
Author : todsacerdoti
Score : 129 points
Date : 2023-09-27 13:00 UTC (10 hours ago)
(HTM) web link (blog.cloudflare.com)
(TXT) w3m dump (blog.cloudflare.com)
| antimora wrote:
| Hooray. Now I can run WebGPU ML on a cloudflare worker using Burn
| framework in Rust (https://github.com/burn-rs/burn). Burn has
| WebGPU backend in Rust.
| celso wrote:
| We'd love to see it!
| millerhooks wrote:
| I've been working on and towards similar for nearly six years
| now. CloudFlare just eliminated so much stack complexity for me
| in one swoop and cleared a considerable amount of my todo list.
| Good work y'all!
| no_wizard wrote:
| This is pretty amazing. This allows you to do all kinds of stuff
| around video, manipulation stands out to me, though they mention
| GPGPU as a use case without further elaboration, which makes me
| wonder if they might be coming out with something soon where you
| can use WebGPU APIs for GPGPU scenarios.
| paulgb wrote:
| If by GPGPU you mean compute shaders, they do support that
| today according to the announcement.
|
| > we fully support the part of the API that the compute shaders
| and the compute pipeline require, but we are not yet focused on
| fragment or vertex shaders used in rendering pipelines.
| alex7o wrote:
| You might be interested in this
| https://github.com/webonnx/wonnx
| no_wizard wrote:
| That could be very interesting in workers!
| celso wrote:
| https://github.com/cloudflare/workers-wonnx
| et-al wrote:
| I couldn't find out the VRAM limit, but I assume for video
| manipulation one would need quite a bit. Also, it looks like
| the workers themselves only have access to 128MB of RAM unless
| this has changed?
|
| https://developers.cloudflare.com/workers/platform/limits/#m...
| latchkey wrote:
| Dawn supports OpenGL, but that was left out of the blog post.
| anonbanker wrote:
| Considering how CloudFlare was just implicated in destroying
| secure email at Defcon [1], the addition of GPUs to that active
| assault is terrifying.
|
| 1. https://www.youtube.com/watch?v=NwnT15q_PS8
| weird-eye-issue wrote:
| It was Mailchannels, and Cloudflare was only one of the many
| companies that they fooled into doing an integration. As far as
| the CF Workers <> Mailchannels integration, the security issues
| are pretty much fixed now, but I still wouldn't trust
| Mailchannel
| stri8ted wrote:
| The CloudFlare worker is just being used to make a POST request
| to MailChannels. How is this a CloudFlare problem?
| drunkendog wrote:
| I believe the secure email issue linked is exploitable by any
| MailChannels customer, not just Cloudflare Workers.
| btown wrote:
| Securing multiple tenants on a GPU is a really, really hard
| problem. As https://gpuweb.github.io/gpuweb/#malicious-use notes,
| there's a tremendous surface for undefined behavior.
|
| Now, it does appear that the security model
| https://developers.cloudflare.com/workers/learning/security-...
| ensures that any nodes to which the GPU is connected aren't
| running the full JS Workers code, just the WebGPU functionality.
| So secrets that are maintained on the JS side aren't likely
| vulnerable to Cloudbleed-esque attacks:
| https://en.wikipedia.org/wiki/Cloudbleed .
|
| But I'd be incredibly wary in sending sensitive information to
| the WebGPU calls themselves - assume any other WebGPU-using
| tenant can arbitrarily access data sent to any GPU and modify
| your outputs.
| fidotron wrote:
| The most surprising aspect of this seems to be that Cloudflare
| would think of deploying it, given the apparent cost of GPUs. It
| almost suggests they are using GPUs for their own workloads
| already.
|
| Honestly the clear case for this will be in online video
| distribution; both the video processing itself (encoding,
| compositing, enforcing privacy masks) and using inference to
| decide how to route it (including choosing bandwidth, configuring
| the internal distribution graph).
| [deleted]
| toddmorey wrote:
| Help: I can wrap my head around WebGPU for in browser & on device
| 3D, video, and ML like face or hand tracking, etc.
|
| I struggle to imagine the use cases where WebGPU runs on the edge
| / servers & makes more sense than other ways to access GPUs.
| Anyone have a good layman's example use case?
| divbzero wrote:
| That's a fair question and I think it can be asked more
| generally: "I struggle to imagine the use cases where _X_ runs
| on the edge / servers & makes more sense than other ways to
| access _X_." You are balancing the cost and speed of _X_ in
| different locations, the cost and speed of accessing _X_ over
| the network, and whether you have existing infrastructure
| deployed there already.
|
| Cloudflare Workers and other edge services tend to make sense
| if you want more power than the client (e.g. phone) can
| provide, low latency to the service, and easy-to-scale
| deployments.
| vineyardmike wrote:
| 1. You want or have a "serverless" function. 2. You want to add
| some sort of ML work to that serverless function. 3. The
| serverless function runs on a maimed browser environment (v8)
|
| Specific examples... you have an API that takes a picture
| returns if there's a hotdog in the picture. Or an API to
| generate images from customer input. Etc.
|
| Why is this better or why does it make more sense? If you're
| already using the above environment for your functions it makes
| sense to keep using it. In this way, you don't have to manager
| or provision GPUs at all.
| andrejguran wrote:
| They have pretty cool examples here:
| https://ai.cloudflare.com/?gpu
|
| Text Generation Speech Recognition Image Classification Text
| Classification Text Embedding Translation
| celso wrote:
| The demos in this page are using Workers AI, not WebGPU APIs.
| However, yes, you can perfectly run an ML inference runtime
| that takes advantage of GPUs using just Wasm and WebGPU APIs.
| That's exactly what we have here:
|
| https://github.com/cloudflare/workers-wonnx
| ramesh31 wrote:
| Serverless single shot inference
___________________________________________________________________
(page generated 2023-09-27 23:02 UTC)