[HN Gopher] My Browser WASM't Prepared for This. Using DuckDB, A...
___________________________________________________________________
My Browser WASM't Prepared for This. Using DuckDB, Apache Arrow and
Web Workers
Author : jjp
Score : 100 points
Date : 2025-04-06 07:31 UTC (15 hours ago)
(HTM) web link (motifanalytics.medium.com)
(TXT) w3m dump (motifanalytics.medium.com)
| __mp wrote:
| I've been toying with the idea of implementing a distributed
| analytics engine on top of Cloudflare workers and DuckDB.
|
| I'm not sure if this goes against the CloudFlare TOS tough (last
| time I checked they had some provisons against processing
| images).
| judge2020 wrote:
| In the past few years there was this blog post[0] that
| clarified this. It moved the restriction on serving a
| "disproportionate percentage of pictures, audio files, or other
| large files" to another part of the TOS dedicated specifically
| to the CDN part[1] and clarified that, if you're using
| Cloudflare add-on services Stream, R2 (their S3), or Cloudflare
| Images, then you won't be at risk of termination.
|
| 0: https://blog.cloudflare.com/updated-tos/
|
| 1: The restriction still exists at
| https://www.cloudflare.com/service-specific-terms-applicatio...
| under "Content Delivery Network (Free, Pro, or Business)".
| httgp wrote:
| BoilingData does something very similar, except with Lambdas on
| AWS.
| bobnamob wrote:
| They've already got one:
| https://developers.cloudflare.com/analytics/analytics-engine...
| tobilg wrote:
| IMO that's not really possible because of the size limits of
| Cloudflare Workers. Neither the WASM nor the Node version are
| small enough.
|
| I'm running it on AWS Lambda functions with some success.
| canadiantim wrote:
| How persistent can you make this data?
| mcraiha wrote:
| Tip for all the blog authors, do NOT post code as image.
| Specially do not add fake editor UI and drop shadow to the image.
|
| In this case 25 lines of code is 50 kB of image binary.
|
| Also it cannot be searched via search engine. Nor can it be read
| with screen reader.
| goda90 wrote:
| This guide can help if you still want the code to be pretty:
| https://www.taniarascia.com/adding-syntax-highlighting-to-co...
| doubled112 wrote:
| Pro tip for everybody: do not post any text as images
|
| Never should I receive a Java exception hundreds of lines long
| as a cut off JPEG file.
|
| Or a screenshot of a Google Sheet missing the information
| you're talking to me about.
| shepmaster wrote:
| We made this to be used as a reply when pictures are misused
| where text would be better:
|
| https://fewer.pics/
| stingraycharles wrote:
| I'm disappointed the rendered picture isn't advanced CSS. I
| honestly expected that to be the case.
| natebc wrote:
| I hope someone accepts this challenge!
| lgas wrote:
| Why ruin the message by writing half of it upside down?
| doubled112 wrote:
| It actually enhances the message by being harder to read.
| shepmaster wrote:
| Yep, as the sibling said, it's to prove the point in a
| snarky manner. The yellow-on-white color, the wobbling,
| upside down path of text, the font choice, the unrelated
| image; all of these serve to make the concept actively
| hard to process.
|
| Hopefully this sparks a little "wow I wonder if my image
| of text was also hard to read like I just experienced"
| moment.
| brulard wrote:
| It should have been animated.
| jasmcole wrote:
| We use the WASM build of DuckDB quite extensively at Count
| (https://count.co - 2-3m queries per month). There are a couple
| of bugs we've noticed, but given that it's pretty much maintained
| by a single person seems impressively reliable!
| jillyboel wrote:
| Looking at your insane pricing page I have to assume that you
| are sponsoring that single person?
| Spivak wrote:
| I'm confused, nothing about their pricing looks that weird.
| Businesses don't typically have large BI teams so you can
| ride that $199/mo $2400/year for a long time which is so
| small most SMBs can probably expense it without approval.
| jillyboel wrote:
| You're focussing on the wrong part
| tobilg wrote:
| Not sure why the comparisons were made with pretty outdated
| versions to be honest.
|
| I'm using a (older) v1.29.1 dev version with https://sql-
| workbench.com w/o any bigger issues.
| azakai wrote:
| > [wasm] is executed in a stack-based virtual machine rather than
| as a native library code.
|
| Wasm's binary format is indeed a stack-based virtual machine, but
| that is not how it is executed. Optimizing VMs convert it to SSA
| form, basic blocks, and finally machine code, much the same as
| clang or gcc compile native library code.
|
| It is true that wasm has some overhead, but that is due to
| portability and sandboxing, not the stack-based binary format.
|
| > On top of the above, memory available to WASM is limited by the
| browser (in case of Chrome, the limit is currently set at 4GB per
| tab).
|
| wasm64 solves this, by allowing 64-bit pointers and a lot more
| than 4GB of memory.
|
| The feature is already supported in Chrome and Firefox, but not
| everywhere else yet.
___________________________________________________________________
(page generated 2025-04-06 23:01 UTC)