[HN Gopher] WebP: The WebPage Compression Format
___________________________________________________________________
WebP: The WebPage Compression Format
Author : Kubuxu
Score : 236 points
Date : 2024-09-07 17:32 UTC (5 hours ago)
(HTM) web link (purplesyringa.moe)
(TXT) w3m dump (purplesyringa.moe)
| next_xibalba wrote:
| If only we hadn't lost Jan Sloot's Digital Coding System [1],
| we'd be able to transmit GB in milliseconds across the web!
|
| [1] https://en.wikipedia.org/wiki/Sloot_Digital_Coding_System
| supriyo-biswas wrote:
| This claim itself is probably a hoax and not relevant to the
| article at hand; but these days with text-to-image models and
| browser support, you could probably do something like <img
| prompt="..."> and have the browser render something that
| matches the description, similar to the "cookbook" analogy used
| in the Wikipedia article.
| Lorin wrote:
| That's an interesting concept, although it would generate a
| ton of bogomips since each client has to generate the image
| themselves instead of one time on the server.
|
| You'd also want "seed" and "engine" attributes to ensure all
| visitors see the same result.
| LeoPanthera wrote:
| You could at least push the work closer to the edge, by
| having genAI servers on each LAN, and in each ISP, similar
| to the idea of a caching web proxy before HTTPS rendered
| them impossible.
| lucianbr wrote:
| Push the work closer to the edge, _and multiply_ it by
| quite a lot. Generate each image many times. Why would we
| want this? Seems like the opposite of caching in a sense.
| sroussey wrote:
| If you are reading a web page on mars and bandwidth is
| more precious than processing power, then <img
| prompt="..."> might make sense.
|
| Not so much for us on earth however.
| roywiggins wrote:
| This sort of thing (but applied to video) is a plot point
| in _A Fire Upon The Deep_. Vinge 's term for the result
| is an "evocation."
| bobbylarrybobby wrote:
| All compression is, in a sense, the opposite of caching.
| You have to do more work to get the data, but you save
| space.
| onion2k wrote:
| Unless you don't actually care if everyone sees the same
| results. So long as the generated image is approximately
| what you prompted for, and the content of the image is
| decorative so it doesn't really need to be a specific,
| accurate representation of something, it's fine to display
| a different picture for every user.
|
| One of the best uses of responsive design I've ever seen
| was a site that looked _completely_ different at different
| breakpoints - different theme, font, images, and content.
| It 's was beautiful, and creative, and fun. Lots of users
| saw different things and had no idea other versions were
| there.
| semolino wrote:
| What site are you referring to?
| 7bit wrote:
| That would require a lot of GBs in libraries in the browser
| and a lot of processing power on the client CPU to render an
| image that is so unimportant that it doesn't really matter if
| it shows exactly what the author intended. To summarize that
| in three words: a useless technology.
|
| That idea is something that is only cool in theory.
| lxgr wrote:
| At least for LLMs, something very similar is already
| happening: https://huggingface.co/blog/Xenova/run-gemini-
| nano-in-your-b...
|
| Currently, we're definitely not there in terms of
| space/time tradeoffs for images, but I could imagine at
| least parameterized ML-based upscaling (i.e. ship a low-
| resolution image and possibly a textual description, have a
| local model upscale it to display resolution) at some
| point.
| ec109685 wrote:
| Similar to what Samsung does if you take a picture of the
| moon.
| everforward wrote:
| Similar ideas have floated around for a while. I've always
| enjoyed the elegance of compressing things down to a start
| and end index of digits of Pi.
|
| It's utterly impractical, but fun to muse about how neat it
| would be if it weren't.
| Groxx wrote:
| Time Lords probably, saving us from the inevitable end of this
| technology path, where all data in the universe is compressed
| into one bit which leads to an information-theoretic black hole
| that destroys everything.
| magicalhippo wrote:
| Reminds me of when I was like 13 and learned about CRC codes
| for the first time. Infinite compression here we come! Just
| calculate the 32bit CRC code for say 64 bits, transmit the CRC,
| then on the other hand just loop over all possible 64 bit
| numbers until you got the same CRC. So brilliant! Why wasn't
| this already used?!
|
| Of course, the downsides became apparent once the euphoria had
| faded.
| _factor wrote:
| Even better, take a physical object and slice it precisely in
| a ratio that contains your data in the fraction!
| lxgr wrote:
| Very relatable! "If this MD5 hash uniquely identifies that
| entire movie, why would anyone need to ever send the
| actual... Oh, I get it."
|
| Arguably the cruelest implication of the pigeonhole
| principle.
| Spivak wrote:
| The real version of this is Nvidia's web conferencing demo
| where they make a 3d model of your face and then only transfer
| the wireframe movements which is super clever.
|
| https://m.youtube.com/watch?v=TQy3EU8BCmo
|
| You can really feel the "compute has massively outpaced
| networking speed" where this kind of thing is actually
| practical. Maybe I'll see 10G residential in my lifetime.
| kstrauser wrote:
| This messages comes to you via a 10Gbps, $50 residential
| fiber connection.
|
| The future is already here - it's just not very evenly
| distributed.
| lxgr wrote:
| > The Sloot Digital Coding System is an alleged data sharing
| technique that its inventor claimed could store a complete
| digital movie file in 8 kilobytes of data
|
| 8 kilobytes? Rookie numbers. I'll do it in 256 bytes, as long
| as you're fine with a somewhat limited selection of available
| digital movie files ;)
| niceguy4 wrote:
| Not to side track the conversation but to side track the
| conversation, has there been many other major WebP exploits like
| the serious one in the past?
| BugsJustFindMe wrote:
| > _the longest post on my site, takes 92 KiB instead of 37 KiB.
| This amounts to an unnecessary 2.5x increase in load time_
|
| Sure, if you ignore latency. In reality it's an unnecessary
| 0.001% increase in load time because that size increase isn't
| enough to matter vs the round trip time. And the time you save
| transmitting 55 fewer KiB is probably less than the time lost to
| decompression. :p
|
| While fun, I would expect this specific scenario to actually be
| worse for the user experience not better. Speed will be a
| complete wash and compatibility will be worse.
| jsnell wrote:
| That size difference is large enough to make a difference in
| the number of round trips required (should be roughly one fewer
| roundtrip with any sensible modern value for the initial
| congestion window).
|
| Won't be a 2.5x difference, but also not 0.001%.
| BugsJustFindMe wrote:
| You don't need a new roundtrip for every packet. That would
| be devastating for throughput. One vs two vs three file
| packets get acked as a batch either way, not serially.
|
| Also when you get to the end, you then see
|
| > _The actual savings here are moderate: the original is 88
| KiB with gzip, and the WebP one is 83 KiB with gzip. In
| contrast, Brotli would provide 69 KiB._
|
| At 69 KiB you're still over the default TCP packet max, which
| means both cases transmit the same number of packets, one
| just has a bunch of extra overhead added for the extra
| JavaScript fetch, load, and execute.
|
| The time saved here is going to be negligible at best anyway,
| but there looks to be actually _negative_ because we 're
| burning time without reducing the number of needed packets at
| all.
| jsnell wrote:
| Those numbers are for a different page. For the original
| page, the article quotes 44 kB with this method vs. 92 kB
| for gzip.
|
| > At 69 KiB you're still over the default TCP packet max,
| which means both cases transmit the same number of packets,
|
| What? No, they absolutely don't transmit the same number of
| packets. Did you mean some other word?
| codetrotter wrote:
| They were probably thinking of the max size for packets
| in TCP, which is 64K (65535 bytes).
|
| However, Ethernet has a MTU (Maximum Transmission Unit)
| of 1500 bytes. Unless jumbo frames are used.
|
| And so I agree with you, the number of packets that will
| be sent for 69 KiB vs 92 KiB will likely be different.
| mananaysiempre wrote:
| I expect what GP meant is the default TCP _window_ size,
| so in a situation where bandwidth costs are dwarfed by
| roundtrip costs, these two cases will end up taking
| essentially the same time, because they will incur the
| same number of ACK roundtrips. Don't know if the numbers
| work out, but they at least sound plausible.
| BugsJustFindMe wrote:
| Yes, sorry
| pierrec wrote:
| Interesting, how does that add a round trip? For the record
| here's what I believe to be the common definition of an
| additional "round trip", in a web development context:
| - client requests X - client gets X, which contains a
| reference to Y - therefore client requests Y
|
| So you're starting a new request that depends on the client
| having received the first one. (although upon closer
| inspection I think the technique described in the blog post
| manages to fit everything into the first response, so I'm not
| sure how relevant this is)
| jsnell wrote:
| Unless a resource is very small, it won't be transmitted in
| a single atomic unit. The sender will only send a part of
| it, wait the client to acknowledge having received them,
| and only then send more. That requires a network roundtrip.
| The larger the resource, the more network roundtrips will
| be required.
|
| If you want to learn more, pretty much any resource on TCP
| should explain this stuff. Here's something I wrote years
| ago, the background section should be pretty applicable:
| https://www.snellman.net/blog/archive/2017-08-19-slow-
| ps4-do...
| crote wrote:
| In reality it's more like: - client
| requests X - server sends bytes 0-2k of X -
| client acknowledges bytes 0-2k of X - server sends
| bytes 2k-6k of X - client acknowledges bytes 2k-6k of
| X - server sends bytes 6k-14k of X - client
| acknowledges bytes 6k-14k of X - server sends bytes
| 14k-30k of X - client acknowledges bytes 14k-30k of X
| - server sends bytes 30k-62k of X - client
| acknowledges bytes 30k-62k of X - server sends bytes
| 62k-83k of X - client acknowledges bytes 62k-83k of X
| - client has received X, which contains a reference to Y
| - therefore client requests Y
|
| It's all about TCP congestion control here. There are
| dozens of algorithms used to handle it, but in pretty much
| all cases you want to have _some_ kind of slow buildup in
| order to avoid completely swamping a slower connection and
| having all but the first few of your packets getting
| dropped.
| Retr0id wrote:
| Why is there more latency?
|
| Edit: Ah, I see OP's code requests the webp separately. You can
| avoid the extra request if you write a self-extracting
| html/webp polyglot file, as is typically done in the demoscene.
| BugsJustFindMe wrote:
| It takes more time for your message to get back and forth
| between your computer and the server than it takes for the
| server to pump out some extra bits.
|
| Even if you transmit the js stuff inline, the op's notion of
| time still just ignores the fact that it takes the caller
| time to even ask the server for the data in the first place,
| and at such small sizes that time swallows the time to
| transmit from the user's perspective.
| Retr0id wrote:
| Here's a demo that only uses a single request for the whole
| page load: https://retr0.id/stuff/bee_movie.webp.html
|
| It is technically 2 requests, but the second one is a cache
| hit, in my testing.
| BugsJustFindMe wrote:
| That's fine, but if you're evaluating the amount of time
| it takes to load a webpage, you cannot ignore the time it
| takes for the client request to reach your server in the
| first place or for the client to then unpack the data.
| The time saved transmitting such a small number of bits
| will be a fraction of the time spent making that initial
| request anyway. That's all I'm saying.
|
| OP is only looking at transmit size differences, which is
| both not the same as transmit time differences and also
| not what the user actually experiences when requesting
| the page.
| jgalt212 wrote:
| I have similar feelings on js minification especially if you're
| sending via gzip.
| edflsafoiewq wrote:
| Well, that, and there's an 850K
| Symbols-2048-em%20Nerd%20Font%20Complete.woff2 file that sort
| of drowns out the difference, at least if it's not in cache.
| lxgr wrote:
| Now I got curious, and there's also a 400 kB CSS file to go
| with it: https://purplesyringa.moe/fonts/webfont.css
|
| I'm not up to date on web/font development - does anybody
| know what that does?
| bobbylarrybobby wrote:
| It adds unicode characters before elements with the given
| class. Then it's up to the font to display those Unicode
| characters -- in this case, based on the class names, one
| can infer that the font assigns an icon to each character
| uses.
| lxgr wrote:
| That makes sense, thank you!
|
| So the purpose is effectively to have human-readable CSS
| class names to refer to given glyphs in the font, rather
| than having stray private use Unicode characters in the
| HTML?
| lobsterthief wrote:
| Yep
|
| This is a reasonable approach if you have a large number
| of icons across large parts of the site, but you should
| always compile the CSS/icon set down to only those used.
|
| If only a few icons, and the icons are small, then
| inlining the SVG is a better option. But if you have too
| many SVGs directly embedded on the site, the page size
| itself will suffer.
|
| As always with website optimization, whether something is
| a good option always "depends".
| marcellus23 wrote:
| Wow, yeah. That kind of discredits the blog author a bit.
| edflsafoiewq wrote:
| I mean, it's all just for fun of course.
| lxgr wrote:
| That's certainly reasonable if you optimize only for loading
| time (and make certain assumptions about everybody's available
| data rate), but sometimes I really wish website (and more
| commonly app) authors wouldn't make that speed/data tradeoff so
| freely on my behalf, for me to find out _after_ they 've
| already pushed that extra data over my metered connection.
|
| The tragedy here is that while some people, such as the author
| of TFA, go to great lengths to get from about 100 to 50 kB,
| others don't think twice to send me literally tens of megabytes
| of images, when I just want to know when a restaurant is open -
| on roaming data.
|
| Resource awareness exists, but it's unfortunately very unevenly
| distributed.
| k__ wrote:
| We need a MoSh-based browser with gopher support.
| sleepydog wrote:
| Seriously, if you're saving less than a TCP receive window's
| worth of space it's not going to make any difference to
| latency.
|
| I suppose it could make a difference on lossy networks, but I'm
| not sure.
| lelandfe wrote:
| If the blob contains requests (images, but also stylesheets,
| JS, or worst case fonts), it will actually instead be a net
| negative to latency. The browser's preload scanner begins
| fetching resources even before the HTML is finished being
| parsed. That can't happen if the HTML doesn't exist until
| after JS decodes it. In other words, the entire body has
| become a blocking resource.
|
| These are similar conversations people have around hydration,
| by the by.
| fsndz wrote:
| exactly what I thought too
| 98469056 wrote:
| While peeking at the source, I noticed that the doctype
| declaration is missing a space. It currently reads
| <!doctypehtml>, but it should be <!doctype html>
| gkbrk wrote:
| > Why readPixels is not subject to anti-fingerprinting is beyond
| me. It does not sprinkle hardly visible typos all over the page,
| so that works for me.
|
| > keep the styling and the top of the page (about 8 KiB
| uncompressed) in the gzipped HTML and only compress the content
| below the viewport with WebP
|
| Ah, that explains why the article suddenly cut off after a random
| sentence, with an empty page that follows. I'm using LibreWolf
| which disables WebGL, and I use Chromium for random web games
| that need WebGL. The article worked just fine with WebGL enabled,
| neat technique to be honest.
| Retr0id wrote:
| I've used this trick before! Oddly enough I can't remember _what_
| I used it for (perhaps just to see if I could), and I commented
| on it here:
| https://gist.github.com/gasman/2560551?permalink_comment_id=...
|
| Edit: I found my prototype from way back, I guess I was just
| testing heh: https://retr0.id/stuff/bee_movie.webp.html
| lucb1e wrote:
| That page breaks my mouse gestures add-on! (Or, I guess we
| don't have add-ons anymore but rather something like script
| injections that we call extensions, yay...) Interesting
| approach to first deliver 'garbage' and then append a bit of JS
| to transform it back into a page. The inner security nerd in me
| wonders if this might open up attacks if you would have some
| kind of user-supplied data, such as a comment form. One could
| probably find a sequence of bytes to comment that will, after
| compression, turn into a script tag, positioned (running)
| before yours would?
| raggi wrote:
| Chromies got in the way of it for a very long time, but zstd is
| now coming to the web too, as it's finally landed in chrome - now
| we've gotta get safari onboard
| CharlesW wrote:
| Looks like it's on the To Do list, at least:
| https://webkit.org/standards-positions/#position-168
| mananaysiempre wrote:
| I'd love to go all-Zstandard, but in this particular case, as
| far as I know, Brotli and Zstandard are basically on par at
| identical values of decompressor memory consumption.
| simondotau wrote:
| Realistically, everything should just support everything.
| There's no reason why every (full featured) web server and
| every (full featured) web browser couldn't support all
| compelling data compression algorithms.
|
| Unfortunately we live in a world where Google decides to rip
| JPEG-XL support out of Chrome for seemingly no reason other
| than spite. If the reason was a lack of maturity in the
| underlying library, fine, but that wasn't the reason they
| offered.
| csjh wrote:
| I think the most surprising part here is the gzipped-
| base64'd-compressed data almost entirely removes the base64
| overhead.
| galaxyLogic wrote:
| Is there a tool or some other way to easily encode a JPG image so
| it can be embedded into HTML? I know there is something like
| that, but is it easy? Could it be made easier?
| throwanem wrote:
| You can convert it to base64 and inline it anywhere an image
| URL is accepted, eg <img
| src="data:image/jpeg;base64,abc123..." />
|
| (Double-check the exact syntax and the MIME type before you use
| it; it's been a few years since I have, and this example is
| from perhaps imperfect memory.)
| TacticalCoder wrote:
| I _loved_ that (encoding stuff in _webp_ ) but my takeaway from
| the figures in the article is this: brotli is so good I'll host
| from somewhere where I can serve brotli (when and if the client
| supports brotli ofc).
| niutech wrote:
| This page is broken at least on Sailfish OS browser, there is a
| long empty space after the paragraph:
|
| > Alright, so we're dealing with 92 KiB for gzip vs 37 + 71 KiB
| for Brotli. Umm...
|
| That said, the overhead of gzip vs brotli HTML compression is
| nothing compared with amount of JS/images/video current websites
| use.
| mediumsmart wrote:
| same on orion and safari and librewolf - is this a chrome page?
| simmonmt wrote:
| A different comment says librewolf disables webgl by default,
| breaking OP's decompression. Is that what you're seeing?
| Dibby053 wrote:
| I didn't know canvas anti-fingerprinting was so rudimentary. I
| don't think it increases uniqueness (the noise is different every
| run) but bypassing it seems trivial: run the thing n times and
| take the mode. With so little noise, 4 or 5 times should be more
| than enough.
| hedora wrote:
| The article says it's predictable within a given client, so
| your trick wouldn't work.
|
| So, just use the anti-fingerprint noise as a cookie, I guess?
| Dibby053 wrote:
| Huh, it seems it's just my browser that resets the noise
| every run.
|
| I opened the page in Firefox like the article suggests and I
| get a different pattern per site and session. That prevents
| using the noise as a supercookie, I think, if its pattern
| changes every time cookies are deleted.
| sunaookami wrote:
| It is different per site and per session, yes.
| toddmorey wrote:
| "I hope you see where I'm going with this and are yelling 'Oh why
| the fuck' right now."
|
| I love reading blogpost like these.
| butz wrote:
| Dropping google fonts should improve page load time a bit too,
| considering those are loaded from remote server that requires
| additional handshake.
| kevindamm wrote:
| ..but if enough other sites are also using that font then it
| may already be available locally.
| pornel wrote:
| This has stopped working many years ago. Every top-level
| domain now has its own private cache _of all other domains_.
|
| You likely have dozens of copies of Google Fonts, each in a
| separate silo, with absolutely zero reuse between websites.
|
| This is because a global cache use to work like a cookie, and
| has been used for tracking.
| kevindamm wrote:
| ah, I had forgotten about that, you're right.
|
| well at least you don't have to download it more than once
| for the site, but first impressions matter yeah
| SushiHippie wrote:
| No, because of cache partitioning, which has been a thing for
| a while.
|
| https://developer.chrome.com/blog/http-cache-partitioning
| butz wrote:
| I wonder what is the difference in CPU usage on client side for
| WebP variant vs standard HTML? Are you causing more battery drain
| on visitor devices?
| lucb1e wrote:
| It depends. Quite often (this is how you can tell I live in
| Germany) mobile data switches to "you're at the EDGE of the
| network range" mode1 and transferring a few KB means keeping
| the screen on and radio active for a couple of minutes. If the
| page is now 45KB instead of 95KB, that's a significant
| reduction in battery drain!
|
| _Under normal circumstances_ you 're probably very right
|
| 1 Now I wonder if the makers foresaw how their protocol name
| might sound to us now
| bawolff wrote:
| They did all this and didn't even measure time to first paint?
|
| What is the point of doing this sort of thing if you dont even
| test how much faster or slower it made the page to load?
| lxgr wrote:
| From the linked Github issue giving the rationale why Brotli is
| not available in the CompressionStream API:
|
| > As far as I know, browsers are only shipping the decompression
| dictionary. Brotli has a separate dictionary needed for
| compression, which would significantly increase the size of the
| browser.
|
| How can the decompression dictionary be smaller than the
| compression one? Does the latter contain something like a space-
| time tradeoff in the form of precalculated most efficient
| representations of given input substrings or something similar?
| gildas wrote:
| In the same vein, you can package HTML pages as self-extracting
| ZIP files with SingleFile [1]. You can even include a PNG image
| to produce files compatible with HTML, ZIP and PNG [2], and for
| example display the PNG image in the HTML page [3].
|
| [1] https://github.com/gildas-lormeau/SingleFile?tab=readme-
| ov-f...
|
| [2] https://github.com/gildas-lormeau/Polyglot-HTML-ZIP-PNG
|
| [3] https://github.com/gildas-lormeau/Polyglot-HTML-ZIP-
| PNG/raw/...
| astrostl wrote:
| Things I seek in an image format:
|
| (1) compatibility
|
| (2) features
|
| WebP still seems far behind on (1) to me so I don't care about
| the rest. I hope it gets there, though, because folks like this
| seem pretty enthusiastic about (2).
| somishere wrote:
| Lots of nice tricks in here, definitely fun! Only minor nitpick
| is that it departs fairly rapidly from the lede ... which
| espouses the dual virtues of an accessible and js-optional
| reading experience ;)
| bogzz wrote:
| Still waiting on a webp encoder to be added to the Go stdlib...
| kopirgan wrote:
| 19 year old and look at the list of stuff she's done! Perhaps
| started coding in the womb?! Amazing.
___________________________________________________________________
(page generated 2024-09-07 23:00 UTC)