https://lcamtuf.substack.com/p/sir-theres-a-cat-in-your-mirror-dimension [https] lcamtuf's thing SubscribeSign in Share this post [https] Sir, there's a cat in your mirror dimension lcamtuf.substack.com Copy link Facebook Email Note Other Sir, there's a cat in your mirror dimension Pets do the darndest things, especially if you teach them a bit of math. Apr 27, 2024 30 Share this post [https] Sir, there's a cat in your mirror dimension lcamtuf.substack.com Copy link Facebook Email Note Other 4 Share A while back, we talked about the frequency domain: a clever reinterpretation of everyday signals that translates them into the amplitudes of constituent waveforms. The most common basis for this operation are sine waves running at increasing frequencies, but countless other waveforms can be used to create a number of alternative frequency domains. In that earlier article, I also noted two important properties of frequency domain transforms. First, they are reversible: you can recover the original ("time domain" or "spatial domain") data from its frequency image. Second, the transforms have input-output symmetry: the same mathematical operation is used to go both ways. In effect, we have a lever that takes us to a mirror dimension and back. Which of the lever positions is called home is a matter of habit, not math. Of course, in real life, the distinction matters -- and it's particularly important for compression. If you take an image, convert it to the frequency-domain representation, and then reduce the precision of (or outright obliterate!) the high-frequency components, the resulting image still looks perceptually the same -- but you now have much less data to transmit or store: [https] Using MS Paint as a cutting-edge compression tool. This makes you wonder: if the frequency-domain representation of a typical image looks like diffuse noise, if most of it is perceptually unimportant, and if the transform is just a lever that takes us back and forth between two functionally-equivalent dimensions... could we start calling that mirror dimension home and move some stuff in? To answer this stoner question, I grabbed a photo of a cat and then calculated its frequency-domain form with the discrete cosine transform (DCT): [https] Time cat, frequency cat. Next, I reused the photo of a woman from an earlier example and placed the mirror-dimension "cat noise" pattern over it, dialing down opacity to minimize visible artifacts: [https] Time woman with a frequency cat. The compositing operation is necessarily lossy, but my theory was that if the composite image is run through DCT to compute its frequency-domain representation, the photo of a woman would be decomposed to fairly uniform noise, perhaps easy to attenuate with a gentle blur; while the injected "cat noise" would coalesce into a perceptible image of a cat. But would it?... Yes! [https] Frequency cat with a time woman. If you want to see for yourself, download the composite image and have fun. In MATLAB, you can do the following: woman = imread("woman-with-cat.png"); colormap('gray'); imagesc(woman, [0 255]); pause(1); cat = dct2(woman); imagesc(imgaussfilt(cat, 1), [-4 4]); Interestingly, the kitty survives resizing of the host document. Upscaling tiles the image; downscaling truncates it. My lingering question was how badly the cat would get mangled by lossy compression; as it turns out, the impact is less than I expected. At higher JPEG quality settings, the image looks quite OK. As the quality setting is lowered, the bottom right quadrant -- corresponding to higher-frequency components -- gets badly quantized: [https] The havoc of JPEG compression, as seen in the frequency domain. This visualization offers a fascinating glimpse of just how much information is destroyed by the JPEG algorithm -- mostly without us noticing. There's plenty of prior art for using audio spectrograms for hidden messages, and some discussion of text steganography piggybacked on top of JPEG DCT coefficients. My point isn't that the technique is particularly useful or that it has absolutely no precedent. It's just that the frequency domain and the time domain are coupled together in funny ways. [ ] Subscribe For more articles about electronics, algorithms, snowplowing, and 19th century repeating pistols, see this categorized list. 30 Share this post [https] Sir, there's a cat in your mirror dimension lcamtuf.substack.com Copy link Facebook Email Note Other 4 Share 4 Comments [https] [ ] lcamtuf Apr 28Pinned Bonus content: the deterioration of a "standalone" frequency-domain cat for various JPEG quality settings: author https://vimeo.com/940487310/8a929a5eb5 Expand full comment Reply Share Omar Shehata Omar's Writing Apr 27 I love this so much!! Makes me want to see an interactive little app where I can "paint" in the frequency domain on the left and see in realtime the "time domain" (that's just the "original" right? back to good old 0-100 monochrome pixels?) (and also drag and drop _another_ image so I can paint bits from one frequency domain into another. this is one thing I [https] was sort of trying to get at with the interactive bits in this article https://parametric.press/issue-01/ unraveling-the-jpeg/) I don't understand why "Time woman with a frequency cat" looks the way it is, is that expected, that it doesn't look any different from original? is it because the "frequency cat" we added in was only a small piece..? like we added in only some high freq components...? Expand full comment Reply Share 1 reply by lcamtuf 2 more comments... Top Latest Discussions No posts Ready for more? [ ] Subscribe (c) 2024 lcamtuf Privacy [?] Terms [?] Collection notice Start WritingGet the app Substack is the home for great culture Share Copy link Facebook Email Note Other This site requires JavaScript to run correctly. Please turn on JavaScript or unblock scripts