[HN Gopher] How does lossless compression in Fuji RAF files work...
___________________________________________________________________
How does lossless compression in Fuji RAF files work? (2020)
Author : dsego
Score : 95 points
Date : 2025-09-26 18:43 UTC (4 days ago)
(HTM) web link (capnfabs.net)
(TXT) w3m dump (capnfabs.net)
| vjvjvjvjghv wrote:
| There are always these crazy discussions in photo forums where
| people are arguing that uncompressed RAW files are better than
| lossless compressed.
| jsheard wrote:
| Glad to hear that audiophiles have some competition in who can
| invent the most pointless things to split hairs over.
|
| The photo guys need to start taping magic rocks to their
| cameras to really keep up though.
| badlibrarian wrote:
| They weren't just rocks. They were brilliant pebbles.
|
| https://web.archive.org/web/20150220012844/http://www.machin.
| ..
| miladyincontrol wrote:
| The worst part is its often in areas people improperly
| understand the benefits and downsides.
|
| Biggest problem with good raw compression is you have a
| linear DNG, half processed essentially. Great, the file size
| is smaller, but now you miss data that processes like AI
| denoise can benefit from as the image is already debayered.
|
| On the flip side, good compression like DNG 1.7 spec's jpeg-
| xl compression is borderline magic. Lossless is actually
| lossless. The lossy flavour is so good even at 105 megapixels
| in 16 bit (per color channel) I would challenge anyone to
| spot a noticeable difference compared to the original, a file
| possibly 20x it's size.
|
| On a tangent, bits per channel is yet another part people
| split hairs over. 14 vs 16 has almost no difference, no the
| colours are not 'better' even in a full 16 bit workflow, the
| only real world perceivable difference is your darkest darks
| are more precise and under extreme editing conditions do look
| a little better if being raised extensibly in post. I digress
| 16 is bigger than 14 and yay marketing.
|
| Looping back to compression, 14 bit raws without compression
| are padded to 16 bit lengths due to word sizes and file
| constraints. This bit throws off the less technically minded
| who make all sorts of assumptions about file sizes and being
| 'more lightweight to edit'.
| adgjlsfhk1 wrote:
| I'm kind of surprised we haven't seen things like 16 bit
| luminence, 12 bit chromanance. (I guess to do that before
| debayering would require RGBW pixels or something like
| that)
| mlyle wrote:
| The problem is camera marketing makes it really unclear what
| you're actually getting. Really lossless? Does it trigger
| more on-body processing? Does it throw away bits to be
| "visually lossless"?
|
| If you shoot a few thousand photos and then find you can't
| fix exposure as well as you'd hoped and the whole batch is
| worse, it's a pretty big disappointment, so it's smart to be
| risk adverse and skeptical.
| jcynix wrote:
| >The problem is camera marketing [...]
|
| IMHO marketing is almost always (part of) the problem. They
| shouldn't drink or smoke that stuff ...
|
| Equating "lossless" with "visually lossless" or some other
| phrase is newspeak. We could call a JPEG of quality >= 95
| (or 96 .. 99) visually lossless too then.
|
| Losslessness is easy to define: compress something, then
| uncompress it again and both the original and the
| uncompressed file should compute to the same
| (cryptographic) checksum.
| beAbU wrote:
| There's probably an untapped market for sd cards with
| "special" holographic stickers on them retailing for $100
| each.
| jsheard wrote:
| Maybe an audiophile SD card would also produce better
| photos in a camera? You never know until you try.
|
| https://www.whathifi.com/news/sony-claims-high-end-sd-
| card-o...
| matja wrote:
| Those discussions I feel are fuelled by manufacturers like Sony
| saying [0] things like: Lossless compressed
| RAW: ... This is a popular format that occupies
| less space with minimal quality loss.
|
| "minimal" "loss"? That's not "no loss", so what exactly is it?
|
| [0]
| https://www.sony.co.uk/electronics/support/articles/00257081
| gappa wrote:
| I actually asked Sony Support about that. Their reply: "We
| can confirm that with Lossless compressed RAW there is a
| minimal quality loss. To have no impact on image quality I
| suggest using Uncompressed RAW files." Lossless isn't what it
| used to be.
| benoliver999 wrote:
| They read it lessloss I guess
| vjvjvjvjghv wrote:
| That's insane. I honestly thought that lossless basically
| means to run zip over the file and not more.
|
| Gotta hate companies these days with their dishonesty.
| "Lossless" means "lossy". "Unlimited" means "limited to
| 50GB".
| smugglerFlynn wrote:
| Seems you just got 1st line robo-reply repeating what
| public resources state. Does not say much about actual
| compression algorithm Sony uses.
| gappa wrote:
| Their first reply was "we have passed your question to a
| higher technical team", then they came back four days
| later with the above reply. I was enquiring about the A7R
| mark V, which introduced the much needed "lossless"
| option. I think I asked because I wondered why they kept
| the uncompressed option and because experts warned that
| Sony did that before with "lossless" formats.
| adgjlsfhk1 wrote:
| It is a shame that Sony has such an obsession with weird
| proprietary formats.
| Scaevolus wrote:
| A lot of cameras write "lossless" to mean "perceptually
| lossless". This is easy to do because ~12-bit ADCs have lots
| of noise in the low order bits.
| matja wrote:
| "lossless" has always referred to compression, not sampling
| - but it seems camera manufacturers want to change that for
| marketing reasons.
|
| Similarly (without starting an audiophile thread):
| Recording a vinyl record and compressing to a MP3 is
| "perceptually lossless" but will be different to
| compressing to a FLAC, never mind that the sampling output
| will always have random noise.
| manoDev wrote:
| It's because manufacturers keep inventing their own formats
| instead of using plain DNG, and you can only figure out the
| difference reverse engineering the file. Even for the same
| manufacturer and file type, they may change encoding details
| between models or firmware revisions.
|
| There are situations where they may decrease the bit depth of
| the final image if there's not enough dynamic range, there are
| situations where even though the file is "uncompressed" the
| camera already does noise reduction and essentially compresses
| detail in the image, and so on...
| danhau wrote:
| Manufacturers not supporting DNG is so annoying. But of
| course they do it. Allows them to sell licenses to software
| vendors like Adobe or Capture One.
| brnt wrote:
| How is DNG tooling these days? It's been ages since I
| checked, but it used have near zero support in (FLOSS) tools
| (including ffmpeg/imagemagick). You even had to use Adobe's
| Windows converter.
| scblock wrote:
| DNG is well supported by the typical raw libraries in my
| experience. My Ricoh records DNG natively, and I work with
| the results in rawtherapee and darktable. I am less sure
| about ffmpeg or imagemagick as I don't really automate raw
| processing.
| tecleandor wrote:
| DNG license might include some legal stuff that probably they
| don't want in their devices.
|
| https://helpx.adobe.com/camera-raw/digital-negative.html#dng
| graemep wrote:
| What is the problem there? It essential provides an
| unlimited license any Adobe patents needed to implement
| DNG, and revocation if you assert a patent against Adobe
| related to DNG.
| agloe_dreams wrote:
| Honestly the best argument for uncompressed is actually nothing
| to do with file quality or loss - it's that Apple only supports
| uncompressed Fuji RAWs.
|
| You cannot preview or process lossless compressed Fuji RAWs on
| iOS natively but the uncompressed files are equal to Apple's
| own RAWs in support. On the field, it is sadly worth every byte
| to be able to grab a file directly off the camera and tweak it
| or send it to an editor. :/
| vjvjvjvjghv wrote:
| The problem is that the size difference between compressed
| and uncompressed is enormous. This adds up quickly if you
| shoot sports or wildlife with 20fps bursts.
| CharlesW wrote:
| > _Honestly the best argument for uncompressed is actually
| nothing to do with file quality or loss - it 's that Apple
| only supports uncompressed Fuji RAWs._
|
| Nitro1 (macOS, xOS), the spiritual successor to Aperture2,
| does support Fuji compressed RAW:
| https://www.gentlemencoders.com/extended-raw/
|
| (I left Lightroom for Nitro + Apple Photos a couple years
| ago, and can strongly recommend Nitro to fellow photo
| takers.)
|
| 1 https://www.gentlemencoders.com/nitro-for-macos/ 2
| https://www.gentlemencoders.com/about/
| jcynix wrote:
| These forum discussions seem unconnected to reality IMO.
| Sometimes funny, more often strange, like audiophile
| discussions about oxygen free copper cables. Why?
|
| The Fuji X-T4 and X-S20 here produce images of 6240x4160
| pixels, but I almost never look at images in 1:1. My 4K
| monitors, mostly set to 2K, display 2560 x 1440 pixels. And
| even when switch them to their full 4K resolution I don't view
| my images in 1:1 obviously. And the tablet l'm typing this
| comment on offers "meager" 1800 x 2880 pixels. Most family
| members look at images on their "smart" devices nowadays, where
| 2K or 4K aren't present. So even decompressed images are fine
| for them.
|
| I have my cameras configured to take both JPEG (fine) + RAW
| (lossless) of course. Fuji JPEGs (and Canon ... etc too) are
| fine for most casual viewers. And if I want to crop certain
| parts, or adjust certain details (esp exposure), I have my RAW
| images as a fallback.
|
| Storage? My SD cards are 256GB and my disks are definitely not
| a problem either.
| LeifCarrotson wrote:
| I almost never look at full images in 1:1, either, but I find
| very often that I have the wrong lens for an application
| and/or the wrong exposure.
|
| You don't have to go very far before you've cropped a 20 MP
| source image into a 4K or 2K image, and if that part of the
| image that you've wanted to highlight is not well-lit, well,
| exposure is logarithmic and I want all of that RAW color
| depth that the camera can find if I'm going to turn black or
| white into perceptually accurate colors.
|
| It's true that when my framing and exposure are great out of
| the box, I probably wouldn't notice or care if JPEG
| compression cut my file size by a factor of 4...but that's
| not always the case.
| cogman10 wrote:
| The main reason my wife shoots lossless is touch-up editing
| in light room. You can rescue some pretty crappy photos when
| all the sensor data is present.
| FireBeyond wrote:
| > My 4K monitors, mostly set to 2K, display 2560 x 1440
| pixels. And even when switch them to their full 4K resolution
| I don't view my images in 1:1 obviously.
|
| macOS, but my understanding of Retina is that even if your
| effective resolution is lower (e.g., my 6K ProDisplay has a
| resolution of 3008x1692), applications can designate regions
| to be "original resolution," so I'm getting the image's
| original resolution in the editing window or region.
| the_lucifer wrote:
| As someone who shoots on Fujifilm (XT-3), this was an intensely
| fascinating read. Thanks, now I have half a mind to sit down and
| re-implement this code, just to get a feel of how it works.
| vadansky wrote:
| I've been thinking of buying that camera for a while, do you
| recommend it? Do you have anything to say that will finally
| push me over the edge to actually but it?
| jagaerglad wrote:
| I have the same one and I can definitely recommend it. It
| depends what your camera experience is, but if you have had
| one that collected dust on a shelf in the past, I can
| guarantee you that this one is more fun to use and has a much
| lower risk of dust collection
| Analemma_ wrote:
| I recently upgraded from an XT-3 to an XT-5, but loved my
| XT-3 and would still recommend it as a good purchase if you
| can find a decent deal on one in good condition. Fuji's AF is
| not the best in the business, so I wouldn't recommend one if
| you're planning on using it for e.g. sports photography, but
| apart from that the XT series has no real downsides. The
| physical dials for ISO+exposure+shutter speed are fantastic
| and Fuji's color processing makes images that I just enjoy
| looking at, even if they're not as strictly neutral and
| accurate as what you'd get from someone else.
| ninjin wrote:
| Fujifilm's whole X-mount series is wonderful and while I
| shoot "full-frame" M mount to remain interoperable between
| digital and film, there is no doubt in my mind that I would
| have a Fujifilm X-mount series if I only shot digital based
| on how much fun they have been when I have borrowed/tested
| them. Great "enthusiast level" cameras, great glass, solid
| build, _everything_ has a button /dial, does not break the
| bank, and I actually know more than one professional
| photographer that shoots them and one of them even shooting
| sports!
| guillego wrote:
| I have an XT-1 from 2015 (still working!) and recently
| started considering upgrading to an XT-5 but I'm a little
| hesitant to buy a "new" camera first released in 2023 that
| still retails for almost the same price as two years ago.
| I'm so torn between just going for it and waiting (who
| knows how long) for the X-T6 to come out. Perhaps I should
| just try to find a good deal on an X-T4.
| Analemma_ wrote:
| If it helps, I pay reasonably close attention to Fuji
| rumors because I'm deep in the ecosystem, and at present
| there appears to be no indication that an XT-6 is coming
| any time soon. They just released the GFX100RF and XE-5,
| plus there are rumors of an X-T30 III soon, and with all
| that in the pipeline I doubt they are also finishing up
| an XT-6. The -4 and -5 are still great cameras, I would
| just go for whichever of those you think is a better
| deal.
| FireBeyond wrote:
| > an XT-6 is coming any time soon
|
| Besides, it's still near impossible to get an X100 VI.
| B&H's backlog must be over a year at this point.
| smugglerFlynn wrote:
| Innovation is very slow in photography world these days,
| X-T5 made a big jump in MP count compared to X-T4, but
| resolution aside image quality is pretty much the same,
| and other improvements were marginal.
|
| I still use X-T2, and it has not really aged, even when
| compared to my X100V. Infamous Fuji AF is where they
| progress slowly but steadily, so that's the primary
| feature that I'd look into when choosing between
| generations.
| PetitPrince wrote:
| Another happy X-T3 owner here (I had in my hand a Nikon D40X,
| D300s, D810 before getting a X-T1 and then upgrading to X-T3
| ; thanks dad).
|
| Yes, this is a very good camera. I love UI of Fujifilm
| cameras; and by that I do not mean the menu system (which
| is... serviceable) but the physical dial for each of the main
| setting. Putting them in "A" for automatic just make sense
| compared to the usual PSAM modes.
| MaxGripe wrote:
| I own 4 Fujifilm cameras and personally, I'd recommend being
| VERY careful and thinking hard about this purchase. This
| isn't the same Fujifilm as it used to be. The company was
| once known for its "Kaizen" approach, which has long since
| disappeared. Prices are now inflated because they're riding
| on popularity. Autofocus in Fuji is simply weak.
|
| The question is whether you actually need such a camera for
| anything. With a new smartphone that has multiple lenses,
| out-of-the-box photos will turn out MUCH NICER than from a
| camera, because initial processing is built into the
| software. Digital cameras don't have this. You need to take
| RAW and work pretty hard on it to make the photo look as good
| as what a smartphone delivers right away.
|
| In tourist destinations, you can often find middle-aged guys
| running around with huge cameras when in reality most of
| their photos are quite poor. Because they don't realize that
| with a regular phone, their pictures would be much nicer.
| crab_galaxy wrote:
| > The question is whether you actually need such a camera
| for anything. With a new smartphone that has multiple
| lenses, out-of-the-box photos will turn out MUCH NICER than
| from a camera, because initial processing is built into the
| software. Digital cameras don't have this. You need to take
| RAW and work pretty hard on it to make the photo look as
| good as what a smartphone delivers right away.
|
| You're completely neglecting to highlight Fuji's film
| simulations. I use Fuji's specifically because they produce
| excellent jpgs out of camera. Not really sure where your
| take is coming from, an xt3 on auto will blow any
| smartphone picture FAR out of the water.
| CharlesW wrote:
| For those who love the Fuji film simulation looks but
| can't or don't want to buy an overpriced-because-
| influencers camera, there are now apps that do great
| Fuji's film simulation:
| https://apps.apple.com/us/app/rni-films-photo-raw-
| editor/id1...
| dsego wrote:
| This aligns with my experience as well. The bigger sensor
| does generate pictures that look more crisp in big prints
| or zoomed in. In theory it should gather more light, but in
| reality, phones stitch together multiple exposures, and
| frequently produce nicer low light images without much
| noise. For sharing on social media, it's hard to notice a
| difference. For me its event worse with the x100 since the
| wide lens doesn't have that signature compression and depth
| of field, so the photos don't really stand out that much,
| no wonder most x100 photographers rely on color filters
| (film sims) and high contrast to draw attention.
| tkcranny wrote:
| It's true that phones cameras are miracles of technology,
| especially considering their size. But I take a modern Fuji
| traveling because the modern phone camera look is so over-
| processed and distinct. There's no faking the real optics a
| large aperture and sensor gives, the portrait mode on
| phones is still a poor imitation of the real thing.
|
| Fuji then has the whole film simulation system with all
| their colour science from the last century. It's a ton of
| fun, and the jpgs it produces are distinct and beautiful,
| and I believe better than 99% of people could achieve from
| post processing the raws, myself included.
|
| The middle-age guy part is accurate though, I got it as a
| thirtieth present.
| krior wrote:
| Even if phone cameras were twice as good, for me its simply
| more fun to take pictures with my camera.
| barnabee wrote:
| I don't find this at all, even compared to my (now rather
| old) X-T1.
|
| For quick shots to remember an event or night out, modern
| phone cameras are fine.
|
| For anything that I'd call photography and actually want to
| print, display, etc. I rarely if ever get results I'm
| really happy with from a phone camera.
|
| If you're in any way interested in photography beyond
| taking a few snaps at parties and on holidays, I highly
| recommend getting a real camera. I've found the Fuji system
| to be great, from the lenses to the out of camera JPEGs and
| film simulations that mean you can pretty much avoid doing
| any significant editing or post-processing if, like me, you
| find that all quite tedious.
| ebbi wrote:
| I have an X-T3 and I love it. I went from an X-E2, to a Sony
| set up, and then quickly went back to Fuji. There's just
| something about Fuji that made it more enjoyable to shoot,
| for me (mostly travel photos).
|
| I will say the only thing that gives me FOMO is the lack of
| the Classic Negative film sim, as a lot of recipes that I see
| online that I really like uses that film sim as the base.
|
| If what appeals to you about Fuji's are the recipes and film
| sims, I'd make sure to research which ones you like, and then
| work out which model has the film sim you need to recreate
| it.
| yboulkaid wrote:
| I wonder if something about this explains why Fuji RAF files are
| the only RAW files that make Lightroom struggle.
|
| I never heard the fans on my M1 mac until I started processing
| those RAF files, they're not even that large but something about
| them makes Lightroom struggle. And judging by posts on
| photography forums, I'm not the only one either
| ImPleadThe5th wrote:
| I moved away from Adobe recently and ended up trying out
| Affinity Photo. Opening a RAF file would regularly seize my
| entire computer.
|
| I always wondered if it was a deadlock situation, because it
| doesn't spit out any overflow errors.
| kjkjadksj wrote:
| Lightroom still is terrible for fuji raws from a worm
| standpoint. Almost looks like they use bilinear debayering.
| Dcraw is night and day better.
| jcynix wrote:
| Lightroom struggles with Fuji's unconventional sensor pattern.
| Other programs like Darktable, GraphicConverter, Luminar Neo or
| FastRawViewer don't make fans of my M1 audible.
| MaxGripe wrote:
| I'm not sure what causes the terrible RAF handling in
| Lightroom, but it's a fairly well-known issue. Lightroom
| produces so-called "worming" and apparently this is related to
| the X-Trans sensor which requires a different decoding
| algorithm. On my end, I can recommend Capture One - the
| handling here is perfect, though you do have to get used to the
| UI unfortunately
___________________________________________________________________
(page generated 2025-09-30 23:01 UTC)