[HN Gopher] Removing Reflections from RAW Photos
___________________________________________________________________
Removing Reflections from RAW Photos
Author : zerojames
Score : 124 points
Date : 2024-04-24 12:56 UTC (2 days ago)
(HTM) web link (arxiv.org)
(TXT) w3m dump (arxiv.org)
| wittjeff wrote:
| I assume this could really help with in-the-wild OCR for blind
| people.
| _ache_ wrote:
| They use a context picture to help with the removing of the
| reflections. It's the first time I see something like that.
|
| But without the context it doesn't seem that good, the S24 AI
| that remove reflections seems better.
| kloch wrote:
| What we need is sensors that can scan polarization on a per-pixel
| basis (like 256 orientations per pixel per image. Then it would
| be much easier to detect and remove consistently polarized
| components of the image (as specular reflections from glass are).
|
| This would just be a fully electronic/computational version of a
| mechanical polarizing filter.
| planede wrote:
| > like 256 orientations per pixel per image
|
| You only need 4 parameters to describe the polarization at a
| single wavelength[1]. Naively this could be 4 parameters per
| color channel, so 12 channels overall. I think you could
| potentially need more color channels though to capture the full
| spectrum. But 12 channels at least looks feasible for a camera.
|
| [1] https://en.wikipedia.org/wiki/Stokes_parameters
|
| edit:
|
| On second thought for dealing with reflections you might get
| away not capturing the "V" Stokes parameter, as you might not
| care about circular polarization.
|
| edit2:
|
| The I,Q and U parameters can be captured fully by a single
| polarization filter at three different rotations. This could be
| feasible with existing cameras with a tripod and a static
| subject. I wonder if this has been done before.
| cornellwright wrote:
| You can buy [1] polarization cameras, both mono and with a
| Bayer filter. They're expensive right now, but I agree it
| would be really cool to see what could be done with a
| consumer grade version in a smart phone.
|
| [1] https://thinklucid.com/product/phoenix-5-0-mp-polarized-
| mode... (among many others)
| planede wrote:
| Interesting. From what I can find the pixel format is 4
| polarization directions per pixel, 45 degrees apart. Even
| though there are 4 channels this doesn't allow to deduce
| the V Stokes parameter (this camera can't capture circular
| polarization). Technically one channel is redundant here,
| but I guess it can be useful for reducing error.
|
| I wonder if an alternative pixel format, with 3
| polarization directions 60 degrees apart and a circular
| polarization channel would be desirable for some
| applications.
| im3w1l wrote:
| I'm pretty sure he means a single byte-valued parameter. As
| you mention a single parameter is not enough to fully
| describe the polarization but maybe it's good enough - I
| guess you would average across colors, and say circular
| polarization would lead to a basically random value.
| kloch wrote:
| I did indeed mean a singe, byte-valued parameter indicating
| angle (similar to the single angle parameter of a
| mechanical polarizing filter)
|
| Full polarization and phase info would be great to have
| also but probably not necessary for reflection suppression.
| And yes purely circular polarization would be undefined in
| this scenario but again not common (possible?) with
| reflections.
| HALtheWise wrote:
| Due to quantum physics, there's actually only two degrees of
| freedom in the ways light can be polarized, referred to as the
| "Jones Vector". In other words, it's impossible even in theory
| to distinguish between light that has exactly two perpendicular
| polarizations mixed together and light that is fully
| unpolarized and has thousands located all around the circle.
| That makes it surprisingly possible to build a camera that
| captures _everything_ there is to know about light at some
| particular frequency.
|
| https://en.wikipedia.org/wiki/Jones_calculus
| ikari_pl wrote:
| i used to have a Lytro camera
|
| very interesting device, also took about 8 angles of every
| photo and built a spatial interpretation, not too advanced
| CharlesW wrote:
| > _" RAW inputs improve prior methods, but our system outperforms
| them."_
|
| I understand why RAW is useful in general and why all methods
| would benefit (i.e. higher dynamic range, >8bpc color depth), but
| I don't understand how this system _disproportionately_ benefits
| from that.
|
| Is it because the models used in this system are trained from
| RAW, where they're not in other systems?
| jlas wrote:
| If you're a photographer, the low tech way of doing this is just
| use a polarizing filter
| user_7832 wrote:
| Is there a reason that reflected light off a vertical plane has
| a particular polarization? I know that light reflected off the
| ground gets polarized (which is why polarized sunglasses help
| so much) but that reflection is at a steep angle and not
| near/at 90 degrees.
| drivers99 wrote:
| It depends on the material as well, I recently learned.
| Specifically, metal does not polarize the light but glass,
| water, etc do.
| yorwba wrote:
| https://farside.ph.utexas.edu/teaching/em/lectures/node104.h.
| .. has a detailed derivation of dielectric reflection, but
| you can also skip it and just look at figure 57 at the bottom
| showing the predicted reflectances for the two directions of
| polarization depending on the incidence angle.
|
| You're right that for perfectly vertical reflection, the
| polarization doesn't matter, but you're unlikely to exactly
| hit that. For angles between 0 and 90 degrees, light
| polarized parallel to the surface is always reflected better.
| If you perfectly hit Brewster's angle
| https://en.wikipedia.org/wiki/Brewster%27s_angle the light
| will be completely linearly polarized, but that is equally
| unlikely. So in general you're going to get mixed
| polarization that's slightly biased in one direction.
| Wistar wrote:
| I spend a few hours a day manually editing photos of shiny stuff
| to remove certain reflections--mostly reflections of the
| photographer.
|
| This is tech I could use.
| ImHereToVote wrote:
| Awesome. Is there code?
| tedunangst wrote:
| I'm still amazed there isn't a simpler popular method that uses
| another shot at an oblique angle to resolve and remove
| reflections. Google's PhotoScan does this, but it's kinda awkward
| to use. I feel like we have the technology that should be able to
| dump a few photos in an app, pick one to refine, and then have it
| use the extras to fill in obscured areas. There was another
| project that removed chain link fences using a similar approach I
| forget the link to.
|
| At least for me, it's really easy for me to take a few steps to
| the side and take another photo. But haven't found a program that
| can use that photo.
|
| https://research.google/blog/photoscan-taking-glare-free-pic...
___________________________________________________________________
(page generated 2024-04-26 23:00 UTC)