[HN Gopher] Cameras and Open Source Smartphones: It's Complicated
___________________________________________________________________
Cameras and Open Source Smartphones: It's Complicated
Author : claudiojulio
Score : 101 points
Date : 2022-05-11 15:26 UTC (7 hours ago)
(HTM) web link (puri.sm)
(TXT) w3m dump (puri.sm)
| marcodiego wrote:
| > And once we get the functionality we need working, it will be
| available to others using libcamera: applications and other
| cameras will have them out of the box.
|
| Reason enough to support purism.
| baybal2 wrote:
| dcz_self wrote:
| Author here. While the essay comes from the side of smartphones,
| it's not really limited to them. As I mention, even some laptops
| use setups that require complex infrastructure to support.
| libcamera itself is also used in the Raspberry Pi, and the
| interfaces in the Linux kernel are used by the Axiom camera,
| which is truly a photo camera.
|
| https://www.apertus.org/axiom
|
| The problem of camera diversity is not limited to open source
| either, because a similar infrastructure to handle all the
| different cases must be replicated by closed drivers as well. I
| don't know about Macs, but the Surface laptop is a Windows beast.
| hedora wrote:
| Can you reuse some of the algorithms provided by open source
| RAW image processing pipelines for SLRs?
|
| Many SLRs are already well supported, though the open source
| stuff doesn't focus on low latency conversions, which is needed
| for viewfinders, or focus control, etc.
| michaelt wrote:
| Cutting edge cell phone camera performance is absolutely mad.
|
| I did a side-by-side comparison of a Micro Four Thirds camera
| (4/3" sensor) and an iPhone SE (1/3" sensor) and the
| performance was... pretty much the same.
|
| And I'm not talking about some ML interpolation wizardry or
| automatic face beautification; I was photographing barcodes
| and testing how many were readable in the resulting images -
| hardly something Apple would have specially optimised for.
|
| The iPhone has a much smaller sensor, a much smaller lens,
| costs less, and manages to pack in a bunch of non-camera
| features. To be competitive in the modern cell phone market
| your camera has to be straight up magic.
| dan_quixote wrote:
| It actually wouldn't surprise me if they optimized for
| bar/qr code readability. I wrote something years ago that
| used industrial cameras to read QR codes as well as very
| precise metrology features. I had to optimize the
| optical/lighting setup for the feature measurement and then
| wrote some finely-tuned operations to identify the QR code,
| window down to the code only, clean up edges/features with
| expensive convolutions (mostly median filter) and then
| finally read the code. None of this was visible to the
| operator, but if you saw the final image of the QR code it
| was essentially binary color space and looked a bit
| cartoon-like.
| dcz_self wrote:
| Totally, RAW processing is planned for after resolution
| changing works correctly. Do you have a recommendation about
| which implementation is easy to understand and work with?
|
| If I have to rewrite stuff for low latency, I'd rather start
| it as an independent library so that other projects can reuse
| the code.
| criddell wrote:
| I don't understand why diversity is a problem for Purism. They
| choose the camera modules they want to buy. Where does the
| diversity come in?
| dcz_self wrote:
| At Purism, our goal is not to build a phone or two, but to
| contribute to the ecosystem as well. That means the Linux
| kernel, and the Linux camera infrastructure. Now, we have two
| choices: either contribute support for our hardware, or use
| some hardware that was already supported.
|
| In reality, UVC is not suitable for a phone, so we can't
| leverage that. There are some camera drivers in the kernel
| already, but not necessarily ones that we could buy or
| meeting our expectations. Even if there were, that still
| leaves us with the problem of connecting the cameras to
| applications in a standard way, so we can't really avoid
| working on libcamera.
| claudiojulio wrote:
| How long will it take for Librem 5 support to be similar to
| Android smartphones?
| dcz_self wrote:
| It's hard to tell because I don't know what Android
| phones do exactly. Does it vary by manufacturer? Do we
| include AI tricks and high speed video encoding?
|
| I think it's going to be a long way to get there, but
| also the openness of the drivers will let us find our own
| strengths (I have high hopes for super-high FPS
| recording).
| criddell wrote:
| > (I have high hopes for super-high FPS recording)
|
| That would be very cool. Google's phone does 4K HDR
| stabilized video at 60 fps. Their slo-mo is 240 fps but I
| don't know what resolution that would be.
| criddell wrote:
| > we can't really avoid working on libcamera
|
| So why not fork it and make libcamera2 (or whatever) and
| concentrate solely on Purism's needs?
|
| My gut instinct would be that unless you ruthlessly narrow
| the scope of the project, progress will be too slow.
| dcz_self wrote:
| That's because we need a standard, and without libcamera,
| there is no standard.
| megous wrote:
| There is a standard that all the normal desktop userspace
| apps are already using, it's v4l2 and in particular a
| single /dev/video# device use case with all the highlevel
| controls exposed on this device directly.
|
| For likes of Librem and Pinephone, the highlevel controls
| either don't exist on HW level, or they exist there, but
| are not exposed on the video device itself, but on
| various v4l2 subdevices that form the video pipeline.
|
| One way to support all the already existing apps would be
| to implement what they already expect. (see above) That
| is, to be able to control the video device by usual means
| they already posess. Instead of extending all the apps to
| use libcamera, and leaving the rest behind, we could
| simply proxy the video controls from ioctls where all
| apps expect them to be to some userspace daemon, that
| would then configure the complex HW specific media
| pipeline behind the scenes (basically all the media
| system subdevices for sensors, sensor interfaces, ISP,
| etc.).
|
| In other words to implement what's implemented by some
| USB microcontroller in UVC webcams in some userspace
| daemon, but keep the existing userspace interface
| expectations for simple camera usage.
|
| This is kinda orthogonal to the libcamera effort, I
| guess. Just wanted to say that there already is a
| standard. :)
| dcz_self wrote:
| It's not orthogonal. In fact, it's a very good
| observation, and even libcamera itself recognizes it by
| providing v4l2 emulation.
|
| It could be a viable way to get the basic use case of
| video streaming where special controls are not needed.
| It's worth considering, although then it makes sense to
| leverage work already in libcamera to implement the extra
| layer.
| alaricus wrote:
| I didn't know cameras were an issue. OpenCamera just works for me
| with no issues.
| seba_dos1 wrote:
| OpenCamera is just a frontend, it has nothing to do with
| hardware support. All the interesting stuff happens in lower
| layers, which are usually closed, undocumented and Android-
| specific.
| Joeboy wrote:
| I've been looking at cheap, low-end movie cameras recently like
| the Blackmagic Pockets, Sigma FP etc. I'm sure this is a naive
| question but I've been wondering why I can't buy a good sensor
| module, attach it to a Raspberry Pi or similar, put it in a nice
| case with an SSD and a lens mount etc and call it a movie camera.
| I guess that's kind of what Apertus are trying to do with the
| Axiom, but they still don't seem to have shipped anything real
| after quite a lot of years.
|
| The low end of professional-ish cameras is great in lots of ways,
| but they all have annoying trade-offs. To be fair I'm sure any
| janky POS I built would too.
| wmf wrote:
| In general, modular products are always going to cost more and
| tend to lag in features/quality. It's easy to get stuck in a
| death spiral of worse product -> low volume -> very high price
| due to no economy of scale -> low volume.
| robto wrote:
| Free Software is so cool - I love how the solutions to tough
| problems can be shared so that everyone benefits from the work.
| It's a long journey, but I'm glad that there are people doing the
| hard work. This is the reason that I'm happy to support Purism!
| StringyBob wrote:
| It's complicated is an understatement ;)
|
| I assume most people have no visibility of the effort of
| _hundreds_ of engineers who work on behalf of the big smartphone
| companies tuning for what happens in the image pipeline between
| camera /raw Bayer data to what your app receives.
|
| There is a reason the quality of pictures in phones has got so
| good - lots of tuning, quite a bit of magic, as well as software
| and hardware co-optimization.
| leni536 wrote:
| Would it make sense to optionally also have a service that
| exposes these cameras through v4l2, so legacy applications can
| still make use of them, without patching them to make use of
| libcamera?
|
| edit:
|
| This would also resolve the dilemma of having complex camera
| drivers either in kernel or in a library. It could be in
| userspace while the kernel provides a small kernel module to make
| it work (like v4l2-loopback, or not unlike FUSE for filesystems).
| dcz_self wrote:
| libcamera already has that functionality via LD_PRELOAD
| megous wrote:
| These cameras are already exposed via v4l2 they are just
| exposed by many v4l2 subdevices (for a separate parts of the HW
| camera and image processing pipeline) and not just as a single
| video device having all the necessary highlevel controls.
|
| So the interface is the same, it's just split over many device
| and platform specific subdevices, which normal apps can't
| really be expected to understand.
| throwawayboise wrote:
| In the meantime is there a Librem phone without a camera? Because
| the camera is like the lowest priority thing for me on a
| smartphone. I could easily live without it.
___________________________________________________________________
(page generated 2022-05-11 23:01 UTC)