[HN Gopher] The growing image-processor unpleasantness
       ___________________________________________________________________
        
       The growing image-processor unpleasantness
        
       Author : pantalaimon
       Score  : 83 points
       Date   : 2022-09-01 18:49 UTC (1 days ago)
        
 (HTM) web link (lwn.net)
 (TXT) w3m dump (lwn.net)
        
       | Aperocky wrote:
       | Does this mean the computer won't be able to record video from
       | the peephole camera when running Linux (or able to but very
       | poorly)?
        
         | jeffbee wrote:
         | Only for narrow definitions of "Linux". As tangentially noted
         | in the article, there are dozens of people working on this
         | problem in the context of ChromiumOS. IPU6 already has support
         | in ChromiumOS, via a binary media driver. If you are
         | ideologically opposed to such drivers, you can expect poor or
         | no support for the latest peripherals. There is a sweet spot
         | for 100% free software peripheral support and that sweet spot
         | varies between 2 and 25 years in the past, depending on the
         | peripheral class in question.
        
       | giobox wrote:
       | I just recently encountered some trouble with this domain the
       | other day. I was trying to port a video project from an intel
       | system that was using their QuickSync solution for hardware h.264
       | decode to a Raspberry Pi 4 using the V4L2 hardware decode API
       | (the Intel system is before the breaking IPU6 change in Alder
       | Lake discussed in the article).
       | 
       | In both cases, it's to hardware accelerate FFMpeg decoding on
       | Linux. This was trivial to accomplish on the system with Intel
       | QuickSync, I still don't have V4L2 working properly on the Pi 4
       | and have had to wade through several complex issues.
        
         | icelancer wrote:
         | Yeah it's complicated to get V4L2 working on Linux and requires
         | recompiling a few things from my experience - we eventually got
         | hardware acceleration on ffmpeg on Ubuntu + RPi working but it
         | was a lot of trial and error, and I really didn't document it
         | too well... can't imagine this won't come back to bite me...
        
       | draw_down wrote:
        
       | an1sotropy wrote:
       | A previous post of the same thing (via different URL) from 15
       | days ago:
       | 
       | https://news.ycombinator.com/item?id=32516826
        
         | jhallenworld wrote:
         | IMHO the key comment from that post is this from kkielhofner:
         | 
         | "I worked on a embedded hardware application using Linux and
         | "cameras". We (like many people) were using sensors from
         | OmniVision[0] and there was a dedicated contractor who's entire
         | area of focus (and life's work) was getting photons from the
         | sensor of the camera into something usable by software. In our
         | case that was an unholy combination of vendor software, custom
         | code derived from very obscure NDA-only datasheets, and a bunch
         | of custom plugins for gstreamer that could actually make use of
         | the sensor, interfaces, and hardware encoding silicon.
         | 
         | It was one of the more eye-opening "WOW, so that's how the
         | sausage is made" experiences of my life."
         | 
         | This matches my experience, bringing up a new MIPI camera on
         | any of the big name SoCs is a nightmare. Actually I've seen a
         | lot of fails with gstreamer, where for one reason or other
         | gstreamer doesn't work, so the software just uses the image
         | sensor driver directly (usually V4L2). But for many SoCs, this
         | means that you get no IPU support, since it's only available as
         | a gstreamer plug-in.
         | 
         | (try to make a webcam using gstreamer: ffmpeg works better...)
        
           | icelancer wrote:
           | Yeah, as someone in this space... what a nightmare it is.
           | Even with off-the-shelf components with standard interfaces
           | like USB3 or Gig-PoE. The supporting documentation, drivers,
           | and software are... well, not so supportive.
        
             | immmmmm wrote:
             | i. can't. agree. more.
             | 
             | been building sensors for research for 5 years with all
             | kind of SDKs, even a 30k camera is a nightmare (that one
             | got decent support tho).
        
           | immmmmm wrote:
           | a sheer nightmare yes: did this for a research project (3
           | mipi modules on jetson tx2) -> worked 6 months 80 hours per
           | week getting the setup working.
           | 
           | ah the joy of guessing device tree parameters (support from
           | the sensor vendor was almost void)
        
       ___________________________________________________________________
       (page generated 2022-09-02 23:00 UTC)