[HN Gopher] Gyroflow: Advanced open-source gyro-assisted video s...
___________________________________________________________________
Gyroflow: Advanced open-source gyro-assisted video stabilization
Author : jonbaer
Score : 263 points
Date : 2022-02-08 15:46 UTC (7 hours ago)
(HTM) web link (gyroflow.xyz)
(TXT) w3m dump (gyroflow.xyz)
| qwertox wrote:
| That promo video is production grade stuff.
| Luc wrote:
| Nice, but in some shots adding back in a sensible amount of
| camera shake would make it look more real and exciting, I feel.
|
| See also - action movies, games.
| aidenn0 wrote:
| That rolling-shutter correction alone on FP is amazing.
| floatrock wrote:
| Not in the graphic arts, but this is the most appropriate use of
| an .xyz domain I've yet seen.
| echelon wrote:
| .xyz is blowing up and gaining a lot of reputation. It's the
| new .io and .ai. Incredibly hip for your startup.
|
| abc.xyz
|
| block.xyz
|
| Look at all the HN articles pouring in with this TLD:
| https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...
|
| But you're right. Spatial domain products should jump all in on
| this.
| birdman3131 wrote:
| And nigh worthless seeing as a lot of phone carriers will
| siliently delete your text if they spot a .xyz domain in it.
| rafale wrote:
| Why would they do that? They will change policy anyway if
| the TLD keeps getting popular. You gotta see things for
| what they are headed not for what they are.
| jmrobles wrote:
| It's awesome and open source!
|
| I made this video comparison between GoPro6 4k video vs the
| stabilized with gyroflow
|
| https://youtu.be/KWiqF8Z8lEg
| Kelteseth wrote:
| This is actually really cool from a code perspective, as this is
| the first app I've seen that uses Qt/QML in combination with
| Rust. I was always searching for alternative cross-platform GUI
| toolkits that use Rust and some form of QML like language for the
| UI. sixtyfps (now slint) is a promising candidate that is
| developed by some ex Qt guys, but their QML style syntax is still
| a bit off in my taste.
|
| [1] https://github.com/slint-ui/slint
| _s_a_m_ wrote:
| Gotta love the word "advanced"
| londons_explore wrote:
| Does the gyro really give much extra information that can't be
| inferred from the video feed itself?
|
| The complexity of simultaneously capturing the gyro output and
| time syncing it to the video seems pretty high if the benefit is
| only marginal.
| WithinReason wrote:
| I remember a colleague working on such a project about 7 years
| ago (he put together a prototype in 2 weeks). The reason the
| gyro makes a huge difference is that the image is not produced
| at the same time, the sensor rows are scanned out gradually,
| and a gyro allows you to assign a correction to each row of
| pixels instead only to the whole image at the same time. Doing
| the same through vision is close to impossible.
|
| edit: A gyro-based algorithm is quite simple, while computer
| vision is still not a fully solved problem in general
| Ancapistani wrote:
| Definitely.
|
| Syncing "blackbox logs" from the flight controller's gyroscope
| is a pain, for sure. There are some cameras on the market now
| that have their own gyroscope and automatically produce sidecar
| files for the same time period as the video, like the RunCam 5
| Orange. For those, syncing the gyro readings is totally a non-
| issue.
| zcw100 wrote:
| You could infer the camera position and angle using Structure
| from Motion (SfM) and then stabilize the video from that but
| I'm guessing you'd get a lot of error built up in the process
| and it would be questionable how much stabilization you'd get
| in the end but it would be interesting to see how well it would
| work.
| quadrature wrote:
| Does gyroflow solve for the camera parameters ?. Im wondering if
| i could attach sensors to my DSLR and then use the sensor data to
| stabilize the camera footage even thought the sensors would be
| offset from the camera.
| jcims wrote:
| Are there any video formats that allow you to stream additional
| data in auxiliary channels.
| pzo wrote:
| > with Android and iOS apps coming
|
| For iOS that's not gonna happen at least with distributing via
| App Store considering GPL-3 license.
| Steltek wrote:
| Depends on who's GPL3 code is being distributed. If it's 100%
| Gyroflow's code, couldn't they distribute in the app store
| under whatever license is necessary while also hosting a GPL3
| repos somewhere?
|
| Admittedly, not a great solution and Android is just always
| going to be more friendly for apps and use cases that fall
| outside of mainstream.
| pzo wrote:
| That's true. It seems a young project but already got some
| PRs so those contributors potentially could always claim
| their rights to apple to remove from App Store.
|
| Some alternative could be just instructions how to build
| yourself or distribution via AltStore.
| liminal wrote:
| But is it better than the ChickenHead S8000? (joke)
| https://www.youtube.com/watch?v=hle5AHZbc8U
| Ancapistani wrote:
| I use this regularly, with good results.
|
| Here's an example comparison of flight footage from a RunCam 5
| Orange on a 5" freestyle quadcopter: https://youtu.be/1ZORzfJ1sog
| WithinReason wrote:
| It does a really good job! Thanks for posting an example.
|
| One decision that such a program needs to make is how to smooth
| out the movement, and what this one seems to be doing is a sort
| of a "piecewise linear" function which is a bit jarring to
| watch when it suddenly transitions between 2 speeds.
| Ancapistani wrote:
| Here's another example - one of the first ones I did, before
| I understood most of the options:
| https://youtu.be/X4R-8BchTd4
|
| That one uses the default smoothing algorithm, which isn't
| really suitable for anything acrobatic.
| Ancapistani wrote:
| > One thing that such a program needs to make is how to
| smooth out the movement
|
| There are several different smoothing algorithms that you can
| choose from - I don't recall the name of the one I used here,
| but it was specifically called out in the docs to be most
| suitable for acrobatic flight.
|
| You can set up key frames, which can give a lot more control
| over where and how the stabilization is applied. I just used
| the ones that gyroflow chose for me here.
|
| > what this one seems to be doing is a sort of a "piecewise
| linear" function which is a bit jarring to watch when it
| suddenly transitions between 2 speeds.
|
| I made this example specifically to learn how to stabilize
| that kind of footage, because that's what I do the most of
| and that's where I think I'll find the biggest benefit.
|
| Also... this is my favorite quadcopter at the moment, and can
| do some pretty crazy stuff. It's got 5x5x3 props, 1,950KV
| motors, and uses 6S batteries. Assuming 24V and no load,
| that's around 47,000 RPM. I don't have a GPS receiver on it,
| but I've handily outrun a friend's flying wing that was
| moving at 120MPH according to its onboard telemetry. All of
| that is to say that a lot of that speed transition isn't an
| artifact of the stabilization. :)
|
| For slow, "cinematic" footage, Premier and DaVinci Resolve's
| image-based stabilization does an adequate job with less
| work.
| WithinReason wrote:
| After a second look it's probably down to the movement of
| the drone rather than the smoothing algorithm
| sbierwagen wrote:
| Why 30fps? The smoothed version gets a little blurry on the
| pans, though that might just be youtube bitrate limits.
| Ancapistani wrote:
| Hardware limitation on the RunCam 5 Orange:
| https://shop.runcam.com/runcam-5-orange/
|
| It can do 4k30 or 2.7k60 - but it's also 56g, and durable
| enough that I "recovered" it from the top of a metal building
| a few days ago by knocking it down with a quadcopter and
| letting it fall 12' onto pavement.
|
| Tradeoffs have to be made somewhere :)
|
| I mostly use it for hobby stuff. While I can record on my
| goggles, the image quality is extremely poor and completely
| unsuitable for any real use. This is what I see while flying:
| https://youtu.be/xywVVS8eLSw
|
| It looks quite a bit better in the moment, and on a screen
| that's optimized for it. I use a Skyzone 04X, which was
| 1280x960 OLED screens with a 46o field of view. You can't
| really get much better for analog video at the moment in this
| form factor. There are a couple of digital options (DJI and
| SharkByte), but I don't want to spend the money on those
| until I feel like the technology has matured a bit more and
| I'm not going to need to spend another $500+ on goggles and
| ~$80 per airframe to upgrade. The RunCam 5 Orange gives me
| the ability to make videos for sharing with others and is
| "good enough" for some light commercial work.
|
| In addition to flying FPV as a hobby, I have my FAA 107
| certificate and do some photography/videography as a side
| hustle. I use an Autel Evo 2 Pro for that most of the time,
| which is capable of 6k30/4k60 and is stupid good for night
| photography. I occasionally mix in some video from my FPV
| quads if the situation calls for it. I rarely need to produce
| output that's over 1080p, and 30fps is pretty standard there.
| If I were doing more interior fly-through and such, I'd pick
| up a more capable camera for that purpose. I'd still want the
| RunCam though, because of its balance of weight and
| durability.
| Chilko wrote:
| Fyi with the 1W VTX soon to be released for Sharkbyte now
| is a great time to jump in with the recent release of the
| HD Zero cameras - your Skyzones are some of the best
| goggles you can use for SharkByte / HDZero
| qwertox wrote:
| I'm surprised on how well it handled the roll at
| https://www.youtube.com/watch?v=1ZORzfJ1sog&t=106s . It's like
| it opted to immediately quit stabilizing for a second instead
| of coming up with some wired results.
| jvanderbot wrote:
| It likely does two-way batch optimization (backward in time
| and forward). So, it has all the information it needs to know
| that a full loop is closed and that minimal-jitter paths
| include the full rotation. That is, a smooth path through
| "image space" is a loop.
| ghostly_s wrote:
| Could you rec a beginner-friendly drone platform that would
| work with a camera like this?
| Steltek wrote:
| What are you looking to do? Tripod-in-the-sky? Acrobatic
| crazy stuff? Cruising around?
|
| Speaking for myself, I wanted to feel like I was piloting
| something, not just stiffly moving a camera around. I thought
| I wanted long flight times but have since realized that I
| really wanted acrobatic ability.
|
| A 2S TinyWhoop drone is about $100-150 and is very durable. I
| wouldn't call it indestructible, because you can definitely
| break it if you try, but it will bounce off things or even
| plummet from the sky. It can't carry a GoPro unfortunately
| but it's a good pick for learning. The costs will come from
| ground equipment like your radio or goggles, which can be
| reused on future drones.
|
| One of the newer "toothpick" drones could carry an Insta360
| Go.
|
| There's currently a few big shifts in drones right now:
|
| - ExpressLRS: Open source radio control protocol. Range will
| exceed your battery.
|
| - HDZero / SharkByte: DJI alternative for HD video. Not as
| good quality but slightly more open (although not truly
| open).
|
| - "Sub 250" - Upcoming US FAA regulations put sub 250g drones
| in a special category, which could have some use.
| Ancapistani wrote:
| I agree with all of the above, but am avoiding work and
| thought I'd chime in as well:
|
| > ExpressLRS: Open source radio control protocol. Range
| will exceed your battery.
|
| This is probably a good call for this audience. I expect
| ELRS will win out in the mid- to long-term for most things.
| A few people will stick with other protocols like TBS
| Crossfire or Tracer - I plan to, but honestly they don't
| offer anything that ELRS doesn't as far as I can tell. I
| just like TBS as a company and some of their products like
| the TBS SIXTY9.
|
| Generally speaking, choose a protocol and stick with it.
| Only change (or add more protocols) if you have a good
| reason to.
|
| Radios come in two broad flavors: protocol-specific and
| "multi-module". I'd go with the latter unless you really
| want a tiny portable controller.
|
| The RadioMaster TX16S is far and away my pick here. It's
| bulky, but it supports _almost_ everything out of the box.
| If you want to add TBS Crossfire /Tracer, ELRS, FrSky
| ACCESS, etc. you only have to add a small module to the
| back.
|
| FrSky radios used to be considered top-end, but they've
| moved toward a more closed protocol that limits
| compatibility for both their radios and their receivers. If
| you have one already, by all means use it - they're good
| radios - but I wouldn't buy one today.
|
| > HDZero / SharkByte: DJI alternative for HD video. Not as
| good quality but slightly more open (although not truly
| open).
|
| This is what I meant earlier when I was talking about
| digital vs. analog - the "protocol war" here is still in
| its early stages. For all its warts and drawbacks, analog
| isn't going anywhere. If my guess on the outcome is
| correct, I'll probably end up with an HDZero setup in about
| a year.
|
| Note that you can put a (digital) SharkByte receiver on
| analog goggles, and you can put an analog receiver on
| (digital) DJI FPV goggles.
|
| > "Sub 250" - Upcoming US FAA regulations put sub 250g
| drones in a special category, which could have some use.
|
| UAS under 250g that are used only for recreational use
| don't require registration ($5, your name in a database,
| and a sticker). I'm not aware of any upcoming regulations
| that will impact the hobby.
|
| For non-recreational use, there is a category for UAS <250g
| with enclosed props that can be flown over people legally.
| There is currently no off-the-shelf offering that meets
| those requirements, but it can be done with a slightly
| modified DJI Mavic Mini or Mini2 using a special battery
| and guards.
|
| At the end of the day, the RPIC ("remote pilot in command",
| the person flying it) must maintain VLOS ("visual line of
| sight") to the UAS at all times to be legal. You can have a
| VO ("visual observer") to help with that, but there is no
| legal way to fly FPV at the moment without a second person
| present.
|
| To be brutally honest, the FPV hobby ignores a lot of the
| laws and regulations around it. Most of our video
| transmitters output quite a bit more power than legally
| allowed with being a HAM, and effectively none of us make
| sure to have a VO when we fly. Being a 107 pilot I do my
| best to comply to the extent that it's practical to do so,
| but even I don't bother with the VO stuff. It makes no
| sense, and serves no purpose with the type of quads I fly.
| If my video feed dies I'm going to crash, period. I'm
| pretty good about being able to put it down gently while
| blinded and I make absolutely sure I'm not putting anyone
| at risk when I fly.
|
| Otherwise, these quads are far too crude - or more
| accurately, "stripped down" - to automatically return to
| home or something. Most of mine are set up to attempt to
| transition to a hover and descend to land in a semi-
| controlled way if I turn off my radio, but honestly doing
| that is more dangerous than just aiming it in the general
| direction of the safest place to land from memory and
| hoping for the best. I've ditched into water more than once
| because that was the best way to be sure I wasn't going to
| hit anything or anyone. Sometimes that means wading into a
| creek or puddle to pull it out and letting it dry for a few
| days, sometimes that means replacing some parts, sometimes
| that means it's 15' underwater and unrecoverable. Such is
| the nature of this hobby.
| Steltek wrote:
| Yeah, I'm less than a year into this hobby so I'm far
| from an authority.
|
| Re: TX16S. I have both the TX16S (running EdgeTX) and the
| T-Lite. I strongly prefer the T-Lite for it's compactness
| and portability. If I succeed in shifting the fleet over
| to ELRS, I'll probably get a nano ELRS module for it. The
| TX16S is a very nice radio but I can fit the T-Lite, a
| whoop or 'pick, and my goggles in my backpack with tons
| of room to spare.
|
| For SharkByte, I bought goggles with HDMI-in (Skyzone
| Cobra-X), with the anticipation of upgrading if my
| interest in the hobby held and the technology evolved.
| People look down on box goggles but I really appreciate
| the compatibility with glasses and being able to hand
| them to anyone with no adjustment needed. DJI's vendor
| lock-in puts right next to Apple in tech that I've just
| written off.
|
| I believe sub250 is exempt from RemoteID for hobbyists
| because they are not required to be registered. If you're
| 107, then you still need it, of course.
| Ancapistani wrote:
| Those two things don't really go together well,
| unfortunately.
|
| If you're dead set on only buying one, go with a durable
| "cinewhoop" like a Diatone Taycan. It'll likely survive the
| abuses can carry a GoPro or RunCam, and will be useful if you
| decide to go deeper in the hobby. Otherwise I recommend
| starting with a "mini whoop" like a BetaFPV 2S Pro or a
| Mobula8.
|
| Realize that you also need goggles and a radio (controller).
| Those will set you back $400 and $200 respectively if you're
| trying to "buy once cry once". If you're seeing if you're
| into it, the BetaFPV 2S Pro "RTF" (ready to fly) kit is a
| good introduction, as is the Emax Tiny Hawk 2.
|
| If you're interested in getting into the hobby, let me know
| and I'll get you my contact information so we can chat.
| dekhn wrote:
| please fix your https cert
| qwertox wrote:
| What was wrong with it? It's a Let's Encrypt issued on Feb 1.
| Maybe you better talk to your MITM.
|
| https://crt.sh/?q=www.gyroflow.xyz
| rvnx wrote:
| Result looks great! Much better than the "stabilisation" of
| YouTube that easily makes you sick (I think they shut it down ?).
| abbaselmas wrote:
| this option is removed from youtube afaik, however, google
| photos now this option on local devices. If you edit videos you
| can see stabilise option, which is actually the youtube option.
___________________________________________________________________
(page generated 2022-02-08 23:00 UTC)