[HN Gopher] Launch HN: Flywheel (YC S25) - Waymo for Excavators
___________________________________________________________________
Launch HN: Flywheel (YC S25) - Waymo for Excavators
Hey HN, We're Jash and Mahimana, cofounders of Flywheel AI
(https://useflywheel.ai). We're building a remote teleop and
autonomous stack for excavators. Here's a video:
https://www.youtube.com/watch?v=zCNmNm3lQGk. Interfacing with
existing excavators for enabling remote teleop (or autonomy) is
hard. Unlike cars which use drive-by-wire technology, most of the
millions of excavators are fully hydraulic machines. The joysticks
are connected to a pilot hydraulic circuit, which proportionally
moves the cylinders in the main hydraulic circuit which ultimately
moves the excavator joints. This means excavators mostly do not
have an electronic component to control the joints. We solve this
by mechanically actuating the joysticks and pedals inside the
excavators. We do this with retrofits which work on any excavator
model/make, enabling us to augment existing machines. By enabling
remote teleoperation, we are able to increase site safety,
productivity and also cost efficiency. Teleoperation by the
operators enables us to prepare training data for autonomy. In
robotics, training data comprises observation and action. While
images and videos are abundant on the internet, egocentric (PoV)
observation and action data is extremely scarce, and it is this
scarcity that is holding back scaling robot learning policies.
Flywheel solves this by preparing the training data coming from our
remote teleop-enabled excavators which we have already deployed.
And we do this with very minimal hardware setup and resources.
During our time in YC, we did 25-30 iterations of sensor stack and
placement permutations/combinations, and model hyperparams
variations. We called this "evolution of the physical form of our
retrofit". Eventually, we landed on our current evolution and have
successfully been able to train some levels of autonomy with only a
few hours of training data. The big takeaway was how much more
important data is than optimizing hyperparams of the model. So
today, we're open sourcing 100hrs of excavator dataset that we
collected using Flywheel systems on real construction sites. This
is in partnership with Frodobots.ai. Dataset:
https://huggingface.co/datasets/FlywheelAI/excavator-dataset
Machine/retrofit details: Volvo EC380 (38 ton
excavator) 4xcamera (25fps) 25 hz expert operator's
action data The dataset contains observation data from 4
cameras and operator's expert action data which can be used to
train imitation learning models to run an excavator autonomously
for the workflows in those demonstrations, like digging and
dumping. We were able to train a small autonomy model for bucket
pick and place on Kubota U17 from just 6-7 hours of data collected
during YC. We're just getting started. We have good amounts of
variations in daylight, weather, tasks, and would be adding more
hours of data and also converting to lerobot format soon. We're
doing this so people like you and me can try out training models on
real world data which is very, very hard to get. So please
checkout the dataset here and feel free to download and use however
you like. We would love for people to do things with it! I'll be
around in the thread and look forward to comments and feedback from
the community!
Author : jashmota
Score : 50 points
Date : 2025-09-24 16:48 UTC (6 hours ago)
| jpollock wrote:
| Congratulations! Looks like an interesting project!
| jashmota wrote:
| Thanks! We enjoyed a lot while iterating the hardware and
| looking at hours of excavator data (mostly done by Mahimana).
| toomuchtodo wrote:
| Would you be able to replicate this with the heavy equipment and
| movements needed to plug orphaned oil wells? In Texas alone, it's
| a TAM of ~$38B, and ~$150B for the entire US.
|
| _The Looming Disaster Under America 's Biggest Oil Field
| [video]_ - https://news.ycombinator.com/item?id=45361022 -
| September 2025
|
| _Texas has thousands of abandoned oil and gas wells. Who is
| responsible for cleaning them up?_ -
| https://www.texastribune.org/2025/05/08/texas-orphan-wells-e... -
| May 8th, 2025
|
| _The Rising Cost of the Oil Industry's Slow Death_ -
| https://www.propublica.org/article/the-rising-cost-of-the-oi... -
| February 22nd, 2024
|
| Well plugging SOP:
|
| https://www.epa.gov/natural-gas-star-program/well-plugging
|
| https://www.osha.gov/etools/oil-and-gas/abandoning-well
| jashmota wrote:
| It is possible. One good thing about our retrofit is it works
| on any hydraulic equipment, and not just excavators, although
| we are focussed on excavators for now. We are able to retrofit
| any hydraulic machinery because we actuate the joystics
| connected to the valves, which are quite standardized and
| usually come from handful of companies like Bosch. Will
| definitely 'dig' more into this though.
| Onavo wrote:
| Your service will need to be cheaper than just lobbying the
| current federal government.
| toomuchtodo wrote:
| Skate to where the puck is going. By the time they're ready
| for prod (2-4 years), regime change might have occurred. Be
| ready with a solution to this environmental problem at that
| time. Lots of oil and gas infra over the next few decades
| that will require remediation after the world transitions
| to low carbon energy and electrification, while declining
| fertility rates globally will mean a shrinking pool of
| workers to pick from for the human labor component.
|
| Locales who don't want their aquifers contaminated will
| also be motivated in the near term to get this work done.
| They would be great test cases for proof of value of this
| automation imho.
|
| (think in systems)
| 91bananas wrote:
| Very optimistic thinking in your first 2 sentences
| there...
| Onavo wrote:
| Says somebody who has clearly never worked in oil and
| gas.
| toomuchtodo wrote:
| I haven't worked in the industry, but a family member
| does and owns several wells (ie the mineral rights) in
| Texas, passed down from their deceased parent who was a
| rig worker.
|
| If you would be so kind as to suggest resources so that I
| can better educate myself in this domain for any missing
| nuance, I am open to any and all resources, including me
| traveling on my own dime to speak with a subject matter
| expert in person (who I am willing to compensate at a
| reasonable hourly rate).
| Onavo wrote:
| Talk to people working in oil and gas at the executive
| level, or even better, try selling stuff to them. Oil and
| gas operates on a very different mindset than the
| nibbling-at-margins product-led-growth models that are so
| prevalent on HN.
|
| The only metric that matters at the end of the day is how
| to print more money i.e. drill more oil. Literally
| nothing else matters, and compliance goes to the lowest
| bidder to make the problem go away. Right now that means
| playing nice with the whitehouse and hopefully bypassing
| the rest of the pesky EPA. I assure you not a single
| person in oil and gas likes the EPA, nor the infamous
| Subpart W.
| westurner wrote:
| Could a mini tunnel boring machine plug a well, from the side?
| ceejayoz wrote:
| That's definitely a thing.
|
| https://en.wikipedia.org/wiki/Relief_well
| westurner wrote:
| Is there a name for resealing an aquifer at each layer?
| Rezonal isolation, Zonal isolation?
| seabrookmx wrote:
| > The joysticks are connected to a pilot hydraulic circuit, which
| proportionally moves the cylinders in the main hydraulic circuit
| which ultimately moves the excavator joints
|
| I've actually spent a decent amount of time running an excavator,
| as my Dad owns a construction / road building company. It was a
| great summer job!
|
| An important note about the pilot hydraulics is that they
| _provide feedback to the operator_. I would encourage any system
| that moves these controls on behalf of a remote human operator or
| AI to add strain gauges or some other way to measure this force
| feedback so that this data isn't lost.
|
| The handful of "drive by wire" pieces of equipment that my Dad or
| other skilled operators in my family have ran were universally
| panned, because the operators are isolated from this feedback and
| have a harder time telling when the machine is struggling or when
| their inputs are not sufficiently smooth. In the automotive
| world, skilled drivers have similar complaints about fully
| electronic steering or braking systems, as opposed to traditional
| vacuum or hydraulic boosting approaches where your foot still has
| a direct hydraulic connection to the brake pads.
| jeffbee wrote:
| My car with its drive-by-wire brakes has a brake feedback
| simulator that gives the driver the kind of feeling associated
| with power-boosted hydraulic brakes. This is by far the most
| expensive single component in the car. Arguably these are just
| expensive accommodations for human flaws. A self-driving car
| wouldn't need them. Can't the self-driving system act directly
| on data like pressure, flow, and displacement?
| 01HNNWZ0MV43FF wrote:
| Maybe it doesn't matter for a car because feeling the car's
| motion tells you most of what you need to know. A car is not
| meant to touch anything but the road, in normal conditions. I
| think steering is the only case where force feedback is very
| important for a car - In the winters up here, I can feel the
| steering go loose when I hit a patch of ice.
|
| I imagine an excavator, meant to touch and dig through
| things, and lift things, benefits from force feedback for the
| same reason VR would.
|
| Have you played those VR sword games? BeatSaber works great
| because you're cutting through abstract blobs that offer no
| resistance. But the medieval sword-slashing games feel weird
| because your sword can't impact your opponent.
|
| I saw a video recently of a quadcopter lifting heavy objects.
| When it's overloaded, it can't maneuver because all its spare
| power is spent generating lift to maintain altitude. If the
| controls had force feedback, the copter's computer could tell
| you "I'm overloaded, I can't move" by putting maximum
| resistance on the sticks.
| jashmota wrote:
| Interestingly, we had some people try out VR teleop:
| https://x.com/Scobleizer/status/1970245161306464667
|
| https://x.com/jash_mota/status/1969091992140304703
|
| I think force feedback is key for small excavators, but not
| really true for 25+ tons excavators. Hence how easy it is
| for operators to accidentally kill someone with it.
| jashmota wrote:
| That's indeed what we're trying to test to the extreme - to
| see how far we could go with just vision. We haven't done
| heavy excavation workflows yet, but we have some early
| success with some excavation workflows with just vision input
| and joystick action output (even without joint angle
| feedback!). We're betting on having really huge data with
| compact observation input and experiment to see if it holds
| water. If not we can always dial it down and add more
| sensors/feedback.
| opwieurposiu wrote:
| Yes, and you also get feedback from your butt as the machine
| tips and wobbles, particularly on smaller machines. Hearing the
| engine straining helps also. Often you can not clearly see what
| you are digging, this feedback lets you know if you are running
| into a rock or something.
|
| One big advantage would be cameras mounted on the boom and rear
| view cameras, as many machines have obstructed views.
| jashmota wrote:
| We're indeed streaming the audio and have haptic feedback. My
| hypothesis is the seat vibration isn't as helpful. It's sub
| optimal and operators would be far more productive without
| it. We would do a paper when we have enough data on this.
| We're also putting more cameras but streaming a lot of them
| at once is tricky.
| Redster wrote:
| Humbly, have you used excavators of varying sizes on uneven
| ground? I have and would suggest it's more important than
| you might think. But if you've operated them, you might
| know better than I.
|
| Also, teleoperation is likely to produce lower-quality
| operation data than hooking up to locally operated
| excavators. Just a thought.
| jashmota wrote:
| I might be less experienced than you - I've operated upto
| 38 tons on maximum 15 degrees incline. I wasn't moving
| tracks a lot when I did that. Would like to hear what
| scenarios you'be been in and how would you describe your
| experience? Maybe I could try those out to learn more!
| jashmota wrote:
| You're right! This is exactly why we like to do mechanical
| actuation - we are able to achieve bilateral telepresence,
| which essentially gives the torque (haptic) feedback over the
| internet! So on small excavators, you can absolutely feel the
| resistance. We also stream the engine audio, which tells you
| how hard the hydraulic pump is working. Operators like our
| system for these reasons :)
|
| I'd like to get a chance to talk to you and your Dad to get
| feedback. How do I reach you? My email is contact at
| useflywheel dot ai
| shepardrtc wrote:
| How much safety training have you done with the models? i.e. does
| it know to stop if it's about to drive over a human?
| jashmota wrote:
| We have some autonomous safety layers (comprised of
| object/human detection like you mentioned and others) even for
| remote teleop. But we haven't deployed autonomy on real sites
| yet. We still have some work to do before we can reliably
| deploy/test it out on the actual sites!
| abraae wrote:
| What a fantastic business. I've watched lots of digger (as we
| call them here) drivers in action and I'm in awe of what these
| machines can do and how incredibly skilled their operators are.
| We have several large rock walls on our property and watching
| these guys delicately picking and placing huge rocks to key them
| in just blows my mind.
|
| There was a YouTube video recently of an AI-assisted digger
| making a wall out of the concrete rubble from a demolition.
|
| I believe the applications for really smart excavators must be
| huge. Sounds like this might be a step on that voyage.
| jashmota wrote:
| You're spot on! The experienced operators are quite skilled at
| moving these machines. The number of operators who could do
| this work are decreasing and it's also pretty dangerous which
| explains the reduction of new people joining in, especially
| with better ways to make money today. We're getting started
| with remote teleop and moving towards full autonomy!
| algo_trader wrote:
| Whats the budget for getting an MVP for heavy equipment ?
|
| I ask in all seriousness since, for example, retrofitting regular
| semis to electric requires millions and millions just to get
| started
| qafy wrote:
| I feel like the inherent bootstrap cost of hardware startups is
| usually reflected in fundraising amounts.
| jashmota wrote:
| It's far cheaper than millions! We're making something
| equipment owners want :) Let's chat? My email is contact at
| useflywheel dot ai
| qafy wrote:
| This scratches the itch I've had since playing with Tonka trucks
| in the back yard when I was 5... Are you hiring?
| jashmota wrote:
| Thanks! Let's chat: contact @ useflywheel dot ai
| jasongill wrote:
| This looks very interesting. I understand that hydraulic retrofit
| is compelling, but you may want to consider CANbus-based
| retrofits as well (similar to how CommaAI works, where it simply
| sends messages to the CANbus network to control steering or speed
| just like ADAS features do). Modern equipment is moving away from
| hydraulic controls toward a fully digital cockpit and being able
| to plug in and teleop (for less than the cost of the
| manufacturers expensive robotics suite) would be amazing.
|
| I have a Cat 289D skid steer I would happily contribute to the
| effort if you guys move into the compact equipment space (compact
| being a relative term, as my machine is only 6 tons compared to
| your 38 ton machine)
| jashmota wrote:
| Thanks! Yes we're testing both CAN spoofing and retrofits. CAN
| spoofing really limits compatibility to a few machines/brands.
| Would love to get in touch with you: my email is contact ai
| useflywheel dot ai
___________________________________________________________________
(page generated 2025-09-24 23:00 UTC)