[HN Gopher] Launch HN: Azalea Robotics (YC S24) - Baggage-handli...
___________________________________________________________________
Launch HN: Azalea Robotics (YC S24) - Baggage-handling robots for
airports
Hey HN! We're David and John B, cofounders of Azalea Robotics
(https://www.azalearobotics.com). We build robots to handle
passenger baggage in airports. Here are some videos to give you the
idea: Unedited autonomous ops:
https://www.youtube.com/watch?v=DuJ3ZORnO1o Teleoperated (sped up,
so no sound): https://www.youtube.com/watch?v=LeK8NQLnYgA The
marketing version: https://www.youtube.com/watch?v=k0SDPm09U6s
Robotics is in an interesting place right now, with many warehouse
automation companies humming along for almost a decade, and a lot
of new effort going to full general purpose hardware with humanoids
and software via generalist robotics foundation models. We love
these efforts (David used to work on one at Google X with Everyday
Robots), but we also see a lot of utility in the current wave of
robotics planning and perception tech that can enable new use cases
today. Airlines in the US compete primarily on efficiency and
customer loyalty, and baggage handling hits both (John B. has
first-hand experience from working on baggage optimization projects
at United Airlines). 2% of flights are delayed by baggage errors,
leading to downstream network delays. Baggage handling is also a
major complaint in customer experience--almost everyone has a
horror story of a missing bag, and sometimes people vow never to
fly an airline again for losing their belongings. Furthermore, it's
a really dangerous job for employees from a repetitive stress
standpoint. EU regulation is coming to reflect this, protecting
workers with a maximum number of bags transferred per shift to
alleviate back and tendon injuries that are inherent to this job.
Unfortunately for airlines, passengers don't package their luggage
in nicely uniform cardboard boxes. If they did, then the airlines
could benefit directly from the recent takeoff in manipulator tech
for warehouses. But airline luggage is way more wacky and
irregular. If robots are going to handle it, they need to reason
about how to grasp each item, handle its deformability, stack it in
a stable way, and do all of this quickly, safely, and reliably.
This is what we're tackling at Azalea. We're bringing our expertise
in deformable object manipulation, perception, robot learning, and
planning, to this logistical problem. We have a few strong bets
behind what we're working on: (1) The hardware to solve this
problem has been available or manufacturable for decades, what's
been missing is perception, planning, and control. (2) Cobots,
robots designed to operate alongside humans, aren't enough for
safety. To do this task efficiently, you need to move up to 50 kg
bags very quickly, which can be dangerous no matter how well the
cobots are designed. Light curtains (arrays of lasers that stop a
machine when interrupted) and machine cages are the current
industrial standard and remain the way to go. (3) Software for
generalist robots needs more data than most people today believe,
and it will be at least 15 years before deployment: we should focus
on specialized problems of economic value. Our core technical
developments are in a few areas: - Grasp synthesis and selection:
From visual data only, how can we identify good candidate grasp
points and rank them? For this, we use a mix of physical reasoning,
heuristics, and a lot of learning from previous data, combined in a
single objective function. Furthermore, success must be evaluated
as both a successful grasp and continual hold throughout the
transfer. - Placement planning: How do we lay out luggage in the
module we're loading? There's a nice ramp-up in difficulty for this
problem, from open-loop "divide the world into a grid" approaches,
to 3d bin-packing optimization, to reinforcement learning. An
interesting aspect of this problem for us is that the bags should
be physically stable when the cart starts driving, and lighter,
deformable objects shouldn't be underneath heavy, hard objects. We
use a similar mix of physics and learning to model this problem. -
Fast collision-free planning: Off the shelf planners work great for
the most part but can fail in heavily cluttered areas or dynamic
scenes. We leverage the fact that we're always solving a series of
similar problems to provide initial guesses for downstream
trajectory optimization algorithms. Since each problem is so
similar, we can use techniques similar to generative models to
propose these initial plans. - Mechanical design: The perfect tool
to pick up everything checked down a conveyor belt isn't an easy
thing to design. We're building tools with multiple modes of
grasping to handle wide varieties of objects. The videos we linked
to are all with suction only - which can be surprisingly powerful!
An interesting aspect of autonomy becomes choosing which mode to
use when, and how to use it. These problems can be deeply
interlinked: where you grasp an object depends on what your tooling
looks like and informs where you can put it- so a perfect solution
would jointly reason about both problems simultaneously. We're
looking forward to getting there as we collect more data and
continue our efforts. Check out our demo videos above! We have a
brand new hardware stack coming soon (and we've added a new end
effector that we're keeping hush), but it's amazing what you can do
with pure suction. We're proud of our progress so far but would
love to hear your thoughts and feedback. Let us know if you've had
a particularly bad baggage horror story and/or have personal
experience with the industry.
Author : dmillard
Score : 73 points
Date : 2024-12-11 18:07 UTC (4 hours ago)
| pj_mukh wrote:
| Super cool! You are additionally going to be saving a lot of
| workers from getting chronic back pains. I thought maybe y'all
| are going too slow, but after looking at some baggage handling
| videos, it seems like you're at a comparable speed already?
|
| In deployment these things are probably going to be on some kind
| of cart system, I presume all your algorithms can handle small
| changes in the XY travel plane (i.e. the robots location w.r.t
| the end of the belt).
| dmillard wrote:
| Thanks! It's a really destructive job for workers' lower backs
| and elbow tendons. This actually puts into perspective the
| blended throughput rate - you can imagine loading a few bags
| really quickly, but moving 20kg bags for 10 minutes straight
| will slow you down. That said - we still have a _lot_ of runway
| on speed for these mechanisms and are still running fairly
| conservatively as we shake out our software.
|
| There are a few ways we plan to deploy (some fixed rails, some
| mobile). Since the carts we're loading aren't placed with much
| precision, even the fixed deployments need to do serious
| environmental perception / localization.
| floren wrote:
| > You are additionally going to be saving a lot of workers from
| getting chronic back pains.
|
| That'll probably be a big comfort to those who lose their jobs
| pj_mukh wrote:
| That's not how this works. Most airports are looking to
| expand, they can keep the same staffing levels with a rapid
| expansion in throughput with these machines. (Assuming the
| unions allow it).
| derektank wrote:
| It's not as though this is a profession which people spend a
| decade learning to perform. And it's not 2009 any more, there
| is demand for labor throughout the economy. I think the
| benefit of getting people out of physically damaging work
| outweighs the pain of having to find a new job in this case
| ztratar wrote:
| How much faster will the robotic arms be able to go? Currently
| this looks far too slow, though as a v1 it's great.
| 0x457 wrote:
| It's about as fast as human does it now.
| bufferoverflow wrote:
| Not even close
|
| https://youtube.com/shorts/KMlCCyQPfME?si=BsyZ9QZgZDJWVh4f
| dmillard wrote:
| Good video! The overall question here is the blended rate
| of bags placed per minute, rather than how fast each action
| needs to be.
|
| That said, the arm itself can move 180deg/s in every joint
| (roughly 5m/s max at the end effector) - these videos are
| still very much v1 and we're looking forward to leveraging
| more of the mechanical capability with a better gripper,
| better perception, and some new planning techniques we're
| rolling out in the next few months.
| ztratar wrote:
| I guess, just my opinion (probably worthless), if you can
| get it to speed up even ~20%, I think that would be
| enough to make the average person agree.
| AustinGrandt wrote:
| Previous baggage handler turned software dev here. Looks like you
| are targeting the bag room right now which is the best spot to
| start to prove the concept. It was surprising to me how manual
| the entire loading & sorting process was when I worked for the
| airlines.
|
| I'm curious if there's any roadmap eventually to get this out to
| the ramp itself. Most of the back injuries seemed to happen in
| the bin itself because you have to often hunch/be on your knees
| in a bin tossing 50+ pound bags. I know airlines would probably
| be very hesitant to have any new equipment around their planes
| but just curious if there's any discussion around that.
|
| Other side note, I also used to work in Cargo and always thought
| there could be way more efficient ways of loading loose packages
| that are on every flight and this seems to be a great possibility
| for those as well.
|
| Awesome work & will keep tabs on it!
| dmillard wrote:
| Thanks for the feedback!
|
| John B was obviously aware from previous experience what a
| manual and injury prone process this was, but I've also been
| really surprised as I've dived deeper into airport operations
| myself.
|
| Bagroom is definitely what we're targeting first - being
| indoors (usually) is a huge plus, and lets us focus on the
| manipulation part of the problem without going fully mobile
| yet.
|
| That said, we're definitely targeting tarmac/ramp operations,
| particularly between a TUG/PowerStow and narrow-body bag carts.
| Inside the bin is much trickier but we agree it's the least
| ergonomic part of the job, you just can't move a massive
| industrial arm in and out of a plane very easily. We have it on
| our longer-term roadmap, though, and intend to leverage the
| baggage dynamics data we collect everywhere else to give us a
| head start on the packing and manipulation problems there, just
| with a different mechanism.
|
| Cargo packing is a huge area of interest for us! Particularly
| around optimizing weight distribution in loaded planes, or just
| optimizing packing efficiency in general.
| AlotOfReading wrote:
| I'm surprised the end effector works on misshapen fabric bags.
| Makes me jealous that I can't test it.
|
| Are you considering dynamic trajectory constraints in the planner
| (e.g. for multiple robots loading simultaneously)? That was a
| thorny problem back when I worked on arms.
| dmillard wrote:
| There are a lot of constraints in our planning, including what
| actions we can do for a particular bag/gripper interaction.
| Multiple robots working in collision range of each other isn't
| on the roadmap right now, but it's always a possibility.
|
| Suction works really well but it's not enough, we're rolling
| out a new gripper soon that covers more cases mechanically
| (there's a very long tail in the distribution of what comes
| through airports).
| riobard wrote:
| Are most suitcases designed to bear its fully loaded weight just
| by sucking on one surface?
| artificialprint wrote:
| Another robot to clean up after the zipper fails is wip
| dmillard wrote:
| Solid question and something we think about a lot! Worst case
| is a weak zipper or similar. We're bringing a new gripper
| online which is more multimodal - some mechanical grasping,
| some suction, and the ability to choose what you use. We're
| moving away from pure suction partly for this reason and partly
| for textiles.
|
| Suction is great though, and ~75% of bags checked through the
| US are hardshells, so it's something we're not ready to ignore
| entirely.
| MaxPock wrote:
| How about a forklift like handling ?
| dmillard wrote:
| Also good question and something we've thought about. The
| difficulty there is actually getting the forklift tines out
| after placement. Actual forklifts in real warehouses rely
| on pallets as an affordance for manipulation, and we don't
| have that luxury here.
|
| There have been some neat attempts with short conveyor
| belts as end effector tools [1]! Generally these systems
| rely on being able to rearchitect a significant amount of
| the process (building a controllable conveyor belt or
| rearchitecting part of the bag-room), and we're focused on
| dropping into existing processes.
|
| [1] https://www.youtube.com/watch?v=n2Wy_tduq5k
| trhway wrote:
| call me disappointed. Not to belittle whatever work was put into
| it as there were probably a lot, just the one-suction-cup way of
| doing it seems so 30 years ago. These days i'd have expected
| [several] hands with fingers.
| dmillard wrote:
| Great point and we 100% agree! We have a new multi-modal
| gripper that we're testing now but are keeping hush while we
| bring it up. (We're actually swapping out a new arm too for a
| variety of reasons)
|
| These videos are more to set a scene for where we're operating
| the the general process.
| Animats wrote:
| This is much like a palletizer. Here's a mixed-case cage
| palletizer, which fills up wire cages open on one side with
| various boxes.[1] For rectangular boxes, palletizing is a solved
| problem with many companies selling systems.
|
| Irregular items are a problem. In the baggage handling video, the
| last bag, the soft one with straps, is sticking out after being
| placed in the baggage container. That's the same problem which
| keeps Amazon from totally automating picking. They keep trying,
| but nothing works well enough yet.
|
| [1] https://www.youtube.com/watch?v=TN-6QaLd3VY
| dmillard10 wrote:
| Yep! We're obviously operating alongside a very well
| established world of palletization and other order fulfillment
| type robots.
|
| We think that due to irregularity that it's not an easy tech
| transfer from the existing logistics world into the aviation
| world. We're very interesting in looking the other direction,
| though!
| jackcampanella wrote:
| Please tell me the robot's name is not Iggy.
| dmillard wrote:
| No - people keep asking us to name our robots, but we haven't
| landed on anything yet!
| dogman144 wrote:
| Was on a plane a few days ago watching someone do this out the
| window 10 yards away.
|
| - Seemed like the baggage handler was required to do a very fast
| cycle of <scan, toss, align, repeat next bag>. Automation seems
| helpful, and certainly it's hard labor.
|
| - This was also a young woman, in presumably a safe union job,
| working in a very pricey city (one of the mountain west towns
| that exploded). Adios union job hello robots.
|
| Tricky ethics! Outside of picking stuff up and putting stuff
| down, not too many automation union-safe jobs left. Saving them
| from back pain is also going to be saving them from a job.
| dmillard wrote:
| We're really focused on health and safety aspects of this job -
| in a repetitive stress sense, these jobs are much more
| dangerous than many people imagine they would be and people end
| up with lifelong injuries.
|
| Generally, regulators seem to be moving in this direction as
| well. The EU has introduced new regulations on the total amount
| of weight someone can move in a shift, and the Dutch government
| has mandated that baggage handling move away from manual
| processes like this in the near future.
| dogman144 wrote:
| Despite the focus difference, do you think it's unlikely that
| automating baggage handlers will replace their jobs?
|
| The regulator focus seems like it'd reduce the max allowed
| weight of a checked bag, not automate the baggage handler
| handing the checked bag. So, I don't see the similarity
| between the regulatory push and your product? Edit - to
| clarify, beyond what Dutch regulators say about Dutch
| markets, which are a very small subset of "regulator focus"
| internationally.
| btbuildem wrote:
| The suction "cup" seems to be doing a good job, but I don't think
| bags are made to be handled that way. Did you alternate test with
| some kind of a grabber (like the claw machine, but actually
| effective)? It would make the grasp selection problem much more
| tractable imo, since all bags have at least one "grasp point"
| built in.
|
| In teleoperated mode, I'm guessing you're using the captured data
| to train the autonomous mode?
|
| Final note, the robot looks beefy enough to lift an entire
| airplane, forget luggage -- is it overengineered on purpose?
| dmillard wrote:
| The cup has taken us very far, which we're excited about, but
| it's definitely not enough - we're currently testing a
| multimodal claw-ish + suction gripper, which we've had good
| results with so far but aren't ready to unveil.
|
| The teleop data is really useful for training data indeed, and
| lets us collect data on current failure points (e.g. with
| suction, just how far can we tilt this fabric bag before it
| peels away, etc). We're not going full behavior-cloned end-to-
| end for a lot of reasons (sample complexity, safety,
| adaptability, etc), but we do a lot of learning in specific
| parts of the system (particularly around grasping and
| placement).
|
| The robot is indeed beefy, as many robots rated for 50kg
| applications are (check them out online). We've accidentally
| stress tested this unit _way_ beyond 50kg without a hiccup, so
| we 're very interested in figuring out what the right-size unit
| is for our application. There are a few other great aspects to
| this unit - it's a 7-DOF arm + 1 more DOF for the linear rail,
| so we have two extra degrees of freedom to play with for
| collision avoidance during planning.
| hluska wrote:
| This is excellent! I live in a small city with an airport so have
| known many baggage handlers during my years here. I honestly
| didn't believe how common back and shoulder injuries were in the
| industry. This could prevent a whole lot of medical problems.
| chfritz wrote:
| I've heard about you guys before. Nice application! What are you
| using for teleop/remote-monitoring? Did you build that yourself?
| dmillard wrote:
| Teleop and monitoring are systems that we've built ourselves
| and are pretty happy with. Since we use MuJoCo for
| simulation/visualization and some kinematics subroutines, to
| visualize, I just keep the MuJoCo GL context open after
| rendering and then throw all of our sensor data into it - it's
| very performant and low latency!
|
| We've since introduced a message-bus layer that makes it
| possible to do it all over the internet etc, but adds the
| associated serialization and transport latency.
| chfritz wrote:
| > to do it all over the internet, but adds the associated
| serialization and transport latency.
|
| I wrote this blog post on that topic a while back after
| having seen various approaches robotics companies take and
| their shortcomings:
| https://transitiverobotics.com/blog/streaming-video-from-
| rob...
| dmillard wrote:
| Excellent post! Curious if WebRTC can be adapted for 3d
| sensor data and would love to chat more about it - I'll
| send an email!
| hemloc_io wrote:
| > Unfortunately for airlines, passengers don't package their
| luggage in nicely uniform cardboard boxes. If they did, then the
| airlines could benefit directly from the recent takeoff in
| manipulator tech for warehouses. But airline luggage is way more
| wacky and irregular. If robots are going to handle it, they need
| to reason about how to grasp each item, handle its deformability,
| stack it in a stable way, and do all of this quickly, safely, and
| reliably.
|
| Curious if you guys have put any thought into seeing if there's
| an operational change you could introduce to airlines, that would
| result in the tech side being a lot easier?
|
| Palletizing logistics for consumer airtravel would be
| interesting...
| dmillard wrote:
| Operational changes for airlines are quite tricky - one of our
| bets is that most of the value for customers here is in
| handling "brownfield" deployments where you drop into an
| existing process, and that intelligence (or at least, good
| perception and reactive planning) really unlocks this ability
| from the robotics side.
|
| For widebody planes, bags are already loaded into Unit Load
| Devices (ULDs), which are large semi-truncated boxes that get
| loaded directly onto aircraft. Narrow body planes don't use
| these (apparently) because they impact turn-time and decrease
| the amount of time a plane can be in the air, and also impacts
| how quickly bags come out, since it adds an extra step to
| unloading.
|
| Many airport conveyance systems also load each bag into a bin,
| but those bins aren't loaded into the airplane because they
| belong to the airport and waste space/weight.
|
| The best case for us would be a customer process change where
| everyone loads their luggage into perfectly regular and very
| sturdy hardshells, but this one's probably out of our hands.
| zk wrote:
| This is awesome! It's amazing how many jobs there are that
| require heavy manual lifting with repetitive motion that will be
| up-skilled in the coming years. New roles will be much more
| monitoring and problem solving which.
| porphyra wrote:
| Using a vacuum to pick up baggage is a very interesting choice. I
| wonder how it would fare with extremely uneven surfaces or even
| porous ones.
|
| Also --- I couldn't see any obvious sensors. What sensors are you
| using to perceive the bag? I am imagining some kind of RGB-D
| sensor like a Kinect (or its successors like the Orbbec).
| dmillard wrote:
| Suction has gotten us pretty far at the prototype level but
| definitely isn't enough - we're testing out some new gripper
| designs that use suction as a broader part of an overall
| grasping system.
|
| For these videos we have lidars and two Intel Realsense depth
| cameras mounted to the safety cage and on a wall. We're working
| on moving as many sensor on-robot as possible in the near
| future to aid with deployability.
| purplezooey wrote:
| Well, this didn't go very well in Denver when they tried it. Good
| luck.
| iancmceachern wrote:
| As part of one of my classes in engineering school we went and
| toured the DIA baggage handling system, studied why and how it
| was such a big failure and what they did to fix it.
| dmillard wrote:
| There have been some solid attempts in this space before - many
| projects take on the whole baggage system design and end up
| very very complex and often over budget. We're focusing on
| introducing tech that plays well in a larger system,
| particularly in "brownfield" existing processes - our bet is
| that recent advances in robot autonomy give us ability to
| handle items that weren't possible before, and therefore our
| units can be introduced in a more flexible way.
| heytakeiteasy wrote:
| This is very cool, thank you for sharing. I work in automation
| and SWE for a certain 4-letter organization that delivers your
| mail. Pick/place is something we've rolled out using articulated
| and delta robots with vacuum end effectors, and it's an
| interesting and challenging space to be in. As in your case, bags
| and other amorphous shapes are always the most difficult. It's
| always an uphill battle to hit throughput targets due to
| exception cases that can stop things until a human gets involved.
| Ultimately, it can be a struggle to avoid overpromising and to
| generate ROI since automation is so costly, especially when
| there's no opportunity to bound the problem by influencing the
| inputs to your system or the output requirements (in your case,
| the cart being loaded). Best of luck and looking forward to
| seeing your new end effector.
___________________________________________________________________
(page generated 2024-12-11 23:00 UTC)