[HN Gopher] Towards robot accelerators, democratizing hardware a...
___________________________________________________________________
Towards robot accelerators, democratizing hardware acceleration in
robotics
Author : vmayoral
Score : 51 points
Date : 2022-07-25 06:12 UTC (16 hours ago)
(HTM) web link (news.accelerationrobotics.com)
(TXT) w3m dump (news.accelerationrobotics.com)
| j-pb wrote:
| While hardware accelerators in robotics are very cool, and
| needed. The fact that the entire workflow is ROS2 centric is a
| bummer. FPGA and Accelerator Programming is already a huge PITA,
| and despite it's success ROS is really one of the worst
| frameworks I've ever used in terms of DX. It makes an already
| hard problem, needlessly harder.
|
| I wouldn't be surprised if they developed their own FPGA
| programming environment based on C++ and CMake, and all kinds of
| other custom language and tool shenanigans.
| pclmulqdq wrote:
| You have to remember that in a professional setting, the
| alternatives to ROS are atrocious pieces of crap like LabVIEW.
|
| From there, ROS is a step up! Both are programming environments
| designed by mechanical engineers, who care more about making
| easy problems really easy than making hard cases any better
| (after all, to a mechanical engineer, hard programming problems
| are really hard on their own).
| octocop wrote:
| Do you mind sharing some of the issues that you are having with
| ROS2? I'm also using it and i find it quite straight forward.
| vmayoral wrote:
| That's exactly what this work does while remaining ROS
| 2-centric (with C++, CMake extensions and other friends). Goal
| in this research was to provide a way for the path of least
| friction for roboticists.
|
| There's a huge gap between robotics development (more sw-
| centric today) and the average hardware engineer in big
| semiconductor companies. Totally different perspectives and
| viewpoints. We felt bridging the gap was necessary.
| pabs3 wrote:
| What would you use instead of ROS?
| vmayoral wrote:
| Use ROS 2. The misconceptions about ROS have been mostly
| addressed with ROS 2.
|
| Check out industry adoption of ROS 2 in various industries
| including automotive, healthcare and warehouse automation.
| ROS 2 is (and will remain as) the de facto standard for robot
| application development.
| j-pb wrote:
| The issue with ROS are mostly cultural, and heavily
| ingrained into the community. ROS2 isn't going to fix that
| and has been shown to repeat the same mistakes of ROS.
|
| The atrocious build system has stayed the same. People
| continue to add even more brittle automagic to the pile.
| (.e.g. automagic compilation of C++ to FPGAs)
|
| The grown semi ad-hoc messaging semantics for services has
| stayed the same. Even if DDS was an acceptable foundation,
| adding a ROS message translation layer and ROS semantics on
| top of a perfectly valid schema language is bringing the
| same pitfalls that Object Relational Mappers have, in that
| you now have to understand three systems, the source, the
| compiler, and the target. And given that ROS1 had a
| 'wontfix' issue on the message schema checksums being
| computed wrongly (ignoring the cardinality of fields),
| which could result in your robot literally ripping your
| head of when it received messages from two different
| library versions, I don't trust anybody from the community
| to get it right this time.
|
| The whole DDS vendor based ecosystem is antithetical to
| open source. The DDS standard is way too big, and a bunch
| of vendors meeting every couple years to demonstrate that
| their systems can send messages about shapes between one
| another simply doesn't cut it to prove correctness.
|
| ROS(2) is an attempt to make C++ more easy by layering
| automagic and complexity on top. Most robot shops I know
| waste most of their time fighting that magic and
| complexity. The other ones don't use ROS(2).
|
| ROS should have been:
|
| * A bunch of libraries (not frameworks) to solve robotics,
| vision, and knowledge representation related tasks.
|
| * A bunch of best practices to work with common
| middlewares, none of which are developed as part of ROS.
|
| * A repository of competing data schemata for the above
| libraries and middlewares to facilitate interoperability
| but allow for experimentation.
|
| * Dataviz libraries for common notebooks, e.g. jupyter,
| observablehq, that work with the common data schemata.
|
| Instead we got one of the hairiest C++ projects and a bunch
| of shell scripts to open your default text editor more
| conveniently. Yay.
| iancmceachern wrote:
| Custom written stack
| thendrill wrote:
| arduino or rpi with something like mqtt, PyRemoteObjects on
| top , just a few off the top of my head.
| cmontella wrote:
| Indeed, ROS set out to create an ecosystem of reusable robot
| software infrastructure, but a monoculture has grown instead.
|
| I started building a ROS alternative called Mech (see my
| profile) after becoming frustrated with ROS. It's getting
| there but not ready for prime time yet.
|
| To answer the sibling comment, my biggest problem with ROS is
| how inscrutable it is for undergrads. I've witnessed students
| struggle to even install it while following directions step
| by step. It is a huge mess of incidental complexity, layering
| the complexity of C++ build systems on top of an IPC system
| on top of an expansive middleware.
|
| The learning curve is so high that when I spend a semester
| teaching robotics, students still struggle with ROS basics at
| midterms. The course turns into more about learning ROS
| rather than learning actual robotics.
|
| I'm hoping one day Mech will work along side and in
| conjunction with ROS to provide an easier way of doing things
| for those who want to get right to the heart of robotics.
| metadojo wrote:
| Bruh, take a step back for a second. ROS is a messaging
| system. Its like Microservices for Robotic components. If
| students are having trouble with ROS you might take a
| different angle on the purpose. When you teach ROS you are
| learning middleware patterns of integration first. Secondly
| you have to teach fundamental geometric concepts like TF.
| So thats why your students are having problems you are
| giving them too broad or too narrow or some convoluted
| context to reason about what they are doing. Try flipping
| the script.
| j-pb wrote:
| If only ROS were that. ROS wasn't a particularly good
| Middleware, and ROS2 isn't a middleware at all (DDS is a
| middleware, ROS is a framework using DDS).
|
| The issue with ROS isn't the general idea of
| communicating nodes over a blackboard and the
| accompanying standard libraries, but the huge pile of low
| quality tools, sparse documentation, ad-hoc architecture,
| bad code, and mediocre introspection tooling. A lot of
| that can be traced back to C++ and CMake, but ROS manages
| to take already complex tools and make them even more
| complex in an attempt to make them easy with automagic.
|
| If ROS2 had just opted for
| ProtoBuffers/FlatBuffers/CaptnProto over MQTT/NATS and
| build TF et.al as a library over that, we could just
| point students to the excellent documentation of the
| Serialisation Format, of the Middleware and to the (with
| all that freed developer time) much better documentation
| of the Robotics libraries, and call it a day.
|
| Instead they spend months with the idiosyncrasies of the
| build system, only to continue struggling with the
| documentation, and the sub-par tooling when something
| goes wrong. _Cough_ only two seconds of time travel in
| ROSBags _cough_
| ragebol wrote:
| Why not provide a pre-set up machine? I would also not have
| them install eg Matlab or LabView themselves at the start
| of classes.
|
| But yes, I recognize the struggle. At my RoboCup team
| (TechUnited Eindhoven), the learning curve is also steep:
| from non-CS students that use Matlab on Windows, to using
| Linux with C++ & Python, ROS, git, not to mention our own
| stuff on top.
|
| It's a filter, to say the least.
| cmontella wrote:
| They all have their own laptops and prefer to use them.
| We also don't have enough money to provide a computer for
| everyone. But I mean the setup is just one part of the
| issue, as you note. What I've found is I've had way more
| success teaching robotics to younger students via
| matlab/windows than Linux/ros/c++.
| j-pb wrote:
| I'm a huge fan of your work. Is there a reason why you
| didn't pursue the triple model of EVE and went with tables
| instead? It feels like the former would be such a great fit
| for the knowledge representation side of robotics.
| cmontella wrote:
| Hey, thanks! That's one of the main and crucial
| differences between Mech and Eve, actually. The main
| reason I went with tables in Mech is because of the
| robotics domain. Robotics has a lot of linear algebra and
| much of the math can be parallelized naturally. This
| means in Mech we can store columns as contiguous arrays,
| and take advantage of SIMD optimizations. The triple
| store in Eve worked well with the whole relational
| algebra nature of that language, so it was a natural fit
| there. Not so much for my use case.
|
| Maybe there's room for an Eve-Mech bridge where Eve can
| be used on the knowledge representation side, and Mech
| for the lower-level motion planning stuff. That's a fun
| thought!
| j-pb wrote:
| Really interesting point. I always figured I'd go with an
| ECS system like specs for the high perf case, but you're
| absolutely right, columnar storage is gonna be orders of
| magnitude faster. (Fun fact, the hibitset of specs looks
| _a lot_ like a worst case optimal join.) A GPU/FPGA
| accelerated Mech would be really really cool :D.
|
| A hybrid system is a really interesting idea. I actually
| wrote my bachelors thesis on a system heavily inspired by
| EVE to be used for knowledge representation in robotics
| and have been experimenting a lot with implementations of
| the idea over the years. You should really open a Mech
| discord channel, I'd love to hang around and help build
| something like that :D.
| cmontella wrote:
| Do you have a link to your thesis? I'd really like to
| read it!
|
| GPU/FPGA Mech has been my dream since like... forever. I
| started taking a crack at it here:
| https://gitlab.com/mech-
| lang/core/-/blob/gpu/src/bin/gpu.rs, but mostly I was
| just figuring out wgpu's capabilities in Rust. But that
| was months ago and I've since decided to push it off
| until after I launch what I have done already.
|
| I have a private slack that I've been using to work with
| my students, I'd love to send you an invite. Send me an
| email at corey@mech-lang.org and I'll send it to you.
| I've been putting off doing a discord channel because I
| wanted to code up a chat app in Mech instead :P I think
| I'm almost there but I've been thinking that for years
| >_<. If I don't have one by the time I launch in October
| I'll probably bite the bullet and do discord then.
| thendrill wrote:
| I have to agree with you on this one.
|
| Yes Ros looks hood on paper but the moment you open it, it just
| screams academia and academics.
|
| Also yes FPGA is faster for in general. But who is gonna
| program it ?
| vmayoral wrote:
| Move to ROS 2 :)!
|
| Regarding who's going to create the accelerators, enablers
| like this allow any ROSser to jump into creating their own
| accelerator from C++ by purely adding some metadata into
| their CMakeLists.txt files (which every ROS package has
| already).
|
| On top of that, organizations are creating professional-grade
| kernels meant for production. NVIDIA and AMD are investing in
| that direction and so are we at Acceleration Robotics.
| Here're a few of the first ones announced: - Accelerated
| perception stack (ROS 2 API-compatible)
| https://accelerationrobotics.com/robotcore-perception.php -
| Accelerated coordinate system transformation (ROS 2 API-
| compatible, tf2) https://accelerationrobotics.com/robotcore-
| transform.php
| LeifCarrotson wrote:
| As a controls engineer who has implemented and assisted with
| dozens of robotic manufacturing cells, the problem isn't the
| motion planning or compute, it's legacy product support and ease
| of onboarding new hires.
|
| Any given cell may be in production for 10 years - some only for
| 5, but some for 20 or more - and you need <24 hour response times
| to any mechanical issues that may arise over that duration. Also,
| one can't expect that the original programmer will be available
| to diagnose and debug issues that arise in the field after a year
| or two, so you need a large pool of technicians and maintenance
| personnel that can understand the software and be productive in
| the first hour after being introduced to the machine. None of
| this is valued by academics, instead, the fundamentals of the
| process of a single lab trying to prove merit as PhD students
| and, once that credential is acquired, move on to greener
| pastures is anathema to both of these ideas.
|
| The problem is that academia is working in ROS (now ROS 2),
| writing Python and VHDL, while the real work in the industry is
| being done in proprietary vendor tooling that's backwards
| compatible with training from the 80s. They're proudly
| advertising touchscreens and full-color 5.7" displays on their
| teach pendants, as if their customers were still 20 years in the
| past.
|
| Yes, academia, I do want sub-millimeter repeatability, and
| flawlessly smooth motion planning. I want collision detection
| that accommodates both the inertia of the arm itself and any end-
| of-arm tooling, including cable harnesses with distributed loads.
| I want automatic singularity avoidance and trivial point and
| frame manipulation. I want 24-bit encoders for resolution, and I
| want angular velocities to the moon, and to make those work
| together I need servo loop rates and high-speed skip responses at
| 1, 2, or 10 kHz. I want 4D stop position prediction to avoid
| intersection of the robot arm with complex assemblies and safety
| zones imported from CAD. I want all these things cheaper and
| faster, with more storage.
|
| But I need these to be accessible by Billy Joe, whose only
| credentials that got him the job in the maintenance department is
| that he helped out his daddy working with a welder and an old
| clapped-out Bridgeport on the farm growing up. Billy is probably
| my best tech on 3rd shift, and he's never learned to type on a
| full-size keyboard, just his smartphone... and on a robot teach
| pendant.
| antoniuschan99 wrote:
| What are your thoughts on OPCUA? That's the direction they're
| trying to get to having different vendors be able to talk to
| each other as it's all pretty much federated.
| metadojo wrote:
| you are leaving out a whole host of use cases bro. maybe in the
| past robotics was just for industrial use cases. but there are
| orders of magnitude other use cases that a ROS can help solve.
| simple things around the house , in the neighborhood, and other
| mechatronic integrations that would not properly called a
| robot. I would say the ones you are addressing with billy bob
| on the 3rd shift are like 1 percent of the cases. There are
| plenty of things you can actuate in the world that dont require
| sub mm repeatability. like what about tending my hydroponic
| vertical garden? theres nothing about the ubiquitous robotic
| future of humankind that needs any of the things you mentioned.
| thats more of an improvement on current use cases.
| vmayoral wrote:
| Well said. Robotics is indeed the art of system integration and
| such remains (and will) one the biggest tech hurdles.
|
| We wrote about this, and raised-funding, and worked for years
| on it. Hell, we even created an extension of ROS for it called
| the "Hardware Robot Operating System" (H-ROS) that aimed to
| address many of these issues you describe using ROS as the
| common language and Programmable Logic (FPGAs) to deal with
| physical interfaces. More on this at
| https://ieeexplore.ieee.org/document/8046383.
|
| Problem was that the market didn't really accept it. I still
| believe on the tech problem landscape in here. I'm just not so
| sure anymore if there's any real market/business for such a
| solution. After all, vendor lock-ins are great business, and
| generate business.
| LeanderK wrote:
| I may be misunderstanding your comment, but as I understood it
| you are talking about industry failures academia can't do
| anything about. If industry is massively behind on outdated,
| closed environments and academia work's with different, open
| tools then that's not academias fault. Like:
|
| > it's legacy product support and ease of onboarding new hires.
|
| I don't see how academia can help with this, this so very much
| an industry problem.
|
| > But I need these to be accessible by Billy Joe, whose only
| credentials that got him the job in the maintenance department
| is that he helped out his daddy working with a welder and an
| old clapped-out Bridgeport on the farm growing up.
|
| While accessible interfaces and forms of human-computer
| interactions are researched by academia, they are not building
| products. Research has to be relevant, but it's still research.
| Those problems, they might be better addressed by a startup
| than academia, if that's even realistic. Academia can't solve a
| problem if it's not a research-problem.
| iancmceachern wrote:
| That's what he is saying. ROS is made, lead, by academia. He
| is saying that ROS isn't terribly useful in industry for the
| reasons he lists.
| LeifCarrotson wrote:
| Exactly. Advancements and capabilities in university press
| releases like this don't result in real-world improvements
| because the two fields are divergent.
| cs702 wrote:
| After reading this, I'm persuaded this space would benefit
| greatly from having a few dominant open-source software
| frameworks and a few dominant open-source hardware standards,
| making product support and new-hire onboarding much easier for
| vendors of all sizes. Alas, I recognize that no established
| vendor in the space would ever want to see the emergence of
| industry standards. :-P
| bismuthcrystal wrote:
| What this field really needs is a massive push. The reason it
| always fail to deliver is that the incentive system is
| screwed. We need something like a Manhattan project capital
| allocation to a few very talented people to make progress.
| Building endless frameworks and trying to get thousands of
| people to cooperate on high level decision making dooms the
| whole endeavor. The risk for product development is too high
| for private industry to take and academia can't deliver
| products.
| iancmceachern wrote:
| What you describe is ROS. The issue is that industrial
| manufacturers of things like arms or motion controllers all
| have their own proprietary stuff.
| cs702 wrote:
| What I describe is what ROS supporters _want it to be._ As
| I wrote, no established vendor in the space wants to see
| the emergence of commodity industry standards. It seems
| that for the foreseeable future, we 're stuck with the
| status quo.
| LeifCarrotson wrote:
| It definitely would. RoboDK is one product that is moving in
| the right direction.
|
| In the PLC space, Beckhoff controllers using commodity x86/64
| hardware and standardized IEC 61131 programming languages are
| gobbling up Rockwell and Siemens' market share.
|
| Given they're perpetually about 20 years behind the times,
| perhaps this year someone at Fanuc, ABB, Kuka, and Yaskawa
| will catch up to the year 2002 and read Spolsky's admonition
| [1] that says:
|
| > _Smart companies try to commoditize their products'
| complements._
|
| Stop trying to turn every division of your company into its
| own product center. I loathe paying $800/year to Rockwell for
| the privilege of reading "update your firmware" (and
| subsequently bricking my controller as an unwilling beta
| tester) on TechConnect. They're a PLC vendor, they should
| focus on selling PLC hardware, not making their customers
| hate them by nickel-and-diming them for software licenses and
| support contracts. They were down for FOUR DAYS this weekend
| for planned maintenance. FOUR DAYS!
|
| Likewise, Fanuc should focus on selling servos and castings -
| robotic manipulator hardware - and stop trying to get the
| training division to turn a profit. If, instead of a 40 hour
| course costing $2,000, it was free, there would be more
| people able to use their $40,000 robot arms! If Roboguide
| software was free instead of $2,650, clients could more
| efficiently build more robot cells, and sell more yellow
| paint! Gah!
|
| [1] https://www.joelonsoftware.com/2002/06/12/strategy-
| letter-v/
| mncharity wrote:
| > IEC 61131 programming languages
|
| A very brief search turned up assorted overviews[1][2][3],
| and a github topic[4]. The PLC _open_ standard[5] is
| paywalled - only $400 single user, with low low per-seat
| multipliers! : /
|
| [1] https://en.wikipedia.org/wiki/IEC_61131-3 [2]
| https://dc-
| us.resource.bosch.com/media/us/products_13/produc... [3]
| https://www.controleng.com/articles/which-
| iec-61131-3-progra... [4]
| https://github.com/topics/iec61131-3 [5]
| https://plcopen.org/iec-61131-3
| iandanforth wrote:
| Did you know it takes almost a hundred ms for a signal to go from
| your fingers to your brain? Did you know that the brain is
| constantly predicting the future to make up for its slow reaction
| time? Did you know even small computers are _really really_ fast?
| If you did then you 'd probably guess that initiatives like this
| are a waste of time and money. The difficulty is in the software,
| not the hardware, or the compute layer. With good algorithms our
| current crop of robots are already fully capable of producing
| trillions more in economic value than they do today.
|
| We do not need hardware acceleration in robotics. Can we use it
| if it exists? Sure, but do we care about "democratizing" it? No.
|
| There is a place for hardware acceleration in robotics that is
| making a huge difference and that's in simulation. GPUs, the once
| and future king, make training robust neural networks possible
| and fast. If you're an investor reading this who needs a
| reminder, bet on software.
| vmayoral wrote:
| I agree that one of the major aspects to improve in robotics is
| software. There's still a lot do there and many are working
| towards it. Most leading initiatives around ROS 2. I also
| believe that we need faster robotics simulation, and hardware
| acceleration will be valuable there, not only with GPUs, but
| also other accelerators (e.g. FPGAs outperform GPUs in many
| benchmarks both performance and energy consumption-wise).
|
| I strongly disagree with the rest and your comments indicate a
| clear lack of (hardware) understanding. I often encounter this
| in the AI world. Where folks building ANNs just "forget" that
| this computational abstraction only "grew in popularity" (there
| were previous accelerators though) when CNNs got implemented as
| an accelerator in a GPU empowering newer results. The same's
| likely to happen in robotics.
|
| Robots are real-time systems. Meeting time deadlines in their
| computations is the most important feature. (robot) Behaviors
| often take the form of computational graphs, with data flowing
| between compute abstractions (Nodes), across physical networks
| (communication buses) and while mapping to underlying sensors,
| compute technologies and actuators. ROS enables you to build
| these computational graphs and create robot behaviors by
| providing libraries, a communication infrastructure, drivers
| and tools to put it all together.
|
| From a more technical compute architecture perspective, ROS 2
| presents an event-driven programming interface. The resulting
| computational graphs built with it are "event-driven software
| architectures". Mapping these event-driven software
| architectures to hardware using CPUs leveraging classic control
| flow architectures (von Neumann architectures) leads to various
| issues. A key challenge in applying classic event-driven
| programming is that CPUs hardly provide real-time and safety
| guarantees. The de facto strategy in industry to meet the
| timing deadlines is a laborious, empirical, and case-by-case
| tuning of the system. This "whack-a-mole" approach is being
| realized by some, but unsustainable and hard to scale due to
| the lack of a hardware-supported timing-safe even driven
| programming interface. This is where accelerators come in. As
| already adopted in other industries including aerospace,
| automotive and healthcare, through the creation of custom
| compute building blocks (accelerators), a hardware/software co-
| design strategy provides clean behavioral specifications,
| describing clearly its guarantees in terms of timing (in other
| words, avoiding memory-centric von Neumann architectures).
|
| Hardware is important. Note the current scarcity of
| semiconductors, which is one of the drivers of this research.
| Note also that creating custom accelerators for robotics
| application not only can lead to performance improvements (i.e.
| software that runs faster!) but also to more deterministic
| responses (which affects the downstream pipeline of software!).
| Rephrasing Alan Kay's quote, if you're serious about robotics,
| you should care about hardware ;).
| fxtentacle wrote:
| Did you know that for predicting what you'll see 100ms into the
| future you need a level of real-time 3D recognition and world
| knowledge that we haven't ever solved acceptably, not even
| offline and parts of it in isolation?
|
| While I agree with you that in 50 years, advances in AI will
| make fast hardware obsolete, it may well be that in 5 years,
| fast hardware will be able to avoid problems that we cannot yet
| solve in software. That would mean this company has 45 years to
| cash in before they get replaced by software.
| [deleted]
| [deleted]
| MichaelCollins wrote:
| The human brain does what it does with roughly 20 watts, if we
| take the popular figure of the brain taking 1/5th the power of
| your whole body and the average human body burning 100 watts.
|
| How much neural net can you run on silicon with 20 watts?
| mynegation wrote:
| TBF it takes human brain anywhere from several months to
| decades to train for its skills. Plus several hundred million
| years spent on pre-training.
| MichaelCollins wrote:
| Sure it's not fair, but there's no handicap in this game.
| Silicon isn't given any leg up for being millions of years
| late. Raw results are what really matters, not results
| weighted by whatever criteria would render the comparison
| subjectively fair.
| meowtastic wrote:
| Looks impressive to a layman like myself. I'm guessing it
| addresses the global shortage of single-board computers like
| Raspberry Pis right? Curious to know how long it'll take to do
| that as well.
| vmayoral wrote:
| Well, yes but not only. There's only so much you can do with
| Pis and other SBCs that offer a CPU-centric compute solution.
| Robots are real-time systems and CPUs don't excel at that.
|
| When optimizing dataflows for lower latencies and higher
| throughputs, you typically seek specialized compute
| architectures and that's wherein GPUs and FPGAs come in. That's
| what this work really enables.
| meowtastic wrote:
| I see, thanks for the explanation!
| archivisti wrote:
| comparison using Jetson TX2 or Xavier could have provided better
| insight instead of Nano 2G which struggles even in basic task but
| otherwise looks quite promising
| vmayoral wrote:
| It was benchmarked against Jetson AGX Xavier as well. See
| https://news.accelerationrobotics.com/hardware-
| accelerated-r....
|
| Results using AGX Orin are in the pipeline and will come out
| soon as well.
___________________________________________________________________
(page generated 2022-07-25 23:02 UTC)