[HN Gopher] Show HN: Copper - Open-source robotics in Rust with ...
___________________________________________________________________
Show HN: Copper - Open-source robotics in Rust with deterministic
log replay
Author : gbin
Score : 154 points
Date : 2024-12-03 00:58 UTC (22 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| a_t48 wrote:
| Nice release - deterministic replay isn't easy - I still need to
| go back and pull our implementation forward to match the new
| features in the core framework. I'm not sure if you've seen
| Basis, but we're somewhat "competing" in the same space. You're
| all in on Rust, we're C++ with bindings to other languages
| (eventually). I don't see this as a bad thing - more tools in the
| space is a win, eventually the industry will settle on something
| better than ROS.
| AlotOfReading wrote:
| Having maintained something similar myself, how has your
| experience been with exact determinism in C++? C++ compilers
| tend to be a bit of a free-for-all when it comes to bit
| determinism, whereas the rustc folks are very good about
| spending the design time to preemptively avoid many of the
| issues you'd need workarounds for in other languages.
| a_t48 wrote:
| My goal is determinism specifically around message
| timings/execution in replay. We don't currently advertise
| runtime determinism, we also don't promise fixes for
| determinism in your code. For message timing, the algorithm
| should be stable - it generates a timeline for when messages
| should be executed by each message handler, and will pause
| further executions to wait on handlers that "should have"
| finished, but haven't due to running too slow. Ties are
| broken by the handler name or data source name. I wrote a bit
| about it here
| https://basisrobotics.tech/2024/09/02/determinism/ but didn't
| go deep into the implementation. If your own code isn't
| deterministic you will have differences, but this eliminates
| the biggest classes of nondeterminism when running tests
| (threading timings, message handling races).
|
| I don't think Copper claims to offer more than this either,
| but I can't speak for them.
| pryelluw wrote:
| Such a great two sentence intro to the project:
|
| " Copper is a user-friendly runtime engine for creating fast and
| reliable robots. Copper is to robots what a game engine is to
| games."
|
| Other projects should take notice.
| gbin wrote:
| Oh wow thanks, I have to tell you something... It is really
| hard to explain what a piece of middleware is and why it is
| important :)
| amacneil wrote:
| It's great to see more innovation in the robotics framework
| space, it's sorely needed.
|
| Deterministic log replay is a killer feature to have baked in
| from the start - many autonomous vehicle & robotics companies
| have invested years of effort to achieve that nirvana, while the
| most popular robotics framework today (ROS) makes it borderline
| impossible.
| jeffreygoesto wrote:
| True. All of it. https://edms.etas.com for example.
| the__alchemist wrote:
| Tangent this got me thinking about: Why are the robotics and
| embedded communities generally separate, with little overlap in
| people and tech? For example, this project seems unrelated to the
| rust embedded tooling. And the components categories are also
| unfamiliar, e.g. the IMUs and ADCs in the readme, using a SBC
| vice a MCU etc.
|
| Another angle: `ROS` and `RTOS` share letters, but not much else!
|
| I gather that robotics is a fusion of embedded and mechanical
| engineering; I refer to the former.
| gbin wrote:
| It almost feels like we would need an "in between" category.
|
| If I paint a broad brush:
|
| - embedded is all about latency, low bandwidth
|
| - computers are all about bandwidth with often terrible
| latencies.
|
| A modern robot needs both low latency and high bandwidth.
| the__alchemist wrote:
| I suppose this lends itself to a hybrid scenario. I suspect
| most mature robotics projects would have both types of
| computation. Likely bare metal or RTOS as the default, with
| delegation to GPOS computer for high bandwidth, and high
| computation tasks such as CV.
|
| I also imagine distributed computing (for example: over CAN
| or another bus) would be a useful pattern.
| ModernMech wrote:
| Typically with robots you need more power than what embedded
| provides. We build on top of commodity x86 systems a lot of the
| time. Other times we use SOCs like the Nvidia Jetson line,
| which also doesn't require a lot of embedded skills; you get a
| full operating system and a userspace so it's more like
| programming on a typical unix-based system. You find more
| embedded in the automation side of things. Wearables, IOT,
| always connected devices -- things that are too small and low
| powered to afford an OS.
| AlotOfReading wrote:
| There's a lot of embedded skills in pretty much any hardware
| platform. Maybe this is selection bias from being an
| embedded-focused person in robotics, but there's plenty of
| jobs out there that need a toolchain wrangler who can deal
| with cost optimization.
| ModernMech wrote:
| I'm not saying skills you pick up doing embedded work
| aren't useful in robotics contexts. I'm saying roboticists
| often have the luxury of an OS, and they take advantage of
| it, so the worries you have doing embedded work without an
| OS are mitigated.
|
| I would say tho there _are_ different "kinds" of
| roboticists. Just as you have backend and frontend devs in
| the web world, you might consider people who work on
| hardware or software to be the robotics equivalent.
| Hardware people _are_ going to be low level and working
| with embedded devices and may even program them. But higher
| up in the robotics stack most of that embedded know-how
| (toolchain wrangling as you put it) is much less important.
| The full stack robotics engineer has hardware and software
| knowledge /experience, but you don't usually see roles that
| ask for both, so people tend to specialize into one or the
| other.
| a_t48 wrote:
| OTOH the OS can very much get in the way. If you're using
| ROS for instance, it's easy to fall into apt package
| dependency hell - upgrading CUDA means upgrading the OS,
| which means upgrading ROS, which means upgrading all deps
| including OpenCV, etc, etc, etc.
| exe34 wrote:
| that's the part I hate about ros and a whole lot of other
| libraries that shouldn't have been so tightly coupled -
| you should be able to run most of ros in one env and then
| for the performance critical or hardware facing stuff,
| run them each in their own env and have lightweight
| communication between them.
|
| I think if something annoys me enough I'll end up using
| nanomsg or mqtt as a bridge, but it's a pain.
| ribadeo wrote:
| Xenomai or rt extensions to linux, basically the rtlinux
| legacy, where the os is a low priority thread while your
| latency critical code runs bare metal or scheduled out of
| band
| AlotOfReading wrote:
| My experience is that there's a major difference between
| "R&D project" mentality to mass production and big
| install bases. The embedded stuff really starts to
| leaking into everything during the latter once you start
| evaluating how much it'd cost to rewrite the high level
| stuff to fit cheaper hardware, or resolve the
| intermittent issues that big customer is complaining
| about.
| dgfitz wrote:
| This is currently part of my $dayjob.
| cchance wrote:
| If this pics up support for like a open-library of assorted
| components and people shared them it has a lot of potential
| especially if the sim env and replays continues to be improved. I
| love the idea and looks gerat!
| cpgxiii wrote:
| This seems to suffer a bit from the same problem that affects a
| lot of "new" robotics frameworks: the beginnings are simple, and
| it's easy at that stage to think that everything else is over-
| complicated.
|
| The comparison with ROS 2 is a bit questionable. Comparing a
| single-process-only (Copper) approach using shared memory with a
| multi-process system (ROS 2) using default DDS settings really
| isn't comparing the same thing. There are ways to make the ROS 2
| system much faster if you're willing to be limited to similar
| constraints (single process with components, or local-system
| shared-memory transport) but most people don't because those
| constraints are very limiting in practical applications.
| gbin wrote:
| Note that Copper can be distributed too, we do support iceoryx2
| for example if you need to split it in 2 or more processes.
|
| We have ~100x less latency and ~12x faster logging also because
| we have adopted a data oriented architecture: the tasks outputs
| are written back to back in memory, all the IOs are linear (we
| could log to a block device straight, we don't even need a
| filesystem). I am not sure it is possible to touch this with
| ROS just because of its "everything is asynchronous" design
| pattern.
|
| So the question is more about those limits in practical
| applications: do you have use cases where you absolutely need
| every single component deployed as a kind of micro service in a
| robot?
| rcxdude wrote:
| I will say the default network-transparency of ROS is probably
| more of a long-term hindrance than a help to applications built
| on top of it. A lot of projects would get a lot further if they
| were designed from the outset with clear boundaries where
| network communication was required (and what can be done to
| deal with what that implies), and it would be nice if
| components intended to be used in those systems were designed
| to be composed with that in mind, instead of requiring a
| network-based pub-sub system with not particularly great
| latency or reliability guarantees, even if it has a mode where
| those are tightened up a lot.
| patrick451 wrote:
| I disagree that the ROS2 comparison is questionable. If ROS
| want's to be seen as performant, they need to use a performant
| middleware by default. You shouldn't have to fiddle endlessly
| with it to get reasonable latencies.
|
| Having worked on both large ROS systems and large monolithic
| robotics systems, the monolith wins hands down. The ROS
| paradigm that every node is it's own process is frankly insane.
| jvanderbot wrote:
| TBF
|
| > every node is it's own process
|
| .. was SOA about 10 years ago. At this point any comparison
| to ROS1.0 is just strawman.
|
| Nowadays nodelets and Ros2 eschew that approach to a larger
| degree. But you're right. Plenty of shops still use ROS1.0.
|
| ROS was ... first, so they set the standard, and were AFAICT
| the introduction to distributed and message-passing systems
| for most roboticists coming out of school/Academia. They all
| graduated and had to relearn things over time (myself
| included). A plain-jane shared memory paradigm is just
| simpler and easier once you get a basic framework together
| for _most_ things. There certainly are situations where you
| want process partitions, e.g., plug-and-play payloads. But
| even there, ROS plug-and-play is atrocious, since this is
| akin to a network drop, which it just doesn 't handle. So,
| everyone just writes bridge nodes anyway.
|
| Then don't get me started on multi-agent systems. At one
| point it was honestly believed that all agents would use a
| common Ros Master. Laughable!
| rgovostes wrote:
| Do I understand that at each tick, every task outputs a single
| message, and can depend on the outputs of other tasks? I.e.,
| nodes and topics are merged into tasks, executed in topological
| order?
| gbin wrote:
| Exactly.
|
| Technically Copper is a compiler taking your graph with those
| constraints in mind and building a game loop out of it.
| BWStearns wrote:
| This reminds me, I've been messing around with Bevy recently
| and have been wondering if ECS systems might be suited to
| hardware. On the surface it seems like a good ergonomic fit.
| Not sure if ECS is a performance/determinism killer but since
| they're used in gaming where performance is often critical I
| imagine it's surmountable?
| gbin wrote:
| I believe that at least the initial version of ISAAC from
| Nvidia was based on an ECS. I don't think it is a
| determinism killer, this is often a well ordered pipeline
| within a game loop.
|
| In terms of architecture though, it looks like it makes a
| ton of sense inside something like a tracker (ie the entity
| is basically a track) but for other parts like a vision
| pipeline, sensor fusion etc... I don't see how it helps.
|
| Tell me if I am missing something.
|
| Talking about Bevy, fitting Copper within Bevy to build
| this little simulation example happened super naturally:
| Copper is a System querying the Entities within the virtual
| world after each simulation tick.
| BWStearns wrote:
| Very cool about ISAAC. I didn't know.
|
| As far as how it helps, I was mostly thinking from a dev
| ergonomics perspective. For things like ROS/mavlink it
| can sometimes be hard (for me) to think through how all
| the systems are interacting but for whatever reason ECSs
| feel like a natural way to think about systems with
| simultaneousish inputs and outputs.
| steve_adams_86 wrote:
| I'm not an authority on the topic, but a somewhat similar
| concept to ECS is state charts, which I find far better
| suited to hardware.
|
| Some might verbally assault me for calling these two things
| similar, but I mean that loosely. Someone who is new to
| both might notice that they compose systems which can
| communicate and respond to inputs, which is conceptually
| similar. State charts offer far more guarantees,
| consistency, reliability, and predictability though.
|
| I suppose one is about propagating data (not state,
| specifically), the other is about state and state control.
| Both are hierarchical in a sense, but ECS doesn't place as
| much importance on hierarchy.
|
| Apologies if I'm dead wrong.
| leonheld wrote:
| > Unlike a game engine we use a data oriented approach to
| minimize latency and maximize throughput.
|
| Don't you mean "like"? I thought game engines were all about data
| oriented approach.
| gbin wrote:
| Indeed thanks. "Not unlike" :)
| ericyd wrote:
| I know nothing about robotics.
|
| Is deterministic log replay really a differentiating factor? My
| naive assumption would be that this is table stakes for pretty
| much any software.
| chippiewill wrote:
| ROS is probably the biggest robotics framework and doesn't have
| deterministic playback.
|
| It's important for safety critical to be sure, but you can get
| surprisingly far without it.
___________________________________________________________________
(page generated 2024-12-03 23:01 UTC)