[HN Gopher] Integrating Rust into the Android Open Source Project
___________________________________________________________________
Integrating Rust into the Android Open Source Project
Author : theafh
Score : 80 points
Date : 2021-05-11 17:38 UTC (5 hours ago)
(HTM) web link (security.googleblog.com)
(TXT) w3m dump (security.googleblog.com)
| gautamcgoel wrote:
| This will be downvoted, but I'm a bit skeptical of the "integrate
| Rust into big C projects" trend. It seems to me that it adds a
| lot of complexity: now there are two toolchains you must
| understand in order to contribute. There will invariably be
| subtle bugs at the interface between the two languages. Yes,
| memory and type-safety are wonderful, but they are not the be all
| and end all.
| wyldfire wrote:
| You have good reason to be wary, it is added complexity that
| should not be taken lightly.
|
| The reason to pay this cost is that the alternatives are: (1)
| do nothing: keep developing solely in C, (2) rewrite the big C
| project in Rust in its entirety. The cost of option (1) is
| significant: memory safety errors contribute to 70% of defects
| not detected by test [1]. Even with static+dynamic checking
| tools, developers keep introducing bugs into C projects that
| aren't possible in safe Rust [2]. The cost of option (2) is
| significant risk. Big C projects have grown over time and have
| many implicit requirements already satisfied. Discovering all
| of those incidental bugs and features that are necessary takes
| a long time and scales with the size of the project.
|
| [1] https://msrc-blog.microsoft.com/2019/07/16/a-proactive-
| appro...
|
| [2] http://blog.vmsplice.net/2020/08/why-qemu-should-move-
| from-c...
| kjs3 wrote:
| I don't think the only options are "rewrite in Rust" or
| "don't". There are other options than Rust.
| throwaway894345 wrote:
| I think it makes the most sense as part of a strategy to
| incrementally migrate a codebase from C to Rust, or else if
| there is a clean separation between components such that people
| are working in one ecosystem or the other but rarely both.
|
| In whichever case, we should be careful not to overstate the
| complexity of integrating Rust and C. Specifically, bringing in
| any given C dependency means integrating its bespoke build
| system (and those of each transitive dependency) with that of
| your project (or more likely, you punt on reproducible builds
| and instead push the dependency hell onto every contributor)
| which doesn't strike me as considerably less work than
| integrating the Rust toolchain (and since Rust has sane,
| standard build/dependency tooling, it's pretty much a one-time
| cost rather than a once-per-dependency cost).
| gregwebs wrote:
| Does there need to be subtle bugs at the interface if Rust
| exposes the interface as a C ABI?
|
| I think the idea of this article is that you don't need to
| understand two tool chains but instead can focus on the harder
| task of understanding two languages.
| Const-me wrote:
| Yes.
|
| Rust does not consume C headers. For every function and data
| structure of the API surface you gonna need two language
| projections, C header and Rust source file. For large API
| surfaces, especially when both sides of the FFI evolve over
| time, bugs are inevitable.
|
| The problem is not specific to Rust. It affects all
| languages, except C++ and Objective-C which are backward
| compatible.
| steveklabnik wrote:
| While Rust itself may not directly, tools like bindgen take
| in those headers and produce the needed glue.
| https://cxx.rs/ is another example of a tool that's used in
| these sorts of cases.
|
| You do not have to write these things, nor keep track of
| changes, by hand.
| Const-me wrote:
| I wonder why so many Rust FFI wrappers don't use these
| tools?
|
| The most popular C library in the world is probably the
| standard one. Yet in https://github.com/rust-lang/libc I
| don't see any glue automatically generated from C
| headers, everything's written manually.
|
| Other examples: https://github.com/servo/rust-png/
| https://github.com/bozaro/lz4-rs
| steveklabnik wrote:
| The first two are older than these tools, not sure about
| lz4. Not 100% sure why but I would imagine that's at
| least a reason.
| jcranmer wrote:
| The libc crate's test does actually parse the C header
| files to generate tests for conformance.
|
| There are a few big reasons to consider avoiding parsing
| the C headers directly:
|
| * The C headers might use features in C that are not
| compatible with the C parser or Rust ABI itself. (C
| unions used to be a big headache, and bitfields probably
| still are.). Manually specifying only what you want to
| import avoids that issue entirely.
|
| * As an extended note to the above, many libraries
| (particularly your system's implementation of libc) are
| actually not written in _any_ standard C dialect but
| instead have a lot of compiler-specific things tacked
| onto it that make it even more difficult to parse and
| handle correctly than regular C /C++.
|
| * Parsing the C headers requires having them installed
| _and_ being able to find them correctly. When you start
| dealing with cross-compilation toolchains, this can be a
| major hassle.
| JoshTriplett wrote:
| > Rust does not consume C headers.
|
| Yet, at least. I hope one day it can, transparently.
| zaphar wrote:
| I would perhaps argue, especially for projects like Android,
| that memory safety should be considered table stakes going
| forward. The class of risks that are mitigated or eliminated by
| memory safety are well worth the effort. There isn't a great
| excuse to not prioritize memory safety when it's an option.
| OmarShehata wrote:
| The reason I'm optimistic about this trend is that it's been a
| common thing in game development for a long time (low level
| language for execution speed, high level scripting language for
| development speed) and it works really well.
|
| Having to maintain this interface between the language does
| come with its own set of problems but is well worth the
| tradeoff especially if you can spend most of your time in one
| language or the other.
| bosswipe wrote:
| That's not what's happening here (Android already has low
| level C and high level Java). What's happening here is two
| low level languages side by side.
| pcwalton wrote:
| Sometimes complexity is worth it for the engineering benefits.
| We could all be running our software on beautifully simple
| vintage 1980s microcontrollers: the 6502 was an incredibly
| elegant and simple design! But then we'd be stuck with
| computers that can't multiply in hardware or add two floating
| point numbers or prevent a misbehaving application from taking
| down the whole system.
| dang wrote:
| Please don't downvote-bait. It's tedious and against the site
| guidelines.
|
| https://news.ycombinator.com/newsguidelines.html
| pjmlp wrote:
| Unless one starts with another CFront, naturally moving away
| from how C compilers see the world has some cost, no matter the
| language.
| jchw wrote:
| If the project is a pure C project using a build system that
| can't integrate Rust very easily, or it would come at a great
| cost, then I can see this being a totally valid concern.
|
| However, AOSP is _massive_. It is HUGE. And complicated to
| build. I really, really doubt this addition will make a huge
| deal.
|
| This is the same for many of the huge projects adopting Rust.
| The worst one so far is the Linux kernel IMO, because at least
| that is mostly C and GNU Make. And yet even there, that still
| isn't that bad.
|
| Also, as far as bugs at the interface, I'm pretty sure it's a
| wash. C and Rust are ABI compatible to the extent that matters.
| The "subtle bugs" are most likely going to be ones that
| could've existed without the additional complexity of a new
| language and an interface between them. It's no worse than say,
| JNI, which doesn't seem to be a serious issue.
|
| I'm pretty sure the benefits of Rust can outweigh this cost. As
| long as you can build the project, you don't really need to
| understand 2 toolchains to contribute unless your contribution
| lands across a boundary between 2 toolchains. On a lot of
| really large projects this may already be the case anyways.
| flakiness wrote:
| One good thing about Android in this perspective is that many
| sub-components of it have their own processes (binary),
| especially around HAL, after Project Treble [1]. This means that
| you can rely on the process boundary as the language boundary to
| start small.
|
| Also note that Android is more C++ than C. And C++-Rust
| integration is something Rust has already done in Firefox. The
| road is paved, at least partially. So I have high hope here in
| long term.
|
| [1] https://android-developers.googleblog.com/2018/05/faster-
| ado...
| thesuperbigfrog wrote:
| >> Dynamic linkage by default >> By default, the Rust ecosystem
| assumes that crates will be statically linked into binaries. The
| usual benefits of dynamic libraries are upgrades (whether for
| security or functionality) and decreased memory usage. Rust's
| lack of a stable binary interface and usage of cross-crate
| information flow prevents upgrading libraries without upgrading
| all dependent code. Even when the same crate is used by two
| different programs on the system, it is unlikely to be provided
| by the same shared object4 due to the precision with which Rust
| identifies its crates. This makes Rust binaries more portable but
| also results in larger disk and memory footprints.
|
| >> This is problematic for Android devices where resources like
| memory and disk usage must be carefully managed because
| statically linking all crates into Rust binaries would result in
| excessive code duplication (especially in the standard library).
| However, our situation is also different from the standard host
| environment: we build Android using global decisions about
| dependencies. This means that nearly every crate is shareable
| between all users of that crate. Thus, we opt to link crates
| dynamically by default for device targets. This reduces the
| overall memory footprint of Rust in Android by allowing crates to
| be reused across multiple binaries which depend on them.
|
| >> Since this is unusual in the Rust community, not all third-
| party crates support dynamic compilation. Sometimes we must carry
| small patches while we work with upstream maintainers to add
| support.
|
| Having a dynamic linking option for Rust would be a tremendous
| improvement.
|
| It would help to solve vendoring issues and hopefully improve
| interop with existing systems and programming languages.
|
| Good dynamic linking support and enhanced interop with C would
| make it much easier to do a piece-wise replacement of C
| components with Rust components, allowing for a gradual migration
| in places where it makes sense to do so.
| esarbe wrote:
| That's very exciting. Rust really seems to establish itself as a
| serious contender for the low-level niche that until so far has
| almost exclusively been occupied by C.
|
| I wonder how much of an obstacle the lack of the stable ABI and
| the inability to transparently use C headers still are.
___________________________________________________________________
(page generated 2021-05-11 23:02 UTC)