[HN Gopher] Welcome to Comprehensive Rust
___________________________________________________________________
Welcome to Comprehensive Rust
Author : eecc
Score : 513 points
Date : 2022-12-22 09:01 UTC (14 hours ago)
(HTM) web link (google.github.io)
(TXT) w3m dump (google.github.io)
| incrudible wrote:
| The first thing I implore everyone to do when evaluating Rust is
| to check out a _non trivial_ project and start hacking on it. I
| have always found the compiler to be unacceptably slow in these
| cases, especially if you come from C rather than C++. If you work
| your way up from Hello World you may not ever notice this until
| you already invested a substantial amount of effort.
| pen2l wrote:
| Look into Apple silicon - lots of cache, great branch
| predictions, pointer-chasing prefetchers and more to subside
| just about all of your compiling worries.
| saagarjha wrote:
| Hmph, I wish...
| wongarsu wrote:
| On the other hand, I've always found it surprisingly easy to
| dive into non-trivial rust projects and start contributing, at
| a speed that would be unthinkable in C or C++.
|
| So I second the advise for non-trivial projects, they give a
| much better idea of the upsides and downsides of rust.
| 9dev wrote:
| Is that really due to the language or that we collectively
| have become better in structuring software projects?
| the__alchemist wrote:
| I'm suspicious of this as well. It's common for OSS C and
| C++ projects (at least in embedded) to be mostly
| undocumented / few comments, have no easy way to debug by
| printing to console, and have a complicated build process
| that requires a certain OS and setup.
|
| Top doc comment: The same license info that's in every
| file. Not helpful for describing what the module does and
| how it fits in with the rest of the project.
| kajaktum wrote:
| Well, I've refactored large portions of Rust code with ease
| and I can't imagine doing it with any C/C++ code because I
| might break some invariant somewhere.
| wongarsu wrote:
| People have become not just better at structuring projects,
| but also at writing standard libs and programming
| languages.
|
| Many features gained a lot of adoption accross languages
| over the last two decades, like anonymous functions (with
| convinient arrow syntax), explicit option types instead of
| implicit null/undefined, match constructs, well-integrated
| library management, for(each) loops using iterators instead
| of indices, map/reduce functional iterators, etc. Rust has
| the advantage of being able to make these core parts of the
| language and standard library, resulting in an overall more
| expressive language.
|
| And in rust's view these features should not only be used
| to make coding more convinient, but also to encode intent
| and make undesirable things impossible. This is reflected
| in the standard library (for example it's impossible to
| access the data protected by a mutex without locking it),
| and the philosophy is copied by the ecosystem.
| nicoburns wrote:
| It's at least partly due to the build tooling. `git clone
| ... && cargo build` (and `cargo test` too) "just works" for
| 99% of the Rust projects I have interacted with. And it
| generally works just as well for projects that have been
| abandoned for 5 years as it does for well maintained
| projects. The ones it doesn't work with it's usually
| because they depend on a C or C++ library!
|
| Build tooling for the C/C++ ecosystem is a mess.
| Sharlin wrote:
| You can trust the compiler to yell at you when you break
| many invariants that would be tacit and implicit in C and
| even in C++ (even with best practices followed) which can
| make you much more productive when you're just
| familiarizing yourself with a codebase.
|
| Also, the two aren't necessarily mutually exclusive. _Many_
| people learning the language have reported being surprised
| by the way Rust seems to make "doing things right" the
| path of least effort, and this applies to best practices in
| general, not just memory safety.
| db48x wrote:
| I've had that experience. I had to write code for a
| complicated reference counted datastructure. The code was
| a mess at first because I was approaching it the wrong
| way, but once I understood it well enough to write it
| correctly all the pieces fell into place. Extraneous
| complexity was eliminated, and all the possible mistakes
| turned out to be impossible. The resulting code is so
| simple and straight forward that anyone can read it and
| see that it is correct. And as is traditional, the first
| time it compiled it was correct in all respects.
| kibwen wrote:
| I work on quite a large Rust project, and my regular `cargo
| check` takes around 5 seconds. Of course, I'd be happy if it
| were instantaneous, but "two deep breaths" worth of turnaround
| time is hardly a deal-breaker, especially given all the bugs
| that the compiler prevents for free that I don't need to spend
| time tracking down.
|
| EDIT: I checked just now and it takes one second to `cargo
| check` the shallowest crate in our workspace, and six to check
| the deepest.
| rowanG077 wrote:
| Could you point me towards a project where this is the case? I
| want to try it out as I never found Rust slow to compile.
| ergonaught wrote:
| "I've never been bothered by the compilation speed" is not
| the same thing as "It isn't slow to compile."
| rowanG077 wrote:
| Sure! I didn't claim anything close "it isn't not slow to
| compile" and I'm not sure how you could interpret my
| comment like that. Notice how I clearly asked to be
| educated.
| deaddodo wrote:
| Rust is most definitely slow to compile compared to GCC or
| LLVM for C/C++. This is a known situation and has been since
| the pre-1.0 days because of the heavily front-loaded macro
| and borrow systems. It's the tradeoff you have for having a
| "safe" language vs a "trusting" one.
|
| I would argue that it's not as bad as it's made out to be,
| especially if you're not doing full recompiles and using it
| on modern (2019+) hardware; but it is a noticeable difference
| in similar large codebases.
| rowanG077 wrote:
| It's slower then C for sure. But slower then C++? I doubt
| it.
| pjmlp wrote:
| C++ is only slower in heavy template metaprogramming.
|
| Additionally thanks to heavy reliance on binary
| libraries, when starting a new project, we only need to
| compile our own code.
| est31 wrote:
| Rust is most comparable to C++ with heavy TMP, as it
| itself also makes use of templates.
| kibwen wrote:
| To be precise, Rust generics perform monomorphization,
| which C++ templates also perform. But I wouldn't go so
| far as to compare the language to "C++ with heavy TMP",
| as that's far too broad of a characterization; TMP
| implies a lot of things, e.g. poor error messages, that
| Rust doesn't necessarily exhibit.
| est31 wrote:
| > TMP implies a lot of things, e.g. poor error messages,
| that Rust doesn't necessarily exhibit.
|
| That's fair. My comment was mostly about performance.
| est31 wrote:
| Rust macros are not the fault for its slowness. expansion
| is one of the fastest compilation phases. Time is rather
| spent in other steps like type checking, llvm optimizations
| and such. The issue is more Rust's _generics_ as they cause
| a lot of code to be passed to llvm (among other issues for
| Rust 's slowness).
| Arnavion wrote:
| Macro expansion _is_ slow, but only noticeably in the
| specific situation of a) third-party proc macros, b) a
| debug build, and c) a few thousand invocations of said
| proc macros. This is because building in debug mode also
| compiles proc macros in debug mode, so while the macro
| itself compiles quickly (because it 's an unoptimized
| build), it ends up running slowly (because it's an
| unoptimized build).
|
| I know this from observing a ~90s difference between
| debug and release builds, of a large (mostly auto-
| generated) crate that had a couple of thousand
| `#[derive(serde::*)]`s. [1]
|
| This doesn't affect most users, because first-party
| macros like `#[derive(Debug)]` etc are not slow because
| they're part of rustc and are thus optimized regardless
| of the profile, and even with third-party macros it is
| unlikely that they have thousands of invocations. Even if
| it _is_ a problem, users can opt in to compiling just the
| proc macros in release mode. [2]
|
| [1]: https://github.com/Arnavion/k8s-openapi/issues/4
|
| [2]: https://github.com/rust-lang/cargo/issues/5622*
| est31 wrote:
| > Macro expansion is slow, but only noticeably in the
| specific situation of a) third-party proc macros, b) a
| debug build, and c) a few thousand invocations of said
| proc macros.
|
| Oh that's interesting. But note that it's an issue caused
| by (from the perspective of the language) user code being
| slow, and also only if you compile in debug mode. You
| probably don't want to unconditionally compile proc
| macros in release mode, as most times they probably don't
| get invoked thousands of times. Ideally you would
| probably a) invoke macros in a parallel fashion, with
| multiple expansion threads, and b) maybe have rustc
| signal to cargo somehow that there are tons of
| invocations of a specific macro and that it might make
| more sense to recompile the proc macro with optimizations
| turned on.
| Gwypaas wrote:
| I have never found this to be the case as long as you are using
| workspaces to split up the compilation units.
| modernerd wrote:
| I class Bevy as a "non trivial" project. It's also an example
| of how to immediately set new Rust devs up with better system
| performance to reduce the problem you mention where compilation
| becomes a bottleneck as a project grows.
|
| https://bevyengine.org/learn/book/getting-started/setup/
|
| The early setup docs offer ways to improve compile speeds. Some
| help all Rust projects (change your linker) and others are
| specific to Bevy (enable dynamic linking if you're not on
| Windows).
|
| Yes, it's a little jarring for an intro to game development
| with Rust to start with, "first, change a bunch of things so
| your compile speeds don't suck by default". But I appreciated
| it because it helps you evaluate Bevy properly (it's as fast as
| it'll get from that point) and it also made my non-game Rust
| project workflows faster.
|
| Bevy also sets expectations well about the initial slow-ish
| build ("This will take some time as you are essentially
| building an engine from scratch. You will only need to do a
| full rebuild once. Every build after this one will be fast!").
| vvanders wrote:
| Yeah Bevy does a fantastic job here(which given the focus on
| iteration time for gamedev makes sense).
|
| With the dynamic link feature it takes about 1s to compile +
| link and Bevy is very much a nontrivial runtime/library that
| makes use of lots of Rust features.
| demindiro wrote:
| > have always found the compiler to be unacceptably slow in
| these cases
|
| How slow is "unacceptably" slow?
|
| Personally I do not find this to be an issue the vast majority
| of the time. Especially in combination with cargo watch which
| can automatically run a check/build/tests/... whenever a change
| is made.
| nicoburns wrote:
| > I have always found the compiler to be unacceptably slow in
| these cases, especially if you come from C rather than C++
|
| I've found this to be highly dependent on the hardware you are
| using. On my 2015 MacBook Pro, Rust was slow enough to be
| annoying. On my new M1 Pro MacBook (which is ~9x faster), it's
| fast enough for me not to notice compile times.
| rapsey wrote:
| Slower compile times are a small price to pay for memory safety
| and much higher productivity.
| saagarjha wrote:
| Slow compile times can reduce productivity.
| rapsey wrote:
| Not nearly as much as chasing bugs that do not happen in
| Rust.
| saagarjha wrote:
| That would depend on what you're doing, of course :)
| rapsey wrote:
| No not at all.
| saagarjha wrote:
| ...perhaps you lack experience of a time where fixing
| bugs that cannot be effectively discovered using tools
| available in other languages took less time than the
| equivalent code in Rust would take to compile?
| rapsey wrote:
| Yet all the tools that exist still don't prevent memory
| safety bugs found regularly in pretty much any C
| codebase. Using safe rust will be a productivity boon
| because of its type system, crate ecosystem and memory
| safety.
| saagarjha wrote:
| > Yet all the tools that exist still don't prevent memory
| safety bugs found regularly in pretty much any C
| codebase.
|
| They can help, but I am well aware of which kinds of bugs
| existing C tooling cannot catch. I still take issue with
| your opinion, which hold through this thread, that claims
| that productivity gains from not having to track these
| down necessarily outweighs Rust's compile times.
| Shinmon wrote:
| Most of the errors can already be caught by running `cargo
| check` which is substantially faster than rust.
|
| Coming from C++ rust's compile times don't feel
| extraordinarily long.
| pjmlp wrote:
| Useless for graphics and GUI programming.
| varajelle wrote:
| For GUI programming, you can use a toolkit that have a
| preview extension that updates in real time as you type.
| For example, Slint does that.
| pjmlp wrote:
| It doesn't do that for Rust code being called from UI.
| saagarjha wrote:
| When I write Rust code most of my errors typically get
| caught by my language server. Unfortunately, this doesn't
| help much with making builds faster, though it may reduce
| the number I may have to do.
| incrudible wrote:
| You can get memory safety and much faster compile times in
| many other languages, so that is not a valid argument on its
| own.
| pjmlp wrote:
| Not an issue on other safer languages, currently on Rust it
| is mostly having the focus elsewhere.
| candiddevmike wrote:
| I get those with Go and I have blazing fast compile times
| too.
| the_gipsy wrote:
| Go isn't very safe and in general is a big mess with
| pointers and those zero-values. It gives you speed, a lot,
| but at a cost.
| rapsey wrote:
| Go is not a C/C++ replacement. Rust is.
| incrudible wrote:
| Rust is its own thing that may share some niches with
| cpp, but it does not replace it. I would really have to
| care about memory safety a lot to pick Rust over Zig (for
| example) as a cpp alternative, but then if I really care
| about memory safety I might pick something other than
| Rust as well. When it comes to performance, you will
| probably end up needing unsafe anyway, which raises the
| question of why you should write an entire application in
| Rust as opposed to mixing a simpler and slower safe
| language with a simpler unsafe language for performance.
| nequo wrote:
| Rob Pike did intend Go to be a C++ replacement for some
| use cases: I was asked a few weeks ago,
| "What was the biggest surprise you encountered rolling
| out Go?" I knew the answer instantly: Although we
| expected C++ programmers to see Go as an alternative,
| instead most Go programmers come from languages like
| Python and Ruby. Very few come from C++. We--
| Ken, Robert and myself--were C++ programmers when we
| designed a new language to solve the problems that we
| thought needed to be solved for the kind of software we
| wrote. It seems almost paradoxical that other C++
| programmers don't seem to care.
|
| Source: https://commandcenter.blogspot.com/2012/06/less-
| is-exponenti...
| incrudible wrote:
| The sort of people who used C++ and thought "if only I
| could have garbage collection!" all moved to Java well
| before Go got a foothold.
| eklavya wrote:
| Really? Do you get compile time deterministic memory
| management while being much more high level than Go?
|
| There are trade-offs, rust makes one, use it if it makes
| sense but these comparisons are just pointless. This whole
| thread is filled with such arguments. Sure use OCaml,
| Haskell, Go whatever suits your needs. But let's not
| pretend that they achieve at compile time what rust does.
|
| Somebody is complaining about compile time, somebody says
| GC alternatives exist. No, if you are getting into Rust you
| pretty much know the trade offs. It's on you if you choose
| Rust somewhere where a GC, REPL providing alternative would
| have been enough. The miracle of Rust to me is that I
| usually find Rust high level enough to not have to bother
| with the alternatives most of the time. I would write my
| UIs in flutter and my web frontends in TS. I am under no
| illusion that a GC language would not be more productive in
| most cases.
| nequo wrote:
| > Sure use OCaml, Haskell, Go whatever suits your needs.
| But let's not pretend that they achieve at compile time
| what rust does.
|
| Are you talking specifically about the memory use of
| rustc vs. ocamlopt? Or do you mean that rustc is faster
| than ocamlopt at compiling a similar program? Or are you
| saying that rustc does more work than ocamlopt because of
| the borrow checker?
| sanderjd wrote:
| I think this is good advice! But YMMV; I've also always found
| the compiler to be slow, but acceptably so for what it's giving
| me.
| neilv wrote:
| Anyone happen to have this in EPUB or PDF format (for an ebook
| reader)?
| shivekkhurana wrote:
| Can Rust make Android apps faster ?
|
| It feels logical because Rust is close to the metal, but I don't
| know enough Android or Rust to be sure.
| pjmlp wrote:
| Android apps are already compiled to native code, being faster
| or slower is a matter of developer skills, and how much they
| paid attention to algorithms and data structures lectures.
| saagarjha wrote:
| Rust can make Android apps faster, but so can Python or Java or
| JavaScript or C++. It's generally not useful to make a
| comparison like this in a vacuum.
| mrtngslr wrote:
| Hi, I wrote the new course. We have been enabling the use of
| Rust in the Android Platform.
|
| Roughly speaking, the Android Platform is the Linux
| distribution running below the Android apps we all know. There
| are a number of daemons running on the system and these are
| often written in C++. We've now made it possible for Android
| engineers to write them in Rust instead (or to link in Rust
| libraries if they want). To do this, the Android build system
| had to be extended with new rules, see
| https://source.android.com/docs/setup/build/rust/building-
| ru....
|
| The overall goal is to have more secure software. Rust removes
| a whole class of possible security vulnerabilities and we
| deploy it to make phones more secure. See
| https://security.googleblog.com/2022/12/memory-safe-language...
| for details on that.
| [deleted]
| 1f60c wrote:
| Is this "just" a 20% project or is this more significant (like
| Android one day supporting Rust to develop apps)?
|
| And I haven't looked at everything yet, but it suggests to
| install Rust like this: sudo apt install cargo
| rust-src
|
| While everyone I know uses rustup^: curl --proto
| '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
|
| ^ https://rustup.rs/
| est31 wrote:
| > Is this "just" a 20% project or is this more significant
| (like Android one day supporting Rust to develop apps)?
|
| Android already uses Rust internally, in fact for new code it's
| a major language. They have made this guide to teach Rust to
| their engineers working on Android, according to this post on
| reddit:
| https://old.reddit.com/r/rust/comments/zrs1of/new_rust_cours...
| ohbtvz wrote:
| I'm tired of installation instructions that consist of "curl |
| sh". Especially since this one just tries to detect the
| platform and downloads the correct installer for me. I know
| what platform I'm running, and I don't want to pipe the
| internet to my shell. In the end this is unpacking a tarball in
| ~/.rustup, I don't need the risk of running some bespoke script
| for that.
| est31 wrote:
| On some distros you can install rustup directly, e.g. NixOS
| or Arch. For Ubuntu, it's sadly not in the default
| repositories, but there is a snap for rustup uploaded by the
| rustup maintainer.
|
| https://pkgs.org/search/?q=rustup
|
| https://snapcraft.io/rustup
| kibwen wrote:
| Since the ultimate objective is to run a binary blob that you
| just downloaded off of the internet, piping a script to your
| shell over HTTPS adds no additional attack surface.
| vbezhenar wrote:
| When I'm installing package from the repository, it's
| signed with GPG. Hopefully in a more secure place than WWW
| server. May be even at offline server with HSM (one can
| hope!). When I'm running code downloaded from HTTPS, all it
| takes is compromising this WWW server (or AWS Cloufront for
| this particular sh.rustup.rs example). HTTPS adds
| additional attack surface.
| jjgreen wrote:
| Actually no, one can detect curl-piping server-side and
| serve hostile blobs only to those foolish enough to do so:
| https://www.idontplaydarts.com/2016/04/detecting-curl-
| pipe-b...
| Arnavion wrote:
| And kibwen's point is that if you're in a situation where
| you're running potentially hostile binaries from a remote
| host, the fact that you used `curl | sh` to obtain said
| binary is not the pressing problem.
| slim wrote:
| also by doing that you don't keep a copy of the script
| for investigation if something smells fishy later
| insanitybit wrote:
| Changing nothing about the relative security of using
| curl | sh.
| kibwen wrote:
| I'm well aware of that attack, it's quite cute, but if
| you don't trust the host not to serve you a backdoored
| binary then you've already lost.
| jjgreen wrote:
| And where is this trust supposed to come from? I
| downloaded the thing manually, looked at the scripts, ran
| the binary in a sandbox, it seemed to be OK. Right, I'll
| recommend that everyone just curl | bash's it ...
|
| I think the worst thing about this is that Rust is
| fashionable, so encouraging inexperienced devs think that
| these dangerous practices are just fine. Look around at
| how many n00b projects now suggest doing exactly the same
| thing. It's simply irresponsible of the Rust crowd to
| keep promoting it.
| pornel wrote:
| The bash timing exploit makes everyone focus just on how
| cleverly evil it can be, and forget the big picture that
| it's about trusting the Rust org not to screw you.
|
| (BTW, you can run `curl | sh` in a VM or with a modified
| bash to intercept the code and catch the bash script in
| the act, so it's not actually as sneaky as people
| believe).
|
| If you think the Rust org is going to pwn you in a clever
| sneaky way, then you can't use Rust or any Rust-
| containing products.
|
| In the end, you're pulling hundreds of MBs of binaries
| that you won't review, they're compiled from over 15
| million lines of code that I don't believe you'd ever
| review either. Reviewing just the first 10 lines of code
| gives you nothing. A smoke test in a sandbox is also
| worthless, since a binary could detect being run that
| way, or delay the attack, or attack by specifically
| miscompiling your code (see Reflections on Trusting
| Trust).
|
| In the end, you have to trust the Rust org, all of it.
| fulafel wrote:
| "Trusting the Rust org not to screw you" is one part,
| another part is trusting the Rust server operators to
| defend against server compromise by any third party. So
| trusting the intentions is not sufficient.
| haradion wrote:
| The same thing applies to any binaries downloaded from
| their site, so unless you you've got signed binaries
| (that use an independently obtained/verified chain of
| trust), trusting the server is your your only option.
| Even with signed binaries, you're still trusting the
| entity that holds the signing key.
| fulafel wrote:
| In real world trust is not so binary. In a risk
| assessment I'd be interested evaluating the level of
| assurance there is in the supply chain of how you get
| your binaries and artifacts. Some of it can be done using
| crypto like you say, some of it could be eg published
| audit reports from a reputable evaluator or other
| credible information about the processes.
| carapace wrote:
| I call this the "Lie back and think of England" stance.
|
| You're not wrong, until the end, it should be: "you have
| to _distrust_ the Rust org, all of it. "
|
| And not just Rust, Python and JS and all the others.
| There are languages and systems that take trust and
| security seriously, but these are not they.
| gernb wrote:
| piping to the shell might be somewhere in the middle but
| node's npx is just asking for trouble. You type `npx cmd`
| and it downloads cmd. Which means any typo could be death
| ohbtvz wrote:
| You can't check the hash of something you immediately
| piping to your shell, for example.
| kibwen wrote:
| Checking the hash isn't relevant here. The content is
| served via HTTPS, you either trust the host or you don't.
| A host can easily serve you a malicious binary, as well
| as the valid hash of that malicious binary.
| posterboy wrote:
| Checking is the only reason that hashes are provided.
| jeroenhd wrote:
| Which is why many people choose to only install software
| from their trusted distro maintainers who add a layer of
| vetting for random software packages, often built from
| source so messing with the package isn't possible without
| leaving some kind of trace that can be detected later.
| kibwen wrote:
| Indeed, by all means, prefer to trust your distro if they
| package a version that's new enough. Alternatively,
| prefer to build from source if you like. But if you trust
| the Rust project to be competent enough and benign enough
| not to include malware in the compiler itself, then it's
| not a stretch to trust their official toolchain juggling
| tool downloaded from their official website. Focusing on
| the curl | bash aspect is a tired meme at this point.
| pornel wrote:
| Your distro's overworked maintainer isn't reviewing 15+
| million lines of code included in Rust.
|
| Most likely they get the precompiled rustc binary just
| like rustup, and LGTM-YOLO the package. If they try to be
| diligent, they maybe take extra 150K lines of mrustc code
| they can't reasonably carefully review for backdoors
| either, and then use it to bootstrap the several sets of
| 15M lines of code they won't look at.
| jeremyjh wrote:
| The one thing you may get in using your distribution is
| protection for the case that the rustup.sh website has
| been temporarily pwned. But I agree that focusing on curl
| | sh is nonsense.
| gpderetta wrote:
| you might have another known-good source of the hash.
| sofixa wrote:
| You meant a binary blob in your distro's repository, so one
| that was checked, tested, approved and verified with a
| hash. Which is wildly different than downloading and
| running random binaries or scripts for that matter off the
| internet.
| kibwen wrote:
| No, it's hardly random, it's an official binary provided
| the Rust project from an official domain managed by the
| Rust project. If you don't trust it, then you shouldn't
| trust the Rust source code either.
| yencabulator wrote:
| In this case, the Google internal course explicitly does
| _not_ trust that random shell script, even if it came
| from rust-lang.org, while their internal apt repository
| _is_ trusted. See
| https://news.ycombinator.com/item?id=34092187
| mhoad wrote:
| I get what you are trying to say here but I could also make
| the argument that you actually doubled it because now you
| have to trust two things rather than one.
|
| Depending on how you want to consider trust in a wider
| sense too it may even be worse than "double" because I do
| not have the same amount of trust for the package I am
| ultimately installing and the script I am using to install
| it.
|
| Edit: it's actually 3 things you need to trust I didn't
| include curl itself which just released a security audit
| that found a number of vulnerabilities here
| https://daniel.haxx.se/blog/2022/12/21/the-2022-curl-
| securit...
|
| ...security is often harder than it looks.
| kibwen wrote:
| _> now you have to trust two things rather than one_
|
| No, you're still trusting one thing: the host itself.
| You're downloading both the script and the binary from
| the host. Both could be backdoored, and of the two, the
| binary is far easier to hide a backdoor in.
|
| As for not trusting curl, you still need to fetch the
| resource _somehow_ , so you're going to be trusting some
| tool to do it for you. That's not relevant to increasing
| the attack surface.
| Someone wrote:
| > You're downloading both the script and the binary from
| the host.
|
| Technically, if you don't read the script, you don't know
| the binary is from the same host.
|
| That doesn't matter, though. The chain of trust is deep,
| including the tooling that produced the binary, your CPU,
| the internet, etc.
|
| Downloading the first file basically says _"I trust this
| site to give me this tool and nothing else"_. Where it
| then gets that stuff from shouldn't matter, even if it is
| from a shady site. You trusted them not to do that, just
| as you trusted them not to open up their own site so that
| hackers can replace files ont it.
| mhoad wrote:
| I'm not actually in the Rust ecosystem at all and only
| just discovered the domain belongs to the official Rust
| project.
|
| That clearly changes the trust calculation in this
| scenario.
|
| I had assumed it was some 3rd party project which would
| have put it in a different category of problems entirely.
|
| But the entire conversation is kind of pointless then.
| "There is a secret backdoor in the official Rust binary"
| is not a useful part of any reasonable threat model.
| dmitriid wrote:
| This is because people are tired of figuring out seventeen
| (hundred) different package managers that may or may not work
| and that may or may not accept your package into their repo?
| mrtngslr wrote:
| To be honest, the course suggests using apt since that's an
| easy security-approved way to install things on our computers
| (see https://en.wikipedia.org/wiki/GLinux).
|
| I'll be happy to update it to suggest using rustup like
| "normal". Could you make a PR for that?
| ohbtvz wrote:
| Are you sure you're replying to the correct comment?
| mrtngslr wrote:
| Sorry, you're right :-) I meant to comment further up
| where people asked if we could reference rustup.rs in the
| installation instructions.
| mrtngslr wrote:
| Actually, there is already an issue for this:
| https://github.com/google/comprehensive-rust/issues/19 and
| I hope someone will fix it soon :-)
| Arnavion wrote:
| https://news.ycombinator.com/item?id=33380755
| zozbot234 wrote:
| Apt gives you the version of Rust that was packaged for your
| distro, so you're not keeping up with the latest releases. It's
| typically configured to reuse the system install of LLVM,
| missing some Rust-specific patches that are in the process of
| flowing upstream. And it references distro-packaged crates by
| default, meaning that again you're not getting the latest
| published versions.
| izacus wrote:
| > Is this "just" a 20% project or is this more significant
| (like Android one day supporting Rust to develop apps)?
|
| It's very unlikely you'll ever see Android supporting apps
| written in 100% Rust (just like it doesn't support apps written
| in 100% C++/C) - the APIs and UI toolkit are exposed and
| written in Java and you'll need to bridge either way.
|
| Supporting Rust as a native code language to augment existing
| support for C/C++ via NDK is much more likely.
| teaearlgraycold wrote:
| I sit next to a guy in Mountain View that works full time just
| on teaching people Rust and integrating Rust into Android.
| There's a whole team dedicated to getting Android developers
| writing Rust.
|
| The difference in installation procedure is probably a security
| measure. Piping curl to bash is a bad move and it's the Android
| security department that's pushing Rust in Android. Plus,
| internally, Google has well maintained aptitude repositories
| with projects built from HEAD. It's nice installing something
| like lldb from the command line and the internal Python
| interpreter is the latest version. And you know all of your
| coworkers are running that version, too.
| mrtngslr wrote:
| Hi, I wrote the course :-) Yes, this is a real thing -- Android
| has had support for Rust in the Android Platform for a few
| years now. Most recently, we shipped support for DNS-over-
| HTTP/3: https://security.googleblog.com/2022/07/dns-over-
| http3-in-an...
|
| We put out the course to make it easy for even more people to
| onboard with Rust.
| carlhjerpe wrote:
| Rust does not guarantee that there are no memory leaks, you
| can have things reference eachother and therefore never count
| to 0. This is why weak pointers exist in rust.
| mrtngslr wrote:
| Right, this is a problem with reference counted data
| structures in general. You're correct that Rust doesn't try
| to solve this particular problem.
|
| Put differently, the borrow checker won't allow you to
| create a cycle of refrences (the & kind). But you can do it
| with library types such as Rc.
| fulafel wrote:
| It's a property of basic RC but real-world RC based GC
| systems can often collect reference cycles. For example
| Python and PHP do this.
|
| (I'm sure you knew this, just adding to the discussion)
| eloff wrote:
| Rust doesn't though. But it does make it much harder to
| leak memory than C. You need to create a cycle between
| reference counted objects versus just forget to free
| something.
|
| Not that it matters as much as people think. Memory leaks
| aren't a correctness problem. You typically want to
| restart a program or service regularly anyway to deal
| with memory fragmentation. You also frequently have a
| watchdog like systemd to restart your program if it
| crashes. Heroku restarts your whole dyno once a day. This
| means in practice that programs with slow memory leaks
| tend to work just fine (there are plenty of them in the
| wild.)
| fulafel wrote:
| Interesting view that memory leaks aren't correctness
| problems. But I would require more convincing to agree.
|
| It's true that memory leaks can be small enough that they
| don't become problems in the end-to-end behaviour of the
| system in regular use. But a lot of bugs are like that.
| For example many memory safety bugs.
| eloff wrote:
| Sorry, I should have said they're not safety problems.
| The commenters below got it right.
|
| Memory safety bugs are very different, because whether or
| not they affect the functioning of your software, they're
| a ticking time bombs that could compromise your system. A
| memory leak will at worst crash your software.
| masklinn wrote:
| > Interesting view that memory leaks aren't correctness
| problems.
|
| The view of rust is rather that they're not _safety_
| problems.
|
| Whether they're _correctness_ problems is more
| complicated: in general they are, but there are lots of
| cases where they're not, like short-running processes
| (once the process terminated it's memory is reclaimed so
| freeing it is unnecessary overhead), or FFI (you're
| moving memory out of your purview, you can't know whether
| it'll be disposed of anymore).
| fulafel wrote:
| Safety and security as terms have an interesting
| relationship especially in Rust context - traditionally
| in engineering safety means defending against accidents
| and security means defending against malicious attackers,
| but "safety" in the term "memory safety" commonly implies
| security as well. Language is hard...
|
| Considing whether memory leaks are a security problem
| brings us to the traditional CIA definition of security -
| the A (availability) is at risk from memory leaks.
| [deleted]
| pkolaczk wrote:
| It does not guarantee, but it is actually quite hard to
| cause a memory leak by accident, because at the same time
| Rust makes creating cycles hard.
| mrtngslr wrote:
| I actually haven't tried this... do you know off the top
| of your head what it takes to make a cycle with Rc?
|
| It should not be possible with normal borrows (and safe
| Rust) since they're statically checked to be acyclic.
| britneybitch wrote:
| You could store a `RefCell<Option<T>>` in an `Rc` and set
| it to `Some(rc)` after creation.
|
| Or you could use the recently-stabilized `new_cyclic`
| method: https://doc.rust-
| lang.org/std/rc/struct.Rc.html#method.new_c...
| jamincan wrote:
| The Rust Book has an example: https://doc.rust-
| lang.org/book/ch15-06-reference-cycles.html
|
| Of course, leaking memory is as easy as using
| `Box::leak`, although that's probably never going to
| happen accidentally.
| bitexploder wrote:
| And when using C libraries / FFI, there are a lot of
| tricky things going in, it is much easier to leak in
| unsafe land.
| junon wrote:
| You're incorrect, sorry. It's very easy to creat cycles
| in inner-mutable data structures (e.g. Rc).
| schipplock wrote:
| They mention rustup on the 3rd slide.
| rob74 wrote:
| They're using Rust in the android _system_ :
| https://security.googleblog.com/2022/12/memory-safe-language...
| eggy wrote:
| I still play with Rust (and Zig), but I have decided to put my
| work efforts into SPARK[1], the subset of Ada, for high-integrity
| software and formal verification. I know AdaCore and Ferrous
| Systems are collaborating in trying to bring a lot of Ada/Spark's
| capabilities to Rust, but this is still going to be some time.
| Ada has a longer legacy in this game.
|
| I am working on safety critical control systems and there is a
| lot to prune from existing work in this area in SPARK and Ada.
|
| This book[2], "Building High Integrity Applications with SPARK"
| really sold me on the benefits. It is a great intro of how SPARK
| was used for the CubeSat program at Vermont Technical College
| (VTC). A CubeSat was successfully launched in 2013. I could not
| find any critical mission software written in Rust yet especially
| that far back.
|
| I found Julia's selection for a flight collision avoidance system
| perplexing[3], but it proves a PLs syntax and evangelism (maybe
| not in this case), can bias a selection of a PL for such tasks.
| People find Ada/SPARK's syntax and methods verbose and similar to
| Pascal or other PLs, however, I was up and running quickly in it
| as opposed to Rust.
|
| You can program to bare metal in SPARK, and the tools are already
| there to provide high-integrity software. The tools for SPARK
| automate most of the verification. Most of Rust's wishlist is
| already in SPARK (and Ada). I hope the Ferrous Systems and
| AdaCore collab bears fruit for Rust, but I have been programming
| since 1978, and I play with many PLs, but I found Rust's initial
| learning curve very off putting and I don't have the time to wait
| for it to mature to SPARK's level. And this is from someone who
| loves J/APL/BQN and Lisp, so I have no issue with different
| programming paradigms. I found Zig much easier to get things done
| from the start, although it does not strive to be a SPARK or
| Rust.
|
| [1] https://www.adacore.com/sparkpro
|
| [2] https://www.amazon.com/Building-High-Integrity-
| Applications-...
|
| [3] https://juliahub.com/case-studies/lincoln-labs/
| shakow wrote:
| > I found Julia's selection for a flight collision avoidance
| system perplexing
|
| From [3]:
|
| > Julia is [...] executable
|
| Indeed, that's a useful feature for a programming language :)
| pjmlp wrote:
| NVidia also went through a similar decision for their high
| security critical firmware.
| galangalalgol wrote:
| People have picked stranger things than julia for safety
| critical control systems. I know people have generated it from
| simulink and have used labview, and it wouldn't surprise me if
| somewhere someone is using an excel macro to control the
| pressure system on a boiler... The world is a scary place.
|
| Reading theough the spark tutorials and examples seems fairly
| similar to the rustlings excercise. I find the ways spark
| prevents problems, like forbidding side effects, is more
| onerous than the borrow checker making me specify a lifetime
| every so often. I'm not saying you are wrong to prefer spark,
| just that people with different backgrounds will find the
| learning curves different. You have been programming longer
| than I've been alive, so you certainly have a different
| perspective. I started with c64 basic then c on the amiga 500,
| then x86 assembler on 386, them turbo pascal, then turbo c++ on
| the 386.
|
| I can't argue that spark isn't more mature for formal
| verification, but it feels like formal verification relies
| heavily on correct contract specifications with no way to
| measure when your contracts are correct. Unit tests and
| coverage at least give me a little feel good that I wrote
| enough tests for my rust.
| ngrilly wrote:
| I started working with functional safety only two years ago,
| and I've been surprised when colleagues expressed that
| Simulink was actually the go-to solution for functional
| safety software where they were working before. The logic
| being that the Simulink toolchain is qualified for functional
| safety, and if you use it in combination with a qualified C
| toolchain and RTOS, then you're good to go, certification-
| wise.
| jfmc wrote:
| My impression when working with people using Simulink is
| that 'safety' is much weaker that for people working on
| formal methods, and certification limited a lot the kind of
| programs that they would write. It made totally sense for
| their domain, but -- as a general practice to write
| software -- it didn't impress me at all. I may be wrong.
| ngrilly wrote:
| I was expecting functional safety standards to require
| the use of formal methods, similar for example to how AWS
| uses TLA+, but I was surprised to discover it was not a
| requirement at all.
| eggy wrote:
| SPARK is expressive and it has a formal specification from
| the get go that makes it easier to stay within the
| requirements and provides automatic verification. GNAT tools
| are pretty cool. I don't think Rust has much of this other
| than its specification. It has momentum and a great
| community, so I think either way it will succeed in many
| endeavors. Like Erlang for distributed computing, I think
| SPARK has carved out and earned its niche, and Rust will not
| soon take over this domain.
|
| For me, Rust seems too complicated at this point. I will
| continue to play and watch it. I use the Helix Editor, which
| I love, and it is written in Rust!
| galangalalgol wrote:
| I'd say most of the truly safety critical control code I
| encounter was generated in simulink, and the remainder is
| all in subsets of c++. Some of that c++ was loosely ported
| from ada (not spark) long ago. So rust wouldn't be taking
| over the domain from spark, it would be taking it over from
| c++ and simulink which took over half of the safety
| critical domain some years ago, many years if you count
| misra c. I would certainly advocate for spark over any of
| those languages, but that is a hard sell as people want
| free open source tooling that is used by google or amazon,
| and if they say no to spark, but yes to rust, well at least
| it isnt c++98 anymore. Or simulink models version
| controlled using folders with dates in the names.
| eggy wrote:
| SPARK being a derivative of Ada, which appeared over 40
| years ago, inherits its legacy, so I would say it is
| older than its 2014 name tag. It was based on the Ada
| 2012 specification. Contracts are relatively new, but it
| has many real world mission critical applications in its
| portfolio other than the CubeSat program I put forth from
| 2013[2]. Rust wasn't even two years old at that point.
|
| If you go back, then assembly was mainly used in mission
| critical software in aerospace, for example the Apollo
| guidance system. To quote an article about NASA
| programmer, Ron Garret about options in 1988[1] and the
| now-famous Lisp troubleshooting from 150 million miles
| away:
|
| "There is Pascal and C and Basic and machine code. And
| that's pretty much it in terms of popular languages. To
| get anything done in any of those languages is just
| really, really hard." The code for most spacecraft ended
| up being written in assembly language.
|
| I want free too, but tooling makes the system, and in
| SPARK2014 it is the tooling, not just the formally
| verified spec. of the PL that makes it really groove.
| Rust has a great build system in Cargo, but it does not
| have 10% of what SPARK2014/Ada/GNAT provide as an
| ecosystem and apps under its belt. I do think there are
| people working on this for Rust (Ferrous Systems with
| AdaCore), but I want to ship a product before that will
| ever happen. Maybe my next project will be Rust or Zig
| with contracts or April (Lisp and APL - my favorite,
| although there is BQN implemented in Rust!)
|
| [1] https://thenewstack.io/nasa-programmer-remembers-
| debugging-l...
|
| [2] https://en.wikipedia.org/wiki/SPARK_(programming_lang
| uage)#I...
| celurian92 wrote:
| what resources can you suggest to learn formal verification and
| implement it to make high integrity fault tolerant services? I
| am quite intrigued by AdaCore. Want to try it out. I dont know
| much about Formal Verification.
| thesuperbigfrog wrote:
| https://en.wikipedia.org/wiki/Formal_verification
|
| AdaCore SPARK Introduction:
| https://learn.adacore.com/courses/intro-to-spark/index.html
|
| https://www.synopsys.com/verification/static-and-formal-
| veri...
| thesuperbigfrog wrote:
| More about Ferrocene, the safety-critical Rust toolchain /
| specification that is in the works:
|
| https://www.adacore.com/ferrocene
|
| https://ferrous-systems.com/ferrocene/
|
| https://spec.ferrocene.dev/
| micronian2 wrote:
| Hi,
|
| Regarding the collaboration between AdaCore and Ferrocene, the
| effort is to produce a Rust _toolset_ that is qualified for
| safety critical usage (e.g. verification of object code that is
| produced, etc). That is _not_ the same as bringing SPARK-like
| language capabilities to Rust. I think too many people get that
| confused.
| bobajeff wrote:
| I've looked a tiny bit into Ada and I like the language. I
| particularly like the pascal-like syntax and the ranges
| feature.
|
| However I love rusts cargo and build system better than Alire.
| As Alire seems to bolted ontop of gprbuild. I also find it
| unfun to go back to forward declaration and definition (ads)
| files.
|
| It also seems to me the rust ecosystem has more libraries in
| things I'm interested in and better support for the platforms
| I'm interested in.
| eggy wrote:
| What libraries are you interested in that Rust has and other
| PLs do not?
|
| What is your main use of Rust? It all depends on what you are
| doing. I use Zig for fun game dev stuff, and not SPARK, but
| for the safety-critical control systems, it's SPARK, not Zig
| or Rust.
| bobajeff wrote:
| I'm using rust to develop a internet client. So I need
| libraries for gui; rust has many to choose from (I'm using
| gtk currently). I see a few that plan on supporting Android
| which I'd like to port to down the line. Of course rust has
| many networking libraries also so I feel covered there.
|
| Safety is fairly important and I like ada/spark for that
| but rust's community seems to be farther along in terms of
| support for Android and looks way more active right now.
| brianzelip wrote:
| Here's a recent podcast episode that looks at Rust and Ada with
| some folks who work on Ada Core, https://rustacean-
| station.org/episode/067-quentin-ochem-flor...
| w4rh4wk5 wrote:
| Why would one use this tutorial over the main https://doc.rust-
| lang.org/book/ ?
|
| Apart from the Android specific additions I mean.
| saagarjha wrote:
| From another comment:
| https://old.reddit.com/r/rust/comments/zrs1of/new_rust_cours...
| w4rh4wk5 wrote:
| Perfect. Thank you.
| mrtngslr wrote:
| Thanks for reposting :-)
|
| In general, I feel that the new course will be useful for
| people who want to teach Rust. If you have a group of
| engineers and you want to teach them Rust, then this is a
| ready-to-go solution. You only need to spend some time
| getting familiar with the material and then you can start
| teaching it. I don't think there was such a resource before.
|
| For self-study, I really like the Rust book and also Rust by
| Example. I've listed a bunch of good resources here:
| https://google.github.io/comprehensive-rust/other-
| resources.....
| saagarjha wrote:
| Based on my personal experience, I think it might also be
| helpful to have an even shorter version tailored to "Rust
| for C++ programmers", which I find to be useful because you
| can mostly just go "ok here's the syntax for basic things,
| here is how lifetimes work and what problems they solve,
| here are the replacements for things you are familiar with
| and why some operations are not available for them". In
| particular, this lets you skim over "What is the stack?
| What is the heap?" that many introductions will spend time
| on.
| warabe wrote:
| It's kind of surprising to see that Rust is so rapidly accepted
| by wide range of developers.
|
| Unfortunately I'm from DataScience field, so I cannot see much
| motivation to learn Rust, but I am considering learning it,
| because language itself seems exciting!
|
| Is there anyone on HN who is from DataScience field like me and
| has learned Rust? It would be much appreciated if you could share
| the experience.
| sharikous wrote:
| We developers love so much to learn computer stuff that we want
| to do it even when we don't need to.
|
| I worked on data science, computational science, some system
| engineering and embedded projects. I tried Rust on all of them
| but it was valuable only for the third.
|
| In data science the focus is more on speed of development and
| as much as Rust is more enjoyable to use than C++, it doesn't
| match scripting languages (namely Python) for the flexibility
| you get, the easiness with which you can adapt to changes and
| the lack of a need to focus too much on machine-related
| details. This is the least suited field for Rust between the
| one I mentioned, in my view.
|
| Computational science needs to focus on algorithms and
| formulas, Rust can hide them a bit too much under "unwrap"s
| "iter"s and so on. Plus still no stellar library support and
| (my) experience with other tools (namely C + OpenMP, Julia,
| C++, numpy...) made me feel using Rust for that was unnecessary
| and slower.
|
| When you are designing systems, however, that's where Rust is a
| complete game changer. Honestly I don't think C++ can stand a
| chance in its current level. Everything, from the package
| system, to the borrow checker, to the safe threading model,
| makes you pity your C++ ancestors for how hard they thought
| this kind of programming had to be. Rust makes it orders of
| magnitude more approachable, saner, and with better results
| too.
|
| Embedded... it's nice but honestly all the dark arts of unsafe
| Rust are not standardized so it doesn't feel as future-proof as
| C already.
| pjmlp wrote:
| Rust still needs to do a lot of catching up in HPC, HFT, GUI,
| game engines, console SDKs, OS drivers on Apple and Microsoft
| OSes, Arduino, LLVM and GCC,...
|
| Rust threading is only safe between threads accessing data
| structures in process own memory. It does nothing for shared
| resources using OS IPC, or external resources shared among
| threads.
|
| It will certainly improve, as hopefully C++ will, even it
| never gets 100% as safe as Rust, the ecosystem has too much
| weight for decades to come.
| adwn wrote:
| > _Rust threading is only safe between threads accessing
| data structures in process own memory._
|
| "Only". Intra-process shared memory is by far the most
| important kind of shared memory. 80% of the time you see
| someone advocating for message-passing multi-process over
| shared memory multi-thread concurrency or parallelism, it's
| because _shared mutable memory_ is incredibly hard to get
| right in just about every mainstream language except for
| Rust.
|
| > _It does nothing for shared resources using OS IPC, or
| external resources shared among threads._
|
| Only partially true. The very same language constructs
| which make shared memory safe can also be used to create
| safe wrappers for other shared resources.
| pjmlp wrote:
| Not in the age of microservices.
|
| Language constructs only help if there are no other
| applications accessing the same resources.
| adwn wrote:
| > _Not in the age of microservices._
|
| There's a _huge_ world outside of microservices.
|
| > _Language constructs only help if there are no other
| applications accessing the same resources._
|
| You're mistaken. Rust's language constructs can often
| help you write safe wrappers around inter-process
| communication and synchronization mechanisms. Of course,
| all bets are off if the other processes don't adhere to
| the agreed-upon IPC protocol, but that's not the point.
| The point is that Rust makes it easier to not
| accidentally screw up the protocol yourself.
| pjmlp wrote:
| That is indeed the point, as most projects don't use a
| single language in isolation.
| vvanders wrote:
| I feel like this was true a few years ago but things have
| changed a lot recently.
|
| In my opinion Bevy may be one of the better ECS
| implementations I've seen, as ex-gamedev it's just very
| well thought through. It's never going to supplant Unity or
| Unreal but that's less of a language issue than just the
| scope of those codebases.
|
| You're seeing a lot more of it showing up in OSes across
| Android, Linux and I've seen a fair bit of movement in the
| windows space as well.
|
| As for IPC, C/C++ doesn't help you their either, that's
| more of a framework level concern. If anything the
| threading model helps when you're doing concurrent IPC
| dispatch(ex: MTA COM or equivalent high throughput IPC) and
| preventing data races.
| rockisch wrote:
| Idk how long ago you tried doing embedded Rust, but nowadays
| it is quite decent. I think I'd choose Rust over C for
| embedded 99% of the time.
| linhns wrote:
| I did. It's a very fun language to learn. If you're comfortable
| with the simplicity of Python, Rust would be too verbose. I'm
| personally fine and actually like Rust more. With the current
| adoption pace it should be competing with Python in the Data
| Science field soon.
| FridgeSeal wrote:
| > Is there anyone on HN who is from DataScience field like me
| and has learned Rust?
|
| Yeah me. I'm data science/data engineering.
|
| Wrote and debugged a lot of R and Python, fair bit of Spark
| too.
|
| Learning Rust was fine. Not trivial, but definitely worthwhile.
| I work on more "data systems" than once-off scripts and
| analysis in my work, so the performance and correctness of Rust
| has proven more valuable to me than the flexibility and
| development speed promises of Python. The former lends itself
| to difficult to maintain, hacky, patchwork code, and the latter
| falls apart after the nth time you have to fix some Python code
| in prod that's failed 3.5 hours into a process, because some
| random package has thrown some as-yet-unseen error.
|
| There's some interesting stuff happening in the data space in
| Rust-polars data-frame library gives you pandas like
| functionality, but with better performance and type checking,
| the data-fusion stuff is making strides as well. There isn't a
| 1:1 replacement for all your ML libs, but it's getting there!
| Linfa and SmartCore are making strides on the sklearn-like
| space.
|
| Would you use Rust to replace your ml stuff in Python at work
| tomorrow? No, probably not just at the moment. Is it worth
| learning because it'll make you a better dev, and give you more
| options and tools the next time you have a non-standard problem
| to solve? Yes, definitely.
| berkes wrote:
| I've had some success with polars.
|
| But: I'm not well versed in Python Pandas either, and my rust
| is better than my python (though Ruby is my daily language, so
| I'm very used to dynamic languages). The thing I miss most is
| the rapid REPL workflow in Python, where I work out the
| pipelines, filters, map-reduce and so forth, on the interactive
| REPL and then extract that into python scripts.
|
| The polars scripts that I worked with, were small enough and
| the builds are incremental, therefore fast: milliseconds. But
| the build-step inbetween is somewhat clumsy when exploring
| data.
|
| But the end-results are just _so_ extremely cleaner, saner and
| especially faster that it certainly is worth it. I worked on a
| dataset where crunching it in python took upwards of 14 hours,
| and the polars under an hour. That feedback loop alone is worth
| days of fighting the borrow-checker. And that immediate
| feedback by the compiler /checker "this column isn't guaranteed
| a String, it could be missing, could be unparsable; deal with
| this _now_ " is priceless. Too often did I have some ETL run
| for 8 hours, and then choke on some column that I swore
| should've been UTF8. Fix it, and re-run. Wasting 8 hours. A
| type-checker in ETL/data flows is priceless.
|
| Edit: to be clear: I am developer. Not datascience. But I do
| need to understand my users, my business, my market.
| Developer/devops today do need to crunch large datasets: at
| least I do.
|
| [1] https://www.pola.rs/
| toyg wrote:
| Rust is a safer C/C++. If you'd never consider writing anything
| in C/C++ (i.e. low-level code where performance is essential),
| you'll never use Rust.
| FridgeSeal wrote:
| That's underselling it quite a lot.
|
| Rust has a number of features (ergonomic and technical) and
| some really nice language design that permits writing quite
| "high-level" code. Personally, having a compiler and type
| system as powerful as Rusts makes it worthwhile alone.
| toyg wrote:
| It's nowhere as ergonomic, for the general population, as
| the likes of Python. References alone add a level of
| bothersomeness to the language that dramatically restricts
| its appeal.
|
| Rust has a great role to play, replacing C and C++ in all
| performance-critical and security-critical underpinnings of
| modern computation (from kernels to UI toolkits). But
| higher up the stack, where those concerns are outweighed by
| overall development costs, other languages will continue to
| rule; and if one's focus is there, learning Rust is
| probably a waste of time.
| bitexploder wrote:
| This. Rust is quite pleasant to use and once some barriers
| are overcome quite productive. Not Python fast dev speed,
| but easily Go levels of productive for me. I rarely fight
| the borrow checker unless building tricky data structures.
| It will still stump me once in a while with lifetimes, but
| the trade off is so worth it to have memory safety
| guarantees and know there are no foot guns like C code
| would have. And if you are just cranking something out you
| can write code a little less idiomatically and of
| efficiently for quick scripts or smaller throwaway CLI
| tools. Rust prefers a more functional style but you can
| just as easily make it not so.
| pjmlp wrote:
| Most of those features are available in ML derived
| languages, without having to deal with Rust features that
| make it usefull C and C++ replacement.
|
| With faster compiler toolchains and REPL like tooling.
| frou_dh wrote:
| The trend is to just assume that using GC is off the
| table, without that being driven by any actual
| requirements or measurements.
| pjmlp wrote:
| The usual cargo cult.
| dagw wrote:
| I'm also in the data science field, and I'm currently teaching
| myself rust as a possible replacement for C and C++ for all
| those times I have to write something fast and novel and that I
| can call from python.
| Lyngbakr wrote:
| That's exactly why our Data Engineering team is also looking
| at Rust, but we're looking at Go, too. I have a hunch that
| Rust is going to be a harder sell to the broader team given
| its learning curve and that most of the team don't already
| have a compiled language in their tool belt. I'm quite open
| to either, though, as they'd both make my pipelines go vroom.
| maleldil wrote:
| Rust has the upper hand for incrementally replacing Python
| modules. With PyO3[0], you can seamlessly replace existing
| Python code with native extensions written in Rust.
|
| If your use case is to replace entire tools, and your team
| isn't familiar with static languages, then Go is probably
| the better option. There's no denying that it's much easier
| to learn.
|
| [0] https://pyo3.rs/
| batterylow wrote:
| I started learning Rust a few years ago, with the intention to
| use it for data science.
|
| I ended up writing a book, "Data Analysis with Rust Notebooks",
| where I demonstrate using Rust + Jupyter Notebooks for tasks
| I'd normally use Python for. It's a good experience, although
| there are some additional limitations when using Rust as a
| notebook kernel.
|
| [1] https://datacrayon.com/shop/product/data-analysis-with-
| rust-...
| sprash wrote:
| It is not rapidly and widely accepted. It is pushed by a small
| but very vocal minority for all the wrong reasons.
|
| When you have to prove your intelligence level (or rather
| "street cred") in a competitive market you have to show that
| you can learn the newest complicated things quickly. Rust is
| pure cryptonite in such a world.
| nicoburns wrote:
| > It is not rapidly and widely accepted.
|
| It has been adopted by most large tech companies and several
| high profile open source projects within ~5 years of it's
| stable release. It certainly has been rapidly and widely
| accepted.
|
| For important projects too: Google is using it in Android,
| Microsoft is using it in Windows, Amazon is using it for
| Lambda, Meta is using it for their source control tool,
| Dropbox is using for their storage backend AND in their
| client apps, Cloudflare is using it for their core proxy,
| etc.
| officialchicken wrote:
| And linux itself is in the process of adding support in the
| kernel.
| azangru wrote:
| Cryptonite? I am... not sure it means what you think it
| means.
| sprash wrote:
| I only know this word from context. And it is actually
| written with "k". So what does it mean? (Context/my
| reasoning: If you have to prove that you are a smart
| programmer you can't stay away from learning Rust. Hence it
| is kryptonite.)
| azangru wrote:
| Kryptonite isn't something you can't stay away from. It
| is something that makes you weak. As it did to Superman
| in the movie. An Achilles' heel.
|
| Merriam-Webster's definition: https://www.merriam-
| webster.com/dictionary/kryptonite
| jamincan wrote:
| You're looking for something like honey that no one can
| resist - kryptonite is Superman's weakness; he avoids it
| at all costs.
| Gwypaas wrote:
| Rust has amazing integration with Python through PyO3 [1] so
| see it like a safe alternative for high performance
| calculations and operations.
|
| The rust data science ecosystem itself is starting to come
| together with projects like Polars [2] (Pandas alternative),
| nalgebra [3], Datafusion [4] and Ballista [5]. Of course
| nowhere near Python but that is hard to beat today.
|
| [1] https://github.com/PyO3/pyo3
|
| [2] https://github.com/pola-rs/polars/
|
| [3] https://github.com/dimforge/nalgebra
|
| [4] https://github.com/apache/arrow-datafusion
|
| [5] https://github.com/apache/arrow-ballista
| wongarsu wrote:
| Also dfdx [1], which is shaping up to become a great neural
| network library (though GPU support is still WIP, give it a
| couple months before production use).
|
| Of course the ecosystem isn't as diverse as python yet, but
| if you're willing to call a bit back and forth between rust
| and python it's great.
|
| 1: https://github.com/coreylowman/dfdx
| danuker wrote:
| I suppose if you're coming from Python, you could use Rust
| (with PyO3) instead of C/C++/Cython in something similar to the
| Numpy/ML lower-level implementations.
|
| But I wish there were a language that solved the two-languages
| problem (fast vs. easy to work with). I find Rust quite
| verbose.
| dagw wrote:
| _But I wish there were a language that solved the two-
| languages problem_
|
| That's the whole pitch of Julia.
| pjmlp wrote:
| Julia, Common Lisp, Scheme, OCaml, Haskell, F#, D,..
| sundarurfriend wrote:
| > But I wish there were a language that solved the two-
| languages problem (fast vs. easy to work with).
|
| That is the motivation and purpose of Julia.
|
| For some people's workflows it already seems to have solved
| the problem, but in general the "easy to work with" part is
| still in progress IMO. Some of it is the (in)famously
| annoying compilation delays, but a large part of it is just a
| tooling and ecosystem thing (both of those sides of the issue
| are being actively worked on).
|
| The difference from C/C++/other low level languages is that
| it's not the language itself that's difficult to work with,
| it's the current tools and workflows that are suboptimal. The
| language is pretty well-designed and fun to work with, so
| there's hope that it can actually solve the two-language
| problem for a wider variety of devs and their needs.
| abiro wrote:
| This is great. I'd recommend the Learn Rust With Entirely Too
| Many Linked Lists [0] tutorial as the next step. It really helps
| you grok the borrow checker.
|
| [0]: https://rust-unofficial.github.io/too-many-lists/index.html
| mrtngslr wrote:
| Yeah, that is a fun tutorial indeed :-D I forgot to add that
| one to the list of https://google.github.io/comprehensive-
| rust/other-resources..... Would you care to open a PR for that?
| nf17 wrote:
| Unrelated: but why does Google use Kotlin(for developing Android
| apps) and Rust when they have a nice little language in Go.
| Genuinely curious: Is Go not a good fit for developing Android
| apps(Is targeting JVM the concern?) or system programming(using
| Rust in Android systems), what does it lack?
| grishka wrote:
| Most system APIs are only exposed in the JVM environment
| without native equivalents. You do need a JVM language to do
| anything meaningful. Either that, or you'll have to use JNI.
| sanderjd wrote:
| My take is that Java -> Kotlin was a very natural progression,
| as they are very similar languages with an identical runtime in
| the jvm. Kotlin is just a better Java, which is incredibly easy
| to adopt into an existing Java ecosystem.
|
| But then they also wanted a more bare metal environment in
| addition to that. So something like C, C++, or Rust.
|
| It's always been unclear whether Go belongs in that list. It
| was conceived or at least positioned as a "systems" language
| like those I listed, and there was noise early on about it
| replacing C++ codebases at Google, but with over a decade of
| hindsight now, I think in practice it's pretty clearly a Java
| replacement rather than a C++ replacement. (I guess there's a
| fun symmetry here in that Java was also conceived as a C++
| replacement and also wasn't.) But Kotlin is just a much more
| natural way to fill the Java replacement niche on Android.
| hedora wrote:
| Not sure why they picked Kotlin (probably JVM interop) or Rust,
| but Go's thread safety is closer to C++ than Rust. I think of
| it as an excellent python replacement, not a C++ replacement.
| [deleted]
| [deleted]
| gernb wrote:
| because some people in charge of android mistakenly believed
| that because some stats claimed Java was the most popular
| language that chosing Java would lead to the most developers
| for android (and then they chose Kotlin later beacuse it's a
| better Java and compatible with the JVM)
|
| IMO, choosing Java was one of the worst decisions they ever
| made. There's a reason why no other platform (PS5, XBox,
| iPhone, Mac, Linux, etc..) don't choose a language like Java.
| Android has so much jank because they chose Java
| optymizer wrote:
| I dont have first hand knowledge, but I've been in the space
| for 10 years and wrote Go, Java, Kotlin, so here is my take.
|
| Kotlin made a huge effort to stay compatible with Java. You can
| call Kotlin code from Java and Java code from Kotlin, so orgs
| could migrate their apps piece by piece. The only real
| incompatibilities that I've experienced were when going from C
| code via JNI into Kotlin. All sorts of really weird issues
| creep up.
|
| There was no boiling the ocean in any of the large apps I
| worked on at Amazon and Meta - just add Kotlin and see how it
| goes, it's a really easy sell to tech VPs. First we added
| Kotlin just for the unit tests, then for small features and
| eventually with Google releasing Jetpack with Kotlin
| extensions, for all new apps and features.
|
| Kotlin also improved upon existing painpoints in Java and
| provided a ton of syntactic sugar for things that were tedious
| in Java. For example, proper get/set syntax for fields, data
| classes with autogenerated equals/hashcode, explicit
| nullability for types, val with 'final' semantics to get rid of
| all the final variables, fields in interfaces, top level
| functions, etc. All the functional stuff in it is nice too.
|
| Go is much leaner than Kotlin. The VM or performance is not an
| issue - Kotlin generates entire classes for simple syntax, to
| make things convenient and compatible, but the bytecode is
| pretty bloated.
|
| I imagine that Google had sufficient resources to make a
| Go<->Java translation layer (though Go would have needed
| generics) but transitioning a huge API like Android is already
| a large effort, and if the language isn't fully compatible with
| the existing code, it would have added an order of magnitude of
| complexity, so Kotlin was a much easier sell.
|
| In my opinion, migrating to Dart would have been more practical
| than Go (as evidenced by Flutter), so thats the real head-
| scratcher. It would have been easy for Java devs to learn Dart
| because the OOP concepts translate directly.
|
| Other factors:
|
| Google already had a relationship with JetBrains because of
| Android Studio.
|
| The Oracle Google lawsuit was happening at the time and I
| speculate that Google needed to show to Oracle they can move
| off of Java. Kotlin happened to be available.
| pjmlp wrote:
| The lawsuit excuse has always been quite a lame one, because
| regardless of Kotlin or Java, the whole Android development
| stack depend on the Java ecosystem for the IDE, Gradle, Maven
| Central based libraries, and Android team never made any
| effort to port anything on top of Kotlin/Native, or even
| improve it.
| omniscient_oce wrote:
| Fantastic timing! I was just going to start looking into running
| Rust on Android
| [deleted]
___________________________________________________________________
(page generated 2022-12-22 23:02 UTC)