[HN Gopher] General Availability of the AWS SDK for Rust
___________________________________________________________________
General Availability of the AWS SDK for Rust
Author : CathalMullan
Score : 136 points
Date : 2023-11-28 14:02 UTC (8 hours ago)
(HTM) web link (aws.amazon.com)
(TXT) w3m dump (aws.amazon.com)
| rusbus wrote:
| Hello! Me and some other folks who work on the Rust SDK will be
| around today answering questions in the comments.
| csomar wrote:
| Will there be any support for Wasm as a target for compilation?
| jupp0r wrote:
| Why would this be specific to the SDK? If the code is Rust
| without any native libraries being linked, WASM should work
| out of the box.
| rusbus wrote:
| Although this is true in theory, in practice you need to be
| very careful when writing code if you want to target WASM.
| One example: `SystemTime::now` will panic on some WASM
| platforms!
| csomar wrote:
| I wish this was true but it isn't.
| rusbus wrote:
| Yep! There are actually examples that use Web Assembly:
| https://github.com/awslabs/aws-sdk-
| rust/tree/main/examples/w...
| csomar wrote:
| So is there an indication/documentation which packages are
| wasm-supported or is this a kind of try and find out type
| of situation?
| belter wrote:
| Thanks for showing up and answering questions. Congratulations
| on the release.
|
| What kind of plans for support of Rust's evolving async
| ecosystem?
|
| Any particular reason why the public roadmap does not show the
| columns similar to "Researching", "We're Working On It" like
| the other similar public AWS Roadmaps? See example for
| Containers: https://github.com/aws/containers-
| roadmap/projects/1
|
| Would be nice to have fully working examples on Github, for
| most common scenarios across most AWS services. This is
| something that historically AWS SDKs have been inconsistent on.
| Just a request not really a question :-)
| rusbus wrote:
| > What kind of plans for support of Rust's evolving async
| ecosystem?
|
| We were hoping async-function-in-trait would land before GA,
| however, we have a plan to add support in a backwards
| compatible way when it's released.
|
| > Any particular reason why the public roadmap does not show
| the columns similar to "Researching", "We're Working On It"
| like the other similar public AWS Roadmaps?
|
| Our roadmap has unfortunately been in a state of disrepair
| for some time. We're hoping to get it cleaned up and accurate
| post GA.
|
| > Would be nice to have fully working examples on Github, for
| most common scenarios across most AWS services. This is
| something that historically AWS SDKs have been inconsistent
| on. Just a request not really a question :-)
|
| There are lots of examples here [1], some simple, some quite
| complex. If there's something you have in mind, please file
| an issue! Having great examples is one of our priorities.
|
| [1]: https://github.com/awslabs/aws-sdk-
| rust/tree/main/examples
| kolektiv wrote:
| No questions from me, just some appreciation and thanks for the
| release. While it is clearly not founded solely on the pure and
| selfless love of AWS for Rust, it is nevertheless very positive
| for the language to have good stable ways to work with major
| platforms. Writing things on AWS in Rust is now a significantly
| easier sell.
|
| Thanks for all the work on this, looking forward to trying a
| few new pieces out!
| habitue wrote:
| Are there any plans to do the same for CDK?
| anentropic wrote:
| Yes, it'd be nice to have a CDK based on Rust with ergonomic
| libs for downstream langs derived from that via FFI (e.g.
| Python/PyO3) instead of the JSII abomination they ended up
| with
|
| I don't hold my breath
| digitalsanctum wrote:
| The blog post mentions support for 300+ services. I have a
| couple of questions:
|
| 1. It would be interesting to see a comparison between the Rust
| service coverage and other language SDKs that have been around
| for a while such as Java. Is there such a place to see this
| comparison? 2. Will the Rust SDK stay up to date with the
| latest services as they're announced?
|
| I'm very excited to see this announcement. It's been a long
| time coming.
| ibotty wrote:
| Disclaimer: I am not working on the SDK nor for Amazon.
|
| As far as I read the code of some AWS SDKs, the SDKs (in most
| languages) are generated from interface files and are thus
| always in-sync and cover the same APIs in every language.
| rusbus wrote:
| The Rust SDK is built on top of the smithy-rs code generator.
| On the service coverage front, you'll find nearly 100% parity
| --There are some legacy APIs that aren't supported. It also
| doesn't have many "high level libraries" (e.g. S3 transfer
| manager) that can find for other languages.
|
| New services will come out the same day as all other SDKs-All
| SDKs utilize the same automated system to deploy new
| releases.
|
| The only exception is services which require extensive custom
| code. We're still catching up on those for the Rust SDK.
| studmuffin650 wrote:
| As a follow-on question, is there plans to integrate the
| AWS CRT under the hood or does the existence of Tokio and
| async/await negate the need for it?
| necubi wrote:
| Thanks for your work on this!
|
| Are there plans to improve the compilation times? Aws sdk
| crates are some of the slowest dependencies in our build--which
| feels odd for what are basically wrappers for http clients.
| rusbus wrote:
| It's on our radar--one of the biggest issues is that some of
| the services like EC2 are absolutely massive. We're
| investigating ways for customers to only compile the
| operations they need, etc.
| evntdrvn wrote:
| thanks, and also thanks for everyone's work on the SDK!
| It's very exciting for it to hit GA :)
| iofiiiiiiiii wrote:
| What are the differences in the design principles of the AWS
| Rust SDK compared to AWS SDKs of other languages? In what ways
| is it special to work best with the Rust ecosystem?
| rusbus wrote:
| Probably the biggest one is "batteries included but
| replaceable." The Rust ecosystem is still maturing, so we did
| a lot of work to make reasonable default choices but still
| allow customers to make different ones.
|
| Some of our other design tenets are here: https://smithy-
| lang.github.io/smithy-rs/design/tenets.html
| say_it_as_it_is wrote:
| What are some valid reasons why people wouldn't now use these
| rust libraries and extend them to their preferred language?
| Maintaining clients is tedious work and prone to abandonment.
| rusbus wrote:
| Not all languages have a great interop story with Rust. Binding
| the JNI is especially tricky, for example. Furthermore, when
| performance isn't important, the need to package and compile
| Rust code may be an unnecessary hassle.
| ufmace wrote:
| I would expect AWS to provide custom libraries for basically
| every language. The cost of a few full-time engineers who are
| experts at any reasonably popular language is probably pocket
| change compared to how much even a few companies using that
| language might spend on AWS services.
| marcinzm wrote:
| Black boxes tend to be problematic to debug and if you're
| wrapping code in a language you don't know then it's a black
| box.
| Aeolun wrote:
| Sounds like that must have been an absolute bitch to build. I
| cannot imagine re-implementing 300 existing API's for Rust from
| scratch...
|
| On the plus side, I guess the work lends itself well to
| parallelization.
| maxwellg wrote:
| A good chunk of the AWS SDKs are codegen'd - it's the only
| possible way to support the sheer volume of stuff they need.
| andrewaylett wrote:
| As with all the other AWS SDKs, the bulk of the code is
| generated. The JSON service definitions are shared, the effort
| (one expects) is in being adding support for all the different
| ways in which the JSON indicates that services behave, and
| making it _look_ like it _could_ have been hand-written.
| ethagnawl wrote:
| I'm a Rust beginner, so please excuse any naivete herein: Does
| this SDK _necessarily_ require an async runtime or is it possible
| to use it in a traditional sync application using whatever extra
| facilities (e.g. block_on) which would be required to "normalize"
| it?
| ursuscamp wrote:
| You can use tokio's block_on to sync-ify. You need to
| instantiate a runtime, but you don't need to do run your whole
| application in it, just the Future.
|
| edit: Tokio can be beefy. You might look at some of the smaller
| single-threaded runtimes to execute your future in the main
| application thread if you're only concerned about serial
| execution.
| ethagnawl wrote:
| Thanks. To further clarify, the SDK can be used from within a
| Tokio runtime or using Tokio's facilities in a synchronous
| runtime. Can other async runtimes be used? (The linked post
| seems to imply that they can.) It looks like Tokio gets
| installed as a dependency and I see the following when trying
| to use the futures package:
|
| > thread 'main' panicked at /home/dev/.cargo/registry/src/ind
| ex.crates.io-6f17d22bba15001f/aws-smithy-
| async-1.0.2/src/rt/sleep.rs:128:20: there is no reactor
| running, must be called from the context of a Tokio 1.x
| runtime
| rusbus wrote:
| if you use other Async runtimes, you need to "wire them
| up", in this case by providing a "sleep" implementation.
| I'd strongly recommend using Tokio, especially if you're a
| beginner. I think the "beefy" statements are not
| necessarily accurate. You can use it as a single-threaded
| runtime if you want. Tokio is not going to have a
| significant impact on your compile times or binary size
| (given you're already using the SDK!)
| jon_richards wrote:
| One thing I love about block_on is that it has a dedicated
| threadpool with a ton of threads. For async code, you want
| around as many threads as cores so the thread can run at full
| speed and have the scheduler handle switching, but for
| block_on, most of the time is sleeping, so the core can just
| switch between them all and take care of any that are done
| sleeping. Just don't use it for CPU intensive tasks.
| insanitybit wrote:
| Huge. I've told people before that the SDK was basically stable,
| but the fact that it wasn't 1.0 was still concerning for them.
| jon_richards wrote:
| To be fair, seeing the big "DO NOT USE IN PRODUCTION" every
| time you view the docs doesn't inspire confidence, lol
| cebert wrote:
| I attended a re:Invent session yesterday on using Rust as a
| Lambda runtime. The potential performance improvements,
| especially with limited memory, was quite compelling. I'm looking
| forward to trying this SDK out with Rust Lambdas.
| jon_richards wrote:
| I've been testing this for a while!
|
| One thing I sorely missed was workers for consuming SQS messages.
| Ended up having an intern adapt a worker for the old community
| AWS SDK (rusoto) into this: https://github.com/Landeed/sqs_worker
|
| Also on my dream list of features: gRPC support for Lambda.
| nailer wrote:
| Hah that reminds me of a decade or so ago - there was an entire
| unofficial node SDK before the official one came out. The
| unofficial one still supported a bunch of features outside the
| main one for a while.
| Joeboy wrote:
| I realize this is a "how long is a piece of string" question, but
| I'm wondering what cost benefits you might realistically see from
| moving lambdas from Python to a faster language like Rust? You
| pay (partly) for execution time so I guess you should see some
| savings, but I'm wondering how that works out in practice. Worth
| it?
| steveklabnik wrote:
| Here's a fun answer to that question: Rubygems saved infinity
| money. That is, they got resource usage down to the point where
| they could move to the free tier.
|
| * https://andre.arko.net/2018/10/25/parsing-
| logs-230x-faster-w...
|
| * https://andre.arko.net/2019/01/11/parsing-logs-faster-
| with-r...
|
| (obviously most people will not realize infinity money)
| odyssey7 wrote:
| This paper is not about lambdas and their typical operations
| specifically, but it shows that across a variety of tasks, as
| of 2017, Rust is more environmentally friendly than Python.
|
| https://greenlab.di.uminho.pt/wp-content/uploads/2017/09/pap...
| steveklabnik wrote:
| I am gonna be honest: I hope this paper never gets cited by
| anyone, ever. There's a number of very weird issues about it,
| but I don't think what it _actually_ shows is demonstrative
| of reality, even if it happens to show Rust in a good light.
| odyssey7 wrote:
| I'd love to know more. Anything you can point to?
| steveklabnik wrote:
| A couple of things off the top of my head:
|
| The title conflates languages and their implementations.
| Different implementations prioritize different things.
| They occasionally do test different implementations, as
| in the main Ruby distribution vs JRuby, but it is still
| annoying.
|
| The second, and I think largest issue, is that they chose
| the Language Benchmarks Game as the set of sample
| programs to test. I do not believe that the kinds of
| programs in the Language Benchmarks Game are
| representative of the broader set of software written in
| most languages. They tend towards math-y, puzzle-style
| programs, and not CLIs, web applications, GUIs, or
| anything else.
|
| A very specific issue I have is that Typescript and
| JavaScript are _very_ different in their analysis, and
| that 's very confusing to me, given that all JavaScript
| is valid TypeScript, and you would execute it in the same
| way. This may be an artifact of issue #2, which is that
| the benchmarks game is only as good as the people who
| wrote the programs, and it's quite possible that the
| folks who submitted the TypeScript code didn't do as much
| perf work as the JavaScript code, but it is still a
| confusing result that's not explained anywhere in the
| paper.
|
| A final one (and this is the one I remember least well,
| so I may be wrong here) is that it is not reproducible.
| They do not mention which date they retrieved the
| programs from the Benchmarks Game, let alone the source
| code of the program, nor released the scripts that were
| used to collect the data, though they describe them. This
| means that these discrepancies are hard to actually
| investigate, and makes the results lower quality than if
| we were able to independently verify the results, let
| alone update them based on what has changed since 2017,
| which is an increasingly long time ago.
|
| In short, I do not think this paper is literally useless,
| though I think that it does not actually demonstrate its
| central claim very well, and is difficult to evaluate the
| actual quality of the results, making it a far weaker
| result than the title would suggest.
| odyssey7 wrote:
| Thanks for writing all of that out! I appreciate the
| analysis.
| Matthias247 wrote:
| Instrument your python code and gather metrics. Maybe use a
| profiler. If it is heavily CPU limited and it spends all time
| in python interpreter calls it might benefit from moving to a
| more efficient language. It it's mostly waiting on IO (eg
| remote services) it might be a negligible difference.
___________________________________________________________________
(page generated 2023-11-28 23:01 UTC)