[HN Gopher] Rust - A hard decision pays off
___________________________________________________________________
Rust - A hard decision pays off
Author : muizelaar
Score : 379 points
Date : 2022-08-24 12:50 UTC (10 hours ago)
(HTM) web link (www.pinecone.io)
(TXT) w3m dump (www.pinecone.io)
| 0xbeefeed wrote:
| amelius wrote:
| Nice to see that Rust is working for your particular usecase!
| throwaway642012 wrote:
| I don't want to be dismissive but how much of this success
| belongs to the team of rock solid senior engineers with enough
| experience under their belt rather than rewrite in Rust?
| hawk_ wrote:
| I would let the same team of "rock solid senior engineers" be
| the judge of that. They seem to think it mattered significantly
| enough.
| amelius wrote:
| Or ... since they are now betting fully on Rust, they now
| want everybody else on Rust too. Because more people == a
| higher probability that if they run into issues with Rust
| someone will fix it.
| farresito wrote:
| That's a really cynical take on the matter...
| robertlagrant wrote:
| The writer seems to give massive credit to the team.
| [deleted]
| [deleted]
| runevault wrote:
| You can give some credit to the fact they were writing it a
| second time certainly, but as far as we know at least a lot of
| the people were the same ones who were central to writing V1.
| So certainly there was some lift from already doing it once,
| but if they went from a lot of subtle bugs they struggled to
| track down to having far fewer of them, and those bugs tended
| to be the types of issue the rust compiler and/or linter
| normally point out to you, it will certainly make it easier to
| mitigate the issues.
|
| Rust cannot make a bad programmer a good programmer, but the
| fact it points out several classes of mistakes (sometimes too
| aggressively, hopefully someday we get Polonia and other
| improvements to the borrow checker to remove some things that
| are hard only because of the current implementation of various
| checking systems) allows far more confidence. I don't care how
| good someone is, having an automated way to validate large
| scales of issues aren't there is a big deal, and unlike
| unit/integration tests you don't have to write these.
| sidlls wrote:
| It's likely they could have chosen to stay with Python, C,
| and C++ and achieved similar results. I'd bet a pretty penny
| that the fact they were building on lessons learned with the
| rewrite accounts for almost 100% of the success here.
| jokethrowaway wrote:
| A team with no rust experience
|
| This mirrors my experience: getting buggy software out is
| easier and faster in JS (or whatever you're familiar without
| compile guarantees), getting something out to production is
| easier with rust.
|
| Sometimes you have enough senior developers and discipline to
| reproduce and catch obscure bugs (which happen more or less
| based on your software complexity) and you can fix your mess
| but that's not always feasible.
| KerrAvon wrote:
| The more senior I get (decades of experience now), the more I
| appreciate languages that simply remove the mere possibility
| of entire classes of bugs. Even in Objective-C -- a dynamic
| language with compile-time type enforcement -- if something
| very weird is happening you have to start with "is that
| object really what I think it is?"
| haskellandchill wrote:
| so how are graphs used for vector search?
| gk1 wrote:
| See: https://www.pinecone.io/learn/hnsw/
|
| Although our graph-based vector index does _not_ use HNSW, the
| concept is similar.
| lucidguppy wrote:
| Python has built in testing and C++ has libraries like catch2. I
| would have liked some initial comment that said "we had a good
| suite of tests". Python has plenty of benchmarking tools
| available.
|
| IMHO - a db is all core. Everything should be fast. Even the CLI
| tools - Rust is pretty good at cli tools. Why is he building a
| database out of python? How much python is in this repo anyway?
|
| Also this is a Python + C++ app - not python. And the article
| critiques python, not c++. Which is weird, because I would have
| thought the source of problems would have been C++ not python.
|
| Don't get me wrong - Rust is great. Cargo is great too. Compiled
| programs are great to deploy rather than python (pip hell) and
| C++ ({CMAKE_HELL}). But this is apples to oranges comparison.
|
| The things that make Rust better than C++ / Python are not the
| things this article talks about.
| mhaberl wrote:
| > ..complex runtime issues which were almost impossible to
| reproduce or isolate
|
| > That's when internal murmurs about a complete rewrite started
| brewing...
|
| > We decided to move our entire codebase to Rust
|
| > there was still one minor problem - no one on the team knew
| Rust
|
| "We have runtime issues so we will fix them by a complete rewrite
| in a language that no one on the team knows."
|
| I would not call that a "hard decision", rather "rolling the
| dice". It awesome that they managed to make it work.
| moonchrome wrote:
| I mean when the tools you know are failing you, you might as
| well roll the dice.
| remram wrote:
| They could have had some team members learn Rust _before
| deciding_ that Rust was the right language for the re-write.
| It feels really risky to do such an important decision only
| on a high-level view of what you think the language will
| provide.
| draw_down wrote:
| zaphar wrote:
| If you read it carefully you'll see that they "rewrote" it by
| replacing components of it. Which:
|
| 1. Allowed them to evaluate if it would actually work. 2.
| Allowed them to train the team on it in a scalable way.
|
| The result is that the team learned Rust and Rust fixed a bunch
| of problems. They were appropriately pragmatic in their
| approach and Rust was appropriately pragmatic in it's solutions
| to their problems. If you are ever considering migrating your
| codebase for some reason this is how you should go about it.
|
| TLDR: You are heavily mis-characterizing their team and this
| article.
| brabel wrote:
| Yeah, I am really surprised they managed to make it work,
| actually.
|
| Rust is one of those languages it takes time to master... if
| you write a lot of code in it while you're learning, you're
| likely to make all sorts of mistakes that later will need to be
| cleaned up... at least, looking at Rust code I wrote when I was
| early in the learning process, I can't even imagine any of that
| code going to production... so I would say it worked for them
| either because they are highly skilled and can learn difficult
| things like Rust quickly and efficiently... or their previous
| code was so completely broken than anything else would be
| better :D...
| jasim wrote:
| As a relative newcomer to Rust, I'm curious to hear about
| those early mistakes
| stouset wrote:
| Rust is very good at telling you, the developer, that
| you've got some flaws in the way you've laid out your
| program.
|
| What it _doesn 't_ tell you is that this is frequently a
| result of having a design that doesn't naturally fit into
| the kind of more restrictive paradigm that Rust wants you
| to write programs in. This is a natural consequence of
| moving from very forgiving languages where almost anything
| goes (e.g., garbage-collected languages). You'll try to
| write your software using paradigms that are familiar to
| you, and many of those don't fit well into the set of
| restrictions Rust forces upon you.
|
| There are escape hatches that let you get around these
| limitations, and people become very accustomed to using
| them early on. This happens because the Rust compiler tells
| you what's wrong and that you can use one of these escape
| hatches to fix it. But as you continue to build, you have
| to use more and more of these to overcome the mismatch in
| your design and the kinds of designs that fit well into
| Rust's ownership model.
|
| I believe it's very important to--early in your Rust career
| --try and understand if and when the compiler is trying to
| tell you about a deeper problem with your design than
| simply throwing surface-level ownership nits at you. I
| think for some people who are very used to RAII in C++ or
| very strict C styles, this can come somewhat easily as a
| lot of the patterns are common (though not enforced at
| compile-time as in Rust). It is much more difficult for
| people who are used to garbage-collected languages that
| tolerate program designs that are entirely unsuitable for
| languages without a GC.
| bluejekyll wrote:
| I think of rolling the dice as taking a chance without any
| knowledge of the chance for it paying of.
|
| Rust fit the need they had pretty well (on paper anyway since
| they didn't have experience with it), and the industry is
| showing it to be a good choice for solving these types of
| problems. That's not rolling the dice, it's taking a chance
| based on research.
| theptip wrote:
| > I think of rolling the dice as taking a chance without any
| knowledge of the chance for it paying of.
|
| Rolling the dice just means relying on getting lucky (instead
| of being skillful or controlling the outcome), usually on an
| unlikely outcome. When you literally roll the dice the odds
| are completely transparent (1/6 chance of each d6 value).
|
| In this case (and in general with big rewrites), the thing
| you can never know about up-front - and therefore are relying
| on luck for - is whether you can build the new version within
| the time/resource budget that you allocate to it. If you get
| the estimation wrong, or hit unexpected problems/delays, then
| the rewrite can easily balloon from an ROI-positive project
| to a massively ROI-negative one. For a startup, this is
| particularly important because you could run out of runway
| before your rewrite pays off. This is why most people regard
| it to be a bad idea (or at least a risky one) to do a full
| rewrite of a product instead of incrementally evolving it.
| (Spolsky's essay on the subject I think is the canonical list
| of concerns with a rewrite:
| https://www.joelonsoftware.com/2000/04/06/things-you-
| should-...)
| meragrin_ wrote:
| > I think of rolling the dice as taking a chance without any
| knowledge of the chance for it paying of.
|
| I think hoping a Rust rewrite would fix "complex runtime
| issues which were almost impossible to reproduce or isolate"
| as rolling the dice. If you cannot reproduce or isolate an
| issue, how can you know a rewrite in another language will
| fix the issue?
| mannykannot wrote:
| "Almost impossible" is not quite "impossible", and once you
| have done the almost impossible a few times and isolated
| the cause of some of the problems, you may have gained some
| apprehension of the scale of the overall problem you are
| facing.
| P5fRxh5kUvp2th wrote:
| I read it as Rust forced the developers to be specific with
| their implementations and problems went away as a result.
|
| I say this because I'm often _AMAZED_ at just how undisciplined
| the majority of developers I work with are wrt stability and
| error propogation.
| nu11ptr wrote:
| This doesn't surprise me and matches my own experience. Rust
| literally makes a codebase nearly void of most bug classes with
| the exception of logic bugs (Unfortunately, in a huge codebase,
| there can still be tons and tons of logic bugs).
|
| Still, when I migrated my Python codebase to Rust I got rid of
| whole classes of bugs and honestly code faster in Rust on a "per
| debugged line of code" basis. In Python, every line MUST be run
| or could trigger an exception (and so many do and it is hard to
| see). This now requires a test harness that tests every single
| line which is a huge pain. In Rust, the things that can obviously
| panic are pretty easy to see and are limited (unwrap, expect,
| indexing, etc.) making it much easier to write robust code, even
| with a more limited test suite.
| wooque wrote:
| >with the exception of logic bugs
|
| this is one big exception, because logical bugs are most common
| and most problematic.
| wazzaps wrote:
| If your main concern is security bugs, then according to
| Microsoft[0] about 70% of their bugs were memory-safety
| related.
|
| [0]: https://msrc-blog.microsoft.com/2019/07/22/why-rust-for-
| safe...
| wooque wrote:
| Most people don't write operating systems or write software
| in C/C++, so memory safety bugs are not big concern.
| jon_richards wrote:
| I strongly disagree. The vast majority of bugs I see are
| unhandled None or exceptions and typos in dictionary keys and
| class attributes.
| AndyMcConachie wrote:
| I find exception handling in Python to be one of its weakest
| spots. I really like Python and I use it often. But I'm never
| quite sure which exceptions a library function might throw.
| This is rarely documented well and so you end up encountering
| new exceptions at run time, which is exactly when you do not
| want to be encountering new thrown exceptions.
|
| Usually I end up looking at source of libraries and making a
| list of all exceptions that can be thrown. But of course this
| is also terribly error prone.
| nu11ptr wrote:
| I think unchecked exceptions in Python probably make sense. I
| think the real issue is people trying to use Python to write
| large systems instead of scripts. I love python for 100-500
| line scripts, but beyond that the ease of use features get in
| the way of writing robust code.
|
| However, I think unchecked exceptions are a mistake in any
| language for writing _serious_ code. Interestingly, everyone
| hated checked exceptions in java so now everyone makes new
| exceptions unchecked. I would say this is not because people
| disliked the idea of ensuring errors were handled, but the
| verbosity required to do so meaningfully was annoying, so
| people took shortcuts to avoid pain by wrapping in unchecked
| exceptions and in the process introduced "exceptions gone
| wild" (and now everyone who runs a known java app is familiar
| with the exception stack trace in logs and in the console).
| brightball wrote:
| That's why Elixir developers like to make NIF callouts with
| Rust. It's dangerous to do anything outside of the BEAM but
| Rust negates all of those concerns.
| johnisgood wrote:
| Is it possible to use Ada instead of Rust?
| brightball wrote:
| I believe you can made a NIF with just about anything.
| johnisgood wrote:
| I wonder if there are any examples though. Would be
| helpful. I suppose I could just make Ada bindings.
| Xorlev wrote:
| Maybe not all concerns, you're still allocating memory
| outside of BEAM and you still need to be careful to yield
| control on longer-running NIFs.
| s17n wrote:
| The unique safety features of Rus are all about memory safety,
| which is not something you have to worry about at all in
| Python. I can only assume that the bug classes you are
| referring to would have been eliminated by using essentially
| any language with a type system.
| charrondev wrote:
| I find Rust to have a particularly expressive type system,
| and a quite strict one to. Not all languages with type
| systems such have strict enforcement of mutability or null
| safety.
| stouset wrote:
| > I can only assume that the bug classes you are referring to
| would have been eliminated by using essentially any language
| with a type system.
|
| That's not entirely true. As an easy example, Go has a type
| system but it's relatively anaemic and recreates the "billion
| dollar mistake" of nil. I have personal experience with Go
| applications that have broken in production due to nil
| dereferences, to bugs related to type-switching off
| `interface{}`, and to the language's inability to enforce
| exhaustive case statements in general.
|
| While the _unique_ safety features in Rust are generally
| about memory safety, it still takes a much more safety-
| conscious approach in general than most languages that are
| statically typed.
| nu11ptr wrote:
| To some extent, possibly, but it is a very rich type system,
| and does not have null which in and of itself gets rid of a
| set of errors. I also find Rust culture tends to be more
| guarded on panicking due to having such a rich error type
| (Result), so panic is reserved for the worst possible "can't
| continue" scenarios (unlike languages with unchecked
| exceptions). Also, enum pattern matching is exhaustive which
| gets rid of errors where you simply don't account for new
| variants. The list goes on and on.
| masklinn wrote:
| > I can only assume that the bug classes you are referring to
| would have been eliminated by using essentially any language
| with a type system.
|
| Rust has a very expressive type system[0], which you won't
| otherwise find before hitting the more functional and
| research-y side of things. Modelling data in terms of enums
| (sum types), move types (affine types), etc... makes it a lot
| more reliable and a lot less faillible as it just removes
| entire swathes of edge cases. See concepts like "making
| invalid states unrepresentable".
|
| [0] though it's still lacking in many ways, the more you get
| the more you want after all
| jmartin2683 wrote:
| This is all directly in line with my experience having ported
| several production applications to Rust.
| ramoz wrote:
| > Simultaneous fetches of thousands (sometimes tens of thousands)
| of objects started becoming inefficient, especially when fetching
| from collections of tens of millions of objects
|
| Why not try leveldb. We're doing random reads of 170,000 vectors
| (/130M) a second. No startup needed.
| benreesman wrote:
| RocksDB started as a fork of LevelDB (Dean/Ghemawat).
|
| The performance imperatives have changed with the hardware, but
| NAND flash at that time had an asymmetry between reads and
| writes in terms of the amount of data: once you were writing
| even one byte you had effectively paid for writing a whole
| block and were therefore incentivized to get your money's worth
| and write to a "log", which then would be "merged", in some
| "structured" way, hence LSM.
|
| This is old news these days but it was quite the novelty at the
| time!
| pas wrote:
| Isn't RocksDB a fork (reimplementation) of LevelDB? :o
| bfrog wrote:
| This very closely matches my own experience nearly 4 or 5 years
| ago now rewriting an ad server in rust from another scripty
| language.
| api wrote:
| We're rusting at the moment too. Encouraging. :)
| t43562 wrote:
| Writing something for the second time is a significantly
| different experience so that makes the conclusion much less
| definitive to me. Did they need python at all the first time -
| maybe not.
|
| Python "velocity" is, to me, more about the ease with which you
| can make changes without breaking other code. You can still break
| other code and if you don't have a proper test suite then you're
| wasting your time - which is something that smells slightly in
| the story.
| jnwatson wrote:
| Yeah. Having runtime errors in a non-trivial Python code base
| is indicative of poor test coverage.
| obviouslynotme wrote:
| This is my feeling as well. The biggest problem with new
| projects is that you don't know what the product will look like
| in advance, even if you think you do. Restructuring code,
| especially with full 20/20 hindsight, will make that code much
| better.
| benreesman wrote:
| So I was one of the 3 hackers who wrote the LSM that predated
| Rocks at FB, and as a result I distinctly remember when Rocks
| overtook it in features by trading off performance, and again
| when it met and exceeded the performance while keeping the
| greater capability. RocksDB kicks ass and has for a long time,
| I'd recommend it to anyone in the market for that sort of thing.
|
| FAISS is also great, and has been for years. I don't doubt that
| someone has done a meaningful rev on it.
|
| But as someone who likes Rust and wants to like it more: don't
| lead with that. The software speaks for itself. Or should.
| bluejekyll wrote:
| > But as someone who likes Rust and wants to like it more:
| don't lead with that.
|
| The original title appears to be "Inside the Pinecone", but
| seeing as they submitted it themselves I'm guessing they wanted
| the uptick on the Rust.
|
| The article itself doesn't discuss their transition to Rust
| until the end. So I wouldn't say they led with it originally,
| but perhaps didn't get traction on the original title.
| benreesman wrote:
| I guess I'm rooting for Rust in the long game sense, which is
| a different set of imperatives than the hang the hashtag on
| peripheral stuff sense.
|
| I use a lot of great software written in Rust, it's
| demonstrably a good vehicle for great software.
|
| But too many of its fans are _advocates_ , this can start to
| seem like an agenda. Don't take engineering advice from
| people with an agenda.
| mwcampbell wrote:
| I don't believe I've been an over-the-top Rust advocate so
| far. But I use Rust, I like it so far, and I want to use it
| more. For me to be able to use it more, it would help if it
| had the good attributes of a popular language, including a
| broad ecosystem of libraries and relative ease of hiring
| developers. So I want Rust to be a popular language. That
| means winning at the short game. Maybe other Rust advocates
| have a similar motivation to mine.
| mxkopy wrote:
| Thank you for pointing this out. Lately I've been getting a
| culty feeling around Rust which has been turning me off to
| learning it.
| fkarg wrote:
| but it's the same with every hype: there's something at
| its core and might be worth checking out
| benreesman wrote:
| Oh I think a hype like this comes along once or twice in
| a career. Java was every bit as revolutionary in its day
| and had Sun Microsystems (they were a Big Deal) pouring
| every spare dollar into fluff pieces about it and it
| still never achieved a true Jehovah's Witness vibe.
| [deleted]
| mwcampbell wrote:
| Well, Java had at least one aspect of a religion: a
| purity test. Remember "100% Pure Java" from the late 90s?
| tialaramex wrote:
| 100% Pure Java is because Java is actually about four
| things. One of those things is a Virtual Machine, which
| means you don't give a shit that the implementer has
| never seen an ARM CPU, because you've got the Java VM,
| and so their code works fine on your ARM CPU.
|
| But if their "Java" software is actually 10% i686 Machine
| Code then that won't run on your ARM CPU, you will need
| to undertake an expensive port or move to Intel. So hence
| 100% Pure Java == Write Once / Run Anywhere actually
| delivered.
|
| You can argue that Rust is more than one thing, but
| certainly not to same extent. Rust's culture is crucial
| to its "A language empowering everyone..." slogan, but
| it's not really separate from the programming language,
| in the way that the Java Virtual Machine doesn't need the
| Java programming language or the Java security model (now
| obsolete) or the Java browser plugin (remember that?) and
| so on.
| stouset wrote:
| IMO that "culty feeling" you're getting is the end result
| of a lot of very experienced engineers encountering Rust,
| going "holy shit", and trying to preach about it from the
| rooftops because it feels _so much better_ than the
| things that have come before it.
|
| I find this particularly compelling because Rust
| generally does not market itself towards novice
| developers. At least until very recently, I've seen Rust
| communities actively discourage novices from trying to
| learn Rust. Not that it's inherently a bad language to
| learn as one's first, but there just haven't been great
| resources for people to learn it if they don't already
| have some previous software experience. The net effect of
| this is that the enthusiasm you're seeing is coming from
| a much higher proportion than usual of people who've been
| in the industry awhile, and not from as many fresh
| college or bootcamp grads.
| dureuill wrote:
| I mean, it's only natural.
|
| I have been using C++, Python and Rust professionally, the
| developer experience is simply one order of magnitude
| better with Rust. The tooling is excellent, a sane
| compilation model brings you a lot, the type system is very
| helpful. Compared with C++, I _measured_ x3 productivity on
| writing initial code on a project (sometimes C++ was the
| rewrite, sometimes Rust was, so not this kind of bias).
| Compounding maintenance, I expect it to tend to 1 order of
| magnitude in time.
|
| Meanwhile, using Python can sometimes feel magical, but I
| have to maintain a library written in the language, and I
| can feel that it hasn't been optimized for this use case.
| It is terribly hard to keep backward compatibility, adding
| typing annotations is somewhat useful, but has terrible
| ergonomics, ensuring no regression is a pain that
| translates to endless suites of unit tests even for trivial
| matters ("be your own compiler" will never be a great idea
| IMO, although tooling like pylance does help a bit).
|
| Meanwhile the rearguard is defending for new projects some
| kind of "No true Scotsman"ish modern C++ delusion, that I
| am still looking for and that moves like goalposts in every
| discussion I have with its proponents. My only explanation
| for the phenomenon is that it is fueled by hubris and sunk
| cost biases, because my experience with the modern idioms
| is that while they bring significant improvements, they are
| still very subpar compared to the expectations brought
| forth by a modern language, and memory safety related CVEs
| are still being written everyday in C++.
|
| So, what should I do? It is true that, in more than 15
| years of coding, Rust represents a revolution in
| programming in my eyes. Should I sit idly while others are
| missing on it and someone is wrong on the Internet?
| gk1 wrote:
| > seeing as they submitted it themselves
|
| I'm grateful to muizelaar but I don't know who that is, so we
| can't take credit for this one.
| bluejekyll wrote:
| Oh, sorry. I misattributed that. I saw the domain origin
| and confused that for the submitter. In that case, the
| submission actually is against the rules of HN, as I
| understand them (use the original title rule).
| jraph wrote:
| The title is "Inside the Pinecone". "Rust - A hard decision pays
| off" is only one of the three major subsections of this article.
| 3a2d29 wrote:
| Gotta mention rust, how else will you get upvotes on HN?
| eterm wrote:
| The title of the section posted is "Rust - A hard decision...".
|
| It's been posted here with the anchor included making it clear
| that it is that section that has been submitted.
| jraph wrote:
| Indeed, I didn't notice. Sorry!
| gk1 wrote:
| Interesting to see the part about Rust getting the most
| attention. We're giving a talk about it in NYC next week[1] and
| will definitely follow up with an in-depth writeup.
|
| [1] https://www.meetup.com/rust-nyc/events/287821884/
| dfinninger wrote:
| The submitted link auto-scrolls me to the section on Rust, so
| I guess the submitter wanted to promote that specific section
| of the article?
| nazka wrote:
| I like this "and an overzealous compiler increased engineers'
| confidence in pushing changes".
|
| If you think about it, you can try to optimize a Rust program
| through trying brute force stuff and trying weird things that you
| are not even sure will break something or not. But you can try
| ~without 100% knowledge or confidence~ and boom the compiler will
| tell you if you are shooting yourself in the foot. Now try to
| that in C++. Even with 95% knowledge and confidence I will be
| very worried as a Lead or CTO.
| lmeyerov wrote:
| Something I still struggle to understand with these systems is
| why indexing 1-10M things is hard / their sweet spot, when that
| is basically a small dataframe in pandas or a small SQLite, vs
| the 10B+ rowa // GBs+ I was expecting them to be discussing
| AtNightWeCode wrote:
| Dev velocity with new projects is often higher so I am not
| convinced by that argument.
|
| With that said. I learned Rust recently. I think it is a viable
| choice for a business core. Rust is not at all that difficult to
| learn as some claim. The tuts are great. Some things are very
| different though.
| ncmncm wrote:
| Leading with several pages of what amounts to smug self-
| congratulation does not inspire finishing.
|
| Rust deserves mention in the title only if actually getting it
| done in Rust was particularly difficult.
| jhgg wrote:
| Writing Rust at scale, I concur with some of these findings.
| Nowadays, my team exclusively writes Rust. Our rationale is two-
| fold.
|
| 1) We want to write software that is fast and inexpensive to run.
| Our rust services are quickly growing, but cumulatively, they
| remain in the sub-1000 core count, while processing many millions
| of requests per second across our service with excellent latency.
|
| 2) We want to write software that won't break and page us in the
| middle of the night, or blow up when we deploy it. We operate
| some of the core persistence primitives at our company, so the
| consequence for error in the worst case could be corruption or
| loss of user data. Our experience thusfar with Rust is that we're
| able to confidently iterate and deploy our services. The
| developer velocity is indeed higher, and continues to grow with
| your experience in the language.
|
| Our team of 10 (with 3 engineers having started in the last 2
| weeks) owns all of the database clusters, and all of the services
| surrounding them. Our on-call is surprisingly quiet, with maybe
| 0-3 low sev incidents a week, with most of them being due to the
| underlying database/hardware, and not the rust services. We are
| actually writing a Rust control plane for our databases soon,
| which seeks to automate most of the response and remediation of
| most of the incidents we experience in a given week, and also to
| automate a bunch of manual work that we currently do around
| operations of the databases.
|
| The learning curve can be a bit rough, but our team of engineers
| is motivated to learn, and we have a great culture of mentorship
| and teaching from the engineers who are familiar with Rust.
| nazka wrote:
| I am curious which web framework do you use? Also do you use an
| ORM like diesel?
| FpUser wrote:
| Frankly on larger projects overall time to create and test the
| whole thing in modern C++ or (I assume) in Rust should not take
| longer than in Python. I do not understand this idea of mixing
| languages where one is enough.
|
| In my own case I rewrote decent size Python app in C++ and it had
| taken me about the same time as for original developers. And no I
| was not porting code but rather working with the specs so did my
| own design.
| kartoolOz wrote:
| Sidenote, been using pinecone for semantic search recently, and
| it's been a joyful experience.
| fb03 wrote:
| Would you guys start a GraphQL Api today using Rust? I'm very
| tempted to hint at that but am still insecure about my 'options'.
| This is coming from a very comfortable python + fastapi +
| sqlalchemy ecossystem
| t6jvcereio wrote:
| > We decided to move our entire codebase to Rust (and Go for the
| k8s control plane).
|
| > there was still one minor problem - no one on the team knew
| Rust.
|
| Is this real or satire? I can't tell.
| drexlspivey wrote:
| How hard can it be for C/C++ devs to learn Rust? Python devs on
| the other hand..
| t6jvcereio wrote:
| That's not the point. If no one knows the language, no one
| knows if it's appropriate. All you know is the language
| marketing claims
| tinco wrote:
| Even though it's software engineering, which supposedly
| isn't really engineering, you can still apply engineering
| principles. Disregarding all the bullshit debates about
| whether Rust is better than Go, there are actually
| differences between the languages that you can objectively
| research.
|
| Maybe no one in that specific company really knew the
| language, but there's loads of people actually using the
| language, it's not like it dropped from the clear blue sky.
| It's not like there's a Rust salesperson that knocked on
| their door with a leaflet. They're not even the first
| persons to write a database engine in Rust.
|
| I wonder if other engineering domains have this same
| silliness. Do the architects have intense debates over
| steel H beams vs wooden LVL beams, and at some point newbie
| architects believe you should only switch from one to the
| other as an architecture firm if your architects have
| experience with it, because the only way to know if it
| would work is reading the marketing claims.
|
| A compiler is just a tool. If you're doing a rewrite you
| better pick your tools correctly, and it seems like they
| did. You could theorise they were lucky somehow, or you
| could recognise that they're experienced, educated and
| skilled and made the right choice based on their
| researched.
| t6jvcereio wrote:
| > Do the architects have intense debates over steel H
| beams vs wooden LVL beams, and at some point newbie
| architects believe you should only switch from one to the
| other as an architecture firm if your architects have
| experience with it, because the only way to know if it
| would work is reading the marketing claims.
|
| That is one way to put it, which is convenient for the
| point you're making. A different way, more convenient for
| the point I'm making is that if you're budding roads and
| you need a bridge, you probably wouldn't hire a firm that
| only has experience building tunnels.
|
| So you see, your analogy is meaningless, because making
| an analogy is not a convincing way of making a point.
| Anyone can make an analogy.
|
| My argument on the other hand that experience is a very
| important factor, is a lot more convincing because anyone
| who has any kind of professional career already agreed
| with it.
| nazka wrote:
| I think at this point Rust has more than marketing points.
| Then if you have some bright people, and I mean doing what
| they are trying to do and feeling that RockDB is a slow
| database... They should be able to pick it up. I don't
| think it's an unreasonable decision as a CTO/CEO.
| remram wrote:
| It's probably exaggerated. I doubt they committed to the re-
| write before anyone knew Rust.
|
| Hopefully what they meant is that Rust was only being
| considered when they started having the team learn it, or that
| the team was not extremely proficient at it when they picked
| (but had sufficient knowledge of the language).
|
| Not saying that this part didn't surprise me as well. And if
| they did commit based only on the marketing material... what a
| gamble.
| RcouF1uZ4gsC wrote:
| > Dev velocity, which was supposed to be the claim to fame of
| Python, improved dramatically with Rust. Built-in testing, CI/CD,
| benchmarking, and an overzealous compiler increased engineers'
| confidence in pushing changes, and enabled them to work on the
| same code sections and contribute simultaneously without breaking
| the code base. Most impressively though, real time operational
| events dropped almost to zero overnight after the original
| release.
|
| I think Rust is really shaping up to be in a sweet spot.
|
| One thing that is not talked about enough with Rust, is that you
| do not need to start with fighting the borrow checker from day
| one.
|
| Making copies and using Rc can bypass a lot of the trickier parts
| of the borrow checker and still allow you to access much of the
| benefits of Rust in terms of performance and correctness,
| especially compared to a dynamic language like Python.
|
| As you get more experience, you can then use references and other
| techniques to eliminated unnecessary copies, etc.
| klabb3 wrote:
| > Making copies and using Rc can bypass a lot of the trickier
| parts of the borrow checker and still allow you to access much
| of the benefits of Rust in terms of performance and
| correctness, especially compared to a dynamic language like
| Python.
|
| > As you get more experience, you can then use references and
| other techniques to eliminated unnecessary copies, etc.
|
| I've always found this advice to be infeasible in practice,
| despite sounding like a reasonable pitch.
|
| Arc allows you to avoid thinking about ownership, which is
| largely what dictates the architecture when writing idiomatic
| Rust. Plastering on ownership later is going to change the code
| so much you might as well rewrite it .
|
| Moreover, if you want to prototype (with Arcs, clones) a new
| part of the code, while integrating with a more mature part of
| the code base, you still need to adhere to the ownership
| structure.
|
| In fact, this is one of my main issues with Rust, that it's a
| terrible language for prototyping. I don't have a solution, and
| I'm not blaming Rust - it's still a great language. But you
| have to have a very solid design/mental model before you start
| coding.
| kevincox wrote:
| I'd disagree. It depends on the use case a lot but I
| encourage people to reach for reference counting when they
| have complex lifetimes. The cost is almost always trivial and
| for most use cases the lifetime analysis is clear and
| unlikely to become a problem in future refactoring.
|
| There is obviously always some nuance, but I find that there
| is a wide useful space between Rcs that can be avoided quite
| simply and making dramatic changes to your code to avoid an
| Rc.
| ncmncm wrote:
| Using Rc unnecessarily is a code smell commonly called Java
| Disease.
| endisneigh wrote:
| Maybe it's just me but it seems irresponsible to move your entire
| team to a programming language none of them knows.
|
| Rust is great, but modern C++ is great too.
| moonchrome wrote:
| >but modern C++ is great too.
|
| No, modern C++ is still cobbled together from tools built in a
| different era. It doesn't even come with a package
| manager/build system.
|
| Just on the fact that cargo/crates.io exist and it has modules
| - it would have to be really bad to make me go back to C++ if I
| ever need to write something at that level.
| adamdusty wrote:
| There are package managers and build systems. MS ships a
| build system with visual studio and msvc and they have a
| fairly capable package manager in vcpkg.
|
| I like cargo better than any c++ tooling but those tools
| don't really make a ton of sense for c++ where every platform
| has a different compiler.
|
| Ultimately, I agree. I don't think I'll be going back to c++
| any time soon, but for me it's more about the tooling
| fragmentation than the tooling itself. I actually like the
| language itself.
| ncmncm wrote:
| Yes. This comes across as "we are so smart, this project wasn't
| hard enough to demand our full attention".
|
| Coding in a manifestly immature language signals that you are
| not really serious about being used where long-term support
| across a variety of platforms may be important. The language is
| still more likely than not to fizzle. If it does, it makes your
| flagship project a niche player.
| diarrhea wrote:
| > The language is still more likely than not to fizzle.
|
| What makes you think that?
| ncmncm wrote:
| Fizzling is the normal fate of any new language, absent a
| miracle. Only a tiny handful of languages get a miracle.
|
| In any given week, more people pick up C++ or Javascript to
| use professionally than the total number now employed to
| code Rust. Not to fizzle, it has to increase its adoption
| rate by orders of magnitude, but its fans are almost
| uniformly hostile toward any measure that could make it
| easier to adopt.
| mynameisash wrote:
| > its fans are almost uniformly hostile toward any
| measure that could make it easier to adopt.
|
| Do you have an example exhibiting what you mean here?
|
| I would self-describe as a fan of Rust, but I also think
| I'm pretty realistic about its limitations and that every
| language has its niche. I will say that as far as
| miracles go, being the most-loved language on SO for
| seven years seems like a bit of a (minor) miracle here,
| and at about 87% loved vs Python's 67%, it's not exactly
| eking out a win. It'll take time for it to grow in market
| share, but it seems likely it's on its way, given that it
| is, for example, breaking into the Linux codebase -
| something C++ wasn't able to do.
| ncmncm wrote:
| > _breaking into the Linux codebase_
|
| Is Linux really your go-to example? Linus is noted for
| his irrational hostility to C++. His complaints are
| proved silly or, at best, obsolete by the example of
| Serenity OS.
|
| An example of a measure to make the language easier to
| adopt is to allow borrow violations to be made a warning
| when building in debug mode. All violations would need to
| be patched up before release anyway, so it does not
| change the language any, but the violations might be in
| code you will delete before that. Forcing borrow
| cleanliness in dead code before you can work on important
| code feels just spiteful.
|
| People invent the most ridiculous objections, including
| that the compiler would not know what code to generate,
| or that it would split the language into two dialects. My
| conclusion is that the loudest Rust fans like that they
| are a select few, and don't want it adopted mainstream.
| They might win.
| nindalf wrote:
| You seem quite certain of your opinions, especially that
| Rust will fizzle out. I'll bet $100 that in 2025
|
| - it's still used at Meta, Google, Amazon and other large
| companies. Once a language gains critical mass at a
| large, profitable company it will be maintained one way
| or another.
|
| - It will be used in the Linux kernel as well.
|
| If you have any other objective measures of
| usage/popularity/usefulness, I'd like to hear it.
|
| > In any given week, more people pick up C++ or
| Javascript to use professionally than the total number
| now employed to code Rust.
|
| If you can tell me how you measure this I'll take a bet
| that this is wrong as well.
| ncmncm wrote:
| Every language you can think of, and many you can't, are
| in use _somewhere_ at every large company. That is no
| distinction at all. So, no bet.
|
| If it gets into the Linux kernel in any capacity,
| obviously it will still be in there as long as anybody
| still uses whatever it does. That, also, is no
| distinction at all. So, no bet.
|
| I am not expressing opinions. I am expressing
| observations, and expectations that seem to follow. If
| you have other observations that bear on the question,
| bring them.
| nindalf wrote:
| You say this language will fail because most languages
| fail. Ok, make a falsifiable statement and I'll take a
| bet on that.
| ncmncm wrote:
| You seem very confused. I have not, in fact, said this
| language will fail.
|
| Bets are not interesting. I would like for Rust not to
| fail, but do not see the things happening that would
| prevent it.
| mynameisash wrote:
| You said, "The language is still more likely than not to
| fizzle." What's the distinction between "fail" and
| "fizzle"?
| ncmncm wrote:
| Not much. But there is a very large difference between
| "will" and "might".
| UncleEntity wrote:
| It seems like a bad idea to allow what is arguably the
| main selling point of the language to be turned off by a
| compiler flag so someone can build the rest of their
| program on top of that shaky foundation.
|
| Whole hog or no hog, this is the rust way.
| ncmncm wrote:
| I.e., a ridiculous objection that demonstrates wholesale
| incomprehension of the topic.
|
| This is an example of why I expect the language may still
| fizzle.
| moffkalast wrote:
| > modern C++ is great
|
| Great in the way that Alexander the Great was great. Achieved
| big things and killed a lot of people.
|
| Though in the case of C++ they're just dead inside.
| benreesman wrote:
| There are better threads to trash C++ hackers on than this
| one. It's literally about insanely successful software
| written in C++, maintained successfully for a decade, that
| seems to have survived a port to another language.
| FpUser wrote:
| >"Though in the case of C++ they're just dead inside."
|
| I think this is pure and memory safe BS
| tialaramex wrote:
| Search your heart, do you think this story ends similarly if
| they decide to all do "modern C++" instead?
|
| My guess is that even with initial discipline in the form of
| code review and style enforcement by one or two people, the C++
| descends into a riot of different opinions about, as usual,
| which are the good bits.
|
| A similar piece of software could be written in C++ but I doubt
| it gets written successfully, in similar time, by this team. I
| reckon the post mortem of such an attempt would be written off
| as "Don't do rewrites, duh"
| UncleEntity wrote:
| > Search your heart, do you think this story ends similarly
| if they decide to all do "modern C++" instead?
|
| It sounds to me they already had a good chunk of it written
| in C++ with python glueing it all together.
|
| From a 'dev velocity' standpoint one would think having the
| senior devs learn a new language and rewrite an entire
| database wouldn't make sense when they could just
| systematically replace the python parts with C++, which they
| already knew.
|
| Or, who knows, maybe all their initial issues stemmed from
| the locations of semicolons around the else keyword?
| rr888 wrote:
| > Search your heart, do you think this story ends similarly
| if they decide to all do
|
| The main difference would be there wouldn't be a blog article
| on the front page. :)
| hu3 wrote:
| > Search your heart, do you think this story ends similarly
| if they decide to all do "modern C++" instead?
|
| https://github.com/facebook/rocksdb
| benreesman wrote:
| You might want to take that up with the authors of RocksDB
| and FAISS. They seem to have shipped some cool stuff.
| tialaramex wrote:
| RocksDB, as I understand it, is a pretty old piece of
| software, forked from a Google product, which was already a
| pretty old piece of software, all written in C++. Doesn't
| seem to have previously been written in Python, doesn't
| seem to have a team of non-C++ programmers who decided to
| just learn as they went, and so it's not clear what
| parallels you expect to draw.
|
| Maybe they're doing a Rust rewrite too? I'd be interested
| to read about that.
| ncmncm wrote:
| This is an exceedingly silly conclusion. "Similar pieces of
| software" are coded in C++ and shipped every single day,
| without anyone saying " _and_ we managed to do it _in C++_ ,
| too!" The emphasis should be on what the software does well,
| and how, not on what hoops the developers were jumped through
| on the way.
| cratermoon wrote:
| > it seems irresponsible to move your entire team to a
| programming language none of them knows
|
| Based on my experiences, I would agree. There seems to be a
| very important difference between my experiences, which
| probably align with most of the industry, and the Pinecone
| team. My career has been working with average developers doing
| ordinary business and internet things. Stuff that takes
| organization and teamwork, but not necessarily advanced or
| esoteric knowledge. So the COBOL programmers that went to the
| "Java in 21 Days" training that I consulted with, or the C#
| programmers who went with Go, they did poor-to-OK, but that was
| Good Enough.
|
| The Pinecone team, though. They were already writing C/C++ to
| solve Hard Problems. As experienced programmers, they probably
| already understood the landscape of systems languages with
| pointers and having to manage their own memory. Rust, while
| imposing, would not be a huge leap for them.
|
| Another aspect worth noting: they did they re-write
| incrementally, never completely abandoning the existing system.
| At least it seems so. I'll be interested in seeing the
| presentation from the upcoming talk about the rewrite that's
| mentioned in the story.
| UncleEntity wrote:
| If you start the article at the top instead of where the hot
| link points (which is its own discussion) it starts by
| talking about mathematically proving why this form of
| database is faster than theory suggests it to be. Definitely
| not your average boot camp devs working on this one.
| cratermoon wrote:
| Precisely. Also, it wasn't one of those cases where
| management decides to move people off of the language they
| know to one they read about in InfoWorld or CIO Magazine.
| Or worse, had a golf buddy tell him how great it is.
|
| In the words, the team wasn't moved, the team moved itself.
| awestroke wrote:
| Seems more like the team moved themselves to Rust
| liminal wrote:
| Any plans to open source the kv store?
| kissgyorgy wrote:
| Expectation: this article.
|
| Reality: It's insanely hard to find talented developers, but Rust
| makes people excited, so they are just using it as a marketing
| tool.
| theptip wrote:
| I'd be interested in a bit more detail on the technical migration
| strategy. Did they just cut over to a version of the project
| built in Rust? (I.e. vN is Python+C/C++, vN+1 is Rust?) Or
| something more gradual?
|
| How did they verify that the new code was conformant with the old
| app's behavior/logic?
|
| > To make matters worse, we would discover issues only after
| deploying (or in production!) due to Python's run time nature.
|
| I don't want to infer too much, but this makes it sound like
| perhaps they didn't have a very robust set of E2E/Acceptance
| tests, which would make a full-cutover migration scary to me. If
| you're finding Python bugs only after deploy, how can you find
| the inevitable rewritten-Rust-code incompatibilities before
| deploying/production?
|
| I've been digging into Rust/Python interop recently using
| https://github.com/PyO3/pyo3 and maturin, and this points to an
| interesting migration strategy; PyO3 makes it quite easy to write
| a Python module in Rust, or even call back and forth between
| them, so you could gradually move code from Python to Rust. A
| migration path to full-Rust might be:
|
| 1. Incrementally replace all your fast-path C/C++ code with Rust,
| still having Python call these compiled modules. End-state: You
| now have a Python/Rust project instead of Python/C/C++.
|
| 2. Gradually grow the surface area of your Rust packages, moving
| logic from Python into Rust. Your existing Python tests can still
| run, and your existing entrypoints are the same Python.
|
| 3. At some point, you presumably need to cut over the entrypoint
| layer (the API?) to use pure Rust, instead of Python. This should
| be a much less scary migration since both versions are calling
| into the same underlying Rust library code. Depending on your
| architecture, if you have an API Gateway you can split your
| service backends to migrate one endpoint at a time to the new
| Rust API service, while keeping the old Python API service around
| to fail back to. (They are using k8s so you can do this with your
| Ingress for example).
|
| I'm interested in others' experiences with Rust/Python interop,
| are there any rough edges worth knowing about?
| throwaway894345 wrote:
| > Dev velocity, which was supposed to be the claim to fame of
| Python, improved dramatically with Rust.
|
| I don't doubt this in the least. I've been a professional Python
| developer for 15 years, and I can't believe Python ever had the
| reputation for "high dev velocity" beyond toy examples. In every
| real world code base I've worked in, Python has been a strict
| liability and the promise that you can "just rewrite the slow
| parts in C/C++/Numpy/etc" has never held (you very often spend at
| least as much time marshaling the data to the target language
| format than you gain by processing in the faster language _and_
| you have all of the maintainability problems of C /C++/Numpy).
| Python trades developer velocity for poor runtime performance.
|
| I don't think Rust is the paragon of high dev velocity languages
| either, but it seems to be more predictable. You don't have
| something that works as a prototype, but then you run into a
| bunch of performance problems (which virtually cannot be worked
| around) when you go to productionize, nor do you get all of the
| emergent quality and maintainability issues that come about from
| a dynamic type system (and last I checked, Python's optional
| static type system was still pretty alpha-grade).
|
| I strongly recommend avoiding Python for new projects. Use Go if
| you're looking for an easy GC language with max productivity and
| a decent performance ceiling. Use Rust if you're writing really
| high performance or correctness-is-paramount software. I'm sure
| there are other decent options as well (I've heard generally good
| things about TypeScript, but I'd be concerned about its
| performance even if it is better than Python).
| sitkack wrote:
| If Go replaces your Python this easily, you have always been
| writing Go. The abstractions available in Python are in a
| totally different class than that of Go.
|
| Nothing you said is verifiable.
| throwaway894345 wrote:
| > Nothing you said is verifiable.
|
| Right, I was pretty explicitly speaking from experience. See
| my first paragraph.
|
| > If Go replaces your Python this easily, you have always
| been writing Go
|
| Go is only a 10 year old language, and I've been writing
| Python _professionally_ for 15 years. I definitely have more
| hours on Python by a wide margin.
|
| > The abstractions available in Python are in a totally
| different class than that of Go.
|
| Python definitely is more abstract than Go, but I don't think
| that's a feather in its hat. Moreover, most of my criticisms
| of Python had nothing to do with its abstract nature.
| fwip wrote:
| I think the intent of saying "you've always been writing
| Go" was along the lines of "you can write FORTRAN in any
| language." E.g, implying that you are not making use of
| Python's abstractions, but writing it in the same
| procedural style you would write Go (or C) code in.
| throwaway894345 wrote:
| Fair enough, but that's still not the case. :)
| 3pt14159 wrote:
| I disagree.
|
| Most startups that YC has funded that became successful (Series
| B or higher) were written in Python or Ruby. Now you can say
| that this is a tradeoff for post Series B, and for that I don't
| know. I've never worked on a massive Python mono-repo for a
| company that size. But I know what I've done in Python and,
| yes, it includes performance optimization in Numpy / Scipy /
| Cython, and other than Ruby, no other language comes close to
| the developer performance I see with Python and the ecosystem
| around it.
|
| That said, there is a good and a bad way of writing Python. You
| absolutely need tests and lots of them since there is no
| compiler for vanilla Python. Mypy helps a ton too and so do the
| linters and even Black. Also, get pdb++ and customize it. The
| tooling helps a _lot_.
|
| Now I haven't ever written production Rust, but I have for Go
| and many other languages, and so Rust may be faster, but other
| than Ruby I haven't seen anything close.
| Nuzzerino wrote:
| > Most startups that YC has funded that became successful
| (Series B or higher) were written in Python or Ruby.
|
| Is there data on startup lifespan ranked by tech stack?
| Genuinely been looking for this for awhile now.
| badpun wrote:
| > I disagree.
|
| > Most startups that YC has funded that became successful
| (Series B or higher) were written in Python or Ruby.
|
| It depends on what your goal is. If you want to get rich off
| of VC money, Python and Ruby might be a good fit. If you, on
| the other hand, want to write good, performant, maintenable
| and (relatively) bug-free software, then there are better
| choices.
| ithrow wrote:
| _If you, on the other hand, want to write good, performant,
| maintenable and (relatively) bug-free software, then there
| are better choices._
|
| You can do that part when the money comes in.
| manfre wrote:
| Dev teams that are unable or willing to properly architect
| and test their projects will end up with buggy and
| difficult to maintainable code regardless of language.
| eatonphil wrote:
| That is unrelated to what the person you're responding to
| said.
| valenterry wrote:
| > Most startups that YC has funded that became successful
| (Series B or higher) were written in Python or Ruby
|
| So, what was the percentage of YC startups that started with
| Python or ruby and what was the share that became successful?
| AtNightWeCode wrote:
| Docker is built with Go. Compare doing things with Docker api
| between Go and Python. I usually recommend people to "script"
| things with Go but I start to think that Go has to go...
|
| https://docs.docker.com/engine/api/sdk/examples/
| rob74 wrote:
| I would say in this case it's more the API designers' fault
| than Go's fault. Looks like the Python API is operating at a
| higher abstraction level than the Go API - using the Python
| API you can just call a "macro" that does everything
| required, in Go you have to call each operation individually.
| throwaway894345 wrote:
| It probably is Go's propensity to return error values
| whereas Python would bubble them up. Every action in Docker
| involves an I/O error at a minimum (since you're talking to
| the Docker daemon) so this is basically a worst-case
| scenario for error handling boilerplate. I don't doubt that
| Python is better for _scripting_ in this case, but if you
| want to make an application that interfaces with Docker, Go
| is probably the better of the two (despite its
| boilerplate). In fact, I vaguely recall running into a
| bunch of Python-related bugs in docker-compose (it was at
| an old gig, and I haven 't used docker-compose since, so I
| don't recall the particulars).
| AtNightWeCode wrote:
| dang wrote:
| Can you please avoid flamebait comments and make your
| substantive points thoughtfully, per
| https://news.ycombinator.com/newsguidelines.html? We
| don't want flamewars here. Programming language flamewars
| are especially tedious.
| youor wrote:
| AtNightWeCode wrote:
| Not sure what you are talking about. When I learned C
| back in the 90's I was learned to handle every error
| state by hand. Exactly as you are expected to do in Go
| about 30 years later. And it sucks.
|
| Maybe you can explain why Gophers are such whiners always
| crying about every freaking comment at any forum.
| dang wrote:
| Can you please just not post like this? We're trying for
| a different kind of forum. I know it feels justified, and
| I'm sure it is, but the problem is the 'other' side
| always has _its_ legitimate justifications too, and then
| everyone goes after each other and there 's no more
| curious exchange.
| AtNightWeCode wrote:
| LOL
| metaltyphoon wrote:
| Event the C# version is simpler :D
|
| https://github.com/dotnet/Docker.DotNet#example-create-a-
| con...
| danwee wrote:
| Wow. That's quite something. I like Go, but more often than
| not I keep stumbling upong these kind of situations (N LOC to
| do something in Y, 10xN to do it in Go).
| reitanqild wrote:
| In addition to a emulating certain teachers dogged
| insistence of making sure students can't have nice things
| (an absolute no-brainer feature like generics only showed
| up recently...!), last time I tried to use Go for a web
| project a few years ago I was also amused by the lack of
| templating libraries - the way to make pages was to string
| together header + main + footer, the old PHP way...! Not a
| single library supported template composition.
| coder543 wrote:
| Umm... what are you even talking about?
|
| Go has had templating built into the standard library
| since... forever?
|
| And Go templates can use other templates, as long as
| they're all loaded into the same context, if that's what
| you were getting at.
|
| simple examples of Go's built-in templates:
| https://gowebexamples.com/templates/
|
| templates using other templates:
| https://levelup.gitconnected.com/using-go-templates-for-
| effe...
| merely-unlikely wrote:
| I've found Go's templating to be pretty great
| reitanqild wrote:
| That doesn't look too bad on a quick glance, but the
| possibility to say that "this content goes in that
| template" still seems to be missing.
|
| Look at this page: https://quarkus.io/guides/qute-
| reference
|
| and read about "Template inheritance" in 3.5.6 to see
| what I mean.
| coder543 wrote:
| In this example, I'm passing an object with a single
| value into the header template, but it could have as many
| values in it as needed: https://go.dev/play/p/kC8pm4Z4WrH
|
| Go doesn't really do template inheritance in the way I
| think you're meaning. Go prefers composition over
| inheritance, and that carries over to the templating
| system. Go templates let you pass a parameter to the
| template you're calling, which I believe addresses the
| need you linked to. Calling another template is not just
| simple string concatenation; the other template can act
| on input provided by the calling template.
|
| There are a lot of different ways to approach templating
| pages in a web app. I think Go's templating system is one
| of the least objectionable ones I've used.
|
| I have no experience with Quarkus, so I can't say how
| good or bad that one is, but my experience with Ruby
| templating systems is that people often start mixing
| complex logic into the templates, including database
| calls, and that causes things to get messy. Go's template
| environment is very minimal by default, and you can
| extend it with custom functions if you want to give more
| power to the templates, but Go encourages you to compute
| your data ahead of time, _then_ pass it into the
| template... and just have your template be a template for
| formatting the data.
| rkwasny wrote:
| Exactly this, in rust it will even more complicated than in
| Go
| nurettin wrote:
| Python is fine, use the typing module, don't be weird with the
| return types and you will be fine. Sure, you will actually have
| to think about your algorithm when writing for speed, and naive
| loops won't work most of the time. But "avoid python in
| production" or "don't write new code in python" is way too
| harsh.
| benreesman wrote:
| I think Rust deserves a lot of credit for _popularizing_ the
| notion of "constraints as power", but excessive enthusiasm
| around it does seem to obscure the rich traditions of PLT that
| it has more than liberally borrowed from.
|
| There are more rungs on the ladder.
| throwaway894345 wrote:
| I say this as someone who has had a lot of criticism for
| Rust: Rust definitely borrows a lot from other languages, but
| Rust deserves praise precisely because it curates the right
| features and composes them in the right way. There are way
| too many languages from the PLT space that add some neat type
| system innovation, but they fail on syntax or performance or
| tooling or etc. That's not particularly laudable IMO, but
| Rust has great tooling (cargo, rust-analyzer, rustfmt,
| rustup, etc), great performance, great safety, _and_ a bunch
| of type system innovations (including its approach to memory
| management) which were probably picked from the PLT world.
| The individual features are neat, but Rust is much greater
| than the sum of those borrowed innovations. Choosing what
| features to use and which to omit is a much more valuable
| task than any individual innovation.
| benreesman wrote:
| Credit where credit is due: Rust has done a far better job
| than most of free-riding on Haskell's type class system and
| clang's optimizer, but rust-analyzer is a dumpster fire
| next to clangd, that's not an example I'd use.
| mustache_kimono wrote:
| Um what?
|
| Maybe it's just that I don't understand what clangd
| provides that's so great, but rust-analyzer, despite a
| few hiccups I've had with it, seems completely amazing at
| what it does.
| benreesman wrote:
| Well we could start with the fact that it crashes process
| IDs more often than Justin Bieber crashes Maseratis:
| https://github.com/rust-lang/vscode-rust/issues/890 (it's
| always LLVM's fault, I know). Or move right along to the
| insane configuration hacks that are mandatory if you want
| anything upmarket of VSCode (who doesn't need a little
| practice with their Emacs Lisp), or the N^2 whatever that
| happens when you point it at bindgen output.
|
| What clangd does is work.
| nindalf wrote:
| The other comments have pointed out this was a simple
| misconfiguration that was easily fixed.
|
| What I take exception with is "it's always LLVM's fault,
| I know". If you're going to trash other people's hard
| work in such a glib manner, it behooves you to know the
| details of what you're talking about. Tell us, why would
| a language server that doesn't generate any code use
| LLVM?
| coder543 wrote:
| You linked to an issue from over a year ago to support
| your case? And the solution in that issue was just
| "switch to the right extension"? That's hardly a
| compelling argument. By all appearances, that person's
| environment was completely broken since they reported the
| other extension did nothing at all, and a broken
| environment isn't really a reflection on the tool. I can
| set up a broken environment for any tool and then
| complain about it. The absence of a bunch of other people
| encountering the same issue in that thread indicates this
| was not some big issue with rust-analyzer.
|
| If I search for "Rust" in the VS Code Extension
| Marketplace, it shows the extension you linked to... but
| it is clearly marked as deprecated, and _VS Code won 't
| even let me install it._ So, it's even more of a non-
| issue today than it was a year ago.
|
| The _actual_ rust-analyzer extension works fine for me. I
| think Go 's tooling is slightly better in my experience,
| but both are far better than _any_ experience I 've had
| with C++ tooling.
| mustache_kimono wrote:
| > it crashes process IDs more often than Justin Bieber
| crashes Maseratis: https://github.com/rust-lang/vscode-
| rust/issues/890
|
| So -- this guy used an extension (named Rust) with rust-
| analyzer, a configuration known to not work, it didn't
| work(!), and the extension author recommends he tries the
| extension made for rust-analyzer.
|
| Hardly a case for the ages. Guy uses unsupported config
| and things don't work?
|
| > What clangd does is work.
|
| Don't doubt it. I'm just saying -- I haven't had any
| problems with the rust-analyzer extension since it became
| the Rust default. But, yes, I had a few hiccups and
| crashes beforehand, no doubt. I just have to imagine it's
| both younger, and doing more/different things than
| clangd.
| throwaway894345 wrote:
| I haven't used clangd since the early days, but rust-
| analyzer hasn't been anything other than reliable for me
| (which is pretty impressive considering how often Rust
| changes as a language and how new rust-analyzer is).
| benreesman wrote:
| I would respectfully submit that time spent pushing your
| toolchain from more than one direction confers many
| benefits, of which a realistic appraisal of the strengths
| and weaknesses of the various tooling options is but one.
| pjmlp wrote:
| Yeah, specially when people refer to stuff taken from ML as
| Rust "features".
| Kinrany wrote:
| "Constraints as power" describes the proposition of all
| statically typed languages. But the compilers of popular
| languages weren't smart enough to cover many common
| scenarios, forcing language designers to offer many unsafe
| escape hatches.
| pjmlp wrote:
| Interestingly Object Pascal was my introduction to data
| type modelling for constraints, followed by Caml Light,
| Modula-3 and Ada.
|
| Naturally one can argue they weren't popular, with
| exception of Object Pascal.
| benreesman wrote:
| It's a very valuable thing to be able to separate the
| mental model one uses from the target machine language.
|
| I worry about the more ardent Rust enthusiasts because it
| seems popular in those circles to embrace an attitude that
| we've somehow arrived at the _right_ conceptual framework.
|
| Rust is a competent, pragmatic embedding of a few of the
| big ideas from serious PLT into a well-optimized C++
| compiler toolchain, and it's cool and useful as a result.
|
| But I hope to God it's not the endgame on fast
| ML/Haskell/Lisp.
| ldng wrote:
| To be successful a language still has to be approachable
| to most dev. I just hope you are not conflating PLT with
| Functional. Look what functionalist did to Scala... It
| had an interesting balance and they destroyed it.
| tialaramex wrote:
| I'd be disappointed if the best language people are
| writing software in when I retire (perhaps a decade or at
| most two in the future) is Rust. We can surely do better.
|
| But I'd be _even more_ disappointed if the best language
| people are writing software in when I retire isn 't even
| Rust.
| queuebert wrote:
| Dev velocity in Rust is directly proportional to how well you
| grok lifetimes and ownership.
| throwaway894345 wrote:
| Eh, I like Rust and started my career doing embedded C and
| C++, so I definitely "grok lifetimes" but I'm probably always
| going to be more productive with Go than Rust (for general
| app development, anyway) if only because I don't have to
| think about memory _at all_ with GC (again, general app
| development, not talking about performance optimizations).
| Even if I do a bunch of unnecessary cloning + Rc /etc in
| Rust, I'm not going to move as fast as I would do in Go.
| cmrdporcupine wrote:
| Yes re: Python.
|
| Story -- I was a very early adopter of Python, back in the
| mid-90s. When other people wrote their CGI scripts in Perl, I
| always reached for Python. The first paid gig I ever had was a
| CGI script ("resume builder") I wrote in Python in 1996. But
| almost nobody was using it back then, and I'd get quizzical
| stares from people in job interviews etc. when it came up.
|
| So for many years back then I really really wanted to get a job
| working in Python. And I was super excited to get a job around
| the 2001 time frame in it on a pretty cool embedded Linux
| project.
|
| But my first experience working in a large codebase was
| disappointing. I could see right away that once many developers
| started working in that codebase together, best practices were
| really hard to enforce, and things started to fall apart into a
| bit of a mess, and lots of problems just showed up at runtime
| in bad explosions. That was the beginning of my disenchantment
| with dynamically typed, late-bound languages. They only give
| you the _illusion_ of fast prototyping. You just shift your
| time to fixing type and binding errors later in your dev cycle.
|
| Python really improved a lot with the 2->3 transition, and the
| community has much better best practices now. But I think the
| core problem with late bound languages remains.
|
| All that said, I'm currently in another window working for my
| current employer wiring an embedded CPython interpreter into a
| Rust-based runtime...
| orangecat wrote:
| I agree with most of this, but I would argue that the 2->3
| transition was on balance a mistake. Python 3 is better, but
| not better enough to justify the huge amount of time and
| resources that otherwise could have been spent on better
| performance, typing, packaging, tools, etc. I'd gladly trade
| the Python 3 improvements for Python 2 equivalents of
| TypeScript and V8.
| cmrdporcupine wrote:
| You are probably right, I'm not invested in enough in that
| ecosystem to know one way or another. I do know lots of
| things remain disrupted by this transition in ways that
| still personally inconvenience me.
| DougN7 wrote:
| > They only give you the illusion of fast prototyping. You
| just shift your time to fixing type and binding errors later
| in your dev cycle.
|
| That is the perfect way to describe it! And sometimes you're
| out of the dev cycle and into production if your test suite
| didn't hit all cases.
| the__alchemist wrote:
| I think Python shines as an interactive calculator, eg an
| iPython REPL. Calculations, plotting etc. Or for website
| backends since Django is very nice. Or scripting OS tasks like
| file manipulation etc.
|
| For full projects outside websites, Rust for almost every case.
| lowbloodsugar wrote:
| I like Octave (gnu matlab) for that.
| zozbot234 wrote:
| Funnily enough, a REPL environment is in the works for Rust:
| https://github.com/google/evcxr
| throwaway894345 wrote:
| I could get down with that, although my experience with the
| Python REPLs has been a mixed bag. Specifically, you can't
| easily deal with async stuff IIRC (not sure if that's been
| fixed or not).
| FartyMcFarter wrote:
| If people only used Python as originally intended (as a
| language to write small scripts in), this kind of problem never
| would have happened.
| pjmlp wrote:
| The only use case that makes sense for Python.
| throwaway894345 wrote:
| Sure, but the Python community itself promoted the idea that
| Python was suitable for general application development and
| so on. And it probably was a really good option back in the
| day when people were using C, C++, Perl, and PHPv4. And I'm
| sure there are still some areas where it's a fine option
| (maybe heavy data science stuff?). But I would only use it as
| an absolutely final resort.
| golergka wrote:
| > I've heard generally good things about TypeScript, but I'd be
| concerned about its performance even if it is better than
| Python
|
| I wouldn't recommend Typescript for CPU-bound tasks, but for
| I/O-bound tasks with complicated business logic it's type
| system is on the order of magnitude better than Go, and makes
| you able to get really close to the "make invalid states of the
| system a type error" ideal of functional languages like
| Haskell, while still being a lot more approachable.
| xmonkee wrote:
| I am launching my startup on Typescript (backend and
| frontend). So far, it's been an amazing experience - I can
| define the problem in terms of types even before I start
| writing any executable code, and refactoring is a breeze if
| you make your types hard enough. There are mature libraries,
| you can share code between the two sides, infinite options
| for PaaS providers who can "just deploy" a typescript
| codebase. As a solo dev, it's saving me an incredible amount
| of time to go to market.
|
| I've had experience with large codebases in Python, Java, Go
| and Javascript before.
|
| Python - as everyone already said, just sucks. Refactoring is
| a nightmare, you have to test every single line of code for
| any sort of dependability and that makes refactoring even
| harder. It allows for too much meta-programming and within
| 3-6 months your codebase is legacy.
|
| Java is just not fun to write for me, but Kotlin is kinda fun
| and I could see myself choosing that. The problem is that my
| product deals with a lot of deeply nested, user-defined
| objects, and converting JSON (or YAML, or XML) to POJOs is a
| verbosity nightmare.
|
| I just fucking really hate Go. I can't explain it, but I find
| the language infuriating. It somehow manages to be less
| expressive, more verbose and more straight up boring than all
| the other options.
| throwaway894345 wrote:
| > I just fucking really hate Go.
|
| As a Go fanboy, this made me chuckle. :)
|
| > I can't explain it, but I find the language infuriating.
| It somehow manages to be less expressive, more verbose and
| more straight up boring than all the other options.
|
| I sort of get it. I think people fixate too much on the for
| loops and the `if err != nil` boilerplate, but there's
| definitely some validity with respect to "how to properly
| annotate errors" and emulating enums (in the Rust sense of
| the word) is pretty error prone and it still doesn't get
| you exhaustive pattern matching.
|
| The stuff I like about Go:
|
| * Single static, native binaries - being able to just send
| someone an executable is pretty nice, not needing to make
| sure people have the right libs and runtime installed is
| phenomenal.
|
| * Great tooling. I love that the Go tool takes a list of
| dependencies. Unlike Gradle/CMake/etc I don't have to
| script my own build tool in a crumby/slow imperative DSL.
|
| * Compilation speed. Go compiles really fast.
|
| * Small language, easy learning curve. Any programmer can
| read just about any Go project with very little experience.
| Any Go programmer can jump into any other Go project and be
| productive immediately (no extensive configuration of IDE
| or build tools or anything like that). You can also write
| surprisingly abstract code without reaching for generics,
| but you will likely have to change your programming habits.
|
| * Value types done right. Most other mainstream languages
| don't have them, and the ones that do very often implement
| them poorly (e.g., IIRC in C# you can define a type as a
| struct or a class and only structs can be value types, and
| classes are more idiomatic).
|
| * Go eschews inheritance and prefers data-oriented
| programming over object-oriented programming (although OOP
| is poorly defined and some people think it means "anything
| with dot-method syntax").
|
| * I'm just more productive in Go than any other language
|
| Stuff I don't like about Go:
|
| * No enums
|
| * I wish there were fewer tradeoffs between abstraction and
| performance
|
| * Needs a better system for annotating errors
|
| * Doesn't run on embedded systems (although I've heard good
| things about TinyGo)
|
| * Doesn't make me feel as clever (this is a nice property
| for professional software development, but for hobbies not
| so much)
|
| On balance, I think Go is the better language, but I can
| also understand why it chafes people. Different strokes. :)
| AmpsterMan wrote:
| I am still fairly early in my career. I've worked mostly
| in Java, but my current job is a mix of Python for E2E
| testing, PHP and JS for application work. Go is my
| preferred language for tools.
|
| My favorite language is by far Java with Go a close
| second. I dislike the others. The only thing I wish Go
| had is more functional support, but only in the style of
| Java's fluent API. I strongly dislike PHP, JS, and Python
| functional style and would almost prefer they not exist.
| zeroc8 wrote:
| I on the other hand like the fact that Go keeps things
| simple and primitive.
|
| Don't need those functional constructs at all, allthough
| I'm using them quite often in Typescript.
|
| But for Go - please no. Go is like C. Adding things just
| makes it worse.
| reitanqild wrote:
| > I can't explain it, but I find the language infuriating.
| It somehow manages to be less expressive, more verbose and
| more straight up boring than all the other options.
|
| I don't hate it like you, I just find it amazingly
| backwards, and I think I can explain why:
|
| The community has the same persistence as certain teachers
| when it comes to locking things down and making things
| harder than necessary for no good reason, all for our own
| good.
|
| Prime example: announcing a supposedly cutting edge
| language without generics in 2009, launching version 1.0
| without it in 2012, 8 years after notoriously conservative
| Java got it, then waiting until 2022 before listening, that
| is some hardcore teacher mentality IMO.
| xmonkee wrote:
| Yeah, honestly I would respect them more if they never
| added generics. Still wouldn't use it. I haven't tried
| go-with-generics but I can't imagine it can be a good
| addition so late in the game.
| Xeoncross wrote:
| > I can define the problem in terms of types even before I
| start writing any executable code
|
| This is how code is supposed to be written. Sort out the
| logic first, don't write the actual implementation details
| until later (models, http handlers, file parsers).
|
| The support for this type of SOLID design is much more
| common in Go than in Typescript projects which often seem
| to inherit bad practices from Javascript developers.
| blueslurpee wrote:
| We have similar experiences. I absolutely love Typescript.
| But I don't like Node very much. So I keep TS in the
| browser.
|
| I've been looking at Nim or Scala for the backend. It's
| either that, or just dive into Rust.
|
| It really seems if you don't want to use Node, Java, or Go
| the available choices for a statically typed backend get
| quite slim.
| [deleted]
| ithrow wrote:
| What don't you like about nodejs? seems like a solid
| choice for most things, the only use one language
| (nodejs/browser) is very attractive (besides SQL).
| xmonkee wrote:
| Look at Kotlin if you haven't. Sure, "it's just Java" but
| really it's not. A good example is the collections all
| have an astoundingly uniform API and everything can map
| into everything else. It's really quite enjoyable.
| nazka wrote:
| Talking about Haskell and stuff. You what I want? Rescript to
| replace entirely Typescript. That will be so much better.
| golergka wrote:
| Personally, I love ML-family languages and would enjoy
| writing in Rescript more, indeed.
|
| But I write software not to enjoy the process, but to solve
| business needs. And using Rescript, Scheme or Haskell
| drastically limits the talent pool and rises the cost.
| Which, BTW, can be very good for some businesses, that
| would want to hire only the best programmers anyway and are
| ready to pay for it. But that's some, not all.
| nazka wrote:
| Yes I agree. Also if you know your business won't need a
| lot of dev, like stuff like Instagram where I think they
| were still at 60 engineers even after hitting 1B active
| users, it can also be a good thing to use a niche (but
| viable) language to attract some bright engineers and
| they will be thrilled to be able to have a job to use it.
| Now I don't have such startup (yet) so I can't do it :)
|
| I just wish ReScript would have gained traction and
| replaced Typescript. But who knows maybe it can still
| happen.
| still_grokking wrote:
| I would endorse that.
|
| TypeScripts's type-system soundness holes kill the fun with
| that language very quickly, imho. That was a really big
| disappointment for me.
|
| A "static" language that compiles fine and than crashes at
| runtime with complete WTF-bugs is not much better than a
| dynamic language. You just can't trust the compiler and
| need to double check every line of code anyway. So there is
| not much additional value in this kind of "type safety".
|
| I would prefer Rescript of Scala.js anytime to TypeScript.
| Got really disillusioned by TS.
| throwaway894345 wrote:
| I sort of agree, but I also think type systems are somewhat
| overrated compared to performance, static compilation, native
| compilation, easy cross compilation, good tooling, expansive
| ecosystem (esp devoid of C dependencies). I think TS and Go
| are on par with respect to tooling and ecosystem, but I would
| rarely trade Go's static, native compilation model
| (distributing a single static binary is pretty awesome) for
| some extra type system features.
|
| Specifically, I posit that returns on type system features
| diminish--there's a ton of value in going from a dynamic type
| system to Go's pre-generics type system, but every addition
| thereafter offers decreasing value.
| golergka wrote:
| > distributing a single static binary is pretty awesome
|
| It is, but does it really matter when you write backend?
| I've written and operated Node-based backends at scale, and
| I can't honestly remember any bugs or outages that would
| have been prevented by this.
|
| At the other hand, there's a lot of potential bugs and
| problems that didn't happen because proper usage of TS's
| type system prevented me from committing them.
| hu3 wrote:
| > I've written and operated Node-based backends at scale.
|
| I'm interested in producing fullstack typesafe web
| projects using typescript on both backend and frontend.
|
| My question is how do you scale nodejs since it is
| single-threaded? Say I need 10k concurrent connections.
| These connections doesn't have to be all hammering the
| system at once but I need to accept 10k connections and
| perform something like 2k requests/s.
|
| How do I approach this? One server with clustered nodejs?
| pm2?
|
| I ask because there are a ton of outdated info on how to
| scale nodejs.
|
| Some rudimentary benchmarking with clustered or pm2
| nodejs tells me that 95 response times go from 30ms to
| 1000ms once I hammer the server. Whereas Go and C# keep
| it under 100ms without any effort or extra tooling, just
| using standard library.
| throwaway894345 wrote:
| I'm not claiming a single static binary is going to
| prevent bugs or outages. It's just a surprisingly nice
| property (though less-so on the backend). For example,
| because Go distributes as a single static binary, we can
| make scratch images that weigh in at only a couple of
| megabytes. This (alongside fast compilation) means we can
| iterate more quickly in the cloud (pulling images onto
| nodes and starting them is almost instantaneous). This is
| particularly helpful when you have an outage and you're
| trying to get things un-stuck (you can try a lot of
| things in a short amount of time). It's one of those
| things that aren't obviously valuable until you've tried
| it.
|
| Again, it's not the end-all-be-all, but it's a
| surprisingly nice property.
| zaphar wrote:
| Python has high dev velocity if you only measure how quickly
| you can get started on a project. The dev velocity drops
| exponentially with the size and complexity of your codebase.
|
| But it is indeed quite fast when you are just trying to get
| your first file of code written.
| lowbloodsugar wrote:
| I would recommend Rust even if writing a simple tool, the kind
| of thing I might previously have used python for. We had some
| data munging to do and someone started it in python. On the
| real dataset it ran for an hour then ran out of memory. I took
| a crack in rust, as a n00b rust programmer, and the rust
| version runs successfully in less than a minute. I haven't
| tried server side yet, because the learning curve for that is
| more of a learning cliff, but for small things I am already
| faster in rust than with my decade of python.
| HeyImAlex wrote:
| I had the same experience with go, 100x performance
| improvements on data crunching scripts with roughly the same
| code. Worst part of writing it in python was simple
| misspellings in the last stages of the computation breaking
| things.
| vonseel wrote:
| > Worst part of writing it in python was simple
| misspellings in the last stages of the computation breaking
| things.
|
| I'm not sure I understand. Was the typo in something like a
| dict key? Or did you add two of the wrong variables
| together? How did another language protect you from this? I
| fail to see how you couldn't also add two of the wrong
| variables (with the correct type) together in a different
| language.
| vonseel wrote:
| >for small things I am already faster in rust than with my
| decade of python.
|
| That seems like a bold claim? I've also been writing Python
| for a decade and it takes no time at all to whip up a small
| script. I have no doubts the Rust version will be much
| faster, but stepping into a new language would take me at
| least longer to write the script than an equivalent version
| in Python. Plus there's all the other stuff to learn.
|
| Once you get past learning the basic syntax (creating
| variables, loop structure, assignment/etc.), you still have
| to pick up new things, like executing your work with
| parallelism, downloading files, parsing and dumping JSON,
| connecting to databases... Each new topic introduces a bunch
| of stuff that takes a little bit to figure out. I guess you
| could get through most of it in a few hours though if you had
| the right problem to work on.
|
| But still, there's the muscle memory aspect. Can a new
| language really sink in so fast that it's faster to use than
| Python after 10 years of writing Python?
| queuebert wrote:
| Completely agree. Rust shines as a language for small tools
| that need to be performant and correct, like Unix utilities.
| Exa and fd are great examples of this.
| peterkelly wrote:
| A lot of people seem to believe that dynamically typed
| languages make development easier, because you don't have to
| worry about writing type annotations or going through the extra
| steps to fix compile-time errors.
|
| This is arguably true for small scripts, but for anything non-
| trivial I find that static typing means I can be _more_
| productive, because type errors are caught straight away
| (rather than potentially in production), and I can be much more
| confident in refactoring code because I know certain kinds of
| errors will be caught by during type checking. This doesn 't
| remove the need for automated testing, but does reduce the
| number of test cases you have to write.
|
| Things have improved a lot in the Python ecosystem of late with
| the introduction of type annotations and mypy, so to the extent
| possible I treat Python as if it's a statically typed language,
| putting type annotations everywhere and always making sure the
| codebase passes type checking before pushing a commit.
|
| However, the fact that this is optional sadly means that not
| everyone does it, and if you're working with parts of a
| codebase written without this level of discipline, you're
| basically back to square one. I've come to the conclusion that
| for medium to large projects, dynamic typing encourages sloppy
| programming, because it doesn't force people to think as
| carefully about what exactly their data types are, and whether
| they're being used correctly.
| jimcavel888 wrote:
| tlrobinson wrote:
| > I find that static typing means I can be more productive,
| because type errors are caught straight away
|
| I agree completely. I feel like there's a pretty common arc
| among programmers:
|
| 1. learn to program using verbose statically typed languages
|
| 2. discover fun dynamically typed languages, eschew
| statically typed languages
|
| 3. discover dynamically typed languages are a shitshow for
| large real-world projects
|
| 4. re-discover fun/less verbose statically typed languages
| that make working on large projects tolerable
| tralarpa wrote:
| I hate that some universities teach dynamically typed
| languages as first language in their curriculum. I think
| for CS students the first language should be as "strict"
| (not in the mathematical sense) as possible, so the student
| is forced to think very carefully what is going where. This
| is of course a controversial opinion, especially among
| proponents of SICP (which is, in my opinion, not a good
| book for beginners. Yes, downvote me).
|
| (Obviously, I am talking about CS students here, not about
| the general public who just want to learn what writing a
| program means)
| amelius wrote:
| Except keep in mind that Rust adds more constraints than
| your typical statically typed language.
|
| It is still more fun _and_ efficient to program in a
| language that is garbage collected _and_ statically typed.
| einpoklum wrote:
| About no. 4: That often translates into noticing C++ has
| gotten `auto` in C++11 with a later improvement in C++14:
| auto x { get_something_complicated() };
|
| or foo(int x) -> auto { return
| get_something_complicated(bar(x)); }
|
| so it's less "not-fun" these days.
| kitsunesoba wrote:
| > for anything non-trivial I find that static typing means I
| can be more productive, because type errors are caught
| straight away (rather than potentially in production), and I
| can be much more confident in refactoring code because I know
| certain kinds of errors will be caught by during type
| checking.
|
| Fully agreed. It's not quite the same as Rust vs. Python, but
| I've pulled off several sweeping refactors and rewrites in
| Swift codebases that I wouldn't dream of even attempting with
| Objective-C.
| throwaway894345 wrote:
| > I can be much more confident in refactoring code because I
| know certain kinds of errors will be caught by during type
| checking.
|
| I'll add to that: static typing also guarantees that your
| annotations are correct and complete. Prior to Mypy, in the
| Python world, we would document types in docstrings, which
| meant that docstrings gradually came to be _incorrect_ --a
| function that returns a string might sometimes return `None`,
| but you called it assuming it always returned a string. In
| other cases, people (including the Python maintainers via the
| stdlib) will add annotations like "argument 'foo' is a file-
| like object", which is ... insufficient (do I just need a
| "read()" method? or do I need to implement "seek()" and
| "close()" as well?). Ultimately, in the absence of static
| analysis, a lot of time is wasted reading source code trying
| to understand the actual type signatures.
|
| > Things have improved a lot in the Python ecosystem of late
| with the introduction of type annotations and mypy, so to the
| extent possible I treat Python as if it's a statically typed
| language, putting type annotations everywhere and always
| making sure the codebase passes type checking before pushing
| a commit.
|
| This hasn't been true in my experience. Last I checked, Mypy
| still couldn't express things like recursive types (think
| "JSON" or "recursive linked list"), and a lot of things still
| required jumping through obscure hoops (defining a callback
| with kwargs). I also couldn't figure out how to export type
| annotations in my packages, and I had a lot of trouble
| getting mypy to pull in type stubs for packages that didn't
| provide their own annotations. In general, Python's typing
| story still feels very shoe-horned, although it's been a
| minute since I last fumbled with it, so maybe some of this
| has improved.
| [deleted]
| rjh29 wrote:
| Having recently used it, my impression is Python's type system
| is pretty damn powerful. It might not be as crazy as
| TypeScript, but it can handle recursive types, dictionaries
| with a fixed set of keys, functions, union types and generics.
| Types can be re-used or exported. There are generic types for
| things like 'Iterable'.
|
| If a package does not supply its own types (increasingly rare
| now), you can generate stubs and put them in typings/$package.
|
| It's picked up a bunch of bugs for me already. And if runtime
| performance is not a priority, I'd rather write Python than Go
| or Rust any day. It's more fun and more expressive.
| [deleted]
| zmix wrote:
| > Use Go if you're looking for an easy GC language with max
| productivity and a decent performance ceiling.
|
| Why not D?
| throwaway894345 wrote:
| I haven't used it. As I mentioned, I'm sure there are other
| languages that fit the bill besides Go and TypeScript.
| salamanderman wrote:
| I've seen the transition from a Javascript codebase to a
| Typescript codebase with all strict checks in place to have a
| similar step change in dev velocity. The speed of debugging
| stupid mistakes, confidence in larger changes, and removal of a
| need for a lot of unit tests that were poor-person's type
| checking, is really significant. And runtime errors are a lot
| less weird than they were.
| mhink wrote:
| > I've seen the transition from a Javascript codebase to a
| Typescript codebase with all strict checks in place to have a
| similar step change in dev velocity.
|
| The important factor here is "with all strict checks in
| place". Whenever I hear fellow FE devs complain about how
| they dislike Typescript, the complaints often seem to center
| around how it doesn't really offer that many guarantees...
| only to find out that their typechecker is set to the least
| strict settings!
|
| I'm not gonna lie, though: the transition from JS to TS
| absolutely comes with growing pains. This obviously includes
| actually updating the codebase and adding types, as well as
| the developer onboarding/learning process. Personally
| speaking, I found it very jarring to go from a permissive,
| lightning-fast JS dev cycle to a more methodical Typescript
| one.
|
| And don't get me started on how frustrating it can be to
| diagnose weird type errors related to generic types :D
| hnfong wrote:
| I think this is a bit unfair. As always, blanket statements in
| software engineering aren't really meaningful.
|
| As I understand it, the article talks about switching from
| python to rust in a non-trivial database service that is
| required to be fast and robust. I can't imagine python to do
| well in this regard, especially when C/C++ extensions are
| required to enhance the performance.
|
| I also agree that 'Python has been a strict liability and the
| promise that you can "just rewrite the slow parts in
| C/C++/Numpy/etc" has never held'.
|
| I just don't agree those points necessarily imply "avoiding
| Python for new projects". Even in the case of the original
| article, was it, in hindsight, a bad decision to start with
| python? I'd argue not necessarily. Python is still great for
| quickly hacking together something that works, not necessarily
| fast or maintainable, but if you're under time pressure or
| you're still trying to feel out the market fit for your
| product, it's not a great idea to waste brain cycles to deal
| with borrow checks for example.
|
| Python's weaknesses only show when you need to scale, both on
| the performance and on the lines of code / number of
| collaborators metric, which is a problem you have only when the
| project succeeds. Avoiding python because of these problems
| reeks of premature optimization, unless you know it's how the
| project will end up in advance.
|
| As other sibling comments point out, python is still great for
| small scripts and weekend projects. Perhaps you do mean
| 'avoiding Python for new projects _that are expected to grow
| non-trivially_ ', which is fine advice. I'm just a bit tired of
| all those over-reaching blanket statements people make
| regarding software design that only really makes sense in a
| limited context (here's a funny one I've heard recently: never
| write nested-for-loops because it's _sloooww_ ). It would be
| unfortunate to have the message "don't use python for large
| projects" be misinterpreted as "python sucks no matter what".
| throwaway894345 wrote:
| > As I understand it, the article talks about switching from
| python to rust in a non-trivial database service that is
| required to be fast and robust. I can't imagine python to do
| well in this regard, especially when C/C++ extensions are
| required to enhance the performance.
|
| The article specifically talked about following the
| conventional Python advice to use C/C++ for the fast parts
| and Python for the "glue". This is precisely where Python
| people say Python shines.
|
| > I just don't agree those points necessarily imply "avoiding
| Python for new projects".
|
| You should avoid Python for new projects because you never
| know at the outset whether you will run into a performance
| bottleneck that can't be resolved by "just rewrite in
| C/numpy/etc" and there are other languages that are better
| than Python at both its strengths and weaknesses. The odds of
| painting yourself into a corner with Python are very, very
| high, and there are other languages that are better than
| Python at just about everything (the only area where Python
| still dominates is data science exploration libraries). Even
| if you don't paint yourself into a corner, you'll be dealing
| with lots of surprising papercuts: if you want reproducible
| builds, you'll have to wade through an ecosystem of package
| managers each with their own pitfalls including long build
| times (e.g., I saw pipenv take 30-45 _minutes_ just to
| `pipenv lock` on a small-medium sized project); developing
| with docker basically doesn 't work on mac (and windows?)
| because the VM uses all available CPU marshaling filesystem
| events over the guest/host boundary; your developers
| accidentally pulled in an API SDK that makes sync I/O calls
| (or else does some CPU-intensive task) which seizes up the
| event loop bringing the process down (and the failure
| cascades to remaining instances bringing the whole app down);
| and then there's the "regular" dynamic type system errors
| like forgetting to "await" things or missing/incorrect type
| documentation.
|
| > Even in the case of the original article, was it, in
| hindsight, a bad decision to start with python? I'd argue not
| necessarily. I'd argue not necessarily. Python is still great
| for quickly hacking together something that works, not
| necessarily fast or maintainable, but if you're under time
| pressure or you're still trying to feel out the market fit
| for your product, it's not a great idea to waste brain cycles
| to deal with borrow checks for example.
|
| My point isn't that Python is bad for _literally every
| project_ --I actually think this project was probably a best-
| case scenario for Python (they didn't describe many of the
| issues I described in my previous paragraph) and they knew
| ahead of time where there performance issues would be so they
| were able to _start_ with an architecture that was amenable
| to "write the slow parts in C/C++". My point is that there
| are languages that are better at fast iteration than Python
| (namely Go or TypeScript) while also excelling at Python's
| weaknesses (performance, tooling, distribution, type system,
| package management, etc etc etc). I think it's incredibly
| reckless to start a new commercial project in Python in
| almost all cases (with a possible exception for data science,
| but even then I would look hard at everything else first).
|
| > Python's weaknesses only show when you need to scale, both
| on the performance and on the lines of code / number of
| collaborators metric, which is a problem you have only when
| the project succeeds.
|
| This might be true for certain Django CRUD apps, but it's
| patently untrue in the general case. I've seen Python work
| fine for a prototype data science app, but fall over when
| real customer data was introduced. Similarly, I'm currently
| an SRE attached to a Python project which falls over
| regularly because of some event loop issue or another (often
| someone pulls in an SDK that makes sync calls under the hood,
| or else some CPU-intensive task blocks the event loop). I see
| stuff like this regularly, and it never happens in our Go
| apps.
|
| > As other sibling comments point out, python is still great
| for small scripts and weekend projects.
|
| Granted. Do what you want in your free time, but it's risky
| to use Python for new commercial apps.
|
| > Perhaps you do mean 'avoiding Python for new projects that
| are expected to grow non-trivially', which is fine advice.
|
| Yes, this is what I meant. I didn't mean "hobby projects".
|
| > I'm just a bit tired of all those over-reaching blanket
| statements people make regarding software design
|
| I don't usually make blanket statements, but Python really is
| so bad that this is a blanket statement I'm pretty
| comfortable with. I've scarcely ever heard of someone getting
| burned for picking Go over Python (there was some Python2
| veteran who wrote an angry blog post once about Go back in
| the early days, but never since). Python just refuses to
| _adequately_ progress because it wants to maintain
| compatibility--that 's fine, but as professional engineers we
| should understand that the rest of the ecosystem is moving
| beyond where Python is willing/able to go, and it's no longer
| an appropriate tool for the overwhelming majority of new
| projects.
| infamia wrote:
| > This might be true for certain Django CRUD apps, but it's
| patently untrue in the general case. I've seen Python work
| fine for a prototype data science app, but fall over when
| real customer data was introduced.
|
| The cognitive dissonance embedded in this statement is
| rather striking, considering Django itself is written in
| Python and is known for its ruggedness and code quality.
| Django manages all of this without even taking advantage of
| somewhat newer features in Python like typing.
| throwaway894345 wrote:
| There's no cognitive dissonance, you just misunderstood
| the claim. :)
| qsort wrote:
| > I think it's incredibly reckless to start a new
| commercial project in Python in almost all cases...
|
| > I don't usually make blanket statements, but Python
| really is so bad that this is a blanket statement I'm
| pretty comfortable with.
|
| Speaking as someone who agrees with you that something
| static is very likely better than python for a large
| project, how do you square this with the existence of a lot
| of Python projects, even "very commercial", that seem to go
| on just fine?
|
| It's a fine theory and something I instinctively support,
| but I don't see measurable evidence in support of it.
| throwawaymaths wrote:
| Those projects were started when python was the least bad
| choice and grew out quickly enough that they could hire
| legions of developers to work around pythons problems.
| pdimitar wrote:
| Just fine, you say? I had dozens of colleagues being in
| medium to large Python projects and every single one of
| them was terrified of every small change in the projects
| because inevitably something fails in an unexpected way.
|
| These guys and girls are heroes. They are like the
| plumbers and electricians working at 3:00 AM so your
| water and electricity never stop while you're awake.
|
| "Just fine" is likely only your external observation. I'd
| give them medals and help them retire at 35 for all the
| horrors they have endured. It's "just fine" because they
| tirelessly made sure of it.
|
| Things were just fine _despite_ Python. Not _because_ of
| Python.
|
| Plus, none of us has statistics that prove a theory
| beyond any doubt. Forget about that; this scarcely (if
| ever) existed in the programming world and likely will
| not exist anytime soon. We should apply our experience
| and the intuition that grows out of it.
|
| Finally: a pile of anecdotal evidence can be treated as
| an objective evidence with, say, 50-60% probability.
| Insisting that the real world must comply with
| academically pure practices is not going to get us
| anywhere (and is misguided).
| throwaway894345 wrote:
| Survivorship bias. You don't hear about the failures.
| Moreover, many of those things (e.g., lots of Python at
| Google) was written at a time when there weren't better
| options, so you weren't struggling to make Python work
| while your competitor is writing Go or TypeScript.
| sharno wrote:
| I'd argue that C# (dotnet core) is a much better option than Go
| for an easy GC language with max productivity and great
| performance ceiling.
| labrador wrote:
| C# 2022 is much improved over earlier versions. For starts
| it's cross platform and compiles to reasonably sized single
| executables (touted as a strength of Go). Moreover, C#
| designers have started reducing boilerplate as much as
| possible, so for example, you don't need to declare a
| namespace, a class or a main to start coding.
|
| That said, Go is a fine language. It doesn't have exceptions
| last I checked, so that could be a deciding factor.
| Exceptions are handy in a lot of cases.
| throwaway894345 wrote:
| I haven't worked with C# since ~2013, but I mostly liked it.
| I'm sure it's a different beast now though. Some things I
| prefer about Go:
|
| 1. Tooling. `go build` just needs a `go.mod` file with a list
| of dependencies. There's no MSBUILD stuff (I think dotnet had
| a similar JSON file format, but then went back to MSBUILD?)
|
| 2. Runtime. I love that Go's default is static, native
| binaries, and that the ecosystem revolves around that. I
| especially love that this enables ~2mb Docker images. (I'm of
| the impression that you can do this in dotnet, but it's not
| clear to me how easy it is in practice or if there are
| performance tradeoffs)
|
| 3. Value types. Go doesn't have a classes; everything is a
| struct, and polymorphism and so on work on structs.
|
| 4. No inheritance. I'm pretty militantly opposed to
| inheritance. I know you don't have to use it in C#, but in Go
| I also don't have to deal with APIs or coworkers that assume
| inheritance.
|
| That said, these aren't major things; I'm sure C# is a great
| language in general, and I'd definitely reach for if I wanted
| to make a Windows GUI app.
| nly wrote:
| Garbage collector is still a steaming pile though
| metaltyphoon wrote:
| Not only that, but greater tooling too.
| keyle wrote:
| I said this for years... no type checking by the compiler? Then
| you are the compiler.
| lijogdfljk wrote:
| > Use Go if you're looking for an easy GC language with max
| productivity and a decent performance ceiling. Use Rust if
| you're writing really high performance or correctness-is-
| paramount software.
|
| I'd add one more addition to use Rust (speaking from an ex-Go
| dev): Use Rust if you want a robust std lib. Go is good, i used
| it for ~5 years, but man Rust was a breath of fresh air with
| the amount of tooling that helped me solve problems. Iterators,
| are a great example.
|
| I also preferred Crates vs Github Repos, but i think that's
| just personal preference.. not objective.
| johnisgood wrote:
| When performance and correctness are paramount, I personally
| stick to Ada/SPARK. Proven to be perfect for the job, and to
| be honest, Rust is quite a difficult language and I am not
| sure it is worth learning it considering that Ada/SPARK ticks
| all boxes when it comes to both low-level and high-level
| programming for critical systems.
| throwaway894345 wrote:
| I have fond memories of using Ada in college. Unfortunately
| it didn't have much of an open source footprint and the
| community was downright hostile toward newbies (more so
| than other communities that I had experienced). I'm sure
| Ada has the stronger story for real-time embedded systems
| today, but (as I have learned Rust in the interim) I look
| forward to the day when Rust breaks into the real-time
| embedded space. In particular, I'm excited about ferrocene
| and the broader Rust certification efforts. I would really
| like to write embedded Rust professionally, and I hope
| certification will boost the number of embedded Rust jobs
| (no the to Ada, I just don't particularly want to re-learn
| it).
| deagle50 wrote:
| My favorite thing about learning Rust is how helpful the
| community is.
| kylereeve wrote:
| Do you work in Ada/SPARK professionally? Curious what
| industry if so.
| dls2016 wrote:
| > but man Rust was a breath of fresh air with the amount of
| tooling that helped me solve problems
|
| The tooling is the #1 reason I'd like to learn rust. I have
| not kept up with C++ and I'm not sure I ever will...
| sometimes plain C seems more straightforward. But those
| languages (C++ especially) have suffered from lack of
| standard tooling, in my opinion.
|
| Even C# being a few years younger than Java seems to have
| made a huge difference in tooling availability (I'm sure
| having a single corporate driver during its lifetime made a
| difference, too).
|
| I'm excited by the prospect of a compiled language with
| modern tooling.
| throwaway894345 wrote:
| Tooling was the reason I left C++. Having to do your own
| package management and script your own build system was
| deeply dull work. The IDE story was also pretty miserable,
| although I think clangd has improved it a bit. Getting
| decent information out of core dumps also sucked a lot. The
| language actually didn't bother me too much after c++11,
| but everything else was such a bear that I jumped ship.
|
| Rust has been very rewarding in that it largely fixes all
| of C++'s tooling _and_ language problems. It very much is a
| significantly improved C++.
| synergy20 wrote:
| I actually filed a ticket at Carbon project saying c++'s
| top 1 problem is tooling, modern c++ is pretty nice to
| use already, no need for Carbon there in fact, at least
| not as needed as tooling.
| Thaxll wrote:
| Rust std lib is subpar compared to the Go one, yes you have
| iterators but that's it, basic things like async are not even
| provided just the interface so everyone has to use tokyo.
| Then for real use cases you're missing
| http/json/compression/crypto etc ...
|
| https://pkg.go.dev/std
|
| Overall tooling and std lib are better on Go, actually there
| are not many languages that are on part with Go to that
| regard.
|
| When you see what you can do with the single go command, it's
| pretty powerful.
| jxf wrote:
| Quick correction for anyone searching: it's `tokio`, not
| `tokyo`. [0]
|
| [0]: https://tokio.rs/
| skrtskrt wrote:
| I think this comes with the caveat that there are a lot more
| mature libraries for Go just as a function of the age of the
| language.
| pdimitar wrote:
| That might have been true 2-3 years ago but nowadays
| whatever I can think of, Rust already has a library for it.
| skrtskrt wrote:
| Most recently I've run into Hyper not having tracing
| support
| lumost wrote:
| Agree, you can see this in https://github.com/rust-
| unofficial/awesome-rust vs.
| https://github.com/avelino/awesome-go
|
| I've noticed however that there has been an uptick in
| _great_ libraries over the last 2 years, with examples like
| pola.rs, rust-bert, tokenizers etc. starting to build
| momentum in the ecosystem.
| throwaway894345 wrote:
| I actually don't care as much for iterators as I thought I
| would. Beyond some simpler map().reduce() stuff they tend to
| fall over pretty fast, and I end up spending too much time
| trying to make iterator chains work before defaulting to for
| loops. I also dislike how anemic Rust's stdlib is. I'm sure
| there are good reasons, but I like that I can just reach for
| Go's stdlib for annotating errors or dealing with JSON. Other
| than that, I like the API design in Rust's stdlib.
|
| Here's an example where the iterator-chain version is way
| more complex: let posts =
| read_dir(source_directory)? .filter_map(|result|
| { (|| { let entry =
| result?; let os_file_name =
| entry.file_name(); let file_name =
| os_file_name.to_string_lossy(); if
| Self::is_bundle(&entry)? {
| self.parse_post_bundle(&file_name, &entry.path())
| .map(Some) } else if
| file_name.ends_with(MARKDOWN_EXTENSION) {
| self.parse_post(
| file_name.trim_end_matches(MARKDOWN_EXTENSION),
| &mut File::open(&entry.path())?, )
| .map(Some) } else {
| Ok(None) } })()
| .transpose() })
| .collect::<Result<Vec<_>>>()?;
|
| Here's the imperative version: let mut
| posts = Vec::new(); for result in
| read_dir(source_directory)? { let entry =
| result?; let os_file_name = entry.file_name();
| let file_name = os_file_name.to_string_lossy();
| if Self::is_bundle(&entry)? {
| posts.push(self.parse_post_bundle(&file_name,
| &entry.path())?); } else if
| file_name.ends_with(MARKDOWN_EXTENSION) {
| posts.push(self.parse_post(
| file_name.trim_end_matches(MARKDOWN_EXTENSION),
| &mut File::open(&entry.path())?, )?);
| } }
| SoftTalker wrote:
| I guess that's subjective, not knowing rust I see a few
| more lines in the first example but beyond that it does not
| jump out at me as being "way more complex."
| throwaway894345 wrote:
| Hah, it took me and a bunch of other people in the Rust
| discord a fair amount of time to figure out how to get
| the first example to compile correctly in the first
| place, and I find the "transpose()" business to be
| particularly convoluted. I think even Rust people would
| prefer the imperative version. I also suspect there's
| some hindsight benefit at play in that it's easier to
| make sense of the first snippet than it was to write it.
| tialaramex wrote:
| Yes, the imperative version looks like the sane choice to
| me. Rust's iterator documentation does suggest that this
| is not a language which favours iterator chains
| everywhere.
|
| However one reason to prefer an iterator chain in some
| cases is that the chain might imply an optimisation you'd
| otherwise have to write manually and might not think of.
|
| For example if I have N things, and I map them, perhaps
| more than once, but I'm definitely getting N of whatever
| the output of the last map was, the iterator chain might
| very well see that I don't actually have any way to
| change N and so when I collect() that into a Vec the Vec
| it gives me is constructed with capacity N, saving the
| (amortized but non-zero) cost of growing it during
| insertions.
|
| Imperatively I can remember to write
| Vec::with_capacity(N), and that's safe of course but it's
| an extra step to remember.
|
| The imperative approach does particularly shine on the
| opposite edge of this, if I know that I'm getting no more
| than 100 items out of this pipeline, despite putting N
| in, Rust almost certainly won't see that and won't make
| Vec::with_capacity(100) for a collect() call whereas my
| imperative code can just explicitly write
| Vec::with_capacity(100) or whatever.
| sgeisenh wrote:
| Depending on how you write it, iterators are much better
| at this since most iterator methods produce iterators
| with size hints.
|
| If you use "take" for example, collect should do the
| right thing: https://doc.rust-
| lang.org/src/core/iter/adapters/take.rs.htm...
| tialaramex wrote:
| Mere hints aren't enough for collect()
|
| It will only care about your hints if you implement the
| unsafe trait TrustedLen (which promises those hints are
| correct)
|
| Take is indeed TrustedLen if the Iterator it's taking
| from is TrustedLen (as in this case it can be sure of the
| size hint)
|
| If you get to ~100 via some means other than take()ing
| 100 of them, chances are you don't have TrustedLen as a
| result.
| Alexendoo wrote:
| TrustedLen will mean it can safely take the upper bound,
| but Vec for example will still use the lower bound of the
| hint when something isn't TrustedLen
|
| https://doc.rust-
| lang.org/src/alloc/vec/spec_from_iter_neste...
| tialaramex wrote:
| Ooh nice, I hadn't seen that. OK, so there are probably
| places where I assumed the iterator won't do as nice a
| job of collect() as I did manually but I was wrong.
|
| I wonder how the hell I didn't see that when I was
| tracking down how collect() ends up caring about
| TrustedLen.
|
| Thanks.
| ywei3410 wrote:
| Honestly the closure + `transpose` is really ugly as sin.
| I had to do a double take and reason through the steps
| which you took to get that particular snippet (while not
| reading the imperative one).
| throwaway894345 wrote:
| Thanks, I feel validated. (:
| mamcx wrote:
| The thing is in Rust this is totally ok. Is NOT un-
| idiomatic to mix imperative + functional paradigms in the
| same block.
| throwaway894345 wrote:
| Agreed, but I think people make too big a deal about
| iterators. The simple for loops that they're fit to
| replace aren't that hard to read or write, so they aren't
| causing real problems, and on the more complex end you're
| writing imperative stuff anyway. Iterators certainly
| aren't a bad thing, but they're not the game changer that
| Go-critics made them out to be in all of the Rust-vs-Go
| debates.
| mustache_kimono wrote:
| > Agreed, but I think people make too big a deal about
| iterators.
|
| I may just have more time to dick around, but I've been
| forcing myself to write everything as an iterator in Rust
| (instead of a for loop) just to learn the paradigm and,
| honestly, it's been pretty great.
|
| First, there is a performance impact. You may think this
| would be small, but, in my experience, it can be
| relatively large. There must be several reasons of which
| I am ignorant, but they probably include mutable no alias
| inside closures, bounds check elision, etc.
|
| Second, there is a mutability advantage to just using
| filter/flatten/map/collect for 90% of your use cases.
|
| Third, the more advanced iterator methods will cover the
| other 5-10% of your use cases, and will make your code
| much easier to read, once you understand how to use them
| (if you're trying to learning something/anything more
| complex at the same time, save your brain power!). I
| don't have a single traditional for loop in my personal
| projects.
|
| My take on iterators is, for iterator/Rust beginners
| (like me!/perhaps not you) -- the best way to learn is to
| get it working as imperative code, and when you have the
| time, go back a rewrite as an iterator. In the beginning,
| it's super frustrating. But you'll get better and better
| and, after awhile, you'll much, much prefer writing as an
| iterator. You won't even think of using a for loop.
| throwaway894345 wrote:
| I mean, I definitely have _tried_ to rewrite stuff as
| iterators even when it isn 't convenient. I only got to
| that working example by dedicating a few hours with the
| good people on the Rust Discord trying to make it work (a
| lot of seasoned Rustaceans also weren't able to get it
| working, it was a collaborative effort that took a fair
| amount of time and rhapsodizing). And even then, the
| result is pretty ugly (I don't think many seasoned
| Rustaceans would call it "idiomatic" or advise the first
| version over the second).
| mustache_kimono wrote:
| Agreed. Your example _is_ a pretty pathological case, but
| yeah.
| efnx wrote:
| The selling point of iterators isn't that they replace
| for loops, it's that you can make new iterators by
| implementing a trait. That at way downstream users can
| use your iterators in a for loop. Of course you also can
| use a more functional style if you like.
| nomel wrote:
| I disagree. If I can avoid an index variable, I'm going
| to have less bugs since I won't have to index.
| throwaway894345 wrote:
| I'm not sure what you mean. You don't have to deal with
| index variables in a for loop: for item
| in iterator { vector.push_back(foo(item));
| }
|
| You're not explicitly indexing the source iterator or the
| destination vector at any point.
| nomel wrote:
| Agreed. That's why I enjoy iterators, and use them
| whenever possible. I think they're important.
| throwaway894345 wrote:
| But earlier you claimed the for-loop version required
| index variables and indexing into iterators/vectors/etc
| ... ?
| lr1970 wrote:
| > If I can avoid an index variable, I'm going to have
| less bugs since I won't have to index.
|
| Not only that. In safe Rust vector indexes are always
| bound-checked that slows execution a little bit.
| Iterators over a vector, on the other hand, do not suffer
| from bound-check overhead.
| lvass wrote:
| The first version isn't what I'd write in a purely
| functional language either, a recursive version would be
| much saner. Isn't recursion idiomatic in rust?
| throwaway894345 wrote:
| In addition to Steve's sibling comment, I'm responding
| narrowly to iterators versus for-loops. Recursion is out
| of scope.
| steveklabnik wrote:
| Rust doesn't guarantee TCO and so recursion is a bit
| dicer than in many functional languages.
| nu11ptr wrote:
| There is no need to force something iterator style. By all
| means use a loop if it is more readable. Also, if your
| iterator style closure body gets very verbose, you can
| always pull the logic out into a function or closure and
| pass it in. In the end, both styles are idiomatic, and the
| best choice is which makes it more readable (subjective of
| course).
|
| I personally use iterator-style for 1-3 simple ops, but go
| into a loop for anything more complicated. The functional
| part of me wants to force iterator style but I fight it for
| complex use cases because it is irrational on my part, and
| I find loops easier to read for stuff like that.
| drran wrote:
| Now, compare a parallel version of both.
| throwaway894345 wrote:
| It's not obvious to me that the parallel version of the
| first is going to be cleaner (or more performant) than a
| parallel version of the second, but even if it is, I
| don't agree with mangling (to put it nicely) single-
| threaded code on the off chance that it might be made
| parallel one day.
| thinkharderdev wrote:
| Yep, agree wholeheartedly with this. I came to Rust from
| Scala where writing an explicit for loop is looked at as
| one step above using goto :) and it took me a while to just
| give in and write the imperative version.
| 0x457 wrote:
| Iterator-chain version would be a lot better looking if it
| was actually was a chain and not everything pushed into a
| single `filter_map`. "Ifs" are weird looking, one branch is
| using `entry`(I take it's a `DirEntry`?) and another is
| using `file_name` I suspect both could use the and look
| like `Self::is_` ?
|
| If you're going to write an iterator chain in an imperative
| way, then you might as well right it imperatively.
|
| A few notes:
|
| - you can compare `&OsStr` with `&str` without
| `to_string_lossy` any explicit conversion (`&str -> &OsStr`
| done in `PartialEq` is basically free)
|
| - you can get extension from `Path` by using
| `Path::extension`
|
| - you can get filename without extension by using
| `Path::file_stem` or `Path::file_prefix`
|
| - making your parse functions take the same arguments makes
| it a lot cleaner
|
| Something like this: read_dir("")?
| .map(|e| e.map(|e| e.path())) .map(|path| {
| path.and_then(|path| { if
| Self::is_bundle(&path) {
| self.parse_post_bundle(&path.file_name().to_string_lossy(),
| &path).map(Some) } else if
| Self::is_markdown(&path) {
| self.parse_post(&path.file_stem().to_string_lossy(),
| &path).map(Some) } else {
| Ok(None) } }) })
| .filter_map(Result::transpose)
| .collect::<Result<Vec<()>, std::io::Error>>()
| // Ignore this, just an example fn is_bundle(path:
| &Path) -> bool { path.extension().map(|ext| ext
| == "bundle").unwrap_or(false) } fn
| is_markdown(path: &Path) -> bool {
| path.extension().map(|ext| ext == "md").unwrap_or(false)
| }
| lijogdfljk wrote:
| Fair, but i'd probably write your iterator example
| differently. Though, besides the point perhaps - as i agree
| there are plenty of cases where imperative approaches are
| cleaner.
|
| I however will often have a dozen maps/filters/etc and that
| style in imperative is what, nesting repeatedly or using
| `continue`. Meh.
|
| If your Iterator consists of a single `filter_map` i can
| see why you don't get much value out of them in that case.
| It's because.. well, there's not much value to get out of
| it.
| Jack_rando_fang wrote:
| Agreed. I still give Python credit for allowing us to write
| software in an extremely efficient way. But after I wrote the
| initial version of the software with Python, I always
| procrastinate on testing, fixing existing issues, and as the
| software grows larger I simply want to give up as the different
| issues keep piling up. In comparison, Rust code usually just
| works out of the box due to the error handling and type system
| designs. With rust I develop much more consistently, and even
| more efficiently as the project progresses due to IDE hints,
| rust's built in `cargo check`, `cargo clippy` commands etc.
| nprateem wrote:
| Don't you find thinking about the borrow checker unnecessary
| overhead while prototyping?
| jhgg wrote:
| No. Not anymore. 4 years in and I maybe run into a borrow
| checker issue very rarely in a given week.
|
| Once you understand how the borrow checker works and how to
| write/structure your code in a way that the borrow checker
| is happy with it sort of becomes second nature.
|
| I think by side effect coding in this style does lend
| itself to simply writing better to understand code.
| Nowadays I just write my code this way without much
| thought. It happens effortlessly. I find myself writing
| programs in other languages like I would in Rust nowadays.
|
| This is compared to when I started and ran into borrow
| checking issues many times a day. Additionally the borrow
| checker has improved since then and NLL borrow checker will
| accept more programs as valid than the previous one.
| pdimitar wrote:
| Of course. That's rarely debatable (unless making sure that
| you have a very small memory footprint is one of the
| project's critically important goals).
|
| The point is that you pay more upfront but can then iterate
| much more confidently and quickly.
| agumonkey wrote:
| Started a small project with rust and serde(toml config).
| Having static checking of config an its parsing felt really
| good.
| Gigachad wrote:
| We started just writing configs in typescript which has the
| benefit of static checking in your editor with
| suggestions/underlines. Could likely do the same in rust.
| _benj wrote:
| Also, +1 for Go. I think that one of the things that makes Rust
| and Go such great language is how strict their compilers are.
| This moves the cognitive load of writing same code from the
| developer/team, to the compiler.
|
| Go is also simpler because of the GC, given that Rust can get a
| bit tricky with lifetimes. Go routines and channels are a
| delight to use together with contexts.
|
| Rust allows for a lot more ways to solve one problem, i.e. you
| can iterate over an array, of use a loop. Go only provided a
| for loop and that is pretty much the only way to iterate over
| something.
|
| Some people might find this limiting but I find it refreshing
| given that I can focus on what I need to do instead of how I
| should do it.
___________________________________________________________________
(page generated 2022-08-24 23:01 UTC)