[HN Gopher] Ruff - a fast Python Linter written in Rust
___________________________________________________________________
Ruff - a fast Python Linter written in Rust
Author : amrrs
Score : 113 points
Date : 2022-08-31 18:05 UTC (4 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| BenGosub wrote:
| Black has been clearly unable to catch up Prettier for years. We
| still don't have format selection and often times formating
| doesn't work for different reasons. On the other hand I love how
| consistent Prettier is, I hope for a similar experience, will
| definitely give Ruff a try.
| LittlePeter wrote:
| > We still don't have format selection
|
| And you never will. From https://github.com/psf/black:
|
| Black is the uncompromising Python code formatter. By using it,
| you agree to cede control over minutiae of hand-formatting. In
| return, Black gives you speed, determinism, and freedom from
| pycodestyle nagging about formatting. You will save time and
| mental energy for more important matters.
| mariocesar wrote:
| The project is testing for 9 rules.
| https://gitlab.com/zapier/team-integrations/zapier-platform-...
| Will you add more with plugins in cargo? DSL? Python? It's not
| much comparable to anything right now
| cercatrova wrote:
| As I mentioned in another comment [0], compiled languages are the
| future of interpreted languages' tooling.
|
| > _Whereas before one might have thought that a language 's
| tooling should be written in said language, there might come a
| time when that's too slow, and we instead need to move to faster
| languages._
|
| > _That 's what has happened to Python (numpy, pandas, scipy etc
| are all written in C and simply provide an interface in Python),
| and now it's happening to Javascript as well, with Deno, swc in
| Rust, Bun in Zig, esbuild in Go, and so on._
|
| [0] https://news.ycombinator.com/item?id=32577837
| pjmlp wrote:
| As I learned during my Tcl days, languages like Python should
| stay as scripting languages, because as you pretty well
| describe, you end up writing everything in C anyway.
|
| A lesson learned 20 years ago, before everything .com went
| bust.
| hardwaregeek wrote:
| I agree, but I do think the big question will be supporting
| plugins without recompiling. The answer is probably WebAssembly
| or one of the native-node interops but it's still not trivial.
| bee_rider wrote:
| Hmm, maybe this is a dumb question, but Numpy, Pandas, Scipy,
| etc are all libraries. Are libraries considered tooling? When I
| think of tooling, I think of linters, debuggers, IDEs, and so
| on.
|
| I can definitely see why libraries should be written in faster
| languages, they are used all over the place. And anyway,
| something Numpy is most importantly an interface to pre-
| existing high performance libraries (BLAS, LAPACK) which were
| already written in faster languages.
|
| I think it is less obvious that a tooling in the second sense
| should be written in a lower level language. There's the
| obvious flexibility tradeoff, the fact that a person interested
| in a linter for a language probably is most familiar with that
| language.
|
| I'm not familiar with the computational problems in linters. Is
| getting better CPU performance a huge concern?
| SAI_Peregrinus wrote:
| Is a compiler "toolchain" package tooling?
|
| Is the standard library included with that "toolchain"
| tooling, or a library?
| bee_rider wrote:
| So, I don't think it is really necessary for there to be a
| perfectly bright line between the two. But for the sake of
| argument, I'll say that my opinion is that the compiler,
| linker, etc are part of the tooling, while the standard
| library is a library.
|
| All of the libraries are linked to the project by some
| tooling, and a particular distribution of software could
| include any number of libraries (e.g., conda), I don't see
| why the standard library needs a special cutout.
|
| It will get even more blurry if you start asking about
| runtimes, macros, and all-template libraries.
| cercatrova wrote:
| A linter like Prettier can take several seconds in large
| codebases while the one by Rome (rome.tools) takes fractions
| of a second, because the former is in JS and the latter is in
| Rust. If I'm a linter user, I really don't care what the
| linter is written in, because I'm never going to open up the
| source code to look at it. It likely exposes a configuration
| file which is agnostic to any programming language, and that
| is what I will primarily be editing.
|
| Same thing with Webpack vs esbuild or swc, the latter two are
| simply instantaneous while Webpack can take minutes, even, to
| build an app. That it's in JS does not matter to me since I
| don't care about Webpack source code, I only care about what
| the tool itself does.
| [deleted]
| agumonkey wrote:
| Honestly I believe the low level core, dynamic shell is as old
| as programming. It's a bit part of the make it right then make
| it fast.
| mikepurvis wrote:
| Is that a bad thing? Libraries like Tensorflow were never going
| to be written natively in Python, but it's a reasonable choice
| for deployment (whls can be installed anywhere), for user
| interface (Python can interop with anything), and for overall
| ergonomics/ecosystem (import antigravity).
| cercatrova wrote:
| No, I'm arguing that it's a good thing. There are some who
| argue that a language's tooling should be written in said
| language, but I argue that that's putting the cart before the
| horse, that the tooling should be fast and efficient first
| and extensible second, because there are many more who are
| tool users than tool creators, so we should optimize for the
| former. There is also no reason that someone who is so
| interested in tooling could not also learn the tool's
| language, like Rust, and contribute that way.
| [deleted]
| mikepurvis wrote:
| I guess there's kind of a vague philosophical argument to
| be made for dogfooding, but that's perhaps more relevant
| early on in the lifecycle of an ecosystem.
|
| On the other hand, if Python had been _really_ committed to
| dogfooding at the level of something like golang, then the
| primary implementation would look a lot more like pypy than
| it would like cpython.
| _hl_ wrote:
| A counterpoint is the typescript compiler, arguably one of the
| most complex pieces of dynamic language tooling out there. It's
| written in typescript itself, and reasonably fast given that it
| has to implement a hugely sophisticated type system.
|
| This shows that it's entirely possible to write fast tooling in
| dynamic language if you know how to.
| tinalumfoil wrote:
| Python is kind of a special case. It's slow, widely used and
| often used as "glue" calling out to other programs and
| libraries that do the real work. There's also a lot of overlap
| for python and systems programmers, so there's tooling and that
| community knows how to do these things. There's more of a
| culture in python that to do anything serious (read fast) you
| write that part somewhere else and call into it.
|
| If you look at JS tooling like eslint, tslint, uglifyjs, tsc,
| webpack, ~~esbuild~~, etc, it's all JavaScript (moving to
| typescript if anything).
|
| EDIT: I stand corrected in that esbuild is not written in
| Javascript, but I think that's a disadvantage to it's success
| if anything and would point out webpack is more popular
| cercatrova wrote:
| As I mentioned above, new JS tooling is beginning to be
| written in non-JS/TS languages. esbuild which you mention
| specifically is in Go, not JS.
| [deleted]
| oefrha wrote:
| esbuild is written in Go, which is a large part of why it's
| way faster than other dog slow tools especially webpack in
| your list.
| guilherme-puida wrote:
| > EDIT: I stand corrected in that esbuild is not written in
| Javascript, but I think that's a disadvantage to it's success
| if anything and would point out webpack is more popular.
|
| Webpack exists since 2012, of course it's more popular. For a
| long time there were no other alternatives, so people had to
| deal with its madness.
| SSchick wrote:
| As much as I'd love the headline to be accurate (we use pylint
| and friends extensively) I think the comparision is pre-mature.
|
| The linters the author is comparing against have significantly
| more features an more overhead given they need to support many
| more complex rules/situations, also I'd argue this linter is far
| from well organized given it seems all checks appear to be
| implemented in a flat file [1].
|
| I would not be surprised if a fully fledged linter written in
| rust outperforms any linter written in python by a factor of
| 20-25x though.
|
| I'd be curious to see functionality & performance comparisions if
| this project continues.
|
| [1]
| https://github.com/charliermarsh/ruff/blob/0b9e3f8b472dc3fd0...
| charliermarsh wrote:
| Thanks, it's a fair comment :)
|
| I've tried to be honest about ruff's limitations: it supports
| only a small set of rules, it's not extensible in any way, it's
| missing edge-case handling, it's significantly under-tested
| compared to existing tools, etc. I don't consider it
| production-ready -- it's a proof-of-concept.
|
| (I _did_ try to build conviction that there weren't inherent
| reasons for ruff to slow down significantly as the set of
| checks extended, but I could definitely be proven wrong as the
| project grows...)
|
| My goal with ruff was partly to build a fast linter, but that
| was really in service of testing a broader hypothesis I have
| about Python developer tools could evolve. If that's
| interesting to you, I wrote about it a bit more here:
| https://notes.crmarsh.com/python-tooling-could-be-much-
| much-.... I'm certainly not here to say that the existing tools
| are bad or you shouldn't use them or that you should use ruff
| instead. I use those tools myself -- a lot!
|
| With regards to code organization: oh well, this doesn't bother
| me given the state of the project. The code will evolve as the
| project grows in scope, I didn't see a need to over-abstract.
| I'd written a small amount of Rust prior to ruff, but much of
| it was a learning experience for me. (Funnily enough: not that
| I want ruff to be organized as such, but pyflakes, pycodestyle,
| and autoflake are also effectively single-file projects :))
| benreesman wrote:
| It was a bear to get set up (oh NixOS...), but I imagine it's
| easier on other platforms: `pytype` from Google is the God Linter
| for Python.
|
| It:
|
| - seamlessly supports everything from old-ass Python 2 code up to
| very recent (like 3.10 or someting)
|
| - it can and does exploit `mypy`-style annotations, but needs
| _none_ to deeply analyze your code and _actually find bugs_
|
| - it seems to do fairly serious deep PLT magicks. i admittedly
| haven't dived deep here but it's doing stuff that looks like
| escape analysis-level whole-program analysis
|
| - it heavily caches and optimizes it's (understandably) heavy
| lifting via Ninja
|
| It _demolishes_ everything else I 've tried. I will definitely
| take `Ruff` for a spin, we've got a lot of Python and I'm always
| up for discovering there's a better tool, but the dark magic PLT
| wizards at Google have probably gotten pretty close to what can
| be done.
___________________________________________________________________
(page generated 2022-08-31 23:01 UTC)