[HN Gopher] Oxlint - Faster than ESLint
___________________________________________________________________
Oxlint - Faster than ESLint
Author : pritambarhate
Score : 268 points
Date : 2023-12-15 10:48 UTC (12 hours ago)
(HTM) web link (oxc-project.github.io)
(TXT) w3m dump (oxc-project.github.io)
| austin-cheney wrote:
| It will be awesome when this gains support for custom rules as I
| have a bunch of custom ESLint rules. The thing that annoys me the
| most about ESLint is that it has too many NPM dependencies.
| kristiandupont wrote:
| This feels like the most important thing about new linters
| (including the one Bun has and others).
|
| If you just use linting for checking a bit of stylistic policy,
| any replacement might be fine. However, linting is much more
| than that and if you are depending on third party rules or
| writing your own (https://kristiandupont.medium.com/are-you-
| using-types-when-y...), there is no way around ESLint.
| WhitneyLand wrote:
| Not sure if I'd be comfortable taking as far as your example.
|
| Adding logic into linters blurs separation of concerns,
| adding unnecessary complexity akin to an extra programming
| language.
|
| Linting in essence should be orthogonal to development -- a
| layer that enhances code quality without being fundamental to
| the code's functionality. By overextending linting, we risk
| creating a maintenance burden and an additional learning
| curve for developers.
|
| Linting is a great tool and but as with any great hammer it's
| easy for lots of things to start to look like nails.
| bakkoting wrote:
| eslint and typescript are the de-facto static analysis
| tools for JavaScript. TypeScript isn't extensible. So if
| you want to do any custom static analysis, you're doing it
| as a custom eslint plugin.
|
| It might be better to have some other tool to do pluggable
| static analysis, but the fact is that there isn't one. And
| eschewing project-specific static analysis entirely would
| be giving up far too much.
| kristiandupont wrote:
| >Linting in essence should be orthogonal to development
|
| I guess that's what I disagree with. Yes, it adds
| complexity of its own, just like types do. And I still
| favor solutions that are based on types for most things,
| but more and more I try to go this route. They are
| surprisingly easy to write.
| DanielHB wrote:
| Not only that, but you also need deps to get eslint to support
| your specific flavour of pre-transpiled JS. Not only
| typescript, but new standard JS syntax (like ?. or ??) often
| requires updating the eslint parser.
| robertlagrant wrote:
| I'm not sure there's a way around that.
| dm33tri wrote:
| I think they have custom rules in the works, using `trustfall`
| query engine and yaml definitions
|
| https://github.com/oxc-project/oxc/tree/main/crates/oxc_quer...
| obi1kenobi wrote:
| Trustfall queries are also how the Rust semver linter `cargo-
| semver-checks` works. It's cool to see more projects putting
| it the engine to good use!
|
| I'm the Trustfall maintainer, happy to answer questions about
| the query engine or how oxlint or cargo-semver-checks use it.
|
| I also recently gave a talk at P99 CONF on how cargo-semver-
| checks used Trustfall's optimizations API to get a 2000x
| speedup: https://www.youtube.com/watch?v=Fqo8r4bInsk
| crossroadsguy wrote:
| Isn't that an NPM/Node thing? I mean I sometimes look at two
| React Native projects and a web project. The dependency
| situation there is downright anxiety inducing and that I am
| saying as an Android developer so please know that I am kind
| acquainted with dependency mess.
| frou_dh wrote:
| "ruff" for Python which is displacing the flake8 linter (and in
| fact the "black" code formatter too) shows that this kind of
| thing can work fantastically well.
| drexlspivey wrote:
| I am hoping that ruff goes after type checking next to replace
| mypy which is pretty slow. One tool to rule them all
| VeejayRampay wrote:
| they've successfully replace pylama and black so yeah I
| really hope it's their next target (though handling types is
| a whole different beast altogether)
| C-Saunders wrote:
| Have you checked out Pyright[1]? It's not one tool to rule
| then all, but it is nice and fast.
|
| [1]https://github.com/microsoft/pyright
| sztomi wrote:
| Pyright is neat but the CLI output makes me want to poke my
| eyes.
| imron wrote:
| dmypy [0] (installed when you install mypy) will give you a
| x10 speedup when running mypy after small regular edits (e.g.
| during general development).
|
| But yeah, I'm also looking forward to the day when I only
| need a single speedy tool for python linting, type-checking
| and formatting.
|
| 0: https://mypy.readthedocs.io/en/stable/mypy_daemon.html
| simicd wrote:
| Have you by any chance used Pyright? If not, I can highly
| recommend it. The VS Code extension makes writing Python
| almost as if it's a statically typed language (+ there is a
| CLI if you want to check types in CI). The docs are claiming
| that it's 3-5x faster than mypy - I haven't run performance
| benchmarks myself, all I can say is that for all my code
| bases it is very fast after the first cold start.
|
| Comparison to mypy:
| https://github.com/microsoft/pyright/blob/main/docs/mypy-
| com...
| agumonkey wrote:
| rust gravity field is getting stronger
| ehutch79 wrote:
| Ruff is better than flake8 for reasons other than speed.
|
| 1) it works better as an lsp/vscode plugin, so I don't need to
| save to get errors popping up. 2) it respects pyproject.toml
| and doesn't need to liter my root dir with another dot file. 3)
| as an intangible, its errors just feel better.
| Alifatisk wrote:
| Did anyone notice? We now have 5 different ways to install this
| package.
| Waterluvian wrote:
| I want to complain but this richness in developer availability
| to implement the same thing many times is why we get a linter
| that's 100x faster or a webpack alternative that's 50x faster.
| jussij wrote:
| So, which one of those five options is the simple download?
| Alifatisk wrote:
| I'd go with pnpm
| thiht wrote:
| So? You don't _have_ to use, or even know all five.
| conartist6 wrote:
| Sure, but if you don't know what all the ways are you'll be
| prone to "just follow instructions" and you may not notice
| that a few years apart your followed instructions regarding
| different ways of installing or uninstalling things and now
| your system is a mess
| never_inline wrote:
| These appear to be same packaging format but different
| installers. Not uncommon these days.
|
| I am a junior developer writing a side project in Golang which
| probably no one on the internet sees, and there are 3 ways to
| install it already.
|
| 1. Compiled executable
|
| 2. Language package manager (go install)
|
| 3. Docker image (super trivial to create a distroless docker
| image).
|
| Same can be said of many python tools (pip, pipx, docker image,
| homebrew or whatever)
|
| It's not that we are doing more work. It's just that we have
| more tools these days. :)
| lloydatkinson wrote:
| This can only be good news. Normally I, like anyone else
| experienced with the JS ecosystem, despair when new tools come
| out like this. However, consider:
|
| - setting up eslint isn't actually that simple
|
| - if you're using typescript you need eslint-typescript too
|
| - there are sets of rules in both eslint and eslint-typescript
| that conflict with each other, so I have countless rules in my
| config like this: 'comma-dangle': 'off',
| '@typescript-eslint/comma-dangle': ['error', 'always-multiline'],
|
| - then if you're doing React there's another set of JS and TS
| rules to apply, I still never figured out how to correctly apply
| airbnb rules
|
| - this is a pretty garbage developer experience
|
| - you can quite literally spend hours or days getting a "good"
| linting/formatting configuration setup, and you often can only
| use pieces of the configs you wrote for other repos because over
| time the rules and settings seem to change
|
| - I hope this will eventually support things such as .astro files
| which is actually a combination of TypeScript and TSX blocks
|
| > At this stage, oxlint is not intended to fully replace ESLint;
| it serves as an enhancement when ESLint's slowness becomes a
| bottleneck in your workflow.
|
| I also hope that eventually it does become a full replacement. I
| like eslint, but holy shit, I cannot bring myself to create a new
| config from scratch that wrestles all the required extras and the
| frequently changing dependencies.
|
| Also, wanted to give a sort of shout out to Deno here. Deno comes
| with a linter/formatter built in that is barely configurable
| (just double vs single quote, 2 or 4 space indentation, minor
| things) and it too is very fast and simply "just works".
|
| ---
|
| Update: I just gave it a quick try and I am _immediately_
| impressed by it. Not only was it incredibly fast like it claims,
| it appears to already have all of the rules I was complaining
| about built in. eslint-plugin-react(jsx-no-
| useless-fragment): Fragments should contain more than one child.
| +-[src/design/site/preact/MobileNavigationMenu.tsx:18:1] 18
| | return ( 19 | <> * -- 20
| | <MenuButton isOpen={isOpen} onChange={setIsOpen} />
| Finished in 17ms on 90 files with 70 rules using 16 threads.
| Found 13 warnings and 0 errors.
| ahuth wrote:
| I am defending eslint/JS's honor in other replies, but you're
| right... setting up eslint is too complicated (and more
| complicated in TS).
| c-hendricks wrote:
| You're right that initially setting up the rules takes time,
| that won't go away with any linter though. But once I set up my
| company's rules 4 years ago, it's just been adding the odd rule
| every year or so, and upgrading various dependencies, then
| publish. I use it across work and personal projects, never
| really noticed "only use pieces of the configs you wrote for
| other repos because over time the rules and settings seem to
| change"
|
| > Fragments should contain more than one child.
|
| What an annoying rule
| namtab00 wrote:
| I don't do JS/TS, but I have no idea why all this hasn't
| converged on editorconfig rules.
|
| I write C# and do "linting" via editorconfig + ReSharper file
| layout formatting at dev time and via precommit hook with their
| CLI tool
|
| I'm surely missing something crucial in that ecosystem that
| editorconfig can't handle...
| leipert wrote:
| editorconfig is mostly about formatting. Parts of the
| JavaScript ecosystem have converged on prettier for that.
|
| These linters do checks on the abstract syntax tree, and so
| they can statically analyze that e.g. you don't use certain
| unsafe APIs or do things that might introduce performance
| issues or bugs.
| lloydatkinson wrote:
| I also write C# and well... good question. Much of it is
| linting though, not just formatting.
| Aissen wrote:
| > 50-100 Times Faster than ESLint
|
| > Our previous linting setup took 75 minutes to run, so we were
| fanning it out across 40+ workers in CI. By comparison, oxlint
| takes around 10 seconds to lint the same codebase on a single
| worker[...]
|
| So it's in fact 18000 times faster on this embarrassingly
| parallel problem (but doing less for now).
| joeldo wrote:
| 75 / (1/6) = 450. Still very exciting!
| Aissen wrote:
| You forgot the 40 workers vs 1 worker.
| anamexis wrote:
| The way I read it, it was taking that amount of time
| _before_ they split it into workers.
| rwilsonperkin wrote:
| Correct, it was 75 minutes total compute time. That was
| spread across workers to make the walltime more
| reasonable
| Aissen wrote:
| Indeed, I really need to improve my reading
| comprehension.
| msoad wrote:
| in a very large codebase, how common it is to run the linter
| for the entire repo? Is this an optimization worth spending
| time on?
| sapiogram wrote:
| Yes, because you lint everything in CI. Otherwise, linter
| warnings will start creeping into your codebase immediately,
| and the tool becomes much less useful.
| mathverse wrote:
| Would not you lint only on files that changed?
| erikaww wrote:
| I'm not sure if Eslint has this, but there could be
| cross-file lints (eg. unused variables). If some file
| changes, you may need to relint dependencies and
| dependent files. This could recursively trickle.
|
| I'm not sure if Eslint does this either, but indices or
| some incremental static analysis sounds like it could
| help the linter minimize rechecks or use previous state.
| msoad wrote:
| if you have one file that every single file across the
| repo imports in a way and you make changes to that file,
| you might run the linter for the entire repo. But again,
| how likely is this scenario?
| erikaww wrote:
| If the index or incremental static analysis object was
| designed well enough, I don't think you would need to
| lint every file, you would just need to look at files
| that consume that variable. Maybe you would look at every
| index?
|
| I'm not sure how well this could scale across (600-
| 1000?) different lints though. I should look into static
| analysis a bit more.
| ehutch79 wrote:
| You can tell eslint about globals in it's config. But if
| you're using variables that arn't declared in the file
| somehow, that might be an issue you want to look at in
| general. That's a potential foot gun a linter should be
| balking at.
| rwilsonperkin wrote:
| As the sibling comment mentions, you may have lint rules
| that depend on checking for the existence of, or
| properties of, another file. A popular set of rules comes
| from https://www.npmjs.com/package/eslint-plugin-import
| which validates imports, prevents circular dependencies,
| etc
| indymike wrote:
| 74 minutes of linting vs 1.3 seconds of linting?
|
| If a file has been linted, is unchanged since it was
| linted, there's literally no need to lint it again. Much
| like if you only need to process one record, you don't
| query the whole table to get the record.
| AndrewDucker wrote:
| File A depends on File B. File B moves. File A is now
| wrong, _because_ it is unchanged.
| sanitycheck wrote:
| I think if my CI was taking 45 mins to lint I'd look at
| linting only the files changed since the previous build
| instead of splitting it across 40+ workers. Or writing a
| new linter in Rust.
|
| But I'm generally working in a (human & financially)
| resource-constrained environment.
| throwup238 wrote:
| Typescript lints are type-aware so you can't just lint
| changed files, you have to relint the entire codebase to
| check if any type changes have impacted the unchanged
| code.
| pcthrowaway wrote:
| Wouldn't an issue with a type change be caught at
| typescript compile/check steps?
|
| I'm not aware of eslint rules which would complain about
| some other untouched file if types have changed in ways
| such that the program still compiles
| anamexis wrote:
| A few examples of typescript-eslint rules that could fail
| when a type in another file is changed:
|
| https://typescript-eslint.io/rules/await-thenable/
|
| https://typescript-eslint.io/rules/no-for-in-array
|
| https://typescript-eslint.io/rules/no-duplicate-type-
| constit...
| arp242 wrote:
| One problem is that a change in a.js may trigger a new
| error in b.js.
|
| ESLint could also cache things fairly trivially:
| hash = hash_file_contents() if
| previously_seen_hashes.contains(hash)
| report_previous_results() else
| run_lint_and_cache_results() end
|
| maybe that already exists. But that has the same problem.
|
| When you've got enough hardware to throw at it, then
| "just run it on the full code" is the safest.
| msoad wrote:
| I thought it would be obvious that in large codebases you
| only lint changed files in CI
| HelloNurse wrote:
| Do you have some source files that are somehow exempt from
| bugs and would be a waste of the linter's time?
|
| Probably not, but it's a trick question: if you try to look
| for exceptions to the rule, you have already wasted so much
| time that running a linter on all files would be faster.
| thfuran wrote:
| >Do you have some source files that are somehow exempt from
| bugs and would be a waste of the linter's time?
|
| Every file not touched in any given diff
| OJFord wrote:
| What if the diff adds a new linter rule, should we only
| run it on the linter config file?
|
| What if the linter uses more context than a single file,
| a type-checker for example or even just checking the
| correct number of arguments (regardless of type) are
| passed to an imported function - or that that symbol is
| indeed even callable? Should we only run the linter on
| the caller's file, or the callee's, when they haven't
| both changed?
| ehutch79 wrote:
| Run the linter on the code base then, when you make the
| change? Not every check-in on the off chance a rule
| changed. Or, add some logic that the CI runs it against
| the whole code base only when the rules changed,
| otherwise just the relevant files to the commit/pr
|
| Also, ESLint doesn't do type checking. That's typescripts
| job, and apparently typescripts runtime isn't an issue.
| Shish2k wrote:
| If I change a function signature, then my code is fine -
| but all the other files which import and use my function
| will break
| zdragnar wrote:
| That's a job for TypeScript, not eslint.
| kristiandupont wrote:
| Linter rules can rely on the type system
| zdragnar wrote:
| What eslint rule would apply to the caller of a function
| after that function's signature changes that wouldn't
| also be picked up by TypeScript?
|
| In particular, the call site itself hasn't changed, as
| this thread assumes the linter is only run on changed
| files
| kristiandupont wrote:
| Anamexis has a couple of examples in this response:
| https://news.ycombinator.com/item?id=38655101
| spenczar5 wrote:
| If a different (unchanged) file depends on the one you
| changed, you could have changed the API in a way that
| makes the unchanged file unacceptable to your linter.
| davedx wrote:
| 75 minutes to generate a bunch of mostly irrelevant nitpicks.
|
| What a colossal waste of compute resources. (1)
|
| IME if you're using TypeScript then ESlint's real value mostly
| approaches zero. For pure JS projects it's useful for finding
| nullref type bugs.
|
| (1) > Our previous linting setup took 75 minutes to run, so we
| were fanning it out across 40+ workers in CI
|
| This. Is. Insane.
| ehutch79 wrote:
| Disagree on ESLint vs Typescript. ESLint and TyepScripts jobs
| should have minimal overlap.
|
| ESLint primary job is linting. It should be finding 'foot
| guns' and code style issues. Things that are absolutely valid
| in the language, but could lead to potential issues. Because
| of that, it's totally valid that you're not finding as much
| value in it. It depends on the rules you enable in it, etc.
| And yeah, it can feel super nitpicky when it's yelling at you
| for not having a radix in parseInt().
|
| Typescript's 'compile' step or whatever, it doing type
| checking and making sure your code is valid. If you're using
| bare JS, your IDE should be doing this job, not eslint.
|
| (but yes, anything more than a few minutes to lint even a
| large code base is insane.)
| padjo wrote:
| "rules of hooks" linting alone prevents a ton of bugs in your
| average React codebase and TS will provide no help there
| davedx wrote:
| Ah yes the "exhaustive dependencies" rule that can trigger
| huge unnecessary refactors for absolutely zero value.
|
| Linting has some value, it's just that in my professional
| experience its costs outweigh its benefits
| clarkdave wrote:
| I think the typescript-eslint plugin in particular has some
| high value eslint rules that complement TypeScript.
|
| For example, the no-floating-promise[0] rule catches some
| easily-made mistakes involving promises in a way that
| TypeScript doesn't on its own.
|
| Other rules can be used to increase type safety further.
| There are various rules relating to `any`, like no-unsafe-
| argument[1], which can be helpful to prevent such types
| sneaking into your code without realising it; TS has
| `noImplicitAny`, but it'll still let you run something like
| `JSON.parse()` and pass the resulting any-typed value around
| without checking it.
|
| [0] https://typescript-eslint.io/rules/no-floating-promises
| [1] https://typescript-eslint.io/rules/no-unsafe-argument
| seanwilson wrote:
| > For example, the no-floating-promise[0] rule catches some
| easily-made mistakes involving promises in a way that
| TypeScript doesn't on its own.
|
| Is there a fast linter that checks for this? I find this
| error easy to make as well, and it usually causes weird
| runtime behaviour that's hard to track down.
| smt88 wrote:
| TypeScript is still permissive because it has to maintain
| compatibility with JS.
|
| We use eslint for formatting and other legal-but-likely-a-
| mistake behavior and it does catch bugs.
| eyelidlessness wrote:
| I get a ton of value from ESLint with TypeScript, and in
| particular from @typescript-eslint. And yes, 75 minutes is
| absolutely bonkers. It would have me rethinking a lot of
| things well short of that time. But automated quality checks
| wouldn't be anywhere near the top of that rethinking list.
| And partly, but not only, _because_ of irrelevant nitpicks.
| Having humans do those nitpicks is vastly worse in time
| elapsed, and likely in compute time in many scenarios as
| well. The more human time is spent on the things linters help
| with, the more that time is _not spent_ on reviewing and
| ensuring correctness, performance, design, maintainability,
| user- and business-implications, etc.
| Kyro38 wrote:
| How much of those 75min are due to @typescript-eslint ?
|
| Requiring the TS AST adds a massive overhead.
| pjmlp wrote:
| XZY written in AOT optimized compiled language, XXX times faster
| than interpreted language (even if with a dynamic JIT).
|
| Maybe one shouldn't have started writting all those tools in JS
| in first place.
| kajaktum wrote:
| It has always been possible to do this kind of things (we have
| C, C++, Java) but I don't think people have been this semi-
| successful with reimplementing a bunch of common tools. Where
| are all the X re-implemented in C/C++/Java?
| afavour wrote:
| > Maybe one shouldn't have started writting all those tools in
| JS in first place.
|
| Sure but rather than scold people perhaps we should examine why
| they used JS instead of a compiled language. And why that's
| changed now.
| shzhdbi09gv8ioi wrote:
| When all you have is a hammer, everything is a nail.
|
| We know why js devs chose js already.
|
| We also know some of them learned rust and used their new
| hammer to bang out better solutions. See OP :)
| afavour wrote:
| > When all you have is a hammer, everything is a nail.
|
| But it was possible to write C/C++ modules for Node very
| early on. So there were always multiple hammers available.
| ahuth wrote:
| Sure, for some tools. Most the time it's fine, though.
|
| It's also nice that people in the JS community can easily
| contribute to tools written in JS (including writing custom
| eslint rules).
| nicoburns wrote:
| Right, but it's only recently that better alternatives have
| become widely available. Previously your options were:
|
| - Something like Java/C#, but that would have required users of
| the tool to manage a second toolchain / runtime just for
| developer tools - quite a hard sell for widespread adoption
|
| - C/C++ which are quite hard to learn (and use correctly once
| learnt) for users of interpreted languages which ends up being
| a barrier to their use.
|
| Now we have Go and Rust which are fast, compile to a single
| binary, and are much easier to learn than C/C++, which is
| leading to a whole new generation of tools.
| mhh__ wrote:
| I know a few rust programmers who say that they don't want to
| learn C++ because it's too hard but I don't know if that's
| actually true.
|
| Learning C++ legalese takes ages but the basic principles
| really aren't that hard. It feels more performative than most
| are prepared to admit.
|
| Rust probably has less bullshit but is borrow checking and
| macros aren't exactly simple in vacuo either
| nequo wrote:
| rustc gives you astonishingly good error messages though
| which helps get through the borrow checking hurdles. And
| all the toolchain you need is cargo.
|
| I don't know C++. Does it have tooling that makes figuring
| things out similarly easy?
| db48x wrote:
| Not nearly enough, and the best tools for C++ (debuggers
| like rr) work just fine on programs written in Rust.
| PoignardAzur wrote:
| It's more that, after years of programming in C++, when I
| run a program I'm still not confident it's not going to
| have major bugs, and if a bug shows up, I know I'll be in
| for a world of pain trying to track it down.
|
| After a few months of coding in Rust, I stopped having that
| problem altogether. I still ran into bugs, mind you, but
| solving them usually became painless.
| maccard wrote:
| Really? Since about c++14, the number of memory errors
| I've experienced in practice is asymptomatically trending
| towards 0. When they do come up, it's usually in some
| awkward C library that has been badly wrapped, and would
| likely necessitate unsafe in rust anyway. I genuinely
| can't remember the last time I introduced a memory stomp,
| use after free, double delete or slicing issue in a
| modern c++ Codebase.
| nicoburns wrote:
| The main problem I had when I tried to learn C++ was
| getting started with the build system. I could compile a
| single source file easily enough but adding others, and
| especially integrating with 3rd party libraries was
| difficult. And there was extra pain if I wanted it to work
| portably across platforms. Whereas with Rust, building is a
| simple `cargo build`. And integrating libraries is as
| simple as adding a line to the manifest file.
|
| The borrow checker wasn't trivial to learn, but I could at
| least bash my head against the compiler, and be pretty sure
| that once it compiled my code was correct. With C++ it is
| much harder to get that feedback while learning as it's
| very easy to compile something that segfaults, crashes, or
| has Undefined Behaviour.
| mhh__ wrote:
| That's fair.
| shzhdbi09gv8ioi wrote:
| Go is 14 years old
|
| Swift is 9 years old
|
| Rust is 8 years old
|
| Zig is 7 years old
| kbknapp wrote:
| TBF, when a 1.0 is released doesn't mean it's viable right
| way for things like this. It takes a certain level of
| market adoption and ecosystem buy-in first.
|
| Also, Zig still isn't 1.0 so if we're measuring languages
| from when they first became public, I believe those others
| in your list are much older as well.
| CyberDildonics wrote:
| Jai is 10 years old and it isn't even released publicly
| dralley wrote:
| Rust is 8 years since 1.0, Zig hasn't even reached 1.0 yet.
|
| If you compare like for like, Rust started development 17
| years ago, as did Golang. Though Rust spent longer in the
| research project phase.
| nicoburns wrote:
| Sure, but when these languages first came out they didn't
| have anything like the library ecosystems they have today.
| In 2023, you can add a JavaScript or CSS parser to your app
| with a single line of code. Back then you'd have had to
| write your own.
| pjmlp wrote:
| As if those were the only options.
| crabbone wrote:
| Languages aren't "optimized compiled" or "interpreted". This is
| nonsense classification.
|
| The words you are looking for are "language runtime". And even
| if you used that, you'd still be wrong. Java is exactly that:
| "even if with a dynamic JIT", and it does perfectly fine and
| even sometimes is the fastest solution for a problem (I think
| Java beat everyone on fastest HTTP server with largest number
| of simultaneous connection, where second in class was an Erlang
| program iirc).
|
| Because you don't understand the problem, you are trying to
| offer a wrong solution: compiling doesn't do anything to speed
| up programs, for example. What people who want better
| performance need is:
|
| * Tools to analyze program performance.
|
| * Tools to alter program runtime ahead of time and during the
| program execution.
|
| * Access to runtime primitives as far down to the "metal" as
| possible with as little undue effort as possible.
|
| ---
|
| The problems with current JavaScript runtimes are that they
| aren't designed for performance-minded developers. The
| developers are given highly engineered "primitives" to work
| with, which make them commit to certain solutions which in turn
| will make automatic or manual optimizations very hard, next to
| impossible.
|
| But it doesn't have to be like that. For example, a variant of
| JavaScript, the version 4 a.k.a. ActionScript had an "escape
| hatch" -- if you wanted to optimize a program you had simple
| unchecked memory access with primitive memory operations, which
| would allow you to side-step all the "bloat" of object
| lifecycle management. This library was often used to implement
| various online tools for dealing with a lot of data-processing
| (eg. image or video compression) and they did just fine.
|
| Current version of JavaScript doesn't have anything like that.
| But it could as the evidence of its previous version doing it
| successfully shows.
| pjmlp wrote:
| I have a Informatics Engineering degree with major in
| compiler design and systems programming, thanks for the
| lesson regarding how languages are supposed to be.
| crabbone wrote:
| Well, it just means that your study was bad... if all you
| have to say about the subject is that you've completed it.
|
| A competent in the subject person would've had something to
| say relevant to the subject.
|
| The problem is that CS studies are, in general bad. Not
| just your specific case. In other fields something would've
| pinched your bubble by now and you'd start wondering what
| other things you might have possibly missed, but because CS
| studies are so universally bad, virtually everyone you
| interact with professionally will share your
| misconceptions.
|
| Ironically, the "hard sciences" as well as math like to pet
| themselves on the back about how these disciplines open
| students to critical thinking, requiring proofs and
| soundness of definitions, and yet your whole taxonomy of
| the thing you interact with professionally is ridiculously
| wrong, contorted and full of magical thinking.
|
| During your studies, I'm sure, you were given a straight up
| definition of what a programming language is. You must've
| taken at least one semester of automata theory -- it's hard
| to imagine a CS degree w/o it. The premise of this
| discipline is that there are languages, and throughout the
| course you discuss their properties, various ways to define
| them, operate on them etc.
|
| And then one day you take an "intro to CS with language X
| 101" course. And that b/s course tells you with a straight
| face that language X is "object oriented" or "functional"
| or "compiled" or "dynamically typed". And you just eat it
| up. You never connect the dots between what you've studied
| in automata course and this intro b/s course. You never ask
| the question like "so how do I get from states,
| transitions, initial and final state to... objects?.. or
| w/e other b/s property the course ascribes to that
| language.
|
| And now you are waving your diploma in my face and making a
| fool of yourself... you should probably ask the academic
| institution who handed you this diploma to reimburse you
| for the time you wasted there instead. Alas, they won't do
| it. They won't so much as understand the reason why what
| they did was a disservice to you. Well... life's unfair.
| db48x wrote:
| No, it's the other way around. Write everything in a safe
| language so that you don't have to worry too much about crashes
| and other problems, but choose a language that is _fast to
| write_ for your first attempt. When you write that first
| program you will not know all of the answers yet, or even all
| of the questions. You want to explore the space of possible
| solutions quickly and efficiently, so a dynamic language like
| Javascript (or Lisp, Python, whatever) is the best choice.
|
| Later once you have figured out how the program should be
| written, that's when you go back and rewrite it in a language
| that is _fast to run_, like Rust. Sure, if you had written it
| in Rust to start with it would have been fast to start with.
| The problem is that the exploration would have been far slower,
| taking years instead of months. And because you haven't done
| the exploration, it is unlikely that you will start with the
| right architecture. That means a lot of factoring and
| refactoring once you figure out what the right architecture is.
|
| In most cases you can gain far more by writing the program
| quickly than you can by writing a quick program.
| pjmlp wrote:
| There are plenty of safe compiled languages to chose from.
| conartist6 wrote:
| Well said!
| d3w4s9 wrote:
| > it serves as an enhancement when ESLint's slowness becomes a
| bottleneck in your workflow
|
| Well, when I need to batch fix errors in files, yes it can take a
| while to run eslint. But that almost never happens. I have the
| plugin and fix errors as I go (which I believe is what most
| people do), and I never feel performance is an issue in this
| workflow. I really doubt how (actually) useful this is.
| jcelerier wrote:
| It's still using more battery
| d3w4s9 wrote:
| True, but eslint energy use would be one of the last things I
| worry about if I am looking for a longer battery life.
| Chances are that TypeScript service used for Intellisense
| costs more electricity.
| sapiogram wrote:
| Their main motivation seems to be CI, where people often lint
| the entire repo on every PR.
| msoad wrote:
| which is a really weird problem to have. Only lint files that
| have changed? How hard that is? our monorepo is 3m lines of
| code and running lint is not a bottleneck by any means...
|
| And once in a while that we have to run lint for entire repo
| (ESLint upgrade for example) we can afford to wait 1 hour
| ONCE
| Aeolos wrote:
| 50-100x faster would turn that 1 hour into 1 minute.
|
| It's not that you can't wait 1 hour, it's that you don't
| _have_ to wait. Think of all the wasted cycles that could
| be put to better use...
| ForkMeOnTinder wrote:
| > Only lint files that have changed? How hard that is?
|
| Quite hard, especially since type-aware rules from e.g.
| https://typescript-eslint.io/ mean that changing the type
| of a variable in file A can break your code in file B, even
| if file B hasn't changed.
| mrkeen wrote:
| > we can afford to wait 1 hour ONCE
|
| If you lint the entire repo, fix every issue in one try on
| the first go, and then lint the entire repo to double-
| check, that's two hours.
|
| But my workflow is usually: lint the repo -> fix one thing
| -> repeat
| zanellato19 wrote:
| Are you kidding? Having someone run faster on the editor is a
| huge gain. I can't believe people are saying this isn't useful.
| d3w4s9 wrote:
| Faster by how much in absolute time? Currently I'm not
| feeling ANY delay in the IDE, so I assume for a regular size
| file linting takes less than 50ms -- likely much shorter than
| that. Let's say it reduces 50ms to 2ms. Guess what? It still
| has absolutely no effect on my everyday work.
| silverwind wrote:
| Likely not worth using currently as it only has like 200 rules,
| while typical eslint setups have 600 or more.
| leipert wrote:
| Why not run both? Run the 200 rules from this one and the 400
| other rules with eslint.
| recursive wrote:
| Now you have 2 problems.
| pzmarzly wrote:
| If I understand it right, we have 3 large projects that aim to
| replace most of JS tools on their own: Bun[0], Oxc[1] and
| Biome[2]. Bun's package manager is great, Biome formatter
| recently reached 96% compatibility with Prettier, and now Oxlint
| is apparently good enough to replace ESLint at Shopify. Exciting
| times ahead.
|
| But it's giving the impression that these projects perhaps could
| be better off collaborating instead of each of them aiming to eat
| the world on their own?
|
| EDIT: I'm not saying it's wrong to write competing tools, it's
| open source anyway, so please do whatever you like with your time
| and have fun. But it looks like out of these 3 projects, 1 has a
| startup behind it, and 1 receives funding from bigger company. I
| assume that money will stop coming in if these tools don't gain
| adoption fast enough, and nobody would want to see that happen,
| especially with so much potential here.
|
| [0] https://bun.sh/
|
| [1] https://oxc-project.github.io/
|
| [2] https://biomejs.dev/
| djbusby wrote:
| Happens loads of times. There is some in-built human condition
| that folk basically see a thing that they could improve but
| then decide to go off and build their own moon-base rather than
| work on someone elses project.
| throwaway894345 wrote:
| In my experience, project maintainers are frequently
| uninterested in changes to their project, especially if those
| changes are a significant departure from their current vision
| or if it involves pivoting away from tools that they like.
| You're often expected to make years of contributions to the
| project to earn the rapport to bring significant suggestions
| before the maintainers. It's often just easier to 'build your
| own moonbase' instead of politicking.
|
| Just a couple days ago, the curl maintainer published a blog
| post about why he wouldn't rewrite curl in Rust and a big
| part of the reason was that he and the other maintainers
| weren't good at it and weren't the right people to lead a
| project that used it--he said that he encouraged other people
| to start their own project in Rust. But then when people
| follow that advise, they're chided for not contributing to
| the more established project! To be clear, I'm not a "just
| rewrite it in Rust" guy, but I think people underestimate the
| difficulty and frustration involved in petitioning an
| established project to make the reforms necessary for
| significant improvements.
| vintermann wrote:
| Well, when they pretty much succeed at building their moon
| base, I say good on them.
| bluGill wrote:
| Except that this is not as good as the original by their
| own admittance. If they had collaborated they could likely
| get more done in the same amount of time. (not twice as
| much, but more)
|
| Maybe this is a better design than the other projects.
| Maybe people cannot get along and so they are forced to
| fork. There are many other good reasons to not contribute
| to an existing project. However we should always look at
| skepticism on such claims: it is easy to start you own
| project and you are in control so the amount of work you
| get done is higher. However working together, while it
| makes everyone slower normally results in many more
| features and higher quality code over the long term.
|
| So please when you have an itch technology can solve look
| to see if you can contribute to someone else's project
| first. It won't be as fun, but the world and you will be
| better for it.
| wg0 wrote:
| More like JS folks are discovering compiled languages.
|
| Now instead of a new JS framework daily, it's going to be new
| reimplementation of an existing tool daily. For a while.
| shzhdbi09gv8ioi wrote:
| About time, js cli apps were never a good idea.
| maccard wrote:
| They exist because it's significantly easier to distribute
| js apps than it is to distribute a compiled app. npm
| install works on Linux, Mac and windows, regardless of what
| libc, Msys, crt, you have installed. It could be python,
| but pip is a usability nightmare
| boredumb wrote:
| Not particularly true especially in this case. You can
| get a rust binary and run it anywhere regardless of libc
| or having cargo installed on the users machine. A
| Javascript CLI requires nodejs and npm to be installed
| before running it.
| rob74 wrote:
| Same goes for Go BTW. I even find it easier to install Go
| (haven't done it for Rust that often yet) and compile a
| binary (of a "pure" project that doesn't involve C
| libraries or other complications) than installing
| node/npm/nvm/whatever to get something up and running...
| maccard wrote:
| In this particular case, you wouldn't be installing
| oxlint unless you had npm installed already?
| boredumb wrote:
| For their main use case they do package it up for npm,
| but the crates folder have each portion available to
| build/distribute as a stand alone binary you can run
| against javascript without node or npm installed.
| lixy wrote:
| I wish the nix programming language wasn't so rough
| because it can be pretty great at this problem. Being
| able to compile from source while just listing out
| package dependencies is powerful.
| bluejekyll wrote:
| Cargo and crates.io is easily as simple as npm for
| installation and distribution. I find it to be more
| reliable than npm in general. Generally it's very easy to
| write system agnostic software in Rust, as most of the
| foundational libraries abstract that away.
|
| So when you say "compiled app" you might be referring
| instead to C or C++ apps, which don't generally have as
| simple and common a distribution model. Rust is entirely
| different, and incorporated a lot of design decisions
| about how to package software from npm and other
| languages.
| andygeorge wrote:
| Cargo is still a dev tool and isn't a great distribution
| solution.
| bluejekyll wrote:
| I disagree. Cargo is a great distribution tool, for Rust
| projects. I just tell people, first install rust, then
| just `cargo install`
|
| Second, this was in response to an npm is simpler
| comment; npm and cargo are absolutely the same category
| of tool.
| satvikpendem wrote:
| I actually recommend cargo install cargo-binstall first,
| then cargo install <crate>. This is because it is quite
| annoying to compile packages every time you want to
| install something new whereas binstall distributes
| binaries instead, much faster.
| bluejekyll wrote:
| Feels like we need a single command for that, I have two
| goals for my workflow (like maybe bininstall should be
| included in Cargo):
|
| 1) what's the easiest way to give people access to a tool
| I just wrote, `cargo publish`
|
| 2) what's the easiest way for someone to use it, as few
| steps as possible, right now it's `install rust` &&
| `cargo install`.
|
| Once I get to three or more steps on 2 I tend to turn to
| just or make depending on the context.
| andygeorge wrote:
| > I just tell people, first install rust, then just
| `cargo install`
|
| local compilation may work for you and other individuals,
| but "just cargo install" can immediately run into issues
| if you're trying to deploy something to things that
| aren't dev workstations
|
| > npm and cargo are absolutely the same category of tool
|
| as a dev tool? absolutely. as a production distribution
| solution? definitely not
| arp242 wrote:
| The overlap between people who want to run something like
| ESLint and people with dev workstations is very close to
| 100%.
| bluejekyll wrote:
| > as a production distribution solution? definitely not
|
| If you're talking about distributing Rust projects, sure
| it's fine. Generally though, if you're orchestrating a
| bunch of other things outside the rust software itself,
| I'd turn to just.
|
| npm is still mainly used in JavaScript and Typescript
| scenarios, so I think you're kinda splitting hairs if
| you're suggesting it's a general purpose tool.
| andygeorge wrote:
| there's a reason `cargo install` is usually the last
| distribution option that maintainers of rust software
| provide -\\_(tsu)_/-
| kbknapp wrote:
| I've had significantly fewer issues with `cargo
| [b]install`ed compiled Rust programs than `npm install`ed
| ones. Getting nodejs/npm installed (and at an appropriate
| version) is not always trivial, especially when programs
| require different versions.
|
| OOTH, _Precompiled_ Rust binaries have the libc version
| issue only if you 're distributing binaries to
| unknown/all distribtuions, but that's pretty trivially
| solved by just compiling using an old glibc (or MUSL).
| Whereas `cargo install' (and targetting specific
| distributions) does the actual compiling and uses the
| current glibc so it's not an issue.
| natrys wrote:
| If only we are so lucky. Still waiting for a faster
| typescript compiler.
| _fat_santa wrote:
| Bun, Oxc and Biome are all great but a typescript rust
| compiler is something I'm really looking forward to. Right
| now my web application I've been building just crossed 25k
| lines of TS code and running `tsc` is becoming a pain
| point. What used to take 2-3 seconds now takes upwards to
| 10s, even with incremental compilation enabled in some
| cases.
| satvikpendem wrote:
| STC by the SWC author should be coming along, I hear. It
| still will take a while though.
| robinson7d wrote:
| Semantic nit: STC is a type checker, SWC already compiles
| TypeScript well. TSC does both (unless flagged to do one
| or the other) so it depends on what needs replacing.
|
| Why it matters: in GP's case it sounds like compiling is
| the problem, so migrating to using SWC as the compiler
| but keeping TSC as the checker (noEmit flag) in a lint
| step may ease that pain a bit. Though it might be nicer
| to migrate both in parallel.
| nonethewiser wrote:
| Can you elaborate? Typescript has existed for a long time and
| has been the standard over vanilla js for a long time. Bun,
| oxlint, and biome are all replacing existing tools with build
| steps. How could it be that their popularity signifies some
| new appreciation of compiled languages?
| wg0 wrote:
| Typescript is not a compiled language. It is a "transpiled"
| language. Transpiled to another interpreted language
| Javascript which in turn again is not a compiled language.
| dragonwriter wrote:
| > Typescript is not a compiled language.
|
| Compilation or not isn't a feature of languages but of
| language implementations, but, yes, the primary
| TypeScript implementation is compiled.
|
| > It is a "transpiled" language.
|
| Transpilation is a subset of compilation.
|
| It's not compiled _to native machine code for the target
| system_ , but that doesn't make it not-compiled.
| wg0 wrote:
| If going with that lax definition and concept wrangling,
| Python is also a compiled language. Python source code
| can be compiled and byte code can be cached and then
| Python runtime can load it.
|
| Just like Typescript compiles the source to Javascript
| which is then loaded by the V8/Node etc.
|
| And thus programming languages can be only of one type -
| Compiled.
| recursive wrote:
| Being compiled or not isn't a property of the language.
| It's a property of whether you compile it or not. Pure
| interpreters can exist. They're not very common for
| "practical" languages. Parse to AST, then call
| evaluate(ast). No target language necessary.
| scotty79 wrote:
| JS seems to be like a great language for discovering what's
| worth to be written. I think rewriting stuff in some compiled
| language is a sweet spot of "build first one to throw away".
| PoignardAzur wrote:
| There's also Deno:
|
| https://deno.com/
| pzmarzly wrote:
| To clarify: I'm also not advocating for merging the codebases
| together, that would be mostly counterproductive (especially
| since Bun is in Zig, and Oxc and Biome in Rust).
|
| When I think why Rust was successful at establishing community-
| accepted standard tooling (clippy, rust-lsp), 2 things come to
| mind:
|
| - Project developers were always promoting each other's tools,
| pointing them out in docs or blog posts
|
| - Good tools were being pulled into rust-lang GH org (for
| visibility) and rustup CLI distribution (for ease of system-
| wide installation)
|
| Both of these things are not technical challenges, they are
| rather more "political" (require agreements between parties).
| In JS ecosystem, what would it take for Oxc to say on their
| website "we are not writing a formatter, please install Biome"
| and for Biome homepage to say "we are not writing a linter,
| please install Oxlint"?
| thatxliner wrote:
| Except Biome can also function as a linter
| conartist6 wrote:
| [3] https://github.com/bablr-lang
|
| I'm its author and focus solely on the collaboration picture. I
| don't generate much press because I only build internal APIs
| for tooling and language authors, where the projects you've
| shared all opted to prioritize fulfilling specific real use
| cases over generalizing their core technology.
|
| Cruel as it is, I think all of them have planted the seeds of
| their own failure by failing to protect their organization's
| mission and day-to-day work from being jailed by a set of
| specific opinions about code style, which cannot possibly be
| "right" or "wrong" but must instead by argued about forever.
|
| I see the core challenge as shifting all editors and tools to
| share a common DOM representation and be interoperable in a
| per-node way, where the current solution is to use siloed and
| reimplemented tools which interoperate mostly in a per-file
| way, with each tool parsing the text, doing some work, then
| emitting text for some other tool to parse...
| conartist6 wrote:
| For example:
|
| "The Oxc AST differs slightly from the estree AST by removing
| ambiguous nodes and introducing distinct types. For example,
| instead of using a generic estree Identifier, the Oxc AST
| provides specific types such as BindingIdentifier,
| IdentifierReference, and IdentifierName."
|
| Already this is getting into matters of style! It is one
| style, yes, but Javascript's shorthand syntax `({ foo })`
| already breaks the mental model: the identifier `foo` is
| technically doing the work of both an IdentifierName _and_ an
| IdentifierReference. OXC chooses IdentifierReference so any
| system built on top of it would need to contain additional
| logic in order to be able to identify all sites in code that
| are used as identifier names.
| art0rz wrote:
| I would really like to speed up my workflow with a faster ESLint
| alternative, but my ESLint configs are often very customized,
| with rules and plugins that are not available (yet) in the
| alternative solutions, making them a non-starter for me. It'll
| take a while for these alternatives to reach plugin/rule parity.
| maccard wrote:
| Would you consider removing your customisations to be closer to
| the workflows supported by these tools? One of the great things
| about go is that you're free to have an opinion, but if you
| disagree with go fmt or go build, your opinion is wrong.
| art0rz wrote:
| No. A linter does more than formatting. Besides, some rules
| may simply not be relevant to what I'm working on while other
| rules are. Prettier works well enough for most people because
| it only covers syntax, and not whether or not you can use
| await in a loop, or should add tracks to your video element,
| or if jsx should be in scope, etc.
| bsnnkv wrote:
| This is one of the real productivity superpowers of
| ecosystems like Go and Rust imo
| IshKebab wrote:
| Python and JavaScript have similarly good formatters (as
| long as your idiot colleagues don't insist on using yapf
| instead of Black, despite yapf producing non-deterministic
| output!). In fact I would say Rust is probably behind
| Prettier in terms of auto formatting. The rustfmt output is
| less pretty (subjective I know), the devs have made several
| strange decisions and it seems to be semi-abandoned (maybe
| partly because the devs were ... shall we say not as
| friendly and welcoming as the Rust community likes to bleat
| on about).
|
| There are a couple of alternative formatters:
|
| * https://github.com/andrewbaxter/genemichaels *
| https://github.com/jinxdash/prettier-plugin-rust
|
| Still, all of them are better than clang-format!
| brundolf wrote:
| Yeah. There have been lots of Rust or Go linters popping up
| with impressive benchmarks, but I don't think any will take
| over the world until they have drop-in parity
| klageveen wrote:
| This is cool of course. But so was Rome. Which only existed for
| about two years. It's one thing to build a cool tool, it's
| something else entirely sustain one over time. I need a bit more
| proof that this is sustainable before I rebuild our toolchain,
| _again_.
| JimDabell wrote:
| The Rome project continued as Biome:
|
| https://biomejs.dev/blog/annoucing-biome/
| klageveen wrote:
| Ok, wow, why did I miss that. Thanks!
| lucideer wrote:
| The spate of rewrites of JS tools in compiled languages
| continues. Here's my problems with them:
|
| 1. The need for a 50-100x perf bump is indicative of average
| projects reaching a level of complexity and abstraction that's
| statistically likely to be tech debt. This community needs
| complexity analysis tools (and performant alternative libraries)
| more than it needs accelerated parsers that sweep the complexity
| problem under a rug.
|
| 2. (more oft cited) The most commonly and deeply understood
| language in any language community is that language. By
| extension, any tools written in that language are going to be
| considerably more accessible for a broader range of would be
| contributors. Learning new languages is cool but diverging on
| language choices for core language tooling is a recipe for
| maintainer burnout.
| rafaelmn wrote:
| How often does an average X developer delve down to compiler
| details and contribute to static analysis tooling ?
|
| Metaprogramming and compilers/language analysis tooling is a
| jump above your run of the mill frontend code or CRUD backends.
|
| Sort of elitist, but IMO devs capable of tackling that
| complexity level won't be hindered by a different language
| much.
|
| And Rust is really tame compared say C/C++. Borrow checker is a
| PITA, but it's also really good at providing guardrails in the
| manual memory management land, and the build tooling is really
| good. Don't know enough about Zig but I get the impression that
| rust guardrails would help developers without C/C++ background
| contribute safe code.
|
| You could argue Go is an alternative for this use case (and
| similar languages) but it brings it's own runtime/GC, which
| complicates things significantly when you're dealing with multi
| language projects. There's real value in having simple C FFI
| and minimal dependencies.
| lucideer wrote:
| > _Sort of elitist, but IMO devs capable of tackling that
| complexity level won 't be hindered by a different language
| much._
|
| Not elitism, just an honest appraisal, though I think flawed
| as competency isn't linear it's heterogeneous - you'll find
| the most surprising limitations accompanying the most
| monumental talent. Language fixation is a common enough one,
| but even beyond that, the beginner-expert curve on each
| language shouldn't be underestimated regardless of talent or
| experience.
|
| In particular when it comes to Javascript there's a tendency
| to believe the above by virtue of the community being very
| large & accessible - bringing in a lot of in-expert
| contributors, especially from the web design field. This
| isn't fully representative of the whole though: there are
| significant solid minorities of hard JS experts in most
| areas.
| arp242 wrote:
| > How often does an average X developer delve down to
| compiler details and contribute to static analysis tooling?
|
| I've done this a few times for Go. One of the nice things
| about Go is that this is actually pretty easy. I've written
| some pretty useful things with this and gotten good mileage
| out of it. Any competent Go programmer could do this in an
| afternoon.
|
| I don't really know what the state of JS tooling on this is,
| but my impression is that it's a lot harder, partly because
| JS is just so much more complex of a language, even just on
| the syntax/AST level. And TypeScript is even more complex.
| davedx wrote:
| Disagree with 1. Most large JS projects I've worked on have
| been relatively high in necessary complexity; probably because
| many JS projects are relatively simple applications and
| relatively new (by the standards of enterprise software).
|
| There is also abundant complexity analysis tooling for JS too.
| When I worked as an architect at a large telco we had this
| tooling in CI. It revealed some code smells and areas needing
| refactoring but didn't really signal anything especially
| terrible.
|
| Software tooling is more productive than ever and product
| requirements have grown to use that capacity. It's definitely
| not a load of tech debt.
| lucideer wrote:
| Not sure where you've worked or what you've worked with but
| everything you've described is the opposite of the JS
| projects I've encountered (multiple companies, multiple 100s
| JS projects).
|
| > _There is also abundant complexity analysis tooling for JS
| too._
|
| I would highly appreciate recommendations here; I wonder does
| your review indicate the projects being analysed had little
| wrong, or that the tools were not very good at identifying
| problems.
| the_duke wrote:
| The rewrites mostly are for tools that run for a short amount
| of time and do lots of AST processing.
|
| Javscript is just inherently suboptimal for this.
|
| * The JIT needs to warm up
|
| * AST data structures can be implemented much more efficiently
| with better control over memory layout
| conartist6 wrote:
| To be fair the AST structure can also be implemented more
| efficiently without better control over memory layout. The JS
| ecosystem standardized on polymorphic ASTs, which in
| retrospect seems dumb, but is not a result of any fundamental
| limitation in JS.
|
| E.g. in ESTree evaluating such a common expression as
| `node.type` is actually really expensive -- it incurs the
| costs of a hashmap lookup (more or less) where you'd expect
| it to be able to be implemented using pointer arithmetic.
| lucideer wrote:
| I get what you're saying but you've missed my point.
|
| You're optimising your execution but there's trade-offs: you
| need to think about optimising your software development
| model holistically. There's little point in having the most
| efficient abandonware.
|
| A JS tool may be technically suboptimal but that's not a
| problem unless AST size is a bottleneck.
|
| > _AST data structures can be implemented much more
| efficiently with better control over memory layout_
|
| I assume you're right but I'm not sure I fully understand why
| this is the case - can you give examples of how a data
| structure can be implemented in ways that aren't possible in
| JS?
| apantel wrote:
| > The need for a 50-100x perf bump is indicative of average
| projects reaching a level of complexity and abstraction that's
| statistically likely to be tech debt.
|
| I don't think this is the right way to look at it. The issue is
| that JavaScript developers have been writing servers, build
| tools, dev ops tools, etc, in JavaScript because that's the
| language they are expert in, but JavaScript was never the right
| choice of language for those types of programs. The whole
| industry is caught in a giant case of "If all you have is a
| hammer...".
|
| I do web development in JavaScript because JavaScript is the
| language of the browser. But I write all of my own build and
| devops tools in Java, including SaSS compiling, bundling,
| whatever you want. There's no contest between the Java runtime
| vs the JavaScript runtime for that kind of work.
|
| I think it's backwards to see this as a 50-100x performance
| boost because Rust was used. That same performance increase
| could be had in a number of languages. The real issue is a
| 50-100x performance hit was taken at the outset simply by using
| JavaScript to write tooling.
|
| Edit: just to put it in perspective, a 50-100x speed up in
| build time means that what would currently take a minute and a
| half using JS tooling could be accomplished in a second using a
| fast runtime. A minute and a half of webpack in the blink of an
| eye.
| jerf wrote:
| As I almost always think to myself whenever I see some
| program braying about its 25x speed improvement in some task,
| the reason you can have a 25x speed improvement is because
| you left that much on the table in the first place.
|
| I don't want to be _too_ hard on such projects; nobody writes
| perfect code the first time, and stuff happens. But this does
| in my mind tend to tune down my amazement level for such
| announcements.
|
| And your last edit is really the important point. That level
| of performance improvement means that you are virtually
| certain to move up in the UI latency numbers:
| https://slhenty.medium.com/ui-response-times-acec744f3157
| Unless everything you were doing is already in the highest
| tier, this kind of move is significant.
| lucideer wrote:
| > _There's no contest between the Java runtime vs the
| JavaScript runtime for that kind of work._
|
| I don't mean to be facetious here, but... citation needed.
|
| There are a lot of assumptions about language performance
| being made throughout comments threads on this page that seem
| more based on age-old mythology rather than being grounded in
| reality.
| apantel wrote:
| Here is a presentation by a team that did benchmarking of
| different runtimes:
|
| https://youtu.be/sRCgu1ng6Bo?si=SV_Mcinuqh_c-nuX
|
| JavaScript is ~8x slower and Python ~30x slower on average
| vs Java / Go / C++ that are all quite close.
|
| A funny aside: I always believed that Java is slow because
| I heard it repeated so many times. I internalized that bit
| of age-old mythology. But lately as I've gotten more
| focused on performance, I've come across a lot of hints in
| various talks and articles that Java has become one of the
| go-to languages for high-performance programming (e.g. high
| frequency trading). So, I hear you about the mythology
| point.
| WhereIsTheTruth wrote:
| I would be surprised and worried if a native tool would be slower
| than a javascript one, even with JIT, wich is useless for short
| lived programs
| msoad wrote:
| This is HackerNews so brace for criticism!
|
| Why would a team of talented engineers focus on solving ESLint
| performance issue? Where is the value in this? If your project is
| small, ESLint is fast enough. If it's super large like ours (3
| millions LOC) then you spend a little time making local and CI
| linters smarter to run only on changed files. Rewriting in Rust
| seems cool and novel but now you lost the entire wealth of ESLint
| plugin ecosystem, you have to keep up with maintaining this new
| linter which has to be updated very frequently for at least new
| syntaxes and so on...
|
| We could put this effort into looking into why ESLint is not fast
| enough and fix the bottlenecks IF we had extra time in our
| hand...
|
| If it was my team, I would not let them spend time on this. I
| don't see the value to be honest.
| Trufa wrote:
| They developed a new tool that reduces their CI from 75 minutes
| to 10 seconds and are offering it for free and open source and
| you really don't see the value? I know you warned this is HN
| but I find this posture ridiculous. If you don't find value for
| yourself, that's one thing but I honestly don't get this place
| sometimes.
| msoad wrote:
| I'm not complaining about free software offered to world for
| free. I'm curious how a leader would justify an investment
| like this? I have engineers reaching to me and asking me to
| all sort of things. My job is to justify it for the business.
| Making this costs more than a million dollar if their
| engineers are paid like ours. Then how you do this? How do
| you get budget for this?
| gregsadetsky wrote:
| 75 minutes to 10 seconds is no joke in terms of speedups.
| Imagine that this time is saved for a small team of 4? 10?
| people who can then inspect/qa/iterate on the build in a
| PR-preview staging environment. Imagine this kind of time
| saving across many teams at Shopify's scale.
|
| Imagine that your pushes to production can happen _an hour_
| faster. At Shopify's scale.
|
| Do you see the pure economic value?
| kbknapp wrote:
| I'm not a fan of trying to put hard numbers on unknowns
| like this because it biases against uncertainty, but if
| they shaved ~74 minutes off their CI time and assuming it
| runs multiple times a day that very quick equates to a
| small teams cost savings over a year.
|
| However, I think trying to find the actual numbers is dumb
| because there's also the intangibles such as marketing and
| brand recognition bump by doing this both for the company
| and individuals involved.
|
| That's not to say all greenfield endeavors should be
| actioned, but ones with substantial gains like this seem
| fine given the company is big enough to absorb the initial
| up front cost of development.
| jerf wrote:
| How big is your business? Facebook has poured immense
| resources into speeding up PHP. It makes sense for them. It
| doesn't even remotely make sense for me.
|
| However, people tend to underestimate this sort of thing in
| general. Even since before programming... we have adages
| about the importance of sharpening the axe precisely
| because people have been hacking away with metaphorical and
| literal dull axes hoping to avoid needing to sharpen them
| for a long time. Sometimes you just won't be able to
| convince a business person of the importance of stopping
| work for a moment to sharpen the axe, because all they see
| is the work stopping. I don't have a solution to that level
| of lack of wisdom in a leader. These are the people who
| save $200 per programmer on computer hardware at the cost
| of 5 hours of productivity lost... per _week_. Some battles
| just come pre-lost.
| wredue wrote:
| It's because everyone time someone says "performance",
| something clicks in most developers brains, forcing them to
| respond with weird things like "being 50x slower is actually
| fast enough".
| thatxliner wrote:
| I don't understand how this is better than Biome. Does it support
| more rules than Biome?
| romanhotsiy wrote:
| Compatibility with eslint. They implement the most common
| eslint rules and looks like esling config support is WIP.
| hexmiles wrote:
| Say what you want about the "rewrite in rust" meme, but it really
| seems that rust started a trend of really caring about
| performance in everyday tools.
| bfrog wrote:
| I think its amazing that all these tools being rewritten in
| rust are likely being done by people that likely do not
| typically code in C/C++ but did code something like this in
| rust.
|
| To be that says Rust is more easily accessible as a language
| than C/C++ and that's great for our environment (speed is
| green) and our joy in computing (speed is happiness).
| ku1ik wrote:
| This.
___________________________________________________________________
(page generated 2023-12-15 23:01 UTC)