[HN Gopher] Speeding up the JavaScript ecosystem part 2 - Module...
___________________________________________________________________
Speeding up the JavaScript ecosystem part 2 - Module Resolution
Author : mhagemeister
Score : 57 points
Date : 2023-01-16 17:07 UTC (5 hours ago)
(HTM) web link (marvinh.dev)
(TXT) w3m dump (marvinh.dev)
| root_axis wrote:
| > _But it gets even better! Lots of projects make use of path
| mapping aliases to save a little bit of typing, so that you can
| use the same import specifiers everywhere and avoid lots of dots
| .. /../../._
|
| This isn't just to "save a little bit of typing", there is a
| significant amount of mental overhead eliminated by not having to
| recall and reason about the relative path relationship between
| each module for every single import, it also prevents the issue
| where moving around files or refactoring your directory structure
| breaks your application, forcing you to do potentially dozens of
| import path updates depending on what got moved around.
| mhagemeister wrote:
| These are good points! There is definitely more appeal to them
| than I gave them credit for in the article.
| robust-cactus wrote:
| Hmm instead of patching node, is there any tech that lets you
| make a virtualized file system on top of existing files? With
| that we might be able to design a system that resolves some of
| these module requests in a better way.
| boucher wrote:
| FUSE will let you create a virtual file system that does
| absolutely anything you want it to (and there are javascript
| bindings out there).
| demurgos wrote:
| Yarn does it with its Plug'n'Play (PnP) mode. It keeps
| dependencies archived, and computes a `pnp.cjs` file caching
| where to resolve dependencies for each package. It works
| transparently by using a virtual FS.
| irrational wrote:
| My takeaway as a developer is always include the file extension
| and the full relative path. Remove all ambiguity.
| hateful wrote:
| Here's to hoping that this article leads to PRs in their
| respective projects!
| stephen wrote:
| Little disappointing that he points out/measures a huge source of
| slowness, but unfortunately we can't really do anything about it
| (for valid reasons, he points out the tool fragmentation, etc).
|
| Originally, I was hoping that "fast module resolution of huge
| projects" is something that Deno would have solved from day 1
| when they said "meh we don't need npm" and had the opportunity to
| clean-slate the design. Granted, they do support npm now, but
| personally I think they only had to back-peddle because their
| non-npm approach didn't have any huge advantages (i.e. perf wise)
| for most users to bother with.
|
| Like if Deno had fixed this "30% module resolution overhead for
| large test suites / app startups" when it first came out, I would
| have gotten our app on to it immediately.
|
| IANAE / I haven' tried it yet, but I believe bun's `bun bun`
| command is the best (only?) innovation trying to tackle this
| problem:
|
| https://github.com/oven-sh/bun#bun-bun
|
| I really should it try, my hope is that it's the "I am
| _immediately_ moving to bun" carrot that Deno never delivered.
| mhagemeister wrote:
| Author here.
|
| Thanks for the feedback. The reason I wrote this article is
| because we _can_ do something about it. Through avoiding
| throwing lots of wasteful error objects and adding a little bit
| of caching, the time it took to lint the project became 30%
| faster. Those changes were applied locally to a couple of
| popular third party tools for module resolution in
| node_modules. That's how the 30% speedup was achieved.
|
| That said if module resolution wasn't as complex in node to
| begin with, the speedup would surely be a little greater. I'm
| hoping that this post sparks a bit of discussion on that and
| some node contributors already voiced interest on twitter to
| think more about that.
| stephen wrote:
| Awesome! I see the PRs you've linked to; sorry, I'd
| misinterpreted the post as experiments/"what if" exploration
| and not "there are patches landing soon". That's great!
| mhagemeister wrote:
| No worries, that's valuable feedback. I should have made it
| more clear in the article that it's not just theory.
| acemarke wrote:
| Great investigation work and excellent writeup!
|
| Out of curiosity: did anyone end up filing issues or PRs
| against these tools as a result of either of your articles?
| mhagemeister wrote:
| Thanks, happy you enjoyed the article!
|
| The PRs for the previous article were all merged. I
| originally wanted to do the same for this one, but I'm not
| sure if I have the time to fix all of them. Updating to a
| newer version of `resolve` already addresses the most
| notable issue with throwing more errors than necessary, but
| many parts of the ecosystem still use an old version.
|
| Manged to land this PR in another package though
| https://github.com/import-js/eslint-import-resolver-
| typescri... . Performance there could be easily improved
| further there.
|
| Another PR to an eslint plugin was unfortunately rejected
| as it broke node 4 support https://github.com/import-
| js/eslint-plugin-import/pull/2654 .
|
| The other popular package that's used for module resolution
| is `enhanced-resolve` by the webpack folks and they expect
| the consumer to deal with passing the appropriate options.
| So there isn't really a single place to fix this.
| epmatsw wrote:
| Ugh. ESLint-plugin-import, I should have known. Not at
| all surprising that a perf fix wouldn't be prioritized
| there unfortunately.
___________________________________________________________________
(page generated 2023-01-16 23:01 UTC)