[HN Gopher] Node.js adds support for direct registry-less HTTPS ...
___________________________________________________________________
Node.js adds support for direct registry-less HTTPS imports
Author : inglor
Score : 118 points
Date : 2022-02-10 14:36 UTC (8 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| mrozbarry wrote:
| My impression is this is really just to have some sort of
| comparable feature to deno. You may think this somehow is more of
| a vulnerability, but the reality is the npm registry is already
| fairly insecure, we've seen npm packages get hijacked regularly.
|
| This also adds less dependence on npm, and maybe with all the
| packages that get hijacked there, this could be a reasonable
| move.
| junon wrote:
| I'd really argue "regularly" is a bit much. It happens rarely
| and moving to URIs is only shifting the problem.
|
| I'd also argue the claim npm is insecure. How would you improve
| its security over what's there now?
|
| Lastly, there isn't really a dependence on npm at all. You were
| always able to 1) put URIs into package.json, and 2) make your
| own package manager. Node does not require npm to work. It just
| looks in node_modules by default, which npm uses to manage
| packages for you. They are completely decoupled.
|
| URI imports, on the other hand, have a lot of security problems
| and surface area and will probably face a load of criticism
| going forward.
| dmitryminkovsky wrote:
| > How would you improve its security over what's there now?
|
| Automated builds on NPM servers based on code in public
| repos, like Docker has (had?). The least they could do is
| require packages to be signed.
| junon wrote:
| How exactly does that help?
| potatoz2 wrote:
| Controlled builds based on public repos prevents a
| malicious person (original author or not) from invisibly
| pushing a packages that doesn't correspond to public
| sources.
|
| Signing packages prevents account takeovers from
| publishing bad packages.
|
| The remaining security problem is the author themselves
| coupled with too few eyes on public repos.
| junon wrote:
| I fail to see how controlled builds would have prevented
| Marak pushing a patch release that broke everyone.
|
| Most malicious package activity I've seen has been at the
| hands of the original maintainers to begin with.
|
| The last thing can't be solved technologically.
| CGamesPlay wrote:
| Evaluating these things at runtime is absurd. It was absurd when
| deno wanted to do it and it's absurd now. People keep equating
| the threat model of npm and url imports, but with npm the
| untrusted code gets downloaded _at build time_ , so tools like
| snyk can scan it and flag problems. Is snyk supposed to intercept
| the live download and patch files now?
|
| Are the policy integrity checks they mention in the PR allow or
| block lists? For example, if my malicious trojan module does a
| dynamic import, can it bypass a saved integrity check by adding a
| cache bust url parameter?
|
| Honestly, I don't think the idea of URL imports is inherently
| bad. Go manages with them pretty reasonably. However, go only
| processes them at build time. My docker image from node might run
| today and break tomorrow because it's allowed to change the
| remote code it runs day to day.
| wperron wrote:
| If you actually read the (admittedly long) thread, it's pretty
| clear that those imports are _not_ resolved at runtime, and
| Deno has _never_ evaluated imports at runtime either.
| Dependencies get evaluated and downloaded _prior_ to startup,
| very similar to Go in fact.
| CGamesPlay wrote:
| I don't see that listed anywhere in the thread, so I'll ask
| here instead:
|
| Does _prior_ to startup mean there 's a separate build step,
| or are you saying it happens during the loading of the
| script. I'm pretty confident that the latter is what deno was
| doing, although I'm happy to be proven wrong.
|
| Are dynamic imports simply disallowed? Since it is impossible
| to download in advance a script to which you don't know the
| URL without running the program, they'd have to be, in order
| for your assertion to be true.
| mstade wrote:
| Direct link to the rendered docs for this feature:
| https://github.com/bmeck/node/blob/ceadb473e6d3a22f38b737108...
|
| It's unfortunate I think that the docs don't mention integrity
| checks or caching, but those things are mentioned in the PR
| thread. Integrity via policies which seems to be a new feature as
| well[1], perhaps designed to essentially obsolete things like
| package-lock.json et al? Caching is punted till later, presumably
| to not hold up progress on this particular feature.
|
| Seems to me these are actually steps in the right direction,
| paving some of the cow paths already created by npm and yarn. I
| for one would love to see package-lock.json etc make way for
| policies. Security is a concern for sure if there's no integrity
| checking on the packages loaded via https, but the same holds
| true for packages regardless of delivery method so I don't see
| how this is much worse security wise than just declaring
| dependencies in package.json. Policies does seem to be an effort
| to solve the integrity issue _regardless_ of how the package is
| installed, which seems like a good idea to me.
|
| If the caching and integrity issues are resolved well then this
| would essentially mean package managers become optional, no?
| Seems pretty good to me!
|
| [1]: https://nodejs.org/dist/latest/docs/api/policy.html
| user-the-name wrote:
| So they are adding a hidden, targetable supply chain attack
| vector on purpose?
| dangerface wrote:
| Loading modules over https because why? are dependancy attacks
| currently too difficult? Thanks I hate it.
| 999900000999 wrote:
| I have mixed feelings here.
|
| On one hand do what you want, on the other hand you can already
| add https imports via package.json.
|
| Why in God's name would you not have a package.json.
|
| I think this would be awesome for hacking out prototypes, but if
| I saw this in production code I'd ether work to get it fixed or
| find a new job. Being reckless is fine in your spare time. It's
| not ok at work.
|
| My fear is this is going to push junior devs into sloppy
| practices.
| sneak wrote:
| I have the distinct impression that whether you put the imports
| in package.json or in the source file itself, ~100% of the
| industry is engaging in sloppy practices now. I am terrified
| when trying to write secure es6.
|
| AIUI, when you add a module in go, the specific and exact
| version of the module is cryptographically locked in your
| project metadata. npm/yarn store the hashes in the lockfile but
| the package.json still uses those (usually) fuzzy matchers, and
| I'm not sure whether "npm i"/"yarn install" uses the lockfile
| exclusively by default. "npm help install" suggests that it
| uses package.json (which does _not_ cryptographically secure
| the deps so that they match what you originally added) by
| default.
|
| (Additionally, tons of people install and deploy docker images
| by tag name alone, which is not cryptographically secure -
| different environment, same problem.)
|
| My mixed feelings about it are that we are slowly blurring the
| line between local and remote software, and pretty soon it will
| be nearly impossible to use most modern software in an offline
| environment (which is a hard requirement for certain security
| circumstances).
|
| Basically every single js developer in the world has granted
| Microsoft (owners of GitHub and NPM domains) the ability to
| remotely execute code on their machine at will. It's a _lot_ of
| tooling and infrastructure to update and change in the event
| that the threat landscape changes significantly (e.g. war,
| national security mandates, etc).
| 999900000999 wrote:
| You can use NPM offline.
|
| Setting up a enterprise mirror isn't hard.
|
| NPM, as far I can tell was not designed to have you run npm
| install every single time you deploy. People decided to do it
| anyway, I still remember the day NPM's SSL certs went down (
| 2013 or 2014). It's like the internet broke overnight.
|
| It's a real catch 22. When you look for a magic npm package
| to save some time, you have no idea what's really in it
| unless you inspect all of it's code, and it's entire
| dependency tree.
|
| I guess you could build an solution to check for any
| vulnerabilities, but then you just move the problem
| elsewhere. Said ML tool would never be perfect
| rectang wrote:
| Manually auditing an update of potentially hundreds of
| packages when package-lock.json gets regenerated is so
| challenging that many projects won't do it -- and for those
| projects, the cryptographic hashes offer limited value.
|
| To make industry-wide improvements, we need to share
| resources for auditing of packages. There are many such
| endeavors underway at various stages of maturity.
|
| But this feature doesn't move that ball forwards. From the
| comments:
|
| https://github.com/nodejs/node/pull/36328#issuecomment-73603.
| ..
|
| > _It is insecure because there is no signature of the
| original file. Anybody that can take control of a domain name
| can take control of a server._
|
| Downloaded arbitrary code can't be added to a pool of pre-
| validated packages, except insofar as you control the server
| and only make pre-validated packages available on it
| exclusively to your own services.
| no_wizard wrote:
| Its also possible that this enables you to deploy all your
| assets prebuilt and optimized to a CDN?
|
| For Node apps running on services like fly.io this may be
| useful, since they're regionally distributed and their claim of
| running closest to the customers using it is at least partially
| true. This makes your deployed images (and therefore disk size
| needed) much smaller
|
| I agree that not having SRI as a requirement is a miss, however
| I see legitimate use cases here.
| numlock86 wrote:
| At this point they just try to make the ecosystem even worse and
| open as many attack vectors as possible, aren't they? Has it
| become some "funny meme" in the maintainer circle or something?
| Why there is no SRI for example?
| bricss wrote:
| More attack vectors for attack | vectors sake !?
|
| Idea for the new toolset start-up: - A cyberware to validate
| source code and its modules against https imports
| mrweasel wrote:
| Won't this effectively prevent an application to start if it
| cannot reach the internet? I haven't done much development with
| Node, but I've seen similar patterns with Java applications that
| attempt to pull in an XSD at start up, only to fail to boot
| completely, because the application server doesn't have direct
| internet access.
|
| It's a bit of a weird feature, which should only be used for a
| very limited number of project, in a niche which I'm not
| completely able to identify.
| BeefWellington wrote:
| My main problem with this is that they punted the integrity check
| work down the line.
|
| If this had been implemented as something akin to the integrity
| HTML attribute, e.g.: express = require(
| 'https://my_local_repo.tld/express/current/4.17.1', '
| sha384-oqVuAfXRKap7fdgcCY5uykM6+R9GqQ8K/uxy9rx7HNQlGYl1kPzQho1wx4
| JwY8wC' )
|
| It would be vastly superior. I'd go so far as to _require_ it
| without an additional --unsafe-import type flag.
|
| However, I think they did not do this because the import of
| dependencies then comes into question. IMO this feature should
| not have been added until developers could guarantee the expected
| integrity hash matches. We've seen time and again what will
| happen now is hundreds of blog posts talking about this new
| feature, none of which will mention the integrity hashing
| functionality (because it's not there). When it does eventually
| get added, those posts will be the most popular anyone can find,
| and thus new users will end up using those eventually-wrong posts
| and implementing things in a less-secure way.
|
| Huge disservice to the developer community.
| zackmorris wrote:
| Came here to say the same thing. In fairness, this isn't
| specifically Node's fault. I feel that a mistake was made early
| in the SPA/mobile craze a decade ago, where the web should have
| used subresource integrity (SRI) everywhere:
|
| https://developer.mozilla.org/en-US/docs/Web/Security/Subres...
|
| But instead went with less-secure Cross-Origin Resource Sharing
| (CORS):
|
| https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS
|
| These operate at different levels of abstraction. SRI is
| mathematically strong, whereas CORS requires a bunch of hoops
| and mostly just gets in the way. I could understand an argument
| for SRI+CORS, but using CORS alone sets off alarm bells in my
| head.
| spankalee wrote:
| CORS and SRI are just completely different. CORS exists to
| allow requests that would be banned under the Same Origin
| Policy, most of which have dynamic responses and could not
| integrate with SRI at all.
| horsawlarway wrote:
| I agree - I'm not really sure what the parent poster is on
| about. CORS and SRI are orthogonal, and designed to solve
| entirely different problems.
|
| CORS is primarily focused on removing csrf (cross site
| request forgery) vulnerabilities. It doesn't really care at
| all about what resources you pass around (or their
| integrity) it only cares about ensuring that the request is
| coming from a known and allowed origin.
| merb wrote:
| > CORS is primarily focused on removing csrf (cross site
| request forgery) vulnerabilities
|
| basically CORS gives you zero defense against csrf. (ok,
| not zero, but also not much)
| sneak wrote:
| It's rubygems all over again. I think only Nix and Go are doing
| dependencies right (cryptographically verified) these days.
|
| One upside is that it's https-only, which _does_ include
| authentication, provided that you trust the holder of the
| certificate for my_local_repo.tld (in your example) to serve
| you the right files.
| francislavoie wrote:
| Don't forget about PHP. Composer is one of the best package
| managers around.
| jeroenhd wrote:
| But what would be the point? Why would you load code over HTTPS
| if you're going to make it static anyway? If the code can't
| change dynamically, then you might as well put it next to the
| JS file on the filesystem.
|
| I think this is a terrible idea that can only end in misery but
| I can understand why they don't add integrity validation by
| default.
| BeefWellington wrote:
| HTTPS doesn't magically protect you from the content of the
| package changing.
|
| They are different security mechanisms designed to defend
| against different threats.
|
| One answers the question: Was this the correct server I got
| this package from?
|
| The other answers the question: Does the package's hash match
| what I expect in my code?
|
| As to why you would want this, I've worked on several
| projects that take advantage of some things that pulling
| dependencies only from a local repository can give you. Take
| the recent Log4J vuln as an example. Immediately on seeing
| that vulnerability, I disable the JAR file, causing builds to
| fail wherever that dependency exists until the updated one is
| added as a dependency and used. From experience having to do
| dependency analysis on an old toothy codebase, it's sometimes
| simpler and more accurate to just let everything break and
| fix the breaking spots.
| jeroenhd wrote:
| I understand that HTTPS doesn't protect you, and that's the
| reason why you'd normally add a hash.
|
| If your goal is to make the build fail for vulnerable
| applications, you can run a local repository with upstream
| proxying functionality like Nexus Repository and kill the
| package there. This won't let you kill production (but why
| would you want to kill it this way?) but it'll give you the
| same benefit without risking remote code execution.
| BeefWellington wrote:
| If you understand HTTPS doesn't protect you, I'm unclear
| on your initial point. Any chance you can elaborate on
| precisely what you meant when you said:
| But what would be the point? Why would you load code over
| HTTPS if you're going to make it static anyway? If the
| code can't change dynamically, then you might as well put
| it next to the JS file on the filesystem. I
| think this is a terrible idea that can only end in misery
| but I can understand why they don't add integrity
| validation by default.
|
| I provided an example of why you would want to still load
| it remotely (specific situations where it's beneficial)
| rather than simply copying the code into the source repo,
| which you have also echoed, though seemingly suggesting
| that the example means you can do away with integrity
| checks?
|
| If my read on that is correct, then point then is to
| worry about the repository - wherever it lives - being
| poisoned, but still wanting some of the benefits of
| having a remote repository (such as centralized
| dependency management and control).
| giaour wrote:
| > I provided an example of why you would want to still
| load it remotely (specific situations where it's
| beneficial) rather than simply copying the code into the
| source repo
|
| Doesn't Node cache imports, though? I think this approach
| would protect you if you were running a script on a timer
| (thus triggering a fresh load at a regular interval) but
| not if you were running an Express server (which would
| only verify the hash of the remote library at application
| startup).
| pc86 wrote:
| > _If the code can 't change dynamically, then you might as
| well put it next to the JS file on the filesystem._
|
| Yeah, you might be on to something with that!
| emiliobool wrote:
| we could call it dpm "Deno Package Manager"
| calvin wrote:
| It's a feature that Deno has and that works on the web (ESM
| imports from URLs). People have use cases for it and they
| want it. That's not to say it's secure by default of course
| but the demand is there.
| paxys wrote:
| The code _can_ update dynamically. You simply have to update
| the version number and verification hash on your end (same as
| you would in a package manager). This is still vastly easier
| than copy-pasting entire repositories into your codebase
| every time.
| danShumway wrote:
| Right, but if you don't want the code to update unless you
| specifically update the verification hash and version
| number, why wouldn't you download and install it locally?
|
| There's two separate questions:
|
| A) is it ever good to have dependencies that can change
| underneath you? (arguably no)
|
| B) if you don't want dependencies that change underneath
| you, why would you ever want to import directly from a URL
| at runtime?
|
| ----
|
| > The code can update dynamically. You simply have to
| update the version number and verification hash on your end
| (same as you would in a package manager).
|
| Code that I import through a package manager can update
| dynamically. I just need to update the version number and
| re-run `npm install`. Why would someone want to use a URL
| if the update process is the same as using a package
| manager?
|
| I'm having a somewhat difficult time imagining a use-case
| for fetching dependencies at runtime that isn't related to
| not knowing until runtime what the dependencies are or what
| version is being fetched. Maybe for making installation
| smoother? But anything that requires compilation or is
| system-dependent isn't something you're going to want to
| re-fetch at runtime anyway. I suspect those dependencies
| will be slow to install.
|
| And we're not dealing with a web situation where you might
| want to import from a CDN or another location that has
| better uptime. It sometimes makes sense not to serve assets
| yourself on the web just because it's expensive/difficult
| to serve them quickly. Node dependencies don't have that
| problem, pulling the dependency in as an install step will
| mean runtime performance is faster, not slower.
|
| I get the argument that you _shouldn 't_ do dynamic imports
| that don't have resource integrity checks, but that seems
| like an argument against this feature in general -- you
| shouldn't really be doing runtime imports at all,
| because... why would you need them except for
| unpinned/unchecked dependencies?
| paxys wrote:
| The advantage of this approach is that you don't need a
| separate package management ecosystem, no central hosted
| registry, no submissions. A bit of code is hosted at a
| URL, and that's it. It uses the same ES standards as
| importing any another JS script from your filesystem.
|
| Secondly, this doesn't mean all this code has to be
| dynamically loaded at runtime. You can still have
| equivalents of npm install which cache the linked
| resources locally on your server, perform integrity
| checks etc (this is exactly how Deno does it).
| danShumway wrote:
| > The advantage of this approach is that you don't need a
| separate package management ecosystem, no central hosted
| registry, no submissions.
|
| But npm has this already. You can already install from
| urls, you can even install from remote git urls. It's not
| even complicated, it's just `npm install
| git+ssh://user@hostname:git_repo.git#commit-ish`.
|
| I get that npm isn't Node, but this feels like a lot of
| duplicated effort for something that is much more safely
| handled with the package manager that ships with every
| single default Node installation. It's not exclusive to
| npm either, Yarn supports this stuff too.
|
| Are there really people building Node projects who are
| using a package manager that won't let them import from
| urls?
|
| ----
|
| > Secondly, this doesn't mean all this code has to be
| dynamically loaded at runtime. You can still have
| equivalents of npm install which cache the linked
| resources locally on your server, perform integrity
| checks etc (this is exactly how Deno does it).
|
| Unless Deno is crawling all of the source files in the
| project and doing this before the code is even run,
| aren't you still going to run into situations where you
| run your project and it hits an import it didn't see
| before, and it then gets dynamically imported at runtime?
|
| On the web, ES imports are designed to be kind of
| inflexible and static so that you can't have those
| dynamic systems, it's all designed to make it harder to
| run into unexpected imports and easier to do static
| analysis of dependencies. And maybe Deno is the same, but
| if so then I don't see what the advantage is over doing
| that instead of putting all of your dependencies in a
| package.json and fetching them as part of an explicit
| step. The web made this stuff super static and inflexible
| to try and make dependencies more explicit, but an
| install step is kind of the end-goal of those
| restrictions, it's the most explicit way possible of
| importing dependencies.
|
| And remember that npm lets you get your dependencies from
| anywhere, and they all get packaged in a predictable way,
| transparently on disk. You don't lose any flexibility in
| regards to where you get your packages from. At best,
| this system does the same thing, in which case why are we
| duplicating something that already exists? At worst, it
| caches resources only for future runs after it fetches
| them the first time, and it fetches them the first time
| at runtime, in which case I just don't understand why
| that would that be desirable.
|
| Maybe conditional installs of packages so you can only
| fetch a subset of the potential dependencies you might
| need based on what is actually used during program usage?
| But this seems like a really error-prone, potentially
| dangerous way to achieve that.
| paxys wrote:
| > But npm has this already. You can already install from
| urls, you can even install from remote git urls
|
| Yes, but npm can only install npm packages. This is a
| step towards reducing/removing the dependency on npm
| itself from the Node ecosystem. It is duplicated effort,
| sure, but that effort is spent in undoing extra
| complexity and getting to a _much_ easier "hello, world"
| state for both app and library developers.
|
| > Unless Deno is crawling all of the source files in the
| project and doing this before the code is even run
|
| That's exactly what it is doing.
| eyelidlessness wrote:
| I'm also concerned about resource integrity, and that was my
| immediate concern initially when I heard of Deno. But it's
| important to also recognize that it's quite a bit more complex
| than a hash for the requested module. Because of course that
| hash has nothing to say about the integrity of transitive
| imports.
|
| I'm willing to give Node the benefit of the doubt that this
| will remain experimental with appropriate warnings until
| resource integrity is addressed to some reasonable level of
| confidence. I'm also glad to see it being a first party
| concern, especially given that it's historically been/still is
| a much higher risk factor as both:
|
| - delegated to their ESM loader API, which is _incredibly
| difficult to get right_ , despite the API itself being
| deceptively simple
|
| - a hilariously insecure example in their docs for same
| jdcaron wrote:
| Exactly, this kind if import without a hash validation is a big
| no for security reasons (unless you 100% trust your import
| source). This feature exists on the browser side with the
| script element: https://developer.mozilla.org/en-
| US/docs/Web/Security/Subres...
| lukifer wrote:
| > you 100% trust your import source
|
| This is a perfect circumstance for the adage "trust but
| verify".
| __s wrote:
| There's already syntax to implement it with imports through
| import assertions: https://v8.dev/features/import-assertions
| FlorianRappl wrote:
| What about TypeScript here? I think it does not yet understand
| such imports. However, especially for types (i.e., evaluation at
| compile-time) imports from a remote source would be good.
|
| Instead of keeping up-to-date via npm I could just write `import
| type { FooType } from 'https://remote-types-
| registry.com/foopackage@2'` and wouldn't have to worry about
| typing updates. (surely this scenario is only valid when types
| are not shipped with the dependency)
| edgyquant wrote:
| This is basically how Deno does things already
| jppope wrote:
| probably just use deno?
| x-shadowban wrote:
| Seems like a lot of new node features are in response to
| competition with deno's offerings. Neat to watch.
| oefrha wrote:
| In my limited experience with Deno, it seems direct URL import
| just leads to a crappy, nonstandard, hard-to-upgrade version of a
| package.json in the form of, say, a deps.ts, once you get past a
| few dependencies. (And it's very easy to get past a few
| dependencies in Deno since std modules need to be imported as
| URLs like everything else.)
| spiderice wrote:
| That is my experience as well. I was really excited about this
| feature in Deno when I first heard about it. Then I started to
| get in to Deno, and it just seemed like it made everything
| harder. I thought it was going to be something like "here is a
| better way to do dependency management". But in practice it
| felt like "we got rid of dependency management. Figure it out
| on your own".
| conaclos wrote:
| I feel the same and import-maps do not solve the issue since
| they are not transitively loaded. In other word, if one of
| your dependency uses an import-map, then you have to copy-
| paste it in your own project to get its import-mapping.
| danShumway wrote:
| Kind of an interesting read.
|
| First thoughts, I generally think this is bad practice. I mean,
| for one thing this guarantees you need an Internet connection to
| run your Node script, which seems pretty problematic.
|
| I'm also reading some really questionable security justifications
| from people on the issue tracker about why dynamic imports are
| actually fine and just as secure as importing packages from NPM.
| There are security improvements that could be made to NPM,
| particularly around package search, and NPM has suffered from
| people replacing existing package versions (although I think
| they're clamping down on this more now).
|
| But there is a really fundamental difference between taking that
| risk once during installation and taking that risk multiple times
| at runtime. Additionally, in regards to some of the comments
| saying that installing code directly from the source
| ("express.com" vs "npm.com") ignores the fact that this is
| already kind of possible with npm dependencies and promoting that
| method wouldn't have any of the other problems.
|
| ----
|
| HOWEVER:
|
| You read further and there are actually some really interesting
| security risk mitigations being proposed that don't just boil
| down to saying "all imports are dangerous, why not do them here
| too." "Scopes" (which I was not aware was a concept that Node had
| any support for) are the idea that packages in a scope or a
| domain can't access any dependencies outside of them.
|
| Additionally, people are talking about mirroring Deno's system
| where these get cached locally and not re-fetched later, which is
| kind of similar to npm's system and reduces some risk. So you can
| load a module from HTTPS and in theory it only gets fetched once
| and it can't access the file system, cool.
|
| Additionally, `node_modules` dependencies can't use them, which
| is cool, means that it's less likely a random dependency starts
| doing this behind your back.
|
| ----
|
| HOWEVER:
|
| Prototype pollution is still a thing, passing objects around
| across closures is still a thing, and as far as I know this is a
| big part of why it's difficult to sandbox Javascript modules.
| Scopes are really interesting, but I can't find a lot of
| information about them online, and their name conflicts with npm
| scopes, which are a different thing. And I can't find any info
| suggesting they've solved the bigger problems of sandboxing
| modules.
|
| And there's still the elephant in the room, which is that all of
| those security measures are stuff that could be done in a more
| general way that applies to normal modules? I'm not sure why I
| would only want scopes for HTTPS imports, assuming they even work
| securely for HTTPS imports in the first place, which I wish I
| could find more information about. It's cool that a module gets
| fetched once at runtime and not later, but that still means I
| need an Internet connection the first time the Node script runs.
| And it seems like "tell me what dependencies you need and I'll
| fetch them" is a good candidate for an install step, which Node
| has.
|
| I'm not totally against the idea, I can in theory think of
| situations where you might want to install something at runtime
| without knowing ahead of time what it is. But that's also really
| dangerous to do, and doesn't really fit with the caching model
| that is being proposed.
|
| ----
|
| So I still feel pretty doubtful that this is a good feature to
| add, and it seems really dangerous at first glance, but maybe
| there are other use-cases that I don't understand or maybe
| there's something to scopes that I don't get. I will say that if
| I actually trusted the security of scopes as a way of sandboxing
| individual modules, then at that point I feel a lot better about
| dangerous imports in general. But I can't find any info on
| scopes, and as far as I know, systems like Deno gave up on this
| because it was really hard, and I also vaguely feel like
| effective module-level sandboxing in Node would be _much_ bigger
| news than any import feature.
|
| But :shrug:, I could be missing something. This is based on only
| a quick read through the entire thread and a tiny bit of extra
| searching/reading the commit docs.
| neals wrote:
| Does anybody have an exmple of this and how / why I could use
| this?
| qbasic_forever wrote:
| Presumably it's just like how deno does dependencies:
| https://deno.land/manual/examples/manage_dependencies
|
| As for why, it's simpler than adding a package.json and
| meticulously curating it. If you only have a few small
| dependencies this might make sense. You can also pull in stuff
| that isn't hosted on npm or properly packaged--for example just
| pull down a js file from someone's github repo.
|
| If you start getting a lot of dependencies then it might turn
| into a new nightmare to maintain. It's an experiment--people
| don't really know if it's good or bad until using it a bit.
| [deleted]
| fredley wrote:
| You can use it to shoot yourself in the foot.
| tehbeard wrote:
| This is, IMO, the weakest part of deno being implemented.
|
| With npm I can easily get metrics for how stale dependencies are
| or quickly check for security issues.
|
| The closest I found for deno was maybe trex? And that felt
| incredibly anemic.
| mgkimsal wrote:
| Am I wrong in thinking this is similar to the below code?
|
| <?php include("https://otherdomain.com/somephpcode.txt"); ?>
|
| PHP got ridiculed for years for having this sort of stuff
| available (even if off by default eventually).
| nexuist wrote:
| PHP got ridiculed for years for doing all the things all the
| other popular frameworks are coming around to support now.
| Server side rendering is particularly hilarious because it's
| like: oh, you're compiling .jsx files to .html now?
| Cool....can't imagine anyone ever did that before...
| eyelidlessness wrote:
| The reason SSR/SSG of JSX (or Svelte, Vue, Marko, whatever)
| is appealing isn't simply that you can run it on
| server/build. The appeal is that you can use _the same code_
| client-side. And of course each such framework /library has
| its own value propositions that are useful in both contexts.
|
| An additional of JSX in particular is that it's render-target
| agnostic. Yeah you can render JS/DOM, and HTML... and CLI,
| and native UI APIs, and smart TVs, and WebGL, and...
|
| (I know there are similar solutions for some of these in the
| others, but JSX was specifically designed to accommodate
| this.)
| wnevets wrote:
| Its always amusing watching people "discover" things that
| already existed.
| jchw wrote:
| CGI script style languages like PHP are only superficially
| similar to SSR.
| nexuist wrote:
| Is it though? I remember injecting <php> tags in the middle
| of JS scripts to fill out variable values. I'm sure someone
| somewhere was using PHP to inject conditionally dependent
| JavaScript onto their page. Is that much different from
| Next rendering a React component on the backend, injecting
| it into HTML, and then injecting its corresponding
| component state into the JS bundle?
| jchw wrote:
| ...Yes? For one thing, PHP code runs _inside_ the process
| of rendering HTML, but JS SSR runs under the process of a
| React JS app, running in Node. That alone is an
| absolutely massive difference. Not just on a technical
| level; React JSX also doesn 't have nearly as much
| hazards surrounding HTML sanitization, for example,
| because JSX is a DSL that protects inputs with proper
| sanitization by default, among other practical
| differences. Another difference is that SSR is more
| flexible than for runtime environment; it is very often
| used as a static compilation technique, where it is run
| as a compilation step for a static site, but either way,
| it is not married to any specific runtime model, whereas
| PHP pretty much is (for now.)
|
| A closer analogue to what React does would be comparing
| XHP[1] to JSX, but even that misses the isomorphic aspect
| of a SPA with SSR, which is almost the entire value
| proposition, because it allows you to only need to write
| your component code once while reusing the same render
| cycle at browser runtime.
|
| It should be pretty evident that PHP-style CGI scripting
| is not ideal given that even early on many PHP apps moved
| to using template systems like Smarty and never actually
| embedding HTML directly into PHP code.
|
| [1]: https://github.com/phplang/xhp
| markstos wrote:
| Sounds like it. These days, HTML is trusting JavaScript that's
| loaded from all over the place already. A "secure" database
| hosting service loads assets from 20 different domains on their
| login page.
|
| Loading code from remote domains can be abused and mis-used in
| any language. In this case, I think it's considered a "feature"
| because code was already being loaded remotely with NPM, who in
| turn got the code from the original author.
|
| Now you can load the code directly from the author. So instead
| of trusting both the author and NPM, you only need to trust the
| author. This could be considered a reduction in your attack
| surface area, and it's definitely helping with the NPM registry
| being a single-point-of-failure in the Node.js ecosystem.
| dangerface wrote:
| Npm won't allow you to require remote code from a user
| touched variable like the below.
|
| myRemoteLib = require(userGeneratedVariable)
| eyelidlessness wrote:
| I don't know where you got that idea. NPM doesn't have any
| say in CJS (or ESM) module resolution other than that they
| conform to relevant aspects of it. Node and other
| implementers of those module systems absolutely do allow
| dynamic require/import.
| that_guy_iain wrote:
| This was literally my first thought, the return of Remote File
| Injections.
| [deleted]
| progx wrote:
| So Deno is obsolete?
| mccorrinall wrote:
| deno scripts need explicit permissions for using the file
| system, network etc
|
| that's a pro in my book.
| qbasic_forever wrote:
| Definitely not, deno does typescript out of the box without
| pulling in a bunch of npm dependencies and tools. It also is
| taking more of a go or python/batteries included view of
| building out a proper standard library vs. relying on the
| community to provide it. And of course deno has a much stricter
| execution sandbox model where you have to opt in to explicit
| permissions like file system access. IMHO we'll probably see
| node get more and more deno-like over time.
| edgyquant wrote:
| Maybe one day, for now typescript with node is still slow
| ravenstine wrote:
| No...
|
| If anything, this shows how ahead Deno is and how features like
| this in Node.js are too little too late.
|
| Deno's standard library is just better than Node's, IMO, and
| it's going to be hard for Node to play catchup, if that's even
| practical. In summary, it doesn't contain the cruft of the pre-
| ES2015 era that is still present in Node, even though Node has
| tried working around it. The global objects in Deno better
| replicate the browser, so it takes a lot less effort to do
| essentially the same things you would in a browser but in a
| DOMless environment. Deno is slightly more performant in Node
| in some areas, although this is usually negligible, and I don't
| have the JSPerf test on hand.
|
| I don't think Node is bad, by any means. If people are
| concerned about security... then don't use so many
| dependencies, or consider just not updating them until they've
| been demonstrated not to contain exploits; you don't actually
| need much code to essentially serve webpages.
|
| But I think Deno, _over time_ , will at least match Node in
| popularity, maybe even more, because it has "modern" approaches
| right out of the box and doesn't need to move as slowly as Node
| in order to maintain a legacy. Although Deno misses some things
| here and there, I've seen features get added very quickly over
| the last couple years; at this point, it would take Node much
| longer to do the same things for fear of breaking something and
| because it's simply much older code.
|
| Then again, maybe I'll be wrong. Perhaps Node gets a second
| "big bang" and makes Deno obsolete. And I wouldn't care,
| either. All I want is a JavaScript language runtime with fewer
| concepts to learn that can also move quickly when something
| needs to be improved. If that ends up being Node, I'd be just
| as glad.
| royjacobs wrote:
| Won't this make leftpad-type scenarios even _more_ likely? What
| is the benefit?
| zja wrote:
| If node shares import syntax with browsers, it's easier to
| write code that runs on both platforms. As far as leftpad, this
| isn't that much different from using a package manager. You
| either cache your dependencies, or rely on a third party (npm
| repo/cdn).
| onion2k wrote:
| When leftpad was in NPM there was an expectation that it
| wouldn't disappear suddenly. Package registries are thought to
| be a constant, reliable source of packages (even when they not
| _quite_ that, as leftpad demonstrated). There would be no such
| expectation for a module that you include directly from an
| https URL.
|
| The main benefit I can see is that you can include code from
| your own "registry" very easily in unmanaged scripts. You can
| already do that because you can add a dependency to
| package.json using an https link already, but with this new
| feature you can include those dependencies in node scripts you
| aren't managing with a package file.
|
| You _can_ do that. I 'm not suggesting you should. I won't.
| jchw wrote:
| A lot of the complaints here claim that this is bad because it's
| insecure. However, allow me to posit a question: what if I made a
| function that just fetches the script and evals it? Node.JS
| already involves running unsandboxed, practically untrusted
| arbitrary code, that nobody audits. I'd have more sympathy if not
| for the fact that you can pretty much emulate this with the
| permission level that Node.JS code runs today.
|
| Also, no... this is not like PHP. PHP include got lambasted
| because it compounded other PHP security issues. If you use ESM
| imports, the obvious way, there's never going to be an accident
| where user input can control the URL, because the grammar makes
| it impossible. You can use the dynamic import statement, but at
| that point you're off into the abyss.
| vorpalhex wrote:
| This is nice for many use cases, especially where running your
| own npm mirror is a pain. Hopefully you are controlling the http
| source though!
| qbasic_forever wrote:
| Nice too in that you might not even need to touch or use npm.
| Imagine a node app without an enormous juggernaut of a
| node_modules folder!
| eyelidlessness wrote:
| A couple observations of interest to me:
|
| 1. This follows closely behind the recent addition of fetch
| support. I expected this would leverage that, but it does not. So
| I assume this was developed in greater isolation than I would
| have hoped. I don't have time to look deeper now, but given
| overlapping responsibilities I would think fetch would be a
| better fit than the older http(s) APIs.
|
| 2. Again following close behind landing fetch, Node is gaining
| more and more feature overlap with Deno. It's not clear yet, to
| me anyway, whether that's a direct motivation... or a somewhat
| convenient coincidence of similar goals. If it is a direct
| motivation, I feel like API compatibility, while valuable, may
| not be the most valuable priority. At least speaking for myself,
| my primary attraction to Deno is its fast and build-/hassle-free
| TypeScript support. Everything else is icing on the cake.
| notpachet wrote:
| The people in the Github thread advocating for this feature seem
| to be pushing really hard on the point that this isn't any more
| insecure than NPM already is. But that's myopic. That's like
| saying that the lock is broken on the second floor window of your
| house, so you may as well unlock all the other windows. If a
| junior developer working for me proposed something like this I
| would pull them away from the computer and have a long coffee
| chat with them about basic architecture principles.
|
| Yes, we know there are problems with NPM. But why broaden the
| possible attack surface area by introducing another kind-of-sort-
| of-equivalent means of remote module loading? Now you're just
| going to have to spend time hardening two systems instead of one.
|
| At the end of the day this just feels like a misguided attempt to
| try and recapture some sense of dynamism from Deno. But if I
| wanted dynamism, I'd use Deno, with all of its pubescent
| awkwardness. I'm using Node.js stable for a reason. I spend a lot
| of cycles trying to defend Node from its many (often reasonable)
| detractors, and things like this make me feel like I'm wasting my
| breath...
| encryptluks2 wrote:
| junon wrote:
| Lul what? First of all, this tone is completely against the
| HN guidelines, so take your flag.
|
| Second of all, centralization is not often a part of a
| security model. Usually in such cases where availability is a
| concern you have a mirror set up, which is ridiculously easy
| to do with CouchDB and completely nullifies your arrogant
| point.
|
| Get off your high horse.
| encryptluks2 wrote:
| notpachet wrote:
| My comment wasn't really about the merits of central
| authorities vs multiple authorities (although I would fall on
| the side of preferring a single package repository than
| having to worry about many of them). It was about having two
| equivalent mechanisms for solving the remote-code-loading
| problem, regardless of where you're actually loading it from.
| wperron wrote:
| Hi, I'm one of those people in the PR advocating for this
| change :) I think you may have missed the point of comparing
| the security aspects to NPM. By saying "it's no less secure"
| it's not saying "well it's just as bad, so what's the point?",
| but rather "given that it doesn't introduce any more
| vulnerabilities, what does this model unlock for us?"
|
| HN comments probably aren't the best place to go in-depth on
| this topic, but just off the top of my head, HTTPS imports
| allow a greater detachment from a centralized registry,
| something we're seeing used a lot in the Go ecosystem, and
| we're moving server-side JavaScript closer to how browser
| JavaScript works, which is a good thing for anyone using the
| language, as it reduces friction across the board. This might
| not be something obvious for people working mostly/only on the
| backend, in which case those frictions aren't as apparent, but
| they _are_ there nonetheless.
| inglor wrote:
| This is off by default due to security concerns but can be turned
| on with a flag. It's mainly in the community feedback phase so if
| there is anything you want to tell Node (be it "I want this!" or
| "This is a bad idea" - please do)
| sdflhasjd wrote:
| I recall when I was a completely PHP n00b (babbys first
| programming language), I might encounter an issue and search
| for the error on expertsexchange. Some of the answers would be
| really complicated, but there'd be that one answer: Just edit
| your php.ini and change `dangerous_option_you_will_die=true` -
| wow, such an easy fix.
|
| I'd know better later, but I'd still find horrendous options
| like these enabled in _real_ software I would come accross in
| my professional career.
| pimterry wrote:
| In terms of feedback, there's an issue thread here covering
| some points already:
| https://github.com/nodejs/node/issues/41920
| mitchellst wrote:
| Longtime node.js developer... I don't think we need this, and I
| don't think it's a good idea to add.
|
| The feature is _strange._ Which we know because it's behind a
| flag, i.e., it will have low utilization. It defies conventions
| --because the convention is to use a registry for packages. And
| it presents significant security concerns. (Which I don't
| really have to explain, as they're already cited as the reason
| to put it behind a flag.)
|
| Use of this feature seems like something you might see once in
| a career. (Indeed, I've seen a similar thing in another
| language employed only once... AND THERE WERE OTHER WAYS for
| that developer to have designed that system that would not have
| been so vulnerable.) When you see it, do you recognize what it
| is, understand and evaluate the risks it poses? Do you validate
| that the library sitting on the other side hasn't been taken
| over by a malicious actor? Do you validate that the library on
| the other side has up-to-date dependencies? If so, on what
| cadence do you do this, realistically?
|
| All the security tooling progress we've made (npm audit, etc.)
| seems shot in the foot by this. And I get that these concerns
| aren't dispositive on their own. I just don't see the argument
| on the other side that clearly. Is there some class of
| application that people really want to write in node that can't
| be written for lacking this? Do we really think this is a
| pattern that would be employed in well-made, secure codebases?
| I don't see the argument. Like I said, I've seen a feature like
| this _used_ in another language. But I haven 't seen it
| _needed_ for any real application.
|
| I love me some good node ugly hacks. Still, the language
| features should be nudging us to write good software.
| oauea wrote:
| > because the convention is to use a registry for packages
|
| You can already pull your dependencies from arbitrary URLs,
| they just have to be tarred up first and go through npm:
| https://docs.npmjs.com/cli/v8/configuring-npm/package-
| json#u...
| encryptluks2 wrote:
| Central authority != Good software. You are essentially
| advocating to provide a single point of failure that can and
| will be abused by nation-state hackers who just have to find
| one popular package and compromise the developer.
| mitchellst wrote:
| The argument is less about central authority than standard
| process. You don't have to use the official NPM registry.
| The point is that having a pipeline for modules with
| established conventions and automated means of audits and
| visibility on the dependency is a net good. Yes, there is
| such a thing as a supply chain attack. But even when that
| happens, the community puts out the call to patch
| immediately and points a finger at the offending package.
| Your HTTPS import from a random URL could fail silently.
| While NPM is a big target for supply chain attack, they
| _know_ they are, so there are a lot of eyes on it. Not so
| with your HTTPS import.
| forty wrote:
| I have been doing nodejs backend developpement for the past 10
| years. I'm not sure why we would need that honestly. Looks like
| it could create some mess if packages on npm all start
| importing files from random servers. Suddenly your CI relies on
| many servers to be up. And I don't know what's the problem with
| declaring things in package.json.
| Andrex wrote:
| My layman's take is that it seems very similar to Log4j's
| vulnerability, but I don't really know for sure.
| oauea wrote:
| Please explain why you think that, because it seems
| completely unrelated and off-topic.
| moritonal wrote:
| Bit harsh? Log4j was caused by a URL leading to bad code
| being imported.
|
| This let's you import code via a URL, seems quite similar?
| sodality2 wrote:
| Log4j was caused when arbitrary log strings were
| interpreted as code imports.
|
| In this case the flag has to be manually set and the code
| manually added as a dependency.
| anamexis wrote:
| The Log4j vulnerability allowed _unsafe user input_ to
| trigger loading of arbitrary code.
|
| Basically every dependency manager out there will allow
| you to load "bad code" from a URL.
| tobyhinloopen wrote:
| Log4j is no dependency manager
| anamexis wrote:
| No, it's not, but this new Node.js feature is for loading
| dependencies
___________________________________________________________________
(page generated 2022-02-10 23:02 UTC)