[HN Gopher] A better streams API is possible for JavaScript
___________________________________________________________________
A better streams API is possible for JavaScript
Author : nnx
Score : 336 points
Date : 2026-02-27 14:02 UTC (8 hours ago)
(HTM) web link (blog.cloudflare.com)
(TXT) w3m dump (blog.cloudflare.com)
| user3939382 wrote:
| " The Streams Standard was developed between 2014 and 2016 with
| an ambitious goal to provide "APIs for creating, composing, and
| consuming streams of data that map efficiently to low-level I/O
| primitives." Before Web streams, the web platform had no standard
| way to work with streaming data."
|
| This is what UDP is for. Everything actually has to be async all
| the way down and since it's not, we'll just completely
| reimplement the OS and network on top of itself and hey maybe
| when we're done with that we can do it a third time to have the
| cloud of clouds.
|
| The entire stack we're using right down to the hardware is not
| fit for purpose and we're burning our talent and money building
| these ever more brittle towering abstractions.
| afavour wrote:
| UDP is a protocol, not an API
| mlhpdx wrote:
| True. But it's also true that trying to shoehorn every use
| case into TCP streams is counter productive.
|
| A stream API can layer over UDP as well (reading in order of
| arrival with packet level framing), but such a stream would a
| bit weird and incompatible with many stream consumers (e.g.
| [de]compression). A UDP API is simpler and more naturally
| event (packet) oriented. The concepts don't mix well.
|
| Still, it would be nice if they browser supported a UDP API
| instead of the weird and heavy DTLS and QUIC immitations.
| kaoD wrote:
| TCP or UDP are orthogonal to this, so the original comment
| feels like a non sequitur. These streams are not network
| streams and could be a file, chunks of procedural audio, or
| whatever.
| delaminator wrote:
| We're too busy building products while waiting for the perfect
| system to arrive.
| user3939382 wrote:
| I'm building everything from first principles, I'm not
| climbing the exponential curve with some billionaire that has
| to finance it.
| delaminator wrote:
| I really doubt you are. you're not visiting the transistor
| shop every time you want to build a react component
| user3939382 wrote:
| Good thing your confidence is a soft requirement :)
| kg wrote:
| It's a real shame that BYOB (bring your own buffer) reads are so
| complex and such a pain in the neck because for large reads they
| make a huge difference in terms of GC traffic (for allocating
| temporary buffers) and CPU time (for the copies).
|
| In an ideal world you could just ask the host to stream 100MB of
| stuff into a byte array or slice of the wasm heap. Alas.
| amluto wrote:
| I wonder if you can get most of the benefit BYOB with a much
| simpler API: for await (const chunk of
| stream) { // process the chunk
| stream.returnChunk(chunk); }
|
| This would be entirely optional. If you don't return the chunk
| and instead let GC free it, you get the normal behavior. If you
| do return it, then the stream is permitted to return it again
| later.
|
| (Lately I've been thinking that a really nice stream or receive
| API would return an object with a linear type so that you
| _must_ consume it and possibly even return it. This would make
| it impossible to write code where task cancellation causes you
| to lose received data. Sadly, mainstream languages can't do
| this directly.)
| dilap wrote:
| > The problems aren't bugs; they're consequences of design
| decisions that may have made sense a decade ago, but don't align
| with how JavaScript developers write code today.
|
| > I'm not here to disparage the work that came before -- I'm here
| to start a conversation about what can potentially come next.
|
| Terrible LLM-slop style. Is Mr Snell letting an LLM write the
| article for him or has he just appropriated the style?
| lapcat wrote:
| You've got it backwards: LLMs were trained on human writing and
| appropriated our style.
| have_faith wrote:
| Partially true. They've been trained and then aligned towards
| a preferred style. They don't use em-dashes because they are
| over-represented in the training material (majority of people
| don't use them).
| lapcat wrote:
| It seems likely that with the written word, as with most
| things, a minority of people produce the majority of
| content. Most people publish relatively few words compared
| to professional writers.
|
| Possibly the LLM vendors could bias the models more toward
| nonprofessional content, but then the quality and utility
| of the output would suffer. Skip the scientific articles
| and books, focus on rando internet comments, and you'll end
| up with a lot more crap than you already get.
| jitl wrote:
| cloudflare does seem to love ai written everything
| azangru wrote:
| What was it specifically about the style that stood out as
| incongruous, or that hindered comprehension? What was it that
| made you stumble and start paying close attention to the style
| rather than to the message? I am looking at the two examples,
| and I can't see anything wrong with them, especially in the
| context of the article. They both employ the same rhetorical
| technique of antithesis, a juxtaposition of contrasting ideas.
| Surely people wrote like this before? Surely no-one complained?
| jsheard wrote:
| The problem is less with the style itself and more that it's
| strongly associated with low-effort content which is going to
| waste the readers time. It would be nice to be able to give
| everything the benefit of the doubt, but humans have finite
| time and LLMs have infinite capacity for producing trite or
| inaccurate drivel, so readers end up reflexively using LLM
| tells as a litmus test for (lack of) quality in order to cut
| through the noise.
|
| You might say well, it's on the Cloudflare blog so it must
| have some merit, but after the Matrix incident...
| azangru wrote:
| > You might say well, it's on the Cloudflare blog so it
| must have some merit
|
| I would instead say that it is written by James Snell, who
| is one of the central figures in the Node community; and
| therefore it must have some merit.
| guntars wrote:
| These AI signals will die out soon. The models are
| overusing actual human writing patterns, the humans are
| noticing and changing how they write, the models are
| updated, new patterns emerge, etc, etc. The best signal for
| the quality of writing will always be the source, even if
| they are "just" prompting the model. I think we can let one
| incident slide, but they are on notice.
| jasnell wrote:
| Heh, I was using emdashes and tricolons long before LLMs
| appropriated the style but I did let the agent handle some of
| the details on this. Honestly, it really is just easier
| sometimes... Especially for blogs posts like this when I've
| also got a book I'm writing, code to maintain etc. Use tools
| available to make life easier.
| eis wrote:
| People are understandably a bit sensitized and sceptical
| after the last AI generated blog post (and code slop!) by
| Cloudflare blew up. Personally I'm fine with using AI to help
| write stuff as long as everything is proof-read and actually
| represents the authors thoughts. I would have opted to be a
| bit more careful and not use AI for a few blog posts after
| the last incident though if I was working at Cloudflare...
| n_e wrote:
| I found your article both interesting and readable.
|
| It doesn't really matter what tools are used if the result is
| good
| silisili wrote:
| I'm not sure any emdash use at all is what people are calling
| out typically(maybe it is?), more the sheer number of them
| typical in LLM written stuff.
|
| Just ctrl-f'ing through previous public posts, I think there
| were a total of 7 used across about that many posts. This one
| for example had 57. I'm not good enough in proper English to
| know what the normal number is supposed to be, just pointing
| that out.
| dilap wrote:
| I think you'd be much better served by writing something
| rough that maintains your own voice!
| hackrmn wrote:
| Just want to raise my hand and say I too have been using em
| dashes for considerably longer than LLM has been on every
| hacker's lips. It's obviously not great being accused of
| being an AI just because one has a particular style of
| writing...
| nebezb wrote:
| The idea is well articulated and comes across clear. What's the
| issue? Taking a magnifying glass to the whole article to find
| sentence structure you think is "LLM-slop" is an odd way to
| dismiss the article entirely.
|
| I've read my fair share of LLM slop. This doesn't qualify.
| shevy-java wrote:
| We deserve a better language than JavaScript.
|
| Sadly it will never happen. WebAssembly failed to keep some of
| its promises here.
| postalrat wrote:
| Where can I find these not kept promises?
| nindalf wrote:
| They haven't yet made languages other than JavaScript first-
| class languages for the web -
| https://hacks.mozilla.org/2026/02/making-webassembly-a-
| first.... I wouldn't call this a broken promise, but it was
| something people were hoping would take less than a decade.
| gejose wrote:
| There's always a comment like this in most discussions about
| javascript.
| krashidov wrote:
| > WebAssembly failed to keep some of its promises here
|
| classic case of not using an await before your promise
| teaearlgraycold wrote:
| As wonky as JS is I really like it. Typescript has done such a
| good job at making it fun to use.
| conartist6 wrote:
| As it happens i have an even better API than this article
| proposes!
|
| They propose just using an async iterator of UInt8Array. I
| _almost_ like this idea, but it 's not quite all the way there.
|
| They propose this: type Stream<T> = {
| next(): Promise<{ done, value: UInt8Array<T> }> }
|
| I propose this, which I call a stream iterator!
| type Stream<T> = { next(): { done, value: T } | Promise<{
| done, value: T }> }
|
| Obviously I'm gonna be biased, but I'm pretty sure my version is
| also objectively superior:
|
| - I can easily make mine from theirs
|
| - In theirs the conceptual "stream" is defined by an iterator of
| iterators, meaning you need a for loop of for loops to step
| through it. In mine it's just one iterator and it can be consumed
| with one for loop.
|
| - I'm not limited to having only streams of integers, they are
|
| - My way, if I define a sync transform over a sync input, the
| whole iteration can be sync making it possible to get and use the
| result in sync functions. This is huge as otherwise you have to
| write all the code twice: once with sync iterator and for loops
| and once with async iterators and for await loops.
|
| - The problem with thrashing Promises when splitting input up
| into words goes away. With async iterators, creating two words
| means creating two promises. With stream iterators if you have
| the data available there's no need for promises at all, you just
| yield it.
|
| - Stream iterators can help you manage concurrency, which is a
| huge thing that async iterators cannot do. Async iterators can't
| do this because if they see a promise they will _always_ wait for
| it. That 's the same as saying "if there is any concurrency, it
| will always be eliminated."
| conartist6 wrote:
| There's one more interesting consequence: you rid yourself of
| the feedback problem.
|
| To see the problem let's create a stream with feedback. Lets
| say we have an assembly line that produces muffins from
| ingredients, and the recipe says that every third muffin we
| produce must be mushed up and used as an ingredient for further
| muffins. This works OK until someone adds a final stage to the
| assembly line, which puts muffins in boxes of 12. Now the line
| gets completely stuck! It can't get a muffin to use on the
| start of the line because it hasn't made a full box of muffins
| yet, and it can't make a full box of muffins because it's
| starved for ingredients after 3.
|
| If we're mandated to clump the items together we're implicitly
| assuming that there's no feedback, yet there's also no reason
| that feedback shouldn't be a first-class ability of streams.
| Joker_vD wrote:
| > Obviously I'm gonna be biased, but I'm pretty sure my version
| is also objectively superior:
|
| > - I can easily make mine from theirs
|
| That... doesn't make it superior? On the contrary, theirs can't
| be easily made out of yours, except by either returning trivial
| 1-byte chunks, or by arbitrary buffering. So their proposal is
| a superior primitive.
|
| On the whole, I/O-oriented iterators probably should return
| chunks of T, otherwise you get buffer bloat for free. The
| readv/writev were introduced for a reason, you know.
| conartist6 wrote:
| As an abstraction I would say it does make mine superior that
| it captures everything theirs can and more that theirs can't.
|
| Plus theirs involves the very concrete definition of an
| array, which might have 100 prototype methods in JS, each
| part of their API surface. I have one function in my API
| surface.
| robby_w_g wrote:
| > So their proposal is a superior primitive.
|
| This lines up with my thinking. The proposal should give us a
| building block in the form of the primitive. I would expect
| the grandparent comment's API to be provided in a library
| built on top of a language level primitive.
| conartist6 wrote:
| How would you then deal with a stream of UTF8 code points?
| They won't fit in a UInt8Array. There will be too many for
| async iterators to perform well: you'll hit the promise
| thrashing issues discussed in the blog post
| Joker_vD wrote:
| No, you'll just need to (potentially) keep the last 1-2
| bytes of the previous chunk after each iteration. Come
| on, restartable UTF-8 APIs has been around for more than
| 30 years.
| conartist6 wrote:
| But those code points were just inputs to another stream
| transformation that turns a stream of code points into a
| stream of graphemes. Rapidly your advice turns into "just
| do everything in one giant transformation" and that loses
| the benefits of streams, which are meant to be highly
| composable to create efficient, multi-step transformation
| pipelines.
| Joker_vD wrote:
| No, it doesn't turn into this. Those two bytes of
| leftovers plus a flag are kept inside the stream
| generator that transforms bytes into code points, every
| time you pull it those two bytes are used as an initial
| accumulator in the fold that takes the chunk of bytes and
| yield chunk of code points and the updated accumulator.
| You don't need to inline it all into one giant transform.
|
| Come on, it's how (mature libraries of) parser
| combinators work. The only slightly tricky part here is
| detecting leftover data in the pipeline.
| conartist6 wrote:
| To quote the article:
|
| > If you want to stream arbitrary JavaScript values, use
| async iterables directly
|
| OK, so we have to do this because code points are numbers
| larger than 8 bits, so they're arbitrary JS values and we
| have to use async iterables directly. This is where the
| amount of per-item overhead in an async iterable starts
| to strangle you because most of the actual work being
| done at that point is tearing down the call stack between
| each step of each iterator and then rebuilding it again
| so that the debugger has some kind of stack traces (if
| you're using for await of loops to consume the iterables
| that is).
| idle_zealot wrote:
| What's stopping you from implementing a stream
| transformation that reads the raw stream like a parser,
| outputting a grapheme or whatever unit you want only when
| it knows it's done reading it from the input?
| paxys wrote:
| There is no such thing as Uint8Array<T>. Uint8Array is a
| primitive for a bunch of bytes, because that is what data is in
| a stream.
|
| Adding types on top of that isn't a protocol concern but an
| application-level one.
| softfalcon wrote:
| > Adding types on top of that isn't a protocol concern but an
| application-level one.
|
| I agree with this.
|
| I have had to handle raw byte streams at lower levels for a
| lot of use-cases (usually optimization, or when developing
| libs for special purposes).
|
| It is quite helpful to have the choice of how I handle the
| raw chunks of data that get queued up and out of the network
| layer to my application.
|
| Maybe this is because I do everything from C++ to Javascript,
| but I feel like the abstractions of cleanly getting a stream
| of byte arrays is already so many steps away from actual
| network packet retrieval, serializing, and parsing that I am
| a bit baffled folks want to abstract this concern away even
| more than we already do.
|
| I get it, we all have our focuses (and they're ever growing
| in Software these days), but maybe it's okay to still see
| some of the bits and bytes in our systems?
| conartist6 wrote:
| My concern isn't with how you write your network layer. Use
| buffers in there, of course.
|
| But what if you just want to do a simple decoding transform
| to get a stream of Unicode code points from a steam of
| bytes? If your definition of a stream is that it has UInt8
| values, that simply isn't possible. And there's still gonna
| be waaay too many code points to fall back to an async
| iterator of code points.
| softfalcon wrote:
| I think we're having a completely different conversation
| now. The parent comment I originally replied has been
| edited so much that I think the context of what I was
| referring to is now gone.
|
| Also, I wasn't talking about building network layers, I
| was explicitly referring to things that use a network
| layer... That is, an application receiving streams of
| enumerable network data.
|
| I also agree with what you're saying, we don't want
| UInt8, we want bits and bytes.
|
| I'm really confused as to why the parent comment was
| edited so heavily. Oh well, that's social media for you.
| whaleofatw2022 wrote:
| Not the person originally replying, but as someone who
| avoids JS I have to ask whether the abstraction you
| provide may have additional baggage as far as
| framing/etc.
|
| Ironically, naively, I'd expect something more like a
| callback where you would specify how your input gets
| written to a buffer, but again im definitely losing a lot
| of nuance from not doing JS in a long while.
| 85392_school wrote:
| A Uint8Array can be backed by buffers other than ArrayBuffer,
| which is where the types [0] come from.
|
| [0] https://github.com/microsoft/TypeScript/blob/924810c077dd
| 410...
| paulddraper wrote:
| Your idea is flatten the UInt8Array into the stream.
|
| While I understand the logic, that's a terrible idea.
|
| * The overhead is _massive_. Now every 1KiB turns into 1024
| objects. And terrible locality.
|
| * Raw byte APIs...network, fs, etc fundamentally operate on
| byte arrays anyway.
|
| In the most respectful way possible...this idea would only be
| appealing to someone who's not used to optimizing systems for
| efficiency.
| conartist6 wrote:
| JS engines actually are optimized to make that usage pattern
| fast.
|
| Small, short-lived objects with known key ordering
| (monomorphism) are not a major cost in JS because the GC
| design is generational. The smallest, youngest generation of
| objects can be quickly collected with an incremental GC
| because the perf assumption is that most of the items in the
| youngest generation will be garbage. This allows collection
| to be optimized by first finding the live objects in the gen0
| pool, copying them out, then throwing away the old gen0 pool
| memory and replacing it with a new chunk.
| conartist6 wrote:
| It's not blazingly fast, no, but it's not as much overhead
| as people think either when they're imagining what it would
| cost to do the same thing with malloc. TC39 knew all this
| when they picked { step, done } as the API for iteration
| and they still picked it, so I'm not really introducing new
| risk but rather trusting that they knew what they were
| doing when they designed string iterators.
|
| At the moment the consensus seems to be that these language
| features haven't been worth investing much in optimizing
| because they aren't widely used in perf-critical pathways.
| So there's a chicken and egg problem, but one that gives me
| some hope that these APIs will actually get faster as their
| usage becomes more common and important, which it should if
| we adopt one of these proposed solutions to the current
| DevX problems
| softfalcon wrote:
| What happens when I send an extremely high throughput of
| data and the scheduler decides to pause garbage collection
| due to there being too many interrupts to my process
| sending network events? (a common way network data is
| handed off to an application in many linux distros)
|
| Are there any concerns that the extra array overhead will
| make the application even more vulnerable to out of memory
| errors while it holds off on GC to process the big stream
| (or multiple streams)?
|
| I am mostly curious, maybe this is not a problem for JS
| engines, but I have sometimes seen GC get paused on high
| throughput systems in GoLang, C#, and Java, which causes a
| lot of headaches.
| conartist6 wrote:
| Yeah I don't think that's generally a problem for JS
| engines because of the incremental garbage collector.
|
| If you make all your memory usage patterns possible for
| the incremental collector to collect, you won't
| experience noticeable hangups because the incremental
| collector doesn't stop the world. This was already pretty
| important for JS since full collections would (do) show
| up as hiccups in the responsiveness of the UI.
| softfalcon wrote:
| Interesting, thanks for the info, I'll do some reading on
| what you're saying. I agree, you're right about JS having
| issues with hiccups in the UI due to scheduling on a
| single process thread.
|
| Makes a lot of sense, cool that the garbage collector can
| run independently of the call stack and function
| scheduler.
| fwip wrote:
| I agree with your post, but in practice, couldn't you get
| back that efficiency by setting T = UInt8Array? That is,
| write your stream to send / receive arrays.
|
| My reference point is from a noob experience with Golang -
| where I was losing a bunch of efficiency to channel overhead
| from sending millions of small items. Sending batches of
| ~1000 instead cut that down to a negligible amount. It is a
| little less ergonomic to work with (adding a nesting level to
| your loop).
| flowerbreeze wrote:
| I think the more generic stream concept is interesting, but
| their proposal is based on different underlying assumptions.
|
| From what it looks like, they want their streams to be
| compatible with AsyncIterator so it'd fit into existing
| ecosystem of iterators.
|
| And I believe the Uint8Array is there for matching OS streams
| as they tend to move batches of bytes without having knowledge
| about the data inside. It's probably not intended as an
| entirely new concept of a stream, but something that C/C++ or
| other language that can provide functionality for JS, can do
| underneath.
|
| For example my personal pet project of a graph database written
| in C has observers/observables that are similar to the
| AsyncIterator streams (except one observable can be listened to
| by more than one observer) moving about batches of Uint8Array
| (or rather uint8_t* buffer with capacity/count), because it's
| one of the fastest and easiest thing to do in C.
|
| It'd be a lot more work to use anything other than uint8_t*
| batches for streaming data. What I mean by that, is that any
| other protocol that is aware of the type information would be
| built on top of the streams, rather than being part of the
| stream protocol itself for this reason.
| conartist6 wrote:
| Yeah it makes sense to me that the actual network socket is
| going to move data around in buffers. I'm just offering an
| abstraction over that so that you can write code that is
| wholly agnostic to how data is stored.
|
| And yes, because it's a new abstraction the compat story is
| interesting. We can easily wrap any source so we'll have
| loads of working sources. The fight will be getting official
| data sinks that support a new kind of stream
| pgt wrote:
| This is similar to how Clojure transducers are implemented:
| "give me the next thing plz." -
| https://clojure.org/reference/transducers
| soulofmischief wrote:
| In the language I've been working on for a couple months,
| Eidos, streams are achieved through iterators as well. It's
| dead simple. And lazy for loops are iterators, and there is
| piping syntax. This means you can do this (REPL code):
| >> fn double(iter: $iterator<i32>) { return *for x in
| iter { $yield( x * 2 )} } >> fn add_ten(iter:
| $iterator<i32>) { return *for x in iter { $yield( x +
| 10 )} } >> fn print_all(iter: $iterator<i32>)
| { for x in iter { $print( x )} } >>
| const source = *for x in [1, 2, 3] { $yield( x )} >>
| source |> double |> add_ten |> print_all 12 14
| 16
|
| You get backpressure for free, and the compiler can make
| intelligent decisions, such as automatic inlining, unrolling,
| kernel fusing, etc. depending on the type of iterators you're
| working with.
| lucideer wrote:
| Other angles of critique & consideration already covered well
| by sibling commenters. One extra consideration (unrelated to
| streams, more general) is the API design & dev UX/DX:
| type Stream<T> = { next(): { done, value: T } |
| Promise<{ done, value: T }> }
|
| the above can effectively be discussed as a combination of the
| following: type Stream<T> = { next(): {
| done, value: T } } type Stream<T> = {
| next(): Promise<{ done, value: T }> }
|
| You've covered the justifications for the 2nd signature, but
| it's a messy API. Specifically:
|
| > _My way, if I define a sync transform over a sync input, the
| whole iteration can be sync making it possible to get and use
| the result in sync functions. This is huge as otherwise you
| have to write all the code twice: once with sync iterator and
| for loops and once with async iterators and for await loops._
|
| Writing all the code twice is _cleaner_ in every implementation
| scenario I can envisage. It 's very rare I want generalised
| flexibility on an API call - that leads to a lot of confusion &
| ambiguity when reading/reviewing code, & also when adding
| to/editing code. Any repetitiveness in handling both use-cases
| (separately) can easily be handled with well thought-out
| composition.
| conartist6 wrote:
| How is it cleaner? I used to actually do that. I wrote
| everything twice. I even built fancy tools to help me write
| everything twice.
|
| But the bigger problem here is that sync and async aren't
| enough. You almost need to write everything three times:
| sync, async, and async-batched. And that async-batched code
| is gonna be _gnarly_ and different from the other two copies
| and writing it in the first place and keeping it in sync is
| gonna give you headaches.
|
| To see how it played out for me take a look at the difference
| between:
|
| https://github.com/iter-
| tools/regex/blob/a35a0259bf288ccece2...
| https://github.com/iter-
| tools/regex/blob/a35a0259bf288ccece2...
| https://github.com/iter-
| tools/regex/blob/a35a0259bf288ccece2...
| hinkley wrote:
| I did a microbenchmark recently and found that on node 24,
| awaiting a sync function is about 90 times slower than just
| calling it. If the function is trivial, which can often be the
| case.
|
| If you go back a few versions, that number goes up to around
| 105x. I don't recall now if I tested back to 14. There was an
| optimization to async handling in 16 that I recall breaking a
| few tests that depended on nextTick() behavior that stopped
| happening, such that the setup and execution steps started
| firing in the wrong order, due to a mock returning a number
| instead of a Promise.
|
| I wonder if I still have that code somewhere...
| conartist6 wrote:
| Here is my test harness and results:
| https://github.com/conartist6/async-perf
| rrr_oh_man wrote:
| > I did a microbenchmark recently and found that on node 24,
| awaiting a sync function is about 90 times slower than just
| calling it. If the function is trivial, which can often be
| the case.
|
| I dabble in JS and... what?! Any idea why?
| hinkley wrote:
| Any await runs the logic that attempts to release the main
| message pump to check for other tasks or incoming IO
| events. And it looks like that takes around 90 instructions
| to loop back around to running the next line of the code,
| when the process is running nothing else.
|
| If you're doing real work, 90 instructions ain't much but
| it's not free either. If you've got an async accumulator
| (eg, otel, Prometheus) that could be a cost you care about.
| conartist6 wrote:
| How did you come up with 90? Can you shed any might on
| the difference between the cost of promise resolution and
| the cost of await? Is there any cost component with how
| deep in the call stack you are when an await happens?
| hinkley wrote:
| Essentially for loop of 10k iterations comparing `fn()`
| versus `await fn()` fed into a microbenchmark tool, with
| some fiddling to detect if elimination was happening or
| ordering was changing things.
|
| I was bumping into PRs trying to eliminate awaits in long
| loops and thinking surely the overhead can't be so high
| to warrant doing this, especially after node ~16. I was
| wrong.
| hinkley wrote:
| I think the context that some other responders are missing is
| that in some functional languages, like Elixir, streams and
| iterators are used idiomatically to do staged transforms of
| data without necessitating accumulation at each step.
|
| They are those languages versions of goroutines, and JavaScript
| doesn't have one. Generators sort of, but people don't use them
| much, and they don't compose them with each other.
|
| So if we are going to fix Streams, an implementation that is
| tuned only for IO-bound workflows at the expense of transform
| workflows would be a lost opportunity.
| amelius wrote:
| How do you send multiple sub-streams in parallel?
| murmansk wrote:
| For gods sake, finally, somebody have said this!
| ralusek wrote:
| I tinkered with an alternative to stream interfaces:
|
| https://github.com/ralusek/streamie
|
| allows you to do things like infiniteRecords
| .map(item => doSomeAsyncThing(item), { concurrency: 5 });
|
| And then because I found that I often want to switch between
| batching items vs dealing with single items:
| infiniteRecords .map(item =>
| doSomeAsyncSingularThing(item), { concurrency: 5 })
| .map(groupOf10 => doSomeBatchThing(groupsOf10), { batchSize: 10
| }) // Can flatten back to single items .map(item
| => backToSingleItem(item), { flatten: true });
| z3t4 wrote:
| I like Node.JS streams. It's very satisfying to rent a 250MB
| memory machine and let it process GB's of data using streams.
| bikeshaving wrote:
| A long time ago, I wrote an abstraction called a Repeater.
| Essentially, the idea behind it is, what would the Promise
| constructor look like if it was translated to async iterables.
| import { Repeater } from "@repeaterjs/repeater";
| const keys = new Repeater(async (push, stop) => { const
| listener = (ev) => { if (ev.key === "Escape") {
| stop(); } else { push(ev.key); }
| }; window.addEventListener("keyup", listener);
| await stop; window.removeEventListener("keyup",
| listener); }); const konami = ["ArrowUp", "ArrowUp",
| "ArrowDown", "ArrowDown", "ArrowLeft", "ArrowRight", "ArrowLeft",
| "ArrowRight", "b", "a"]; (async function() { let i
| = 0; for await (const key of keys) { if (key
| === konami[i]) { i++; } else {
| i = 0; } if (i >= konami.length) {
| console.log("KONAMI!!!"); break; // removes the keyup
| listener } } })();
|
| https://github.com/repeaterjs/repeater
|
| It's one of those abstractions that's feature complete and
| stable, and looking at NPM it's apparently getting 6.5mil+
| downloads a week for some reason.
|
| Lately I've just taken the opposite view of the author, which is
| that we should just use streams, especially with how embedded
| they are in the `fetch` proposals and whatever. But the tee
| critique is devastating, so maybe the author is right. It's
| exciting to see people are still thinking about this. I do think
| async iterables as the default abstraction is the way to go.
| boilerupnc wrote:
| Off topic - But just wanna say - Love the cheat code! 30 Lives
| added :-) Nostalgia runs deep with that code. So deep - in
| fact, that I sign many of my emails off with "Sent by hitting
| Up, Up, Down, Down, Left, Right, Left, Right, B, A"
| sfink wrote:
| Off topic to the off topic, but that logic doesn't look
| right. It seems like if up is pressed, you might need to
| reset i to 1 or 2, not 0.
| bikeshaving wrote:
| Not accepting PRs for this. I think the logic is sound
| (Easter Eggs should be difficult to trigger).
| pcthrowaway wrote:
| In the repeater callback, you're both calling the stop argument
| and awaiting it. Is it somehow both a function and a promise?
| Is this possible in JS?
|
| edit: I found where stop is created[1]. I can't say I've seen
| this pattern before, and the traditionalist in me wants to
| dislike the API for contradicting conventions, but I'm
| wondering if this was designed carefully for ergonomic benefits
| that outweigh the cost of violating conventions. Or if this was
| just toy code to try out new patterns, which is totally legit
| also
|
| [1]:
| https://github.com/repeaterjs/repeater/blob/638a53f2729f5197...
| bikeshaving wrote:
| Yes, the callable promise abstraction is just a bit of
| effort: let resolveRef; const promise =
| new Promise((res) => { resolveRef = res; });
| const callback = (data) => { // Do work...
| resolveRef(data); // This "triggers" the await };
| Object.assign(callback, promise);
|
| There's a real performance cost to awaiting a fake Promise
| though, like `await regularPromise` bypasses the actual
| thenable stuff.
| paulddraper wrote:
| Just use AsyncIterator<UIntArray>.
|
| The objection is
|
| > The Web streams spec requires promise creation at numerous
| points -- often in hot paths and often invisible to users. Each
| read() call doesn't just return a promise; internally, the
| implementation creates additional promises for queue management,
| pull() coordination, and backpressure signaling.
|
| But that's 95% manageable by altering buffer sizes.
|
| And as for that last 5%....what are you doing with JS to begin
| with?
| tracker1 wrote:
| One minor niggle on freeing resources... I'm hoping it becomes
| more popular with libraries, but there's using/await using with
| disppse/disposeAsync which works similarly to C#'s use of using.
|
| I'm working on a db driver that uses it by convention as part of
| connection/pool usage cleanup.
| halfmatthalfcat wrote:
| The Observables spec should just get merged and implemented.
|
| https://github.com/tc39/proposal-observable
| bakkoting wrote:
| Observables has moved to WHATWG [1] and been implemented in
| Chrome, although I don't know if the other browsers have
| expressed any interest (and there's still some issues [2] to be
| worked through).
|
| But Observables really do not solve the problems being talked
| about in this post.
|
| [1] https://github.com/WICG/observable [2]
| https://github.com/WICG/observable/issues/216
| spankalee wrote:
| Async iterables aren't necessarily a great solution either
| because of the exact same promise and stack switching overhead -
| it can be huge compared to sync iterables.
|
| If you're dealing with small objects at the production side, like
| individual tag names, attributes, bindings, etc. during SSR., the
| natural thing to do is to just write() each string. But then you
| see that performance is terrible compared to sync iterables, and
| you face a choice: 1. Buffer to produce larger
| chunks and less stack switching. This is the exact same thing you
| need to do with Streams. or 2. Use sync iterables and
| forgo being able to support async components.
|
| The article proposes sync streams to get around this some, but
| the problem is that in any traversal of data where some of the
| data might trigger an async operation, you don't necessarily know
| ahead of time if you need a sync or async stream or not. It's
| when you hit an async component that you need it. What you really
| want is a way for only the data that needs it to be async.
|
| We faced this problem in Lit-SSR and our solution was to move to
| sync iterables that can contain thunks. If the producer needs to
| do something async it sends a thunk, and if the consumer receives
| a thunk it _must_ call and await the thunk before getting the
| next value. If the consumer doesn 't even support async values
| (like in a sync renderToString() context) then it can throw if it
| receives one.
|
| This produced a 12-18x speedup in SSR benchmarks over components
| extracted from a real-world website.
|
| I don't think a Streams API could adopt such a fragile contract
| (ie, you call next() too soon it _will_ break), but having some
| kind of way where a consumer can pull as many values as possible
| in one microtask and then await only if an async value is
| encountered would be really valuable, IMO. Something like
| `write()` and `writeAsync()`.
|
| The sad thing here is that generators are really the right shape
| for a lot of these streaming APIs that work over tree-like data,
| but generators are far too slow.
| jauntywundrkind wrote:
| I liked conartist6's proposal, type Stream<T> =
| { next(): { done, value: T } | Promise<{ done, value: T
| }> }
|
| Where T=Uint8Array. Sync where possible, async where not.
|
| Engineers had a collective freak out panic back in 2013 over
| _Do not unleash Zalgo_ , a worry about using callbacks with
| different activation patterns. Theres wisdom there, for
| callbacks especially; it's confusing if sometime the callback
| fires right away, sometimes is in fact async.
| https://blog.izs.me/2013/08/designing-apis-for-asynchrony/
|
| And this sort of narrow specific control has been with us
| since. It's generally not cool to use MaybeAsync<T> = T |
| Promise<T>, for similar "it's better to be uniform" reasons.
| We've been so afraid of Zalgo for so long now.
|
| That fear just seems so overblown and it feels like it hurts us
| so much that we can't do nice fast things. And go async when we
| need to.
|
| Regarding the pulling multiple, it really depends doesn't it?
| It wouldn't be hard to make a utility function that lets you
| pull as many as you want queueing deferrables, allowing one at
| a time to flow. But I suspect at least some stream sources
| would be just fine yielding multiple results without waiting.
| They can internally wait for the previous promise, use that as
| a cursor.
|
| I wasn't aware that generators were far too slow. It feels like
| we are using the main bit of the generator interface here,
| which is good enough.
| conartist6 wrote:
| Yeah I think people took away "It's better to be uniform"
| since they were trying to block out the memory of much-feared
| Zalgo, but if you read the article carefully it says in big
| letters "Avoid Synthetic Deferrals" then goes on to advocate
| for patterns exactly like MaybeAsync to be used "if the
| result is usually available right now, and performance
| matters a lot".
|
| I was so sick of being slapped around by LJHarb who claimed
| to me again and again that TC39 was honoring the Zalgo post
| (by slapping synthetic deferrals on everything) that I
| actually got Isaacs to join the forum and set him straight:
| https://es.discourse.group/t/for-await-of/2452/5
| spankalee wrote:
| That's an amazing thread, thanks for posting it! I've
| wanted `for await?()` for exactly these situations.
|
| I feel like my deep dives into iterator performance are
| somewhat wasted because I might have made my project
| faster, but it's borderline dark magic and doesn't scale to
| the rest of the ecosystem because the language is broken.
| conartist6 wrote:
| Yeah that problem you have is pretty much what I'm offering a
| solution to. It's the same thing you're already doing but more
| robust.
|
| Also I'm curious why you say that generators are far too slow.
| Were you using async generators perhaps? Here's what I cooked
| up using sync generators: https://github.com/bablr-lang/stream-
| iterator/blob/trunk/lib...
|
| This is the magic bit: return
| step.value.then((value) => { return this.next(value);
| });
| conartist6 wrote:
| You know now that I look at it I do think I need to change
| this code to defend better against multiple eager calls to
| `next()` when one of them returns a promise. With async
| generators there's a queue built in but since I'm using sync
| generators I need to build that defense myself before this
| solution is sound in the face of next();next(). That
| shouldn't be too hard though.
| matheus-rr wrote:
| The practical pain with Web Streams in Node.js is that they feel
| like they were designed for the browser use case first and
| backported to the server. Any time I need to process large files
| or pipe data between services, I end up fighting with the API
| instead of just getting work done.
|
| The async iterable approach makes so much more sense because it
| composes naturally with for-await-of and plays well with the rest
| of the async/await ecosystem. The current Web Streams API has
| this weird impedance mismatch where you end up wrapping
| everything in transform streams just to apply a simple operation.
|
| Node's original stream implementation had problems too, but at
| least `.pipe()` was intuitive. You could chain operations and
| reason about backpressure without reading a spec. The Web Streams
| spec feels like it was written by the kind of person who thinks
| the solution to a complex problem is always more abstraction.
| zarzavat wrote:
| It's news to me that anyone actually uses the web streams in
| node. I thought they were just for interoperability, for code
| that needs to run on both client and server.
| apitman wrote:
| You need to use them for things like Cloudflare and Denos
| HTTP servers, which is actually a fairly common (and nice)
| pattern:
|
| https://blog.val.town/blog/the-api-we-forgot-to-name/
| cogman10 wrote:
| Seems pretty similar to the design of OKIO in java [1]. With
| pretty similar goals ultimately. Here's a presentation on the
| internal details and design decisions. [2]
|
| [1] https://github.com/square/okio
|
| [2] https://www.youtube.com/watch?v=Du7YXPAV1M8
| notnullorvoid wrote:
| There's a lot I like about this API, mainly the pull-based
| iterator approach. I don't really see what the value of the sync
| APIs are though. What's the difference of just using iterators
| directly for sync streams?
| jonkoops wrote:
| It avoids the overhead of Promises, so I can imagine that this
| would be quite useful if you know that blocking the thread is
| fine for a little while (e.g. in a worker).
| notnullorvoid wrote:
| I mean the APIs like `Stream.pullSync` you could do that with
| a regular (non-async) iterator/generator.
| socalgal2 wrote:
| Promises should not be a big overhead. If they are, that seems
| like a bug in JS engines.
|
| At a native level (C++/rust), a Promise is just a closure added
| to a list of callbacks for the event loop. Yes, if you did 1 per
| streamed byte then it would be huge but if you're doing 1 promise
| per megabyte, (1000 per gig), it really shouldn't add up 1% of
| perf.
| conartist6 wrote:
| I'm fairly sure it's not Promises that are actually the heavy
| part but the `await` keyword as used in the `for await` loop.
| That's because await tries to preserve the call stack for
| debugging, making it a relatively high-level expensive
| construct from a perf perspective where a promise is a
| relatively low-level cheap one.
|
| So if you're going to flatten everything into one stream then
| you can't have a for loop implementation that defensively
| awaits on every step, or else it'll be slooooooooow. That's my
| proposal for the change to the language is a syntax like
| for await? (value of stream) { }
|
| which would only do the expensive high-level await when the
| underlying protocol forced it to by returning a promise-valued
| step.
| adamnemecek wrote:
| It might be a good idea to look into the research on streams as
| coalgebras, there is quite a bit, for example here
| https://cs.ru.nl/~jrot/CTC20/.
|
| Coalgebras might seem too academic but so were monads at some
| point and now they are everywhere.
| rhodey wrote:
| the pull-stream module and its ecosystem is relevant here
|
| the idea is basically just use functions. no classes and very
| little statefulness
|
| https://www.npmjs.com/package/pull-stream
| nottorp wrote:
| Well, it's also possible to replace JavaScript with a better
| language, it's just too late for it...
| bennettpompi1 wrote:
| I really enjoyed reading this article however I can't help but
| feeling that if you need anything described within it probably
| shouldn't be writing JS in the first place
| szmarczak wrote:
| > This pattern has caused connection pool exhaustion in Node.js
| applications using undici (the fetch() implementation built into
| Node.js), and similar issues have appeared in other runtimes.
|
| That's an inherent flaw of garbage collected languages. Requiring
| to explicitly close a resource feels like writing C. Otherwise
| you have a memory leak or resource exhaustion, because the
| garbage collector may or may not free the resource. Even C++ is
| better at this, because it does reference counting instead.
| etler wrote:
| There are many use cases where having a value stream is very
| useful. I do agree having a separate simpler byte only stream
| would make sense though. I think the current capabilities of web
| streams should be kept and an IOStream could be added for
| optimizing byte streams.
|
| Ideally splitting out the use cases would allow both
| implementations to be simpler, but that ship has probably sailed.
| steve_adams_86 wrote:
| I ran into a performance issues a few months ago where native
| streams were behaving terribly, and it seemed to be due to bad
| back-pressure implementation.
|
| I tried several implementations, tweaked settings, but ultimately
| couldn't get around it. In some cases I had bizarre drops in
| activity when the consumer was below capacity.
|
| I have a feeling it could have been related to the other issue
| they mention, which is the cost of using promises. My streams
| were initiating HEAPS of promises. The cost is immense.
|
| I eventually had to implement some complex logic to accomplish
| batching to reduce the number of promises, then figure out some
| clever concurrency strategies to manage backpressure more
| manually. It worked well.
|
| Once I was happy with what I had, I ported it from Deno to Go and
| the result was so stunningly different. The performance
| improvement was several orders of magnitude.
|
| I also built my custom/native solution using the Effect library,
| and although people claim it's inefficient and slow, it out-
| performed mine by something like 15% off the shelf, with no fine-
| tuning or clever ideas. I wished I'd used it from the start.
|
| The difference is likely in that it uses virtual threads (fibres)
| rather than promises for concurrency.
___________________________________________________________________
(page generated 2026-02-27 23:00 UTC)