[HN Gopher] Ode to a Streaming ByteString
___________________________________________________________________
Ode to a Streaming ByteString
Author : jsnell
Score : 51 points
Date : 2021-10-26 22:14 UTC (1 days ago)
(HTM) web link (blog.sumtypeofway.com)
(TXT) w3m dump (blog.sumtypeofway.com)
| lmm wrote:
| So isn't ByteStream just Stream (Of ByteString)? And isn't all
| this just iteratees?
| Arnavion wrote:
| Minor nit:
|
| >ByteString represents a byte buffer and its associated length;
| in this it is similar to Go's []byte or Rust's &[u8] (though it
| has one fewer datum to track, as the Rust and Go types offer
| mutable access to the associated byte buffer and thus must keep
| track of its total capacity).
|
| Rust's &[u8] and &mut [u8] don't track capacity. Mutable access
| only allows changing the data in the slice, not changing its
| length. It couldn't even if it wanted to, since it has no idea of
| what the underlying storage is. It's only Go's slices that do
| double-duty as a ranged view into an existing allocation but also
| with the freedom to become the owner of a new allocation.
| bennofs wrote:
| How does rust do bounds checking for slice accesses if the
| length is not stored?
| steveklabnik wrote:
| Length and capacity are two different things; a slice is a
| pointer and a length, but a Vec<T> (and Go's slices) are a
| pointer, a length, and a capacity.
|
| That Rust and Go both have "slices" that are slightly
| different is, unfortunate, but that's just how it goes
| sometimes.
| wyager wrote:
| Having a bunch of string types is something people like to rag on
| Haskell for, but I think the reality is that strings _are_ that
| complicated and most languages kind of sweep the complication
| under the rug, while Haskell chooses to accept it. Thankfully the
| practical downside is quite manageable; the most annoying thing
| you'll occasionally have to do is call toStrict /fromStrict
| across library boundaries.
| assbuttbuttass wrote:
| Using [Char] as String was mistake. If Haskell used Data.Text
| everywhere strings would be fine
| throwaway81523 wrote:
| If you mean a Text-like representation (basically ropes of
| codepoints) then sure, but Data.Text has a bunch of i/o
| including lazy i/o stirred in with it. Lazy i/o in particular
| is part of the still-lingering schizophrenia in Haskell about
| how to do i/o cleanly and safely. The current Streaming
| library is yet another try at that. I hadn't seen it before
| and it looks interesting, though I'll have to examine it more
| closely, and it still seems to be Iteratees deep down.
|
| I had the impression a few years ago that Pipes was the most
| cleanly designed of these schemes, but haven't been keeping
| track.
| wyager wrote:
| Streaming is definitely the most cleanly designed today and
| also in my experience the fastest. It's a shockingly simple
| design. Main downside is that it's unidirectional, but I
| don't think anyone really cares. Bidirectional pipes are
| tough to use.
| whateveracct wrote:
| True - going from Text to [Char] costs far less than the
| other direction all told.
|
| But [Char] is quite nice for certain uses.
| arianvanp wrote:
| The article only seems to concern handling effectful input. But i
| can't really imagine what the type for writing to stdout would
| look like.
|
| Would it be something simple as:
|
| writeStdout :: ByteStream IO a -> IO a
|
| ?
| wyager wrote:
| Sure, that could work, or something like
|
| mapM_ :: (ByteString -> m a) -> ByteStream m b -> m b
|
| And then you just have
|
| writeStreamToStdout = mapM_ writeBytesToStdout
| WJW wrote:
| Maybe I misunderstand what you mean, but why not just:
| writeStdout :: ByteStream a -> IO ()
|
| Maybe you could even leave out the `a` type since arguably a
| `ByteStream` should only contain bytes anyway. When writing to
| stdout you don't really need a return value usually, so that
| could just be (). The IO monad can encapsulate any amount of
| side effects in the same function, so you could fit your
| additional side effects in besides writing to stdout.
| defanor wrote:
| I find all those streaming I/O libraries (and lazy bytestreams as
| well) to be more neat-looking than useful: they can produce
| slightly cleaner code when used for basic tasks (such as those in
| examples), are fun to play with, and generally seem nice at
| first. But once you have a bunch of dependencies with their own
| (byte)string/text types, errors/exceptions, and ways to do I/O,
| all of which should be combined and work together, those become
| rather annoying. Additionally, it becomes easier to shoot
| yourself in the foot (which is mentioned in the alternative
| title, but doesn't seem to be expanded on much) -- say, by
| closing a Handle or even merely not draining the input before
| proceeding with other I/O, if the outside system expects you to.
|
| At the same time, it's not a big deal to handle potentially
| infinite streams with strict I/O and explicit chunks.
|
| I imagine it is possible to build projects around such libraries,
| possibly wrapping all the other I/O into it. Similarly to how in
| principle it's possible to unify all the error-yielding
| functions, and those working with text and/or byte streams. But I
| guess it doesn't happen often.
| jdonaldson wrote:
| There's some completely different programming patterns that are
| useful when dealing with streams. Streams really come into
| their own within DAGs. DAGs are to streams what module
| structures are to programs. You organize streams within a DAG,
| and can manage different use cases and goals with different
| branches. Exceptions can be handled flexibly if they are turned
| into data for the stream. Same goes for configuration.
| vmchale wrote:
| > it becomes easier to shoot yourself in the foot (which is
| mentioned in the alternative title, but doesn't seem to be
| expanded on much) -- say, by closing a Handle
|
| lazy I/O (such as file reads) is suspect and even bad, but
| laziness + IO (or something like it) is a great way to stream
| from an API: http://blog.vmchale.com/article/lazy-io
|
| Makes it possible to transcode for one. Basically makes
| streaming "compose" nicely - bzip2 and lzip otherwise have very
| different ways of streaming! And one would have to line a lot
| of things up.
| nnamtr wrote:
| I once wrote a brainfuck interpreter in Haskell which executes
| even incomplete programs as far as possible. It was really easy
| thanks to streaming ByteStrings and lazyness in general.
___________________________________________________________________
(page generated 2021-10-27 23:02 UTC)