[HN Gopher] Understanding Concurrency, Parallelism and JavaScript
___________________________________________________________________
Understanding Concurrency, Parallelism and JavaScript
Author : kugurerdem
Score : 95 points
Date : 2024-09-09 12:42 UTC (4 days ago)
(HTM) web link (www.rugu.dev)
(TXT) w3m dump (www.rugu.dev)
| necovek wrote:
| The one missed distinction is that concurrent tasks _can_ be
| executing in parallel, it just doesn 't imply they are or aren't.
|
| Basically, all parallel tasks are also concurrent, but there are
| concurrent tasks which are not executed in parallel.
| yazzku wrote:
| That is how I learned it at uni too. Unfortunately, I have
| found these definitions are all over the place in some
| literature.
| austin-cheney wrote:
| Think about it not in terms of timing but in terms of
| instruction. Parallelism requires that different instructions
| can occur with resource independence. In JavaScript that
| requires something like web workers or Node clusters.
| Concurrency merely means one task does not block another,
| which is less restrictive and more commonly available through
| the event loop.
| jerf wrote:
| Pragmatically it doesn't feel like there's a lot of
| difference to humans, because even single-threaded concurrent
| code _feels_ like it 's parallel because computers are just
| so, so much faster than humans that switching rapidly _feels_
| parallel. And you often are well advised to treat it as
| parallel anyhow, at least to some extent, for safety 's sake.
| That is, you may be able to get away with less "locking" in
| concurrent-but-not-parallel code, but you still can't write
| it exactly the same as conventionally single-threaded code or
| you will still get into a lot of trouble very quickly.
|
| So pragmatically I don't find a lot of value in
| distinguishing between "concurrent" and "parallel". Some. But
| not a lot.
|
| There is a difference for sure. It just isn't useful in
| proportion to the way people like to jump up and correct
| people about the differences.
| derefr wrote:
| To sort through the "dialect of jargon" that a piece of
| educational CS/SWEng writing is using, it helps to put it in
| the context of its publication. Consider both _when_ it was
| published, and _what kind of work_ the author of the writing
| does.
|
| Why "when it was published"?
|
| Well, mainframes were single-core until the 70s, and then
| multi-core (really, massively multi-socket NUMA) thereafter.
| PCs were single-core until the late 90s, and then multi-core
| thereafter. Thus:
|
| * Anyone writing about "concurrency" in some old Comp Sci
| paper, is probably thinking about "concurrency" as it
| pertains to their local single-core Minix mainframe, or on
| their single-core Sparc/NeXT/SGI university workstation --
| and so is inherently thinking and talking about some form of
| cooperative or pre-emptive multi-tasking through context-
| switching on a single core, with or without hardware-assisted
| process address-space isolation.
|
| * Anyone writing about "concurrency" in some old "industry-
| sponsored" Software Engineering or (especially) Operational
| Research paper, was likely working with the early parallel
| batch-processing mainframes, or perhaps with HPC clusters --
| and so is much more loose/sloppy with their definitions. In
| their mental model, there is no context-switching -- there is
| just a cluster-level workload scheduler (itself bound to a
| core) which assigns workloads to cores; where these workloads
| are essentially single-threaded or shared-nothing-multi-
| threaded virtual machines, which own the cores they run on
| until they finish. (Actually very similar to writing CUDA
| code for a GPU!) To them, the only kind of concurrency that
| exists _is_ parallelism, so they just use the words
| interchangeably.
|
| And why "what kind of work the author does"?
|
| Well, anyone writing about "concurrency" _today_ , is writing
| in a context where everything -- even the tiniest little
| microcontrollers -- have both multiple cores _and_ the
| inherent hardware capability to do pre-emptive multitasking
| (if not perhaps the memory for doing so to make any sense --
| unless your "tasks" can be measured in kilobytes); and yet
| everything does it in a slightly different way. Which in turn
| means that:
|
| * If such a person is writing _product_ software, not in
| control of the deploy environment for their software -- then
| to them, "concurrency" and "parallelism" both just mean
| "using the multi-threading abstractions provided by the
| runtime, where these might become OS threads or green-
| threads, might pin their schedulers to a core during CPU work
| or not, might yield the core during IO or not, who knows."
| None of these things can be _guaranteed_ -- even core count
| can 't be guaranteed -- so their definition of what
| "concurrency" or even "parallelism" will _do_ for them, has
| to be rather weak. To these people, "parallelism" is "nice
| if you can get it" -- but not something they think about
| much, as they have to write code under the assumption that
| the software will inevitably get stuck running on a single
| core (e.g. on a heavily-overloaded system) at some point; and
| they must ensure it won't deadlock or livelock under those
| conditions.
|
| * Meanwhile, if such a person comes from a background of
| writing SaaS software (i.e. software that runs in a _big_
| knowable environment), then anything they say about
| concurrency / parallelism is likely founded on the
| assumption of having large distributed clusters of big
| highly-multicore servers, where the chief concern isn't
| actually in achieving higher throughput through parallelism
| (as that part is "easy"), but in resolving distributed data
| races through write-linearization via artificial concurrency-
| bottlenecking abstractions like channels, message queues, or
| actors that hold their own linear inboxes. For these types,
| "parallelism" puts them in mind of the "multi-threaded with
| shared state behind semaphores" model that they want to avoid
| at all costs to keep their software scalable and ensure
| distributed fault-tolerance. So this type prefers to talk
| about designing architectures made of little actors that are
| individually intentionally concurrent-but-not-parallel; and
| then hand-wavingly introducing shared-nothing instances or
| pools of these trees of little actors, that can live and move
| within greater parallel distributed clusters.
|
| * And if such a person comes from a background of writing
| embedded software (i.e. software that runs in a _small_
| knowable environment), then their assumption will likely be
| founded on a concern for achieving realtime dataflow
| semantics for at least some parts of the system -- requiring
| _some_ , but not all, of the cores to sit there bound to
| particular tasks and spin-waiting if they finish their
| processing step early; while other cores are free to be
| "application cores", executing arbitrarily-long instruction
| sequences. To these people, "concurrency" is mostly the
| frustrating low-level process of handing off data between the
| real-time and non-realtime worlds, using lockless
| abstractions like shared ring buffers; and "parallelism" is
| mostly just getting the single most expensive part of the
| application to schedule a pool of almost-identical expensive
| operations onto a pool of specialized identical cores. (This
| is the design perspective that made the PS3's Cell
| architecture seem like a good idea.)
| weinzierl wrote:
| I like to use _" interleaved concurrency_" for the later
| category.
| lioeters wrote:
| That reminds me of the JavaScript interpreter written in
| JavaScript (which is used for Scratch, I think) - that
| supports stepping through multiple instances of the
| interpreter to achieve a kind of interleaved concurrency.
|
| https://neil.fraser.name/software/JS-
| Interpreter/docs.html#t...
|
| > JavaScript is single-threaded, but the JS-Interpreter
| allows one to run multiple threads at the same time. Creating
| two or more completely independent threads that run
| separately from each other is trivial: just create two or
| more instances of the Interpreter, each with its own code,
| and alternate calling each interpreter's step function. They
| may communicate indirectly with each other through any
| external APIs that are provided.
|
| > A more complex case is where two or more threads should
| share the same global scope..
| duped wrote:
| If you prefer to learn by video, here's an excellent talk on the
| same subject by Rob Pike that I link all the time to people
|
| https://www.youtube.com/watch?v=oV9rvDllKEg
| donatj wrote:
| I had posted this as well before realizing you had beaten me to
| it!
|
| For the uninitiated, this is a seminal and oft cited Go talk
| "Concurrency is not Parallelism" by Rob Pike - well worth your
| time for anyone, even outside the Go community.
|
| I'd encourage anyone who finds themselves in the comments here
| to check it out, it expands on the content of the article
| beautifully.
|
| Another link, with some added context
|
| https://go.dev/blog/waza-talk
|
| As well as the slides for those who may not want to watch the
| video
|
| https://go.dev/talks/2012/waza.slide#1
| kitd wrote:
| Good video.
|
| (My paraphrase of) Rob Pike's tldr:
|
| Concurrency is a characteristic of the design that allows
| multiple tasks to execute, coordinate and be scaled.
|
| Parallelism is a characteristic of the runtime that allows
| _simultaneous execution_ of those tasks.
| rdtsc wrote:
| I like to think of them as different levels. Concurrency is at a
| higher abstraction level: steps that can execute without needing
| to wait on each other. Parallelism is a bit lower and reflects
| the ability to actually execute the steps at the same time.
|
| Sometimes you can have concurrent units like multiple threads,
| but a single CPU, so they won't execute in parallel. In an
| environment with multiple CPU they might execute in parallel.
| CalRobert wrote:
| I just learned something! I realize now I was talking about
| parallelism in a recent interview question about concurrency. Oh
| well.
| bugtodiffer wrote:
| Learn Go and you will understand concurrency
| bryanrasmussen wrote:
| thinking about this - is there a term for tasks which are
| partially parallel, that is to say X starts at 0.1 and ends at 1
| and Y starts at 0.2 and ends at 0.9 - X and Y are not parallel,
| but they are something that I'm not sure what the technical term
| for is. (this is assuming they are not executed concurrently
| either of course)
| meiraleal wrote:
| If there are 2 processes/threads, they are parallel, if there
| is just one it is concurrent. For example, the event loop in JS
| is concurrent, offloading computation to workers is parallel.
| bryanrasmussen wrote:
| sure but parallel comes anyway from the English word,
| inheriting the meaning "occurring or existing at the same
| time or in a similar way; corresponding.", 2 processes that
| exist in the same time all time are, to coin a phrase,
| classically parallel (by the original English usage), but two
| processes where they exist only part of the time at the same
| time are not parallel in the original meaning although they
| may be for CS - but the question is if there is a special
| technical term for this condition that anyone knows of?
| duped wrote:
| > 2 processes that exist in the same time all time are, to
| coin a phrase, classically parallel
|
| This is not physically realizable.
|
| > but two processes where they exist only part of the time
| at the same time are not parallel in the original meaning
|
| It doesn't, both formally and colloquially one would still
| refer to them as "in parallel."
|
| ---
|
| In a very loose way you're talking about structured vs
| unstructured parallelism but it has nothing to do with
| time, and you can have parallel processes that start at the
| same point but do not join at the same point and it's still
| considered "structured."
|
| Structured parallelism is where the distinction of where a
| process starts/joins is meaningful semantically, but there
| is not really a distinction between "these N things live
| for the same amount of time" and "N things have different
| lifetimes" in terms of vocabulary. They're all still
| parallel or concurrent processes.
| dragontamer wrote:
| Concurrency often is about running your I/O routines in parallel,
| achieving higher bandwidth. For example, one computer handling 50
| concurrent HTTP requests simultaneously.
|
| No single HTTP request uses all the CPU power or even your
| Ethernet bandwidth. The bulk of your waiting is latency issues.
| So while one task is waiting on Ethernet responses under the
| hood, the system should do something else.
|
| Hard Drives are another: you can have random I/O bandwidths of
| 5MB/s or so, but every request always takes 4ms on the average
| for a 7200 RPM drive (aka: 120 rotations per second, or about 8
| miliseconds for a complete rotation. So 4ms on the average for
| any request to complete).
|
| So while waiting for the HDD to respond, your OS can schedule
| other reads or writes for the head to move to which improves
| average performance (ex: if 8 requests all are within the path of
| the head, you'll still wait 4ms on the average, but maybe each
| will be read per 1ms).
|
| ----------
|
| Parallelism is often about CPU limited situations where you use a
| 2nd CPU (today called a core). For example, if one CPU core is
| too slow, you can use a 2nd, or 8 or even 128 cores
| simultaneously.
|
| ------------
|
| Hyperthreads is the CPU designer (Intel and AMD) that the above
| concurrency technique can apply to modern RAM because a single
| RAM read is like 50ns or 200 clock ticks. Any RAM-latency problem
| (ex: linked list traversals) would benefit from the CPU core
| doing something else while waiting for the RAM latency to
| respond.
|
| -----
|
| Different programming languages have different patterns to make
| these situations easier to program.
___________________________________________________________________
(page generated 2024-09-13 23:01 UTC)