fiber

4. Blocking threads, no async runtime

Date: 2026-09-21

Status

Accepted. Settled by The threading and streaming model. The contract is the concurrency section of docs/architecture.md.

Context

Fiber has to stream a model response off a socket, run tool subprocesses alongside it, append to a log with an fsync on the critical path, draw a terminal, and let a person cancel all of it mid-stream. Rust offers two ways to write that: plain blocking std::thread, or an async runtime.

Idle CPU does not decide it. A parked tokio runtime costs the same idle CPU as a parked std::thread (0.114 ms against 0.122 ms over 60 s, #8). Dependency graph size and thread count needed measuring too.

Three measurements decided what the argument could rest on. All were run on macOS arm64 (Darwin 25.6.0, Apple M3 Pro, rustc 1.98.1), then on Linux x86_64 and arm64 as the static musl binaries Fiber ships (#16, figures in research/concurrency/README.md). Linux confirmed all three.

Parked threads are cheap at any count Fiber will reach. research/concurrency/threads_scale, 10-second idle windows: 1 thread 1.6 MiB RSS, 32 threads 2.3 MiB, 128 threads 4.2 MiB, 512 threads 11.6 MiB and 0.35 ms of CPU, at 0.1 wakeups per second. A 64 KiB stack changed nothing, because RSS counts committed pages rather than reserved stack. "One parked thread per blocking thing" therefore survives every MCP server, background job and child session v0.0.1 could plausibly have. On Linux musl, 512 parked threads cost 3.2 MiB and 0.16 to 0.18 ms, with zero context switches. This removed thread count as an argument.

The same program, written three ways, differs by almost nothing. research/concurrency/mini_blocking, mini_smol and mini_tokio each stream a response, run a subprocess alongside it, fsync per event, and take a cancellation that interrupts the in-flight read. Medians of three runs:

stripped binarypeak RSSwallcancelcratesSLOC
blocking threads517 KB2.03 MB506 ms88 µs2132
smol862 KB2.33 MB502 ms42 µs34137
tokio915 KB2.57 MB502 ms27 µs15133

On Linux musl the gap to tokio is the same 400 KB and all three peak within 0.3 MiB of each other. 400 KB is 2% of a 20 MiB budget, the cancel latencies are three orders of magnitude below human perception, and the wall times are identical because the work is I/O. The line counts came out within five lines of each other, so "easier to write" had no measured winner either. Two incidental findings: smol pulls more crates than tokio, not fewer; and async-std is discontinued — crates.io describes it as "Deprecated in favor of smol".

Cancellation does not require writing an HTTP client. #8 concluded that escape-cancels-a-request "requires Fiber to own the TcpStream", because ureq exposes no way to interrupt a blocked read. research/concurrency/cancel_ureq_connector shows that ureq 3.4's custom-connector API gives Fiber the socket handle while ureq keeps doing HTTP, chunked decoding and TLS: a thread stuck in ureq's body reader returned after another thread closed the socket, with the stream still arriving decoded. Over TLS the median of 20 cancellations was 67 µs on macOS and 154 to 174 µs on Linux musl.

So the numbers do not decide it. Three things that are not performance do.

Most of Fiber's blocking work is threads underneath in every design. tokio's own source says tokio::fs "will use ordinary blocking file operations behind the scenes... using the spawn_blocking threadpool", and "currently, Tokio will always use spawn_blocking on all platforms". crossterm's async EventStream spawns a thread and wakes a future from it. Of Fiber's four blocking activities, only the model socket is genuinely async; a runtime wraps the other three rather than removing them.

The design is already synchronous by decision. docs/architecture.md on the hook seam: "Synchronous: the loop stops, asks, waits and honours the answer, under a timeout Fiber enforces." One turn at a time, one writer per session, and loop as the only module that decides what happens next. The overlap an async runtime is good at is overlap three earlier tickets chose not to have.

The two styles fail differently, and one failure is invisible. Calling something blocking from inside an async task stalls the executor for the duration. It compiles, it passes tests, and it surfaces as the screen stuttering while a tool runs. Fiber is unusually exposed to it: mlua is synchronous, the hook seam is synchronous by contract, and fsync is on the critical path by decision. The blocking equivalent — a deadlock or a missed join — is a hang with a stack trace naming the function, which a gate with a timeout catches and a delegate can fix. Fiber's Rust is written by delegates and checked by gates, so which bugs a gate can catch is a first-order concern.

Decision

Fiber uses blocking std::thread and no async runtime. HTTP is ureq over rustls, behind a custom connector that keeps the TcpStream handle so a second thread can close the socket to cancel a read.

Consequences

Every "wait for whichever of these happens first" is written by hand with a channel rather than with select!. Cancelling a read means handing a socket clone to a second thread before the read starts, which is the one place the blocking implementation is fiddlier than the async one, and that pattern recurs in background jobs, subagents and MCP. This is accepted, not overlooked.

The argument that would reopen this is a measurement on a shipped target contradicting these, or a v0.0.1 requirement to hold enough concurrent sockets that thread-per-socket stops being free. Neither is an argument from taste, and neither is available today.