Part of #8811 (production disk-full incident). Companion to #8812 — this
is the systemic backstop so no future never-settling job can wedge the
system.
Two changes:
**Queue timeout was dead config.** `enqueue()` passed `timeout:
undefined` to p-queue's `add()`; p-queue spreads per-call options over
its defaults, so this overrode and *disabled* the queue-wide timeout
configured from `compilationEnvTimeoutMs` (default 300s). A job whose
promise never settled therefore occupied a queue slot forever — and
since temp dir cleanup only runs when the queue is fully idle, one
wedged slot permanently disabled cleanup and filled the disk. Removing
the override makes the timeout effective: in p-queue v9 a timed-out task
rejects with `TimeoutError`, freeing the slot. (The timeout doesn't kill
underlying work — that remains the exec layer's job — but the system
makes progress again.)
**Uncaught exceptions now actually stop the process.** The handler set
`process.exitCode = 1` assuming the app would "exit naturally", but a
process with live server listeners never does: during the incident the
instance limped on half-dead for hours, passing healthchecks while every
compilation failed. Now it exits after a 1s delay (letting winston flush
its transports), and the load balancer replaces the instance.
The new queue test fails against the previous code: a never-settling job
is never rejected and `status().busy` stays true forever. Also adds
basic enqueue/nested-enqueue coverage.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: mattgodbolt-molty <mattgodbolt-molty@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
9db0c4d28b