The Worker interface in QM polls for unclaimed runs, claims and processes them with a lease token, and exposes methods to start/stop polling and release in-flight runs; createWorker generates workers that poll every 50ms and invoke the orchestrator to handle each turn. On failure or shutdown, the worker safeguards against lease orphaning by releasing leases back to the store, and protects against transient heartbeat errors while enforcing non-retryable errors via explicit failure markers. A lease token is a unique credential issued to a worker when it claims a run, proving exclusive ownership; no other worker can process a run without holding its valid lease token.
The Worker interface in src/runs/worker.ts exposes four methods: start(), stop(drainMs?), releaseInFlight(), and busy() — where busy() returns true when a run is currently in flight.[1] createWorker generates a worker ID of the form w-<8-char UUID prefix> when deps.workerId is not provided, and polls for new runs every 50 ms by default (pollMs = 50).[1] WorkerDeps extends ProcessDeps with optional pollMs, workerId, required sessions: SessionStore, and optional canClaim and onClaimed callbacks; canClaim gates whether the worker will attempt to claim a new run on each poll cycle.[1]
processRun in src/runs/worker.ts requires the run to already hold a lease (leaseToken !== null) before it is called; it throws synchronously if the token is absent.[1] Before invoking the orchestrator, processRun computes queueMs as run.startedAt - run.createdAt (clamped to 0) and passes it to orchestrator.handleTurn only when startedAt is non-null.[1] On failure, processRun calls deps.runs.fail with retry: false when the thrown error is an instance of NonRetryableTurnError, preventing the run from being requeued.[1]
A transient heartbeat error resets the consecutiveLost counter to 0 and is not counted against the lease-loss threshold, allowing intermittent network failures to be tolerated.[1] When createWorker is stopped while a run has just been claimed, src/runs/worker.ts releases the lease back via deps.runs.releaseLease before breaking out of the loop, preventing lease orphaning.[1] Worker.releaseInFlight is idempotent: it tracks releasedLeaseToken and returns early if the in-flight run's lease token matches the previously released token, and coalesces concurrent calls behind a single releasing promise.[1] Worker.releaseInFlight also force-releases the associated session lease via deps.sessions.forceReleaseLease before releasing the run lease, ensuring both session and run locks are freed together.[1]
The worker-main.ts entry point starts the run-draining worker, logs the org, run-store type, and worker count on startup, and registers SIGINT/SIGTERM handlers that call stopWithBackstop for a graceful shutdown.[2] The shutdown() function guards against double-invocation: a shuttingDown flag is checked at the top and the function returns immediately if already set.[2]
Sources