BetaRefusalFallbackMiddleware is a middleware layer for client.beta.messages that automatically retries refusals against a chain of fallback models; it patches the original request params and splices fallback seam blocks into the response, while BetaFallbackState pins subsequent requests to whichever fallback accepted via a context-local token. The middleware injects the fallback-credit beta into requests, strips fallback seam blocks from conversation history to avoid re-processing, and degrades gracefully when all fallbacks exhaust — replaying the original refusal with recommended model metadata and logging the failure.
BetaRefusalFallbackMiddleware, located in src/anthropic/lib/middleware/_fallbacks.py, only processes client.beta.messages requests to /v1/messages; first-party client.messages refusals carry no fallback_credit_token and pass through untouched.[1] The module's __all__ exports exactly two names: BetaFallbackState and BetaRefusalFallbackMiddleware.[1]
BetaRefusalFallbackMiddleware.__init__ accepts a fallbacks iterable (the ordered chain of model patches tried on refusal) and a betas keyword argument; an empty fallbacks iterable disables the middleware entirely.[1] The betas argument defaults to DEFAULT_BETAS — the tuple ("fallback-credit-2026-07-01",) — which is injected into the anthropic-beta header of every /v1/messages request the middleware handles, including the original request, because refusals only carry a fallback_credit_token when the beta is enabled; pass () to suppress it entirely.[1] Log output from BetaRefusalFallbackMiddleware is emitted under the logger name "anthropic.lib.middleware" — the public package path, not the private submodule path.[1]
Each fallbacks entry is a patch against the ORIGINAL request params: a field set to a value overrides it, a field explicitly None unsets it, and an absent field keeps the original value; output_config patches its subfields the same way one level deep. Hops never compound — every hop patches the original params, never the previous hop's patched request.[1] The fallback_credit_token carried on retries is always wrapped as {"token": ..., "mode": "best_effort"} so that token-layer failures degrade gracefully rather than returning an HTTP 400.[1] A refusal before any output has streamed causes BetaRefusalFallbackMiddleware to retry even without a credit token, and the serving hop's message_start opens the wire carrying the primary model's message id.[1] When every remaining fallback entry fails over HTTP, BetaRefusalFallbackMiddleware replays the suppressed refusal to the client with recommended_model stamped from the final failure — the failed model for capacity errors, null otherwise — and reports the event through the anthropic.lib.middleware logger.[1]
In non-streaming mode, BetaRefusalFallbackMiddleware prepends a fallback seam block per model boundary to the serving hop's content when the chain succeeds; the served hop's usage is left verbatim.[1] In streaming mode, fallback events are spliced onto the still-open original stream so the client sees one continuous message: a fallback content block at each model boundary, monotonic block indices, and per-hop usage.iterations on the final message_delta.[1]
BetaFallbackState is the only stickiness mechanism — fallback seam blocks replayed in the request history are stripped from the outgoing request and never read back as a pin.[1] An assistant turn left empty after seam-block stripping is dropped whole from the outgoing request history.[1]
BetaFallbackState is a context manager that pins subsequent requests in the same with block to the fallback model that accepted; it can be shared across sync and async clients and works correctly across threads and tasks via a ContextVar.[1] BetaFallbackState.index holds the index into the fallback chain the requests are pinned to; None (or -1) means the original request params are targeted, and the middleware sets it to the index of the fallback that accepted.[1] Reset tokens for BetaFallbackState context managers are stored in a ContextVar — not on the state instance — so a single state shared across threads and tasks has per-context enter/exit; a Token can only be reset in the context that created it.[1]
Canonical usage of BetaRefusalFallbackMiddleware with a BetaFallbackState pin for conversation stickiness:
client = Anthropic(middleware=[BetaRefusalFallbackMiddleware([{"model": "claude-opus-4-8"}])])
state = BetaFallbackState()
with state:
message = client.beta.messages.create(**params)
Sources