AgentCore Gateway Model Fallback and InvokeModel: A UK SRE Playbook for Resilient Inference
On 22 October 2026 AWS added Bedrock InvokeModel support and model fallback routing to the AgentCore Gateway inference endpoint. Here is how I would design fallback chains that keep agents up without quietly breaking UK data residency or your cost model.

On 22 October 2026 AWS added Bedrock InvokeModel support and model fallback routing to the AgentCore Gateway inference endpoint. Here is how I would design fallback chains that keep agents up without quietly breaking UK data residency or your cost model.
- AWS added InvokeModel and model fallback to the AgentCore Gateway inference endpoint on 22 October 2026.
- Existing Bedrock SDK clients can point at the gateway, which translates between API dialects.
- Fallback uses ordered chains, configurable failure types, first-byte timeouts and a default-on circuit breaker.
- Only chain destinations that meet the same UK data residency requirement as the primary.
- Fall back on throttling and capacity errors, not on client errors or guardrail blocks.
- Set first-byte timeouts from measured p99 time-to-first-token.
- Evaluate and guardrail every fallback model to the same standard as the primary.
- Monitor fallback rate and cost by actual destination, and game-day the chain before production.
What changed
On 22 October 2026 AWS announced two additions to Amazon Bedrock AgentCore Gateway inference routing:
- InvokeModel support. The gateway's aggregated
/inferenceendpoint now accepts the Amazon BedrockInvokeModelandInvokeModelWithResponseStreamwire format, in both directions. A client using the Bedrock SDK can reach any inference target, including non-Bedrock providers, and a client using the OpenAI or Anthropic SDK can reach a Bedrock target that serves models throughInvokeModel. Where dialects differ, the gateway translates the request and re-frames the streamed response so the client SDK works unchanged. A newbedrock-runtimeconnector configures a target for the nativeInvokeModelAPI. - Model fallback. A gateway rule can select requests with a
matchModelscondition and define an ordered fallback chain on itsrouteToTargetaction. When the primary destination is throttled, out of capacity or unreachable, the gateway tries the next target or model. You choose which failure types trigger fallback and set a per-destination first-byte timeout. A service-owned circuit breaker, on by default, skips destinations that are currently failing.
AWS says both are available in all Regions where AgentCore Gateway is available, except China. The same release notes also list a built-in AWS Step Functions connector that exposes state machines as MCP tools, and AgentCore CLI 1.0.
From the client's point of view, fallback is invisible: one request in, one response out. That is exactly why it needs designing carefully.
Why this matters for SRE teams
Until now, most teams I work with handled throttling in each client: retries with backoff, maybe a hard-coded second model. That logic drifts between services, rarely gets tested, and is almost never visible in one place. Moving it into the gateway gives you one governed place to define what happens when a model is unavailable.
The flip side is that the gateway can now send a request somewhere you did not explicitly call. In a UK context, that raises three questions straight away: where does the fallback run, what does it cost, and does it behave the same?
The UK playbook
1. Make every fallback chain residency-aware
If your primary target is chosen for UK or EU data residency, every destination in its fallback chain must meet the same requirement. A chain that falls back to a different provider or a model served from another geography could move prompts containing personal data outside the boundary you documented in your DPIA.
Practical rules:
- Group targets into residency classes (for example UK-only, UK-or-EU, global) and only chain within a class.
- Check regional availability for AgentCore Gateway and each model in the AWS documentation before you design the chain; do not assume a particular Region is supported.
- Record the chain in your record of processing and DPIA, not just the primary model.
2. Separate chains by workload sensitivity
Use matchModels and separate rules so that a customer-facing agent handling personal data has a different, stricter chain from an internal summarisation tool. Not every workload deserves the same fallback; some should simply fail and alert.
3. Choose failure types deliberately
Fallback on throttling and capacity errors is usually right. Fallback on server errors is often right. Fallback on client errors (a bad request, a guardrail block) is usually wrong, because the next model will either fail the same way or, worse, answer something the first one refused. Configure only the failure types you actually want to absorb.
4. Set first-byte timeouts from real data
Pull p95 and p99 time-to-first-token for each model from your existing telemetry and set the per-destination first-byte timeout a little above p99. Too short and you fall back constantly and pay for wasted attempts; too long and users sit watching a spinner before the fallback even starts.
5. Keep behaviour consistent across the chain
A fallback model is a different model. Before adding it:
- Run your evaluation set against it and record the scores next to the primary's.
- Make sure the system prompt, tool schemas and output format work on it. Streaming re-framing helps the SDK, but it does not make two models answer the same way.
- Apply the same guardrails to every destination, so a fallback is never the soft path.
6. Make fallback visible
The client sees one response, so your observability has to see the rest. Track, per rule:
- Fallback rate by destination and failure type.
- Circuit-breaker open events.
- Cost per request by actual destination, not requested model.
- Quality scores from online evaluation, split by which model answered.
Alert when fallback rate is sustained above a threshold you agree. A chain that is permanently on its second destination is not resilience; it is a silent migration.
7. Test it like a failover
Treat a fallback chain like any other failover: game-day it. In a non-production gateway, force the primary to fail (for example by pointing it at an unreachable target or exhausting a low quota) and confirm the chain, the circuit breaker, the logs and the alerts all behave as designed. Keep the evidence for change approval.
8. Use InvokeModel support to consolidate, carefully
The InvokeModel compatibility means existing Bedrock SDK code can point at the gateway without a rewrite. That is a good way to bring scattered direct-to-model calls under one policy point. Migrate service by service, compare latency and cost before and after, and remove the old per-client retry logic once the gateway rule is proven, so you do not end up with retries stacked on retries.
A simple starting pattern
- Rule A (customer data, UK-only class): primary model, one fallback in the same residency class, fallback only on throttling and capacity, alert on any sustained fallback.
- Rule B (internal, non-personal data): primary model, two fallbacks across providers, fallback on throttling, capacity and server errors.
- Everything else: no fallback until someone owns it.
Bottom line
Gateway-level fallback and InvokeModel compatibility are genuinely useful: one place to define resilience, and existing Bedrock code that can move behind it without rewrites. For UK teams the job is to make sure the chain respects the same residency, guardrail and quality bar as the primary, that fallback is visible and alerted, and that it has been tested like any other failover before it carries customer traffic.
Fallback moves retry logic into one governed place, which is a big improvement. But the client never sees which model answered, so the gateway can quietly send personal data to a destination outside your documented boundary. Build chains within residency classes, fall back only on the right failure types, and alert on sustained fallback.
