Original AI-generated editorial illustration for SPKE. A conceptual metaphor, not an engineering diagram or measured result.
The missing receipt
Consider a fictional support assistant that prepares a replacement shipment. The application sends the approved shipment request, but the connection drops before the receipt arrives. A recovery routine switches to another model, supplies the conversation, and asks it to finish the job. The backup sees no success message and proposes another shipment.
The first shipment might already exist. A fluent recovery message could hide a duplicate operation. The dangerous step is the application's decision to execute again, not the nationality of either model.
For teams combining US and Chinese AI services, this suggests a useful acceptance question: after a failover, can the system establish what actually happened? A second provider adds another route to generation. It does not automatically preserve the state of the surrounding workflow.
Compare behavior, not just request formats
Three primary sources help locate the boundary. The current OpenAI Python SDK documents two automatic retries by default for certain request failures and configurable retry settings; consuming an interrupted stream is not automatically retried. Official SDK
DeepSeek documents account-level concurrency limits, plus empty lines or SSE comments used to keep waiting connections alive. Its documented ten-minute closure applies when inference has not started, not to a universal task deadline. Rate Limit & Isolation
AWS's discussion of idempotent APIs explains the underlying distributed-systems problem: a missing response can leave completion uncertain; retry safety needs a service contract. Making retries safe
These are documentation facts checked on October 26, 2026, not a fresh outage report or a comparative reliability test. They describe different layers: SDK behavior, provider connection handling, and business-service semantics. None establishes that an arbitrary AI tool can be repeated safely.
Before pairing providers, document the exact endpoint, model and SDK version, supported input and tool formats, retry configuration, timeout behavior and data-handling permission. A familiar request shape is not enough. Also check whether the fallback may receive the same documents; switching suppliers must not silently expand who receives confidential data.
Separate an attempt from an operation
Our proposed recovery design starts with two records. An attempt records a particular model request: provider, model, start time, outcome and available request identifier. An operation records the user-authorized business action: its purpose, exact approved parameters, status and evidence of completion. Several attempts may serve one operation.
For the fictional shipment, the operation should retain its identity when a different model takes over. A newly generated tool-call identifier must not be treated as permission to create a second shipment. Conversely, two separately authorized identical shipments must remain distinguishable.
Keep operational state outside the conversation. A model can summarize the record, but the executor should check it before acting. Make that check and the claim to execute atomic, so concurrent primary and backup workers cannot both proceed. A useful minimum is planned, submitted, confirmed, rejected, and outcome-unknown. Unknown is a real state with an owner and a resolution path; it must not become failed merely because that makes a retry loop simpler.
Where the destination service provides an idempotency mechanism, follow its documented scope, parameter rules and retention window. A local identifier alone cannot make a remote effect idempotent. Without an adequate service contract or reliable reconciliation, hold uncertain writes for review rather than promising exactly-once execution.
Recover at the boundary that failed
Use three distinct recovery paths.
Generation only. If the failed step only drafted text and no action was submitted, another model may generate a replacement within the allowed time and cost budget. Keep partial output separate from the replacement; do not quietly splice two incompatible answers into one apparently continuous result.
Known rejection. If authoritative evidence establishes that the business service rejected the operation without applying it, address the reason before any new authorized attempt. An invalid destination or missing permission is not repaired by choosing a more willing model.
Uncertain write. If an action was submitted but its receipt is missing, query the destination's authoritative operation record where available. Reuse the recorded result if confirmed. If the outcome remains unresolved, report that uncertainty and stop automatic writes for that operation. An empty search result is not necessarily proof of nonexecution when visibility can lag.
The fallback model can help explain what is pending or prepare a draft for review. It should not invent confirmation, reinterpret approval, or decide that an unknown shipment never happened.
Give recovery one shared budget
Retries can be useful, but independent layers can multiply them. In a hypothetical configuration, three application attempts, each permitting three SDK attempts, can produce nine requests before any further provider switch. This is arithmetic for the stated configuration, not a measured SDK trace or a universal upper bound.
Assign one component responsibility for the overall deadline, attempt budget and escalation policy. Count actual downstream attempts, not just the top-level task. Respect service retry guidance, apply appropriate backoff, and avoid immediately replaying an overloaded queue against the backup. Do not use fresh credentials to evade an account-level limit.
Measure time to a usable answer separately from time to a confirmed action. A keep-alive signal establishes connection activity, not task progress. Similarly, a user-facing timeout does not prove that a remote action was canceled. Recovery should account for late results instead of allowing them to trigger another execution.
Test the lost receipt before the real customer
Use a sandbox with simulated external actions. Write expected outcomes before running the tests. At minimum, exercise these cases:
- A generation request fails before any business action is submitted.
- The destination applies an action but its response is lost.
- The primary finishes after the backup has already produced an answer.
- A stream stops after some text has been displayed.
- A backup proposes changed parameters for an already approved operation.
- An operation remains invisible briefly in the destination's read interface.
- The destination's deduplication window expires before recovery.
For each case, inspect the external effect and the operation record, not only the assistant's final sentence. In the lost-receipt case, an acceptable result may be one confirmed action or an explicitly unresolved hold. A second shipment is not successful recovery. A false success message is not either.
Report duplicate effects, unresolved operations and recovery time alongside completion rate. Keep the denominator as original user tasks; retries are attempts, not additional successful tasks. Record review workload and provider costs as well, so a design that defers everything does not appear to offer effortless automation.
What a second provider should buy
A useful fallback preserves the original task's constraints while offering another chance to complete it. For a drafting task, that may mean a replacement answer. For a workflow with external effects, it may mean a trustworthy account of what is confirmed, what is still unknown, and what must happen next.
Start with one consequential action and one controlled lost-receipt test. If the system cannot recover that case without duplication or invented certainty, adding more models makes the routing diagram richer before it makes the workflow dependable.
This article is SPKE's original engineering analysis informed by the linked primary sources. The shipment scenario, proposed state model and test plan are illustrative. We did not run provider benchmarks, inject faults into production systems or measure reliability gains. Verify current documentation and the contracts of your actual tools before deployment.