The Second Request Is the Real Test: Designing APIs That Survive Retries

Every API contract has a hidden clause: the client may send the same request twice. A mobile network drops the response after the server commits a charge. A queue worker times out and retries a job that already ran. A load balancer replays a request after a backend node disappears. The client is not being careless; it is behaving correctly in an unreliable environment. The server, however, must make duplication harmless.

That is the promise of idempotency keys. They are not a performance optimization or a compliance checkbox. They are a coordination mechanism for turning ambiguous retries into deterministic outcomes. In 2026, as more American teams build multi-service systems with aggressive retry policies, the second request is the real test of API design.

Exactly-Once Is a Coordination Problem, Not a Header

Exactly-once delivery is usually a fantasy at the network layer. Messages can be lost, responses can be dropped, and clients can crash after committing a request but before recording success. What teams can build is effectively-once processing: the client attaches a stable key to each logical operation, and the server guarantees that repeated attempts with that key produce the same result without repeating side effects.

The key is not a request identifier generated fresh on every attempt. It must represent the business intent. If a client retries a payment, the key should remain the same across all attempts for that payment. If the client creates a new payment, it needs a new key. Confusing those two ideas is the first crack in the design.

The Key Must Be Part of the Contract

An idempotency key that appears only in an SDK is not an API feature; it is a convention. The contract should state which endpoints accept keys, which methods require them, how long keys remain valid, and what happens when a key is reused with a different payload.

For state-changing operations, the safest posture is to require the key on create and other non-idempotent verbs. The client sends it in a dedicated header, and the server treats it as part of the operation identity. Documentation should also make clear that the key is scoped to a tenant or account. A key that is unique only within a single client process will collide at scale, and a global namespace invites accidental cross-tenant interference.

Storage Is Where Most Implementations Fail

A common implementation stores the key in application memory, checks for a duplicate, performs the work, and returns a response. That works until the process restarts, the service scales horizontally, or two requests arrive at nearly the same moment. Idempotency state belongs in durable, shared storage with atomic semantics.

The record should include the key, the caller scope, a fingerprint of the request, the operation status, and the stored response. The fingerprint matters because a client that reuses a key with a different body should not silently receive the old result. The server should reject that mismatch with a clear conflict response. Without the fingerprint, key reuse becomes a data integrity bug rather than a safety feature.

TTL and Scope Are Policy Decisions

How long should an idempotency key live? There is no universal answer. The TTL must exceed the client's maximum retry window, including backoff, queue delays, and manual intervention. A short TTL creates a dangerous gap: the original record expires, a late retry arrives, and the side effect runs again while the client believes it is still protected.

Scope is equally important. Keys should be isolated by tenant and, where relevant, by operation type. A broad namespace forces unnecessary coordination and raises the cost of storage. A narrow namespace prevents a key from protecting the same logical operation across services. Teams should define the boundary explicitly and document it alongside the endpoint.

Concurrency: The Race Between Check and Insert

Two retries can arrive at the same time. If both read the key as absent and both proceed, the system has built a race condition rather than an idempotency guarantee. The storage layer must provide an atomic claim: insert the key if absent, or return the existing record. A unique constraint or conditional write is usually the right primitive.

The winning request performs the work. The losing request waits, polls, or receives a conflict response depending on the API's semantics. Neither outcome is inherently wrong, but the behavior must be deliberate. Silent double execution is the only unacceptable result.

Replay the Outcome, Not Just the Status Code

When a duplicate request arrives after the original completes, the server should return the stored outcome, including the response body and status code. Replaying only a generic success message breaks clients that need the created resource identifier or the final state of the operation.

For long-running operations, the record may hold an in-progress state. In that case, the duplicate should receive a clear indication that the operation is still underway, not a false success. The client can then query the operation status or retry according to the documented contract.

Failure Modes That Deserve Explicit Responses

A robust idempotency design distinguishes between a duplicate in progress, a duplicate completed with the same payload, and a key reused with a different payload. Each deserves a different response. Collapsing them into one error code makes client behavior guesswork.

Teams should also decide what happens if the stored response cannot be recovered. Returning a server error is safer than pretending the operation failed and inviting another attempt. The goal is not to eliminate every ambiguity; it is to make the ambiguity visible and bounded.

A Practical Checklist for 2026

Before shipping a state-changing endpoint, confirm that the key is required and documented, that the storage is durable and atomic, that the request fingerprint is checked, that the TTL exceeds the retry horizon, and that duplicates receive a consistent replay or conflict response. Test with concurrent retries, delayed retries, and key reuse with altered payloads.

Idempotency keys are not optional in systems that retry. They are the difference between a resilient API and one that quietly duplicates work under pressure. The second request will arrive. The only question is whether the design is ready for it.