Retries without backoff will take down your dependencies

You've seen the pattern. A network call fails, so you wrap it in a loop: catch the error, wait a second, try again. Ship it. Done.

Except it's not done. That innocent loop is a loaded gun pointed at whatever service you depend on.

Why immediate retries make things worse

When a downstream service starts struggling, it slows down or starts dropping requests. Every client talking to it gets errors. Every client with a naive retry loop fires again immediately. The struggling service, which was already overloaded, now receives a second wave of requests right on top of the first. Then a third. The load spikes rather than subsiding, and what would have been a 30-second blip becomes a full outage.

This is the thundering herd. It's not a hypothetical. It's how incidents escalate in production, and it happens because retrying immediately treats a transient failure as if it were a fixed bug, rather than a signal that the system needs time to recover.

I've watched this happen in real incidents. A database connection pool exhausts during a deployment, clients retry instantly, and the pool never gets a chance to drain. The rollback fails for the same reason. It takes twice as long to recover because the retries are actively fighting the recovery.

Exponential backoff with full jitter

The fix is to wait longer after each failure, and to add randomness so clients don't all retry at the same moment. The algorithm is simple:

wait = random(0, min(cap, base * 2^attempt))

The base is your initial wait (say, 100ms). The cap is the maximum you're willing to wait (say, 30 seconds). The 2^attempt grows the window exponentially. random(0, window) picks a value inside that window, which is what "full jitter" means.

Full jitter is better than adding a small random offset to a fixed backoff. It spreads retries across the entire window, which gives the downstream service genuine breathing room rather than a slightly less synchronized thundering herd.

Here's a practical Node.js helper built on p-retry, which handles the retry loop and exposes the backoff config cleanly:

import pRetry, { AbortError } from 'p-retry';

type RetryOptions = {
  maxAttempts?: number;
  baseMs?: number;
  capMs?: number;
};

export async function withBackoff<T>(
  fn: () => Promise<T>,
  { maxAttempts = 5, baseMs = 100, capMs = 30_000 }: RetryOptions = {}
): Promise<T> {
  return pRetry(fn, {
    retries: maxAttempts - 1,
    randomize: true, // full jitter
    factor: 2,
    minTimeout: baseMs,
    maxTimeout: capMs,
    onFailedAttempt(error) {
      // Non-retryable: abort immediately
      if (error.response?.status && error.response.status < 500) {
        throw new AbortError(error);
      }
      console.warn(
        `Attempt ${error.attemptNumber} failed. ${
          error.retriesLeft
        } retries left. Waiting before retry.`
      );
    },
  });
}

Call it like this:

const data = await withBackoff(() => fetchExternalApi(userId));

The AbortError path matters a lot, which brings us to the next point.

Not all errors deserve a retry

Retrying a 400 Bad Request is pointless. The request is malformed. Sending it again won't change the outcome, it will just add load and delay the error reaching whoever needs to handle it. Same for 401, 403, 404, 422. These are client errors: the server understood the request and rejected it for a reason.

Retryable errors are the ones where the server couldn't complete a valid request due to its own state: 500, 502, 503, 504, and network-level failures like connection reset or timeout. The distinction matters because bad retry logic turns a fast, actionable client error into a slow, confusing one.

A rough rule: retry on anything that looks like a temporary infrastructure failure. Abort on anything that looks like a logic error.

Retry budgets

Even with backoff and jitter, unbounded retries are a problem. If every operation can retry five times and you have a thousand concurrent requests, you can generate up to five thousand requests to a service that's already struggling. That's a lot of amplification.

Retry budgets set a ceiling on how many retries a service (or a request handler) will attempt within a time window. The idea comes from Google's SRE practices. At the service level, you track total retries as a ratio of total requests. If retries exceed, say, 10% of traffic, you stop retrying and fail fast instead.

In practice, most applications don't implement service-level retry budgets. But at minimum, set per-operation limits that reflect how much latency your callers can tolerate. If your API has a 10-second timeout, a retry config that can accumulate 45 seconds of waits is going to produce timeouts that look like bugs, not retries.

Cap your maxAttempts to something that fits inside your caller's patience. Three to five attempts is usually enough. If the service isn't recovering after five attempts with exponential backoff, you're not dealing with a transient blip anymore.

The practical checklist

Before shipping any retry logic:

• Check whether the error is retryable before retrying it
• Use exponential backoff with full jitter, not fixed delays
• Set a cap on both the maximum wait and the number of attempts
• Make sure total retry time fits inside your caller's timeout budget
• Log failed attempts so you can see retry storms in your metrics

None of this is complicated. The hard part is remembering to do it, because a simple retry loop feels like it solves the problem in the moment.

Seaworthy flags retry logic without backoff as a resilience issue.