AI-generated HTTP clients have no circuit breakers

The client that kills your app

When I ask an AI assistant to write a service client, it writes something that looks fine. The function signature is clean. The error handling catches HTTP errors. It even destructures the response nicely. But there's something missing from every single version I've ever seen come out of an AI tool: any protection against the upstream being slow.

Not down. Slow.

A downed service throws an exception quickly. Your app catches it, logs it, moves on. A slow service is far more dangerous, because it holds your threads, your connections, your workers, whatever concurrency primitive your runtime uses. One slow API becomes a queue. That queue fills up. And now a payment gateway timing out at 30 seconds has taken your entire application offline.

This is a well-understood problem. Circuit breakers, timeouts, and retry budgets have been described in engineering literature for decades. The Hystrix library popularised the pattern in the Java world around 2012. It's not new. But AI coding assistants generate HTTP clients as if none of that exists.

What the generated code actually looks like

Here's a representative example. This is the kind of client you get when you ask an AI assistant to wrap a third-party API:

async function getUser(userId: string): Promise<User> {
  const response = await fetch(`https://api.example.com/users/${userId}`);
  if (!response.ok) {
    throw new Error(`Failed to fetch user: ${response.status}`);
  }
  return response.json();
}

This code has no timeout. If api.example.com takes 90 seconds to respond, your thread waits 90 seconds. If ten requests come in while the first one is still waiting, you have ten threads blocked. In a Node.js application with a limited worker pool, or a serverless function with a concurrency cap, you've just created a denial-of-service vector where the attacker is your own dependency.

The fixed version needs at least a timeout signal:

async function getUser(userId: string): Promise<User> {
  const controller = new AbortController();
  const timeoutId = setTimeout(() => controller.abort(), 5000); // 5s hard limit

  try {
    const response = await fetch(`https://api.example.com/users/${userId}`, {
      signal: controller.signal,
    });
    if (!response.ok) {
      throw new Error(`Failed to fetch user: ${response.status}`);
    }
    return response.json();
  } finally {
    clearTimeout(timeoutId);
  }
}

That's the floor. It still isn't a circuit breaker. It doesn't track failure rates or open the circuit after repeated timeouts. But at least it won't let a single call hang indefinitely.

Why the AI gets this wrong

AI assistants are trained on code that prioritises making the happy path work. The happy path of an HTTP client is: connect, send, receive, return. Most tutorial code, most Stack Overflow answers, most README examples show exactly that. The resilience layer, when it exists at all, tends to be in a separate library configured elsewhere in the application. So when an AI generates a client, it generates the tutorial version, because that's what most of its training data looks like.

There's also a subtler problem: the AI has no idea what the caller's concurrency model is. It doesn't know whether this function will be called from a single async queue or from 500 concurrent requests. Adding a 30-second default timeout is a reasonable suggestion for a function called once. It becomes a necessity when the same function is called by hundreds of parallel workers. The AI doesn't ask.

What you actually need

A production HTTP client for an external dependency should cover things that generated code skips.

Timeout at the connection level and at the total response level, separately. Connecting in 1ms but waiting 60 seconds for the first byte is still a hang.

A retry strategy that is not "retry forever." Back off exponentially, cap the number of attempts, and only retry on transient errors: network timeouts, 429s, 503s. Don't retry on 400s or 401s. Retrying a bad request a hundred times just multiplies your problem.

A circuit breaker. After N consecutive failures or a failure rate above a threshold, stop calling the upstream entirely for a short window. Let it recover. Libraries like cockatiel (Node.js) or opossum handle this with composable, testable policies rather than hand-rolled state machines.

Bulkhead isolation, if you have multiple external dependencies. Don't let a slow payments API starve your user-lookup API by consuming the same connection pool.

None of this is exotic. It's the same pattern described in Michael Nygard's Release It! from 2007. It just doesn't appear in AI-generated code because the training data doesn't model it at the call site.

The cost of skipping it

I've seen this failure mode play out in production. A third-party geolocation API slows down. Requests start queuing. Within two minutes, the application's own health check can't get a response because every available worker is waiting on the geo API. The load balancer marks the instance unhealthy. Auto-scaling spins up new instances. They all immediately start calling the geo API. They all queue. Now all instances are down.

One slow third-party call. Total outage. Nothing in the generated client to stop it.

If your codebase has AI-generated service clients, audit them for timeouts before you're writing an incident retrospective. The fix is not complicated. The failure, when it happens, absolutely is.

Seaworthy flags missing timeouts and absent circuit breaker patterns as part of its resilience checks.