Your AI-generated service dies loudly: the missing graceful shutdown

Your AI-generated service dies loudly: the missing graceful shutdown

I keep seeing the same bug in code that AI assistants generate. You ask for an HTTP server, a worker, a queue consumer. You get something clean, readable, and completely unprepared for the moment it has to stop. The model writes the fun part (accept connections, process requests) and skips the boring part (what happens when the process gets a SIGTERM). That skip is not harmless. It is the difference between a deploy that users never notice and one that drops requests, truncates writes, and leaves locks held.

Look at what a typical generated Node service looks like:

const http = require('http');

const server = http.createServer(async (req, res) => {
  const data = await doSomeWork(req);
  res.end(data);
});

server.listen(3000, () => console.log('up on 3000'));

That code has no SIGTERM handler. When Kubernetes or your process manager sends a termination signal, Node's default behavior kicks in and the process exits. Any request that is mid-await is gone. The client sees a connection reset. If doSomeWork had already written half a row, you now have a partial record. If it held a database lock, that lock may linger until the connection times out, and then a retry hits it again. Nobody wrote this bug on purpose. The assistant just never considered shutdown a requirement unless you asked.

This is not a theoretical edge case. Rolling deploys, autoscaling scale-downs, and container restarts all send SIGTERM as a matter of course. Every deploy is a chance for the bug to fire. I have watched teams spend hours chasing "random" 502s that were really just an unhandled signal during rollout.

The fix is not complicated, which makes the omission more annoying. You track in-flight work, stop accepting new connections, wait for the current work to finish (with a timeout), then exit.

const http = require('http');

let shuttingDown = false;
const inFlight = new Set();

const server = http.createServer(async (req, res) => {
  if (shuttingDown) {
    res.writeHead(503, { Connection: 'close' });
    return res.end('shutting down');
  }
  const job = doSomeWork(req).then(
    (data) => res.end(data),
    (err) => { res.writeHead(500); res.end('error'); }
  ).finally(() => inFlight.delete(job));
  inFlight.add(job);
});

server.listen(3000);

async function shutdown(signal) {
  if (shuttingDown) return;
  shuttingDown = true;
  console.log(`${signal} received, closing`);
  server.close(); // stop new connections
  const timeout = setTimeout(() => process.exit(1), 10000);
  await Promise.allSettled(inFlight);
  clearTimeout(timeout);
  process.exit(0);
}

process.on('SIGTERM', () => shutdown('SIGTERM'));
process.on('SIGINT', () => shutdown('SIGINT'));

A few things matter here. server.close() stops accepting new connections but lets active ones finish. The inFlight set lets you know what still needs to drain. The timeout ensures a stuck request cannot block the deploy forever. The 503 path handles requests that slip in during the gap. None of this is clever. It is just the operational floor.

One detail worth understanding is how the shutdown window interacts with your orchestrator's own timers. Kubernetes gives a pod a termination grace period, usually 30 seconds by default. If your app's internal timeout is longer than that, the orchestrator will kill the process before your drain finishes, and you are back to dropped requests. Set the app timeout comfortably below the platform limit: if the grace period is 30 seconds, a 10 to 15 second app timeout leaves room for the container runtime to stop the process cleanly. The same reasoning applies to load balancer deregistration delays. Many platforms need a few seconds after the pod is marked terminating before traffic actually stops arriving, so sleeping a short time at the start of shutdown prevents a burst of 503s from clients that have not yet learned the pod is going away.

It is also worth checking what your framework does when the signal fires during startup. A process that receives SIGTERM before it has finished binding the port should exit promptly rather than hang waiting on a connection pool that will never be used. Register the signal handlers early, before any long initialization, so a slow boot sequence does not ignore the stop request. The same rule applies to workers: a queue consumer that is halfway through fetching its first batch should be able to abort cleanly instead of completing work nobody asked for.

And this pattern is not Node-specific. Python's uvicorn has --timeout-graceful-shutdown. Go's http.Server has Shutdown(ctx). Rails has Puma's graceful stop. The mechanism differs; the requirement is the same. If your application owns a resource (a lock, a file handle, a database transaction, a job lease), you need to release it on shutdown, not rely on the kernel to clean up.

My reaction is impatience, honestly. AI assistants are very good at writing the 20-line happy path and very bad at the 20 lines of failure handling around it. They optimize for the demo. Shutdown handling does not show up in a demo. It shows up at 2 a.m. during a failed rollout, when your health check says healthy and your users say otherwise. If you use an assistant, you have to ask for this explicitly, and then read the result carefully. "Add graceful shutdown" is now a prompt I paste into every service skeleton before I trust it.

Seaworthy is a security scanner that checks codebases for issues like this; you can find it at seaworthy.dev.