There's No Such Thing as a Transient Issue

Kyle Georgeme

• October 9, 2026
• 5 min read
• Engineering Culture, Reliability Engineering

There are no transient issues, only issues you don’t understand yet.

If you work in technology you’ve no doubt seen numerous problems that were dismissed with a hand wave and a transient issue or transient error comment. “Closed, transient issue.” “Resolved, transient error.” Flapping alert? Transient. Fixed by restarting service? Transient. Except that there’s no such thing. There is no such thing as a transient issue, there are only issues that you don’t understand yet.

These supposed transient issues are telling you something. Are you listening? That flapping alert might be telling you that your system can’t handle peak load and could fall over with a spike in traffic. That problem that was fixed by restarting that service could be masking a memory leak. That periodic HTTP 429 error you get from the AWS API could be telling you that you’re one scale-up away from a widespread outage. That transient 429 is telling you that your call rate is periodically exceeding the token bucket and refill rate, and your system is flying too close to the sun. But hey, at least your KPIs are green because you resolved that alert within the SLA.

Transient errors are signals, but it’s easy to ignore or paper over them. Google’s AI Overview for transient error is …

A transient error (or transient fault) is a temporary, self-correcting failure caused by brief disruptions like momentary network drops, a busy service, or a temporary resource lock. It resolves itself quickly without needing a permanent fix.

… and it dutifully links to a wall of text from Microsoft preaching about retries, retry strategies, exponential backoff, etc. Sure, good engineering requires that you consider these things as part of the end-to-end principle of system design. Yes, you do need things like intelligent retries. But these are things that get you through the night, they aren’t meant to be in use perpetually. If your software is retrying operations all the time you’re missing a cry for help. Are your retries instrumented? Do you alert on them?

I completely understand that during a busy on-call shift the flapping alert gets closed in favor of giving the cycles to higher immediate priorities. Teams must have a process for individuals or managers to recognize patterns or metrics that deserve greater attention.

Many years ago after observing more than one “resolving, transient error” across multiple on-call shifts, only for that warning sign to later blow up into a bigger problem, I added a Slackbot autoresponder at my company. Anyone who sent messages containing transient error or transient issue would get a friendly reminder:

There are no transient issues, only issues you don’t understand yet.

That helped, but it wasn’t enough on its own. I created a wiki page where people could put known issues. I called it NOTAMs, or NOtice To All Members, inspired by aviation’s use of the same technique.¹ Team members were asked to check this at the start of their shift, and record any notable anomalies. Finally, we backstopped the process with alert review, on-call shift handoff procedures, and encouraging people to log tickets for follow-up when they couldn’t immediately give it time. Overall, I’m happy to say the combination of these techniques greatly reduced the missed occurrences of transient issues blowing up in our faces.

Here’s an instance — pun intended — where it paid off. Multiple jobs across different parts of our environment were having elevated retries for the EC2 API DescribeInstances. Runtimes were delayed (as is frequently a symptom of retries) but the system was still operating. Pattern recognition and having a culture that values understanding transient issues led us to discover that we were exceeding our DescribeInstances rate limit within a single account, and we were able to take a combination of actions to request a rate limit increase and reduce our utilization (caching). If we had waited much longer the retries would have caused time-sensitive systems to fail. By recognizing the transient pattern, wanting to understand it, and taking timely and decisive action, we were able to head off disaster.

These days with AI agents and anomaly detection in open source and vendor monitoring products there are a lot more options at your disposal. You can point your agent at, say, a month’s worth of alerts and ask it to find the patterns that you missed. Whether you’re doing it with human eyes or AI augmentation, the point is to do something.

In summary:

• Transient issues are a chance to catch problems before they become severe. Don’t waste the opportunity. Understand them and take action on transient issues before they blow up.
• Instrument your retries and error handling and alert on their overuse. Teach operators that repeated manual actions aren’t normal and to surface anomalies.
• Build a culture and put a process in place to recognize patterns and recurrences and not lose track of them.

What do you have to learn from your transient issues?