Steadwing SDK: Every Error Arrives With a Fix Attached

The alert is no longer the start of an investigation. It's the end of one.

Here's the incident that made that concrete for us. One SQL query, one deploy, and an API that stopped answering. 12 minutes from broken to fixed.

The Time Our Database Choked on One Slow Query

It started the way database incidents always start. Not with an error. With slowness.

A query had gone out in a recent deploy. It ran on every request to one of the tool's endpoints, and under real traffic it sat on the database long enough to matter. Requests queued up behind it. Connections stopped coming back. Then the failure stopped being about that one endpoint at all. Unrelated routes started timing out too, because there was nothing left in the pool for them to use. This is connection pool exhaustion, and it's why one slow query can look like a total outage.

This is what makes database saturation so hard to debug.

The symptom shows up everywhere except the place that caused it.

By the time anyone noticed, a dozen endpoints were failing and not one of them was the culprit. The tool was unusable, and everyone who depends on it was blocked.

How an Incident Like This Normally Goes

Someone notices the timeouts. Someone else opens the APM, sees latency up across half the service, and learns nothing about the cause. A third person pulls up the database dashboard, sees CPU pegged, and now there are two theories and no evidence for either.

More often on a small team, one person does all of it. They work through the APM, the database metrics, the error logs and the last few deploys, and they come out the other side holding two or three theories that all fit the symptoms equally well. That's the worst place to be. Every one of those theories needs concrete evidence before it's safe to act on, and getting that evidence means another pass through the same tools while the API is still timing out.

Then the real work starts: figuring out which of the day's deploys did it. That means reading commit diffs while the API is still down and Slack is filling up.

We've written before that roughly 70% of incident response time goes to coordination rather than engineering. A saturated database is that statistic in its purest form. Every dashboard is lit up and none of them is pointing at a commit.

How It Actually Went

Steadwing paged us in Slack. The alert didn't ask us to go and look at anything. It already had:

• The failing requests with the endpoint, method and request context that produced them.

• The query, and how long it took to run. The SDK keeps recent database queries as breadcrumbs, so the slow statement was already in the payload. Nobody had to go digging through logs for it.

• The commit that added the query. Every event the SDK sends carries the git SHA of the running build.

• The database's own numbers for the same window, lined up against the deploy. You could see load climb the moment it went out.

• And the fix. Don't run this query on every request. Run it only when the condition that needs it is true. That was clear enough to act on, so we turned it into a pull request.

The correlation is what separates a stack trace from an answer. Anyone can tell you a query is slow. What we opened was a slow query, the commit that added it, the database's own metrics agreeing on the timeline, and a change we could review.

8 minutes, error to root cause.

The fix took 4 more. We didn't optimise the query, add an index or touch the schema. The query just didn't need to run on every request. It only mattered when one particular condition was true, so we gated it behind that condition and shipped. Latency went back to normal on deployment.

One line of control flow. 12 minutes of degradation instead of an evening of it.

What Those 12 Minutes Bought

The number we keep coming back to isn't 8, or 4. It's zero. That's how many wrong holes we went digging in.

That part is hard to appreciate until you've had it. In the normal version of this incident, the first 20 minutes produce two or three explanations that are plausible and wrong, and each one costs you a detour. A query optimised that didn't need optimising. An index added to the wrong table. A rollback of the wrong deploy. Getting a finished analysis doesn't just save you the minutes on the clock. It saves you from the confident wrong turn.

We didn't get faster at debugging, we just arrived at the solution.

Both SDKs are open source under Apache 2.0, and instrumenting a service takes 3 lines at startup.

Python: pip install steadwing. Flask, FastAPI and Django are auto-instrumented, with SQLAlchemy and Django ORM query breadcrumbs.

Node: npm install @steadwing/node. Express and Fastify are auto-instrumented, and winston and pino error logs get picked up automatically.

Want your alerts to arrive solved? Check more here.

Github codebase:

• Python repo

• Node repo