The AI Wrote Code That Compiles, Runs, and Is Wrong: A Debugging Workflow
A practical three-tier workflow for catching the subtle bugs, boundary omissions, and security gaps AI coding assistants keep shipping.
The AI Wrote Code That Compiles, Runs, and Is Wrong: A Debugging Workflow
More than 90% of developers now use an AI coding assistant daily. Over 65% of them will also tell you, if you ask, that the code it hands back is often "almost right" — and almost right is the expensive kind of wrong. You don't catch it on read-through. You catch it three sprints later when someone's pagination silently drops the last row.
I've hit this enough times across client SaaS builds to have opinions about it, so here's the actual workflow I run instead of just eyeballing the diff.
Why "almost right" is worse than "obviously wrong"
Broken code that doesn't compile gets caught in five seconds. The dangerous stuff is different: it's syntactically clean, idiomatically styled, and passes a casual review because it looks like something a competent engineer would write. That's exactly the problem — AI models are pattern-matching against millions of examples, not executing your code against your actual database schema or your actual third-party API version.
Three patterns show up over and over.
Hallucinated APIs and parameter assumptions. The model invents a config property that sounds plausible — timeoutMs on a client that actually expects timeout in seconds — and if your types are loose, it compiles fine and fails silently at runtime. This is the one that burns the most hours, because you're debugging a value that was never wrong syntactically, just wrong semantically.
Edge case blindness. AI assistants write the happy path beautifully and then quietly skip the boundaries: off-by-one errors in pagination loops, unhandled null/undefined from a database query that didn't find a match, no retry or backoff on a third-party call that will eventually rate-limit you.
Permissive-by-default security gaps. Nobody's model is suggesting SQL injection outright anymore. But it will default to origin: "*" on your CORS config because that's the fastest way to make the demo work, skip sanitization on a dynamic ORM filter, or reach for a non-cryptographic random generator for something that's actually a token.
None of these are exotic. They're boring, unglamorous, and exactly the kind of thing a rushed human reviewer skims past.
A three-tier verification workflow
The fix isn't "review AI code more carefully" — that's not a process, that's a hope. What works is putting mechanical checkpoints between the AI's suggestion and your main branch, in this order:
AI Suggestion → Type System Validation → Automated Unit Harness → Security Linter
Tier 1: make the compiler do the first pass
This is the cheapest tier and it should be non-negotiable. Turn on noImplicitAny, strictNullChecks, and exactOptionalPropertyTypes in tsconfig.json. If you're in Python, that means actual type hints plus mypy --strict, not decorative annotations nobody enforces.
A strict compiler catches the hallucinated-method and wrong-parameter-type class of bug before a human even opens the PR. It won't catch a wrong-but-valid boundary condition — for that you need tier two.
Tier 2: flip the order and make the AI write tests first
This is the tier that actually moves the needle, and it's the one most teams skip because it feels slower. It isn't, once the AI is generating both halves.
The sequence: ask the assistant to write unit tests covering the happy path, empty input, boundary values, and error states — before it writes any implementation. Review those tests yourself. Then, and only then, have it write code to satisfy the tests you already approved.
The reason this works better than reviewing finished code is that it forces the edge cases into a form you can actually evaluate quickly — a list of test cases — instead of forcing you to mentally simulate every branch of someone else's (or something else's) logic. You're reviewing intent before you're reviewing implementation.
Tier 3: let static analysis catch what review misses
Run security scanners — Semgrep, Bandit, ESLint's security plugins — on every PR that touches AI-generated code, no exceptions. Also audit your dependency lockfile. AI assistants occasionally hallucinate a package name that doesn't exist, and if that name happens to get squatted by someone malicious, you've just added a supply-chain vulnerability to your build (this pattern has a name now: slopsquatting).
This tier is your safety net, not your first line of defense. If tiers 1 and 2 are doing their job, tier 3 should mostly come back clean.
Prompt constraints that prevent the problem upstream
Some of this is fixable before generation even happens. When you're asking for anything beyond trivial glue code, put boundaries in the prompt itself:
- No external dependencies outside an approved list you specify.
- Explicit error handling required for every null/undefined return path.
- Unit tests required, covering empty arrays, boundary limits, and timeout scenarios.
- Strict typing — no
any, no loose type assertions.
None of this is exotic prompt engineering. It's just refusing to let the model default to the laziest version of "working."
The mental model that actually matters
Treat the AI assistant as a fast junior engineer, not an autonomous author. A junior who ships clean-looking code that's subtly wrong isn't a junior you stop reviewing — it's a junior whose code you review more carefully, at the exact places juniors reliably get wrong: boundaries, error paths, and defaults. Strict types, test-first prompting, and CI-level security scanning aren't extra overhead on top of AI-assisted development. They're what makes the speed actually usable instead of a debt you pay back later, with interest, in production.
Accelerate your Software Engineering Modernization Roadmap
Need custom architecture auditing, automated OpenAPI contract generation, or zero-downtime microservice migration guidance for your engineering team?
Frequently Asked Questions
Why does AI-generated code look correct but fail in production?
AI models predict plausible-looking code from patterns in training data, not from actually running it against your specific APIs and edge cases. The syntax is fine and the logic reads cleanly, but assumptions about method signatures, null returns, or default configs often don't match reality.
What's the single highest-leverage fix for catching these bugs?
Making the AI write tests before implementation. It forces the boundary cases into the open before any code exists to hide behind, and it gives you something concrete to review instead of trusting the logic on read-through.
Subscribe to RenovateAPI
Get weekly architectural guides, API refactoring strategies, and technical SEO updates delivered directly to your inbox.
Discussion (2)
Extremely helpful breakdown of the Strangler Fig pattern! We're currently refactoring a legacy Java monolith at work and the OpenAPI gateway routing tips saved us weeks of experimentation.
The schema JSON-LD and FAQ block structure really helps with indexing. Great technical detail on entity mentions too.
Suggested Related Articles
Your AI Assistant Isn't Getting Dumber Mid-Session — It's Running Out of Context
Why long AI coding sessions quietly degrade, and four concrete practices — repo rule files, short sessions, symbol ingestion, ADRs — that fix it.
Building a Google Maps Business Location Picker in React (With Nearby Business Search)
A practical React + TypeScript guide to a Google Maps location picker: reverse geocoding, nearby business discovery via Places API (New), custom markers, and the errors that trip everyone up.
Your Blog Isn't Getting Cited by AI Search — Here's the Structure That Fixes It
A practical breakdown of Answer Engine Optimization (AEO): how to format headings, code blocks, and tables so ChatGPT Search, Perplexity, and Google AI Overviews actually cite your content.