What Claude Code got wrong building the apidays demo (and how I caught it)

In part one of this series I detailed a structural flaw an adversarial review caught in the Auth0 FGA model behind my apidays Australia 2026 workshop demo, and why fixing it meant restarting the plan from the requirements rather than patching around it. That post was about planning. This one is about building - specifically, two moments where Claude Code, doing the actual implementation while I directed and reviewed, got something wrong in ways I only caught by asking a plain question rather than trusting the summary in front of me. I'll be explicit throughout about who did what, the same as last time.

The MCP server that never needed to stay open

The build plan called for a Model Context Protocol server hosted somewhere other than Vercel's own serverless functions, on the reasoning that Vercel Functions have a hard execution-time cap (15 seconds on Hobby, 60 on Pro) and a live vignette needs a connection held open for longer than that. Claude had already spent real effort testing Vercel Sandbox as the alternative to Render or Fly.io, including a soak test that found the first version of that test had actually validated a Vercel Function, not a Sandbox at all, then a second test that fixed a deploy-script bug and produced a result recorded as passed: true - except the raw output showed all 20 test clients logging errors, because the test script's own pass condition only required one of two checks to succeed, not both, and it had miscounted a clean shutdown as a failure.

I asked Claude to explain that discrepancy, and while it was digging through the scoring bug I asked a different question: why does this MCP server need to hold a connection open for hours in the first place? The FGA model re-checks delegation on every single tool call - that's the whole point of the workshop's premise, an agent's access can be revoked mid-conversation and the very next call has to reflect it. If every call is independently re-authorised anyway, there's no session worth keeping alive between calls. Claude checked two other projects I have running - an MCP server in one, a long-lived webhook processor in another - and both turned out to be short, stateless, on-demand function invocations dressed up to look persistent from the outside. Neither held a connection open at all.

The fix reversed the entire hosting decision. The MCP server now ships as a plain, stateless Vercel Function, same deployment as the rest of the app, no Render or Fly.io account, no Sandbox account, nothing new to provision. The build plan's own "MCP hosting account" open item - something to decide before launch - got deleted outright rather than resolved, because the question it was asking no longer applied. Stateless doesn't mean limitless - a real spike in tool-call volume would still need the usual serverless-scaling answers, rate limits and concurrency included - but that's a much smaller, better-understood problem than keeping a bespoke long-lived process alive for a five-minute vignette.

None of the Sandbox testing was wasted effort exactly - Claude's writeup of it stays in the plan as a note for if a genuinely long-lived connection is ever needed somewhere else in this build - but it's a clean example of testing an assumption harder without ever checking whether the assumption itself was true. A rigorous test of the wrong premise still gives you the wrong premise, just with more confidence behind it.

The database I said I didn't want, twice

The second miss is the one I'm less comfortable with, because it's a case of Claude explaining a decision well instead of questioning whether the decision should have existed.

The build plan is specific about the stack: Auth0 for identity, Auth0 FGA for authorisation, Firebase/Firestore for app state, all deployed on Vercel. I'd said more than once, in plain language, that I didn't want extra services beyond that set. Somewhere during implementation, a fourth provider - a hosted Postgres database on Neon, wired up through Drizzle ORM - had been added directly into the codebase. Not proposed, not flagged, just present. It was installed as a dependency and given a real schema for the citizen's case record, their AI agent's identity, and the audit log of every FGA decision. Nine separate files ended up depending on it.

I found the DATABASE_URL line and asked what it was for. Claude answered accurately - it explained Neon's pooled versus direct connection strings, why the pooled one matters for a serverless workload that can spin up many concurrent function instances at once, all of it technically correct. What it didn't do was notice that the question underneath my question was "why do we have a database the plan never asked for, using a provider I've explicitly ruled out." I had to ask that directly before it got addressed at all.

The fix was a full migration, not a patch: Neon, Drizzle, and the DATABASE_URL variable came out entirely, and the case record, agent identity, and audit log moved to Firestore collections with the same shape and the same behaviour. One thing came out of the correction that hadn't existed before the mistake was caught - the Auth0-to-Firestore identity bridge the plan had specified since the very first draft, which nobody had actually built yet, because the Neon detour had been quietly standing in for it. Fixing the wrong decision surfaced a piece of real work the wrong decision had been letting us skip.

Turning this into something that outlasts the session

Both misses share a shape: Claude did competent, technically accurate work on the thing directly in front of it - testing Sandbox properly, explaining Neon's pooling model correctly - without stepping back to check whether the thing in front of it should have existed. Competence at the task isn't the same as noticing the task was the wrong one, and an AI collaborator will generally answer the question you asked rather than the constraint you'd already stated. That's not a reason to stop delegating the actual work; Claude wrote the Firestore migration cleanly once redirected, and the stateless-MCP fix held up under a type-check and a full sweep for leftover references. It's a reason to keep asking "why does this exist" out loud, on a schedule, rather than only when something breaks.

A lesson that only lives in my head until the next session starts isn't much of a lesson, so I had Claude write both of these into my global CLAUDE.md - the file it reads at the start of every project, not just this one. Two of the new rules map straight onto the stories above:

Before spending effort validating or implementing a technical choice, check it against the user's own stated constraints first, not just against whether it will technically work. If the user has said "no extra services," "only use X," or similar more than once, and something in the codebase or plan contradicts that, say so before doing more work on it - don't wait to be asked what it's for.

Before accepting a test result as fact, especially one written into a plan or memory as a settled finding, check the pass/fail condition itself, not just the headline verdict. This is more likely to slip through on tests I wrote myself, since I have no independent check on my own scoring logic - treat self-written test harnesses as needing the same scrutiny as the code they're testing, not less.

A third, smaller rule came out of the repo-hygiene pass that led into this build: check that anything meant to be a generic template doesn't have this project's real client IDs and store IDs baked into it, and treat any secret that's ever been pasted into a chat transcript as compromised whether or not anyone confirms it leaked. Neither of those is a guarantee against the next version of the same mistake - a written rule only works if something actually checks it against, and the checking still has to be a habit, not a file. But it's a cheaper place to start than hoping I remember to ask the right question next time.

Next Steps

Part three will cover the parts of the build that didn't need correcting - the FGA delegation model actually working end to end, and what the live vignette looks like once an agent's access gets revoked mid-conversation. If you're building something similar, the OpenFGA modelling post I wrote earlier in this series is a good companion piece for the authorisation side of things.