Why an apidays workshop proposal had to be replanned from the requirements up
I'm currently building the live demo behind a workshop proposal for apidays Australia 2026 - a session on securing AI agents and MCP tool calls with Auth0 and OpenFGA-style authorisation. I'm building it with Claude Code doing the actual typing: I set direction, review what comes back, and make the calls; Claude writes the code, runs the tests, and drafts the plans I then pull apart. Partway through turning the proposal into an actual build plan, I had Claude run an adversarial review of the plan itself before either of us wrote a line of application code, and the review didn't come back with a punch list of nits. It came back with a structural problem in the authorisation model at the centre of the whole talk, one that meant the demo as designed could never have shown what the proposal claimed it would show. In this post I will walk through what that flaw was, how two independent reviews both landed on it from different angles, and why the fix wasn't a patch but a genuine restart from the requirements - and I'll be explicit throughout about which parts of this were me making a call and which were Claude doing the work.
The premise the demo was supposed to prove
The workshop's core message is "who is this AI agent acting for, and is that authorised right now" - a citizen signs up for a fictional government portal, talks to an AI assistant about their own tax record, and across three acts the demo shows an insecure baseline, a fix that retires it, and a scoped delegation grant the citizen can revoke live on stage. I set that structure. The mechanism doing the actual authorisation work in that story is Auth0 FGA, Auth0's hosted relationship-based access control product, which I chose because it's the product the workshop is meant to be about.
Claude wrote the first draft's FGA model, and the check it produced was user-only: it asked whether the signed-in citizen owned the tax record they were asking their assistant about. That's a perfectly reasonable check for a session-authenticated web app, and it will always return true for a citizen's own data, because an owner has trivial access to their own record by definition. Which is the entire problem: there was nothing for the AI agent to be denied, and nothing to delegate, because the model only ever asked about the human, never about the agent making the call on the human's behalf. I didn't catch that when I read the draft. Neither did Claude, when it wrote it.
What a solo adversarial pass caught, and what it missed
Before bringing in outside review, I had Claude run its own adversarial pass over the first draft, and it did catch real things - a mismatch between a stated 50-minute session and a described 60-minute run of content, and a contractor/admin MCP vignette that broke the citizen persona the rest of the talk had built. Both were genuine issues and both got fixed. Neither was the flaw that mattered most. A single reviewer, even a careful one, tends to catch the problems adjacent to what it's already looking at, and misses the ones sitting one layer deeper in the model itself - and that was true whether the reviewer was Claude checking its own work or me reading it afterwards. I'd read the same draft and not seen it either.
Two independent reviews, one shared finding
The next pass sent the full build plan to two reasoning-tier models via the internal LiteLLM proxy - gpt-5.4-pro and gemini-3.1-pro-preview. I picked those two specifically because they're the reasoning/architecture tier, not the fast/mechanical tier I have Claude reach for elsewhere for things like Auth0 Management API scripting - that model selection is my call, made in advance, not something decided per-task. Both came back, independently, with the same core finding, in almost the same words. One put it about as plainly as it can be put: "you aren't doing agent authorization, you're doing standard user session authorization, and the LLM just happens to be the UI calling the API."
Neither model needed to be told to look at authorisation modelling specifically - both traced the actual relation chain in the FGA model and noticed there was no second principal anywhere in it. A user-only model checking a citizen against their own document has exactly one identity in the whole check. Delegation needs two: the person granting access, and the thing being granted access to act on their behalf. Without a distinct agent principal, "the citizen delegates access to the assistant" isn't a permission the model can express, let alone deny and then grant live on stage.
The fix, once named, is a small change to the schema and a much bigger change to which principal gets checked at runtime. I decided on the shape of the fix - a distinct agent type, delegation as a revocable tuple - and Claude implemented it:
type user
type agent
type document
relations
define owner: [user]
define delegate: [agent]
define viewer: owner or delegate
The check the secure chat tool makes is check(agent:smartgov_assistant, viewer, document:tax_<citizenSub>), where citizenSub is the citizen's own subject identifier from their login token - the agent's relationship to the document, never the citizen's own. Before delegation, that check is denied even though it's the citizen's own data, because the agent has no grant yet. Flipping "Delegate access to AI Assistant" writes a scoped tuple; flipping it off deletes it and the very next agent-mediated request is denied again. That's the only version of this model where "revoke access live on stage" is actually revoking anything.
The other findings, and why they mattered less
Both reviews also surfaced a longer list of real infrastructure risks that hadn't shown up in the first draft at all: a thundering-herd risk if roughly a hundred phones unlock the secure chat path within the same few seconds and all fire Bedrock requests at once, conference Wi-Fi triggering Auth0's bot detection on a burst of signups from a single shared egress IP, Vercel's serverless function limits being flagged as a poor fit for a long-lived MCP server connection during a five-minute live vignette, and a closing QR code that would have pointed post-event visitors at the same deployment used live on stage - inheriting whatever room state the presenter had left behind. Three of those got fixed as flagged, with Claude implementing each: staggering load behind a human-click unlock rather than an automatic trigger, allowlisting the venue's IP range ahead of time, and splitting the live event and the public standalone copy into two separate deployments.
The Vercel one is worth a longer look, because it turned into its own small version of the same mistake - and this time it's one I need to own directly, because the mistake was mine, not Claude's. The connection the MCP server holds open for the live vignette doesn't need to carry any state between messages, which made "serverless function limits are a poor fit for a long-lived connection" sound like settled wisdom rather than something to check. I asked Claude to validate Vercel Sandbox specifically - the ephemeral-compute product built for exactly this kind of longer-running workload - because that's the primitive I'd read about and wanted tested. What Claude actually built and tested first was a Vercel Function with a 300-second maxDuration, streaming a heartbeat every five seconds over SSE - a different product entirely. I didn't catch the substitution when I reviewed the result, and it was never flagged to me as a scope change. A 60-second run held 14 of 15 expected heartbeats with zero errors; a 5-minute run went the distance at 301 seconds, 59 of 60 heartbeats, zero disconnects. On the strength of that, the plan's own summary - which I approved - marked the hypothesis validated and moved the MCP server onto Vercel Functions.
It wasn't validated. A follow-up soak test against the same Functions endpoint, run for the workshop's actual multi-hour duration, failed outright: every one of 100 simulated clients disconnected at almost exactly the 300-second mark, because maxDuration is an architectural ceiling on Vercel Functions, not a setting you can tune around. Only after that failure did Claude build a real Sandbox test - and the first attempt at that failed too, on a deploy script bug that only started the sandboxed server on resume, so a freshly created sandbox never launched it at all. The corrected run, once Claude fixed that, delivered 99.8% of expected heartbeats over fifteen minutes and got recorded as a pass. But the raw result for that same run shows all 20 clients logging errors and none still connected when it ended, and once I asked about it, the reason turned out to be sitting in the test script itself - the one Claude had written. The pass/fail logic marked a run successful if either every client stayed connected or the heartbeat ratio cleared 90%, not both, and it counted the deliberate destroy() call at the end of the timed run as a client error. So a clean, intentional teardown got misclassified as 20 failures, and the heartbeat ratio alone was enough to call it a pass regardless. Claude wrote that test, ran it, and reported the pass without either of us noticing the scoring bug underneath it. I only found it because I went back and asked why a "passed: true" result had 20 errors sitting right next to it - that's the part that was on me, and it should have happened before the result went into the plan, not after.
The Sandbox result may well be a genuine pass in substance - the heartbeats really did arrive - but the test that's supposed to prove it has a scoring bug, and neither of us caught that before "pass" went into the plan. One of the plan's own approval documents still lists the full multi-hour Sandbox test as outstanding rather than closed. None of that discredits Sandbox as the right primitive; it's a genuinely different piece of infrastructure with its own timeout model, and the theory behind switching to it is sound. "We ran a test and it passed" is a claim about the test, not about the thing you meant to test. That gap survived several rounds of documentation, and it survived the person who wrote the test also being the one reporting the result. It only closed when someone downstream went back and read the raw numbers instead of trusting the summary.
All four risks were worth taking seriously, and three are closed. The Vercel question is still technically open pending a proper multi-hour Sandbox run, which is a normal thing for a build-in-progress to have outstanding. None of the four is why the plan had to restart from the requirements rather than get patched, though - a hosting risk is the kind of thing you measure and fix (or leave as a tracked open item) in the build phase without touching what the demo is claiming to prove. A model that can't express the thing you're demonstrating is a different category of problem entirely - it's not a bug in the plan, it's a hole in the premise.
Ruling out the obvious explanation
Given how deep the FGA flaw sat, the obvious next question is whether it traces back to using a cheaper model somewhere in the planning chain - the classic false economy of reaching for Haiku on a task that needed more reasoning. Checking that was my call, not Claude's: every model reference in the build plan's own review notes is gpt-5.4-pro or gemini-3.1-pro-preview, both explicitly the reasoning/architecture tier per my own model-selection guidance, and Haiku doesn't appear anywhere in this stage of the work at all. I've deliberately scoped Haiku to later, mechanical Auth0 Management API calls during tenant provisioning - a separate and much less consequential job. The real cause was simpler: the plan had only ever been reviewed by its own author, first Claude and then me, until an independent pass happened. Even a strong reasoning model would have had nothing to disagree with if it had only been shown the finished demo script instead of the actual FGA schema underneath it.
Final Thoughts
The lesson I'm taking into the rest of this build is narrow: when a demo's core claim rests on an authorisation model, review the model directly, not the narrative wrapped around it. When a test reports a pass, read what it actually measured before trusting the label. Claude did the writing, the testing, and a good share of the catching in this story - the FGA fix, the soak test that disproved the first Vercel result, the deploy-script bug in the Sandbox retry. The two places it fell short were also the two places I hadn't set up a check: nobody had reviewed the FGA model against its own relation chain until I asked for that specifically, and nobody had read the Sandbox test's raw numbers against its own pass condition until I went looking. Both gaps were mine to close, and closing them is what's now driving the live build ahead of the workshop in Melbourne this October.