Skip to content
DAagentic-workflowengineering-judgment

The gate is you, part 2: production voted no

We shipped the 'right' fix and I wrote a blog post about catching the AI. Next morning production was still broken — and the root cause was myth number three.

Yesterday I wrote a post about catching the AI in a wrong diagnosis. Three gates, a whoami test, source-code research — and finally the "real" root cause: Next.js doesn't trust the Host header by default. I published the post in the evening, we committed the fix and deployed.

22:07: "yes commit push and deploy". 05:22 the next morning: "where are we?"

Production was still redirecting to https://host:3000/en.

The rabbit hole, day 2

There is a particular feeling to watching your own verified, source-code-backed, blog-published fix fail in production. The fix wasn't guesswork, either: we had read the line in Next.js' source, runtime-tested the helper against the real NextRequest constructor, gotten green from tsc and eslint. All of that still held. It was just the answer to the wrong question.

The morning went to more digging in the same session — and this is where I started suspecting something other than the code:

human promptprompt
6 lines2026-07-03 07:57
I am abit conserned here that we really dont undestand nextjs and intl stuff here. can we make a small reproduction where we setup a minimal newest version nextjs and we can show the diffrent cases to make sure we understand. i dont get why experimental.trustHostHeader did not solve the problem as expected? I am wondering if we have mudded our context and we should do a prompt for a fresh agent/session to reporudce it, so can you generate the prompt for me that we want a second agent to run and reproduce our hyposisis.

"Have we mudded our context" is the central sentence. After a day and a half in the same conversation, the context was full of half-true conclusions: the disproven ACA theory, the built-and-discarded middleware, the found-but-untested trustHostHeader line. Every new turn in the conversation stood on top of all of it. The AI was no longer reasoning freely — it was reasoning inside our shared history of mistakes. And so was I, the way you do when you've stared at the same problem long enough.

The gate this time wasn't a question to the AI. It was throwing the context away.

The clean-room agent

Instead of fixing onward, I asked the muddied session for one last thing: write the prompt for a fresh agent that will re-examine everything we think we know. The result was an investigator brief built on falsifiable hypotheses — here is the opening and the five claims (the full brief is an artifact):

human promptprompt
29 lines2026-07-03 11:56
You are a clean-room investigator. Do NOT trust any prior analysis — I will give you a hypothesis to confirm or refute with your own empirical evidence. If you cannot reproduce a claim, say so. Every conclusion must be backed by a command output or a quoted line of source code you actually read, not by reasoning alone. ## Background (the real-world symptom) A Next.js app that uses next-intl for i18n routing is deployed as an output:'standalone' container behind Azure Container Apps (ACA). The container listens on port 3000; ACA's ingress terminates TLS on 443 and reverse-proxies to it. In production, GET / returns a 307 redirect whose Location is: https://<public-host>:3000/en ← the internal port :3000 leaks into the public URL ## Hypothesis to test (confirm or refute each point independently) 1. next-intl builds the locale-redirect's PORT solely from the x-forwarded-port request header (not from Host, not from request.url). 2. In an output:'standalone' server, Next.js BACKFILLS x-forwarded-port with its own listen port (3000) using nullish-assignment (??=) — it only fills the header when the proxy did NOT send one. Proxy sends x-forwarded-port → Next respects it; proxy omits it → Next injects 3000. 3. ACA's ingress sends X-Forwarded-Proto but does NOT send X-Forwarded-Port (nor X-Forwarded-Host). That omission is what triggers the leak. 4. experimental.trustHostHeader does NOT fix this in current Next.js standalone — its code branch is effectively dead when the standalone server sets HOSTNAME+PORT. (This is the point I am most unsure about — investigate it hardest and tell me the truth.) 5. The correct fix is to normalize x-forwarded-port in middleware (proxy.ts) from the trusted x-forwarded-proto (https→443, http→80) — the Next equivalent of ASP.NET Core's UseForwardedHeaders.
See what Claude did next

Notice the form. It isn't "fix my bug". It's five numbered, falsifiable claims, a requirement that every conclusion be backed by command output or quoted source code, and an explicit invitation: "Be skeptical. If any hypothesis is wrong, I specifically want to know — that's the point." The brief even carried the gotchas the old session had paid to learn (Turbopack + symlinked node_modules), so the new agent wouldn't pay for them again.

Then a fresh session with an empty context ran the brief like a scientific protocol: minimal reproduction from scratch, four curl cases, source-code quotations, a version matrix.

The verdict: myth number three

The new agent came back with something neither of us had on the list. The leak is version-bound to next-intl 4.10.1–4.11.1. Introduced in PR #2281 ("Set redirect domain if x-forwarded-host header exists"), reverted in PR #2322. Our deployed app pinned next-intl 4.11.0 — right inside the window. The latest stack (next 16.2.10 + next-intl 4.13.1) doesn't leak at all; there, the Location comes out relative (/en), so a port can never appear.

The case matrix from the reproduction, on exactly the combination we had in production (verbatim from the clean-room report):

┌─────┬───────────────────┬───────────┬──────────┬─────────────────────────────────────────────────────┐
│  #  │       Host        │ X-F-Proto │ X-F-Port │                  observed Location                  │
├─────┼───────────────────┼───────────┼──────────┼─────────────────────────────────────────────────────┤
│ A   │ (direct)          │ —         │ —        │ http://127.0.0.1:3201/en                            │
│ B   │ public.example.io │ https     │ — (ACA)  │ https://public.example.io:3201/en ← leak reproduced │
│ C   │ public.example.io │ https     │ 443      │ https://public.example.io/en                        │
│ D   │ public.example.io │ https     │ 3000     │ https://public.example.io:3000/en                   │
└─────┴───────────────────┴───────────┴──────────┴─────────────────────────────────────────────────────┘

And then the two humbling details.

One: experimental.trustHostHeader — yesterday's published root cause — was refuted empirically. Next 16's build rejects the key ("Unrecognized key") and bakes false into the standalone output. The agent even force-edited true into the generated server.js and required-server-files.json — the leak persisted. The switch I was so pleased to have found in the source code cannot even be turned on.

Two: yesterday's whoami test was correct — and the conclusion we drew from it was still wrong. whoami proved that ACA doesn't send x-forwarded-host to the container. But Next.js' standalone server backfills x-forwarded-host and x-forwarded-port itself, internally, before middleware ever sees the request. One debug line from the reproduction tells the whole story — curl sent neither a forwarded host nor a port, yet the middleware saw both:

DEBUG_MW {"requestUrl":"https://0.0.0.0:3201/","host":"public.example.io","xfHost":"public.example.io",
          "xfProto":"https","xfPort":"3201","location":"https://public.example.io:3201/en"}

We had measured the right layer and still concluded about the wrong one. next-intl 4.11.0 always saw an x-forwarded-host (Next's own backfill), so it always took the unconditional forwarded branch, and set the port from the backfilled x-forwarded-port — the listen port. Which is also why yesterday's normalizeForwardedUrl didn't work: it rewrote request.url, but that version of next-intl reads the headers, not the URL.

The verified fix: normalize the x-forwarded-port header in the proxy (https → 443, i.e. empty port) before next-intl runs — B, C and D all yield a clean https://public.example.io/en, and a POST body survives the recreated request. Or simpler still: upgrade next-intl to ≥ 4.11.2. Sometimes the right answer to the rabbit hole of the week is npm update.

So who won, round 2?

Yesterday I closed with a tally: I caught the AI, the AI found the right line, Anthropic got tokens for both. That tally needs a revision. The "right line" was myth number three, and my own sense of victory — published on this blog the same evening — was part of the mud.

The lessons, as I see them now:

The context is part of the system. After enough hours in the same conversation, the session's memory has itself become technical debt: layer upon layer of almost-right conclusions that every new thought builds on. The gate instinct from part 1 — "this smells" — applies to the conversation itself. When you can no longer tell what you know from what you've said many times, the cheapest move is a fresh context.

Falsifiable hypotheses are the handover format. What made the clean-room run valuable wasn't just the empty context — it was that the muddied session was forced to distill a day and a half of mess into five claims that could be wrong. The writing is the debugging. And ask explicitly to be told when a hypothesis falls; otherwise you get confirmation instead of investigation.

"Verified" has layers. The whoami test was good empiricism and is still true. But between "what the proxy sends" and "what the library reads" sat a server that makes up headers. Every time you verify something, ask: did I measure the layer my conclusion is about?

And the tally: the engineer caught the AI (part 1). Production caught the engineer (part 2, night). The clean-room agent caught them both (part 2, day). Anthropic invoiced all three rounds — this time including an ultracode run. I'm starting to sense a pattern in who wins.

But here's what I'm actually buying for the money: a failure process that converges. Each wrong theory became more expensive to maintain and cheaper to topple, because every layer left evidence behind — a whoami dump, a version matrix, a debug line. That's not infallibility. It's science in small portions, billed per token. And unlike yesterday, I won't promise there's no part 3.