Google Cloud Claude demo vs reality: five gates a live product has to ship that a Vertex AI showcase cuts

Gate one. The 15 second time-to-first-token watchdog.

When the user submits a prompt, the most common live failure is not a wrong answer. It is no answer at all. The ACP subprocess accepts the request, then never emits a notification. From the browser the stream is open but empty. The tab is staring at a blinking cursor.

At src/app/api/chat/route.ts line 539 a setTimeout fires 15 seconds after the request begins. If the loop has not seen any notification by then, the code evicts the session, fires the chat_ttft_timeout PostHog event, and sends the user the literal string “Agent did not respond within 15s. Please retry.” That string does not appear in any Vertex AI showcase, because no showcase ever waits 15 seconds for a token that never comes. It exists because in production sometimes the upstream is queued or the local subprocess is wedged, and refusing to time out is worse than admitting it.

Gate two. The 800 second per-turn ceiling, inside Cloud Run's 3600 second sync max.

Cloud Run documents the maximum synchronous request timeout at 3600 seconds (60 minutes). The deploy step in cloudbuild.yaml passes --timeout=3600 at line 41 explicitly. That is the ceiling. The per-turn cap lives below it, at maxDuration = 800 on src/app/api/chat/route.ts line 25. 800 seconds is 13 minutes 20 seconds.

Two reasons for the gap between 800 and 3600. The first is that the rest of the request lifecycle (auth checks, Firestore writes, PostHog flush, retry margin) needs headroom under the Cloud Run ceiling. The second is product judgment: anything a user will sit through synchronously caps out around 13 minutes. Longer than that and the work belongs on a scheduler, not on a chat handler. mk0r's sandbox has one at/opt/scheduler-mcp.js for that case.

A Vertex AI showcase that shows “an autonomous agent runs for an hour” is either on a longer-lived transport (no browser tab waiting), batched, or cut. Production on Cloud Run cannot have it both ways: synchronous chat tops out at 60 minutes, period.

Gate three. flushServer() before every response close.

This one is the most Cloud Run-specific gate of the five. The comment at src/app/api/chat/route.ts line 202 says it directly: “Cloud Run throttles CPU to ~0 after the response closes, so posthog-node's queued HTTP POST never fires unless we wait for the network call here.”

Translation: on a normal server, libraries like posthog-node queue events and flush them in the background. Background CPU exists. On Cloud Run with default CPU allocation, the moment your response closes, CPU drops to almost nothing for that instance. The flush task that posthog-node scheduled never actually runs. Your telemetry silently disappears.

The fix is one line: await flushServer(); before every return Response.json(...) or every controller.close(). It blocks the response by a few hundred milliseconds in the worst case. The alternative is gates one and four going dark in production because they fire telemetry events that never land.

You will not see this in any Vertex AI showcase notebook because a notebook is not a request handler. The kernel keeps running. CPU is whatever you provisioned. This gotcha is exclusively a Cloud Run problem, and it bites every team that migrates a working handler from a long-running server to serverless.

Gate four. A six-class SDK error classifier.

At src/app/api/chat/route.ts line 48 the type union is six names long:

type ErrorKind =
  | "credit_exhausted"
  | "auth_required"
  | "invalid_request"
  | "image_error"
  | "stale_session"
  | "generic";

The classifier at line 50 maps from the SDK's structured api_retry info (HTTP status, errorType) plus a regex fallback onto one of the six. Each class has a different next action:

• credit_exhausted: top up. The SDK's reset-time message is surfaced verbatim so the user knows when service resumes.
• auth_required: re-authenticate. Not retry the same prompt.
• image_error: resize the image. Not refresh the page.
• invalid_request: edit the prompt. The original error message is shown.
• stale_session: the only kind the code automatically retries, and only before any text has streamed.
• generic: the fallback for anything the classifier did not match.

A page that collapses every failure into one banner pushes every user down the wrong recovery path. The cost of being honest about which kind of failure happened is six branches in the error handler and six rendered states in the UI. No demo pays this cost because no demo ever fails.

Gate five. Stale-session retry exactly once, only before text streams.

Internal errors from the ACP subprocess usually mean the SDK lost track of the session, often after a long idle. The code at src/app/api/chat/route.ts lines 621 to 653 handles this: restart the ACP subprocess, reload the session state, re-prompt once. The retry only fires if textDeltaCount === 0, meaning the user has not seen any output yet.

Once text is flowing, retry corrupts the turn (the user sees half an answer, then a different half an answer). So the retry is a narrow guardrail, not a panacea. The Vertex AI marketing line about “global endpoint with dynamic routing for maximum availability” is true at the transport layer. It does not absolve a Claude-backed product of having to ship its own retry policy at the application layer, because session state lives in the SDK, not in the global endpoint.