Vertex AI Gemini 3.7 Flash: one video prompt repeatedly returns delayed HTTP 500, while a short prompt succeeds

Google Cloud Build with AI Generative AI & Foundational Models
1

Hi, I’m seeing a reproducible failure with gemini-3.7-flash on Vertex AI Standard PayGo. These are generateContent calls through aiplatform.googleapis.com in the global location, using the Google Gen AI Python SDK with agentic media processing enabled. Each request references one of three fixed GCS-hosted MP4 videos, each about 30 minutes long.

I compared requests using the same videos, project, model, endpoint, settings, and maximum concurrency of three:

  • A short, 45-character prompt succeeded on 42/42 requests across two runs (HTTP 200).
  • Our production prompt, approximately 9,084 characters, returned HTTP 500 on all 14 completed requests in the latest run. Each call ended after approximately 6–18 minutes with 500 INTERNAL: Internal error encountered.

SDK retries were disabled, and HTTP transport logging recorded one HTTP request per attempt. These were actual API responses, not client-side timeouts. We’ve also seen this prompt fail at different times of day, so it does not appear limited to a peak-traffic window. The short prompt succeeds with the same videos. Also, both prompts succeed reliably if the video part is media_processing="static", the error only occurs with media_processing="agentic"parts.

Has anyone encountered similar behavior? Are there known interactions between Gemini 3.7 Flash, agentic video, and particular prompts that could explain a delayed 500? The generic error gives me no visibility into what happened within the API call, and I would appreciate any help figuring this out. Thank you!

2

Hi @agupta, That delayed 500 error is definitely an internal timeout / OOM on Google’s backend. Running media_processing=“agentic” on a 30-minute video forces the model into iterative sweeps over the timeline. When you combine media that long with a ~9k character prompt, the internal TPU worker runs for 15+ minutes until it exceeds its execution boundary and collapses. Since media_processing=“static” works cleanly for both prompts, that is your best bet right now. Another solid approach is doing it in two steps: use static mode first to get a quick summary/transcript with timestamps, then run your heavy 9k prompt against that extracted text. That runs in seconds and avoids hitting the backend timeout ceiling.

3

When media_processing is set to "agentic", Gemini 3.7 Flash doesn’t execute a single forward pass over the video. Instead, it enters an iterative orchestration loop: sampling frames, evaluating scene cuts, running tools/code internally, and checking progress against your instructions.