Error when use context caching with gemini-1.5-flash-001

Google Cloud Build with AI Vertex AI - Custom ML & MLOps
1

I am getting the following error when using context caching following vertextai’s tutorials:

InternalServerError: 500 Model gemini-1.5-flash-001 does not support cached content.

Please see the code snippets enclosed.

Name: google-cloud-aiplatform
Version: 1.57.0

SCR-20240702-juan.png
SCR-20240702-juan.png860×739 48.9 KB

image.png
image.png860×380 32.8 KB

Appreciate any help.

1 Like
2

Yeah same issue here, seems to be a totally random restriction they’ve placed on the Vertex AI version of gemini-1.5-flash but not the Google AI Studio version. See the difference in listed model support between

https://cloud.google.com/vertex-ai/generative-ai/docs/context-cache/context-cache-overview#supported_models and https://ai.google.dev/gemini-api/docs/caching?lang=python. What a pain, I just migrated a whole lot of my stuff over to Vertex AI because I thought it was the better supported version for larger scale. Guess not.

3 Likes
3

Thanks! I noticed as well. What a drag… I though vertax AI is supposed to be for enterprise use, thus had less restrictions. Google PM strikes again…

4 Likes