We are using Vertex AI’s DeepSeek-V3.1 API service. Recently, we have discovered that when the prompt exceeds a certain length (possibly around 30k tokens), the API ceases to return any valid results, despite the HTTP status code being 200.
Hi @Zhouze ,
thank you for sharing this!
I will reach out the Vertex AI Inference team to collect feedback about it.
Keep you posted.
Best
1 Like
The DeepSeek-V3.1 API on Vertex AI stops returning results when prompts exceed roughly 30k tokens, even though the HTTP status shows 200. To avoid this, keep prompts under the token limit or split longer inputs into smaller chunks. The Vertex AI team is aware and reviewing this behavior.
Hi @ilnardo92 ,
Thank you for the prompt fix. I have tested the API with an extended prompt (48,417 tokens), and it is now working perfectly. The responses are complete and as expected.
We really appreciate the swift resolution.
For our internal understanding and to help prevent similar issues in the future, would it be possible to briefly share what the root cause was? Any insight would be helpful.
Thanks again for your support.
Best regards,
Zhouze
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.