Hey Community! Exciting news from #GoogleCloud for those building with #GenerativeAI.
We’ve just announced the Public Preview of enhanced Apigee API management features specifically for GenAI Serving. This release is all about giving you more control, better performance, and enhanced security for your LLM-powered applications.
Key features include:
Server-Sent Events (SSE): Stream LLM responses token-by-token, dramatically reducing perceived latency for end-users.
Model Armor Policy: Native protection against prompt injections, sensitive data leakage, and offensive content for your AI workloads. Critical for responsible AI!
Semantic Caching Policy: Intelligently cache LLM responses based on meaning, not just exact matches, saving you costs and boosting performance.
No extra Apigee cost for these! Just enable the dependencies.