Hello everyone,
With the upcoming restrictions and deprecation of the Custom Search API endpoints, many of you are likely exploring or implementing “Vertex AI Search (Grounding with Google Search)” as your next-generation search integration.
However, simply swapping out the API is not enough to create a product that withstands a production enterprise environment. During my migration process, I realized that a true “paradigm shift” in how we approach search engines is required.
Today, I’d like to share a use case regarding structural design and how to safely extract output results.
1. The Decisive Difference Between Custom Search API and Vertex AI Search
The legacy Custom Search API was designed to return a direct list of related URLs (JSON) based on a keyword.
Vertex AI Search (Grounding) is completely different. The AI reads the pages behind the scenes, interprets the context, generates an answer, and attaches groundingMetadata as proof. The biggest structural difference is that the source URLs (uri) inside the metadata are obfuscated as Google Cloud redirect links (e.g., https://vertexaisearch.cloud.google.com/url?q=``...) for security and tracking purposes.
2. Architecture for Building an “AI-Embedded Search Engine”
If you output these obfuscated URLs directly to the user’s screen (frontend), they look like suspicious phishing links, completely destroying the business UX.
Therefore, to integrate Vertex AI Search as a “Web Search Engine,” a backend layer (middleware) responsible for “UX Formatting” and “Safety Auditing” is absolutely essential between the AI and the frontend.
[Architecture Diagram of the AI-Embedded Search Engine]
Plaintext
[1. User Input]
│
▼
[2. Gemini 3.1 Pro + Vertex AI Search] (Inference & Grounding)
│
│ ≪Raw Metadata: Raw data including obfuscated redirect URLs≫
▼
[3. Extraction & Safety Middleware] (Custom Backend Layer)
├─ ① URL Extraction: Rescue the "true official URL" from redirect parameters
├─ ② Domain Formatting: UX optimization (e.g., auto-appending "www.")
└─ ③ Security Auditor: Dynamic filtering (Purging dangerous TLDs and sensitive info)
│
│ ≪Clean Data: Only verified, official, and safe URL lists≫
▼
[4. Frontend UI] (Safe UI maintaining Zero Trust UX)
3. How to Safely Extract the Generated Artifacts (Code Example)
Here is a Node.js backend example for “① URL Extraction” in the diagram above, safely extracting the original URL from the Google redirect link and formatting it for the frontend.
JavaScript
// Example of looping through candidate.groundingMetadata.groundingChunks
let uri = chunk.web?.uri || chunk.retrievedContext?.uri || "";
let domain = "";
if (uri) {
// ① Safely extract (rescue) the "true URL" from the Google redirect link
if (uri.includes("vertexaisearch.cloud.google.com") || uri.includes("GoogleGenAIsearch")) {
const tempUrl = new URL(uri);
// Extract the original URL from query parameters (url, q, etc.)
const realUrl = tempUrl.searchParams.get("url") || tempUrl.searchParams.get("q");
if (realUrl) {
uri = realUrl; // Overwrite with the extracted real URL
}
}
// ② Domain parsing
const parsedUrl = new URL(uri);
domain = parsedUrl.hostname;
}
4. [CRITICAL] Strict Output Governance in Enterprise Search
As indicated in “③ Security Auditor” in the architecture diagram, there is an absolutely unavoidable critical point when deploying AI search in production.
The search capability of Vertex AI Search is incredibly powerful, which is exactly why designing strict quality standards to “block information that must not be exposed” is vital. The greatest fear is the risk of the AI outputting malicious sites or scam landing pages that could entangle users in cybercrime. Furthermore, there is a severe danger that the AI might indiscriminately pick up gray-zone information directly linked to others’ privacy or corporate secrets (such as raw financial documents or leaked corporate registry data).
It is required in AI-era system design to embed strict rules into your custom middleware: “Sites linked to crime, or information sources that threaten to infringe on the privacy or trade secrets of others, must absolutely never be included in the AI search engine’s output.”
5. Why I Chose This Architecture/Option
The reason I stopped merely looking for a simple alternative tool to the Custom Search API and chose this “Vertex AI Search integration via middleware” is clear.
I am convinced that what is required in the coming era is not a search tool that merely returns a list of links, but the operation of an “AI-Embedded Search Engine” that manages everything from information gathering and interpretation to the governance of safety standards in a one-stop manner.
Instead of connecting the mighty search power of AI directly to the user, the developer must design (handle) the “compliance safety standards” and “UX elegance” at the backend layer. Only then is a truly practical AI search completed for the enterprise environment.
I hope this serves as a helpful reference for anyone hitting a wall migrating from the Custom Search API or integrating grounding data into your UI/security.
If anyone here is struggling with the same dilemmas, please leave a comment! Also, if you’d like to discuss deeper architectural safety and quality standards, feel free to reach out to me directly.