Vertex AI RAG Engine: Metadata Filtering not working

Google Cloud Build with AI Agents
, ,
1

I’m struggling to get metadata filtering working correctly using the Vertex AI RAG engine (google-cloud-aiplatform). My goal is to restrict retrieval results based on a category field assigned to different files within a single RAG corpus.

My Setup:

  1. Storage: Files are stored in a GCS bucket folder.

  2. Metadata File: I have a metadata.jsonl in the same folder.

    • Example entry: {"file_gcs_uri": "gs://my-bucket/folder/doc1.pdf", "metadata": {"category": "HR"}}

  3. Import: I imported the entire folder into the RAG Corpus using the import_files API.

The Problem:

Despite the filter, the retrieval returns results from all categories (ignoring the filter).

Is there a specific schema requirement for the metadata.jsonl? Any guidance on this would be greatly appreciated!