Model Armor Not Detecting PII When Sensitive Values Lack Context

Google Cloud AI Infrastructure
1

Hi everyone,

I’m working on a GenAI agent and want to add a security layer before prompts are sent to the LLM. My goal is to detect and mask PII/sensitive information in user prompts using Google Cloud Model Armor.

During testing, I noticed that Model Armor seems to detect and mask sensitive values only when there is contextual information indicating what the value represents. For example:

Aadhar 4991-1866-5246
He paid with 4012-8888-8888-1881.

In these cases, the Aadhaar number and credit card number are masked because the surrounding text provides context.

However, if the prompt contains only the value itself, such as:

4991-1866-5246
4012-8888-8888-1881

the values do not appear to be masked because there is no contextual keyword indicating what they represent.

My questions are:

  1. Does Model Armor support context-independent detection of sensitive data (e.g., detecting card numbers, Aadhaar numbers, account numbers, etc. based solely on patterns)?

  2. Is there a configuration or policy setting that enables pattern-based PII detection without requiring descriptive keywords?

  3. Are there any recommended approaches for protecting prompts that may contain standalone identifiers before they reach the LLM?

  4. Would combining Model Armor with Cloud DLP (Sensitive Data Protection) be the recommended architecture for this use case?

Any guidance, best practices, or examples would be greatly appreciated.

Thanks!

1 Like