How to Track User-wise LLM Usage in Vertex AI

Google Cloud Build with AI Custom ML & MLOps
,
1

Hi Community,

We have multiple GCP projects, and in most of them, Vertex AI APIs are enabled along with some models from the Model Garden.

I’m looking for a way to monitor usage at the individual user level—specifically, to understand how each user is consuming LLMs and how much cost is being incurred per user.

Any suggestions or best practices would be greatly appreciated.

2

I’m not sure if this is going to help or not. Did you try looking at the API Library section, where we can see who is consuming the Vertex APIs? This won’t provide the direct cost, but it will give us some level of usage information linked to accounts and service accounts.

Thanks
Darwin.

3

Thanks but this does not provide user specific consumption.

4

Hi @Vishal_Solanki,

Try tracking usage and billing by API calls using labels, where the caller (service account) can add metadata labels. These labels allow you to break down your billed charges. You can check this documentation for complete details.

5

Yes, this what I’m planning not sure how much this can help.
For tracking LLM usage via Vertex AI, we might be able to leverage metadata labels. However, this doesn’t seem like a fine-grained solution, as it would require each user to manually pass these labels when invoking the Vertex API.

Step 1: use metadata labels while calling vertex api

Step 2: enable “Detailed usage cost export” to a BigQuery. (Admin access required)
We need it because billing dashboard does not support metadata based filtering.

Step 3: write query to fetch the details from gcp_billing_export_resource_v1_<BILLING_ACCOUNT_ID>
Example queries for Cloud Billing data export  |  Google Cloud

1 Like