EDW consumption options

Google Cloud Compute Infrastructure
,
1

Hello all,

We have a Bigquery EDW and we are making it available for consumption to different departments in the enterprise.

We need to either

1- Avoid getting billed by other team members queries(compute costs)

and if we can’t

2- Link the compute costs to the team/domain that the query running individual belongs to or running queries for

As far as I know, there’s no way of enforcing using labels on Bigquery jobs.

Information schema could be cumbersome with just collecting the job runner username.

How to best approach to this issue?

2

Off the top of my head, I’m imaging you owning a Google Cloud project called MasterEDW. I’m then imagining other departments that have their own Google Cloud Projects. Users in these other projects have the ability to access some of your Datasets owned by the MasterEDW project … however, they do NOT have the authority to run BigQuery Jobs “in” your MasterEDW project. Instead, they would run BigQuery Jobs in their own projects (with their own bills). So … MasterEDW owns the BigQuery storage costs and compute costs for building the tables/views … but other teams do NOT have permissions to execute jobs IN your MasterEDW project.