We have a log sink setup to store all queries run on bigquery.
In the log sink we have logged dataform querying the INFORMATION_SCHEMA.TABLES which seems to cost around € 3.9 and process 9 tb of data, although if the query is run from the bigquery interface itself only processes 10 mb.
It seems we are also being billed for this costs that are being logged.
Does anyone know what is going on here?
Hello @Fabian_Peterstorfer,
When you say that the query cost €3.9, how are you doing the calculation?
How many total_bytes_processed and total_bytes_billed do you see?
Does your table contain 9 TB?
Querying INFORMATION_SCHEMA.TABLES does not query your table but only its metadata. (see)
Concerning the 10 MB part, you can learn more on INFORMATION_SCHEMA.TABLES Pricing:
For projects that use on-demand pricing, queries against
INFORMATION_SCHEMAviews incur a minimum of 10 MB of data processing charges, even if the bytes processed by the query are less than 10 MB. 10 MB is the minimum billing amount for on-demand queries. For more information, see On-demand pricing.
Perhaps the page about Understand compute pricing for BigQuery may help you.
Hello @LeoK ,
The cost is coming directly from the log sink, please see screenshot.
The table contains more than 9tb and yes from my understanding querying metadata should not be expensive. Thanks
