BQ Multiple FROM statements

Google Cloud Data Analytics
,
1

I have sink setup where our GCP logs are streamed to BQ.

These logs are separated by date where the end of the log file is YYYYMMDD for example:

LogName20220801, LogName20220802 (August 1, August 2 etc).

I would like to query a month’s worth of logs, however when I add more than one of these in my FROM statement then it doesn’t work. I tried:

FROM

'LogName20220801'
'LogName20220802'

However, the estimated data that will be processed doesn’t go up when I add the second log so it appears to only be querying the first log.

Any ideas?

2

You can use wildcards or [UNION](https://cloud.google.com/bigquery/docs/reference/standard-sql/migrating-from-legacy-sql#comma_operator_with_tables) statements.

Wildcards:

SELECT
FROM
<project-id>.<dataset-id>.LogName2022080*
WHERE _TABLE_SUFFIX BETWEEN ‘1’ AND ‘2’

UNION:

SELECT * FROM
(SELECT * FROM ..table1 UNION ALL SELECT * FROM ..table2)

NOTE: In legacy SQL, the comma operator has the non-standard meaning of UNION ALL when applied to tables. In standard SQL, the comma operator has the standard meaning of JOIN.

1 Like