import Google play data to BigQuery

Google Cloud Data Analytics
,
1

Hello,

I have managed to import Google play data using Data transfer service to Bigquery.

i noticed that from Google play there are reports from various years 2022,2021 etc but when i checked the tables in Bigquery there are data only from 2023.

Is there any way to verify that all reports have been imported to BigQuery?

Thanks

1 Like
2

To ensure that all reports from Google Play have been successfully imported into BigQuery, follow these steps:

  1. Check Transfer Logs:

    • Navigate to the Data Transfer page in the Google Cloud console.
    • Locate the transfer configuration for Google Play data.
    • Click on “View Logs” to examine the details of each transfer run. Pay attention to the success/failure status and the date ranges of the imported data. Look for any error messages or warnings that might indicate problems.

  2. Review BigQuery Tables:

    • Inspect the BigQuery tables created by the data transfer.
    • Check the “Last modified” timestamp to determine when the tables were last updated.
    • Execute queries to verify the data, especially the range of dates. For example, use SELECT MIN(date_column), MAX(date_column) FROM your_table; to confirm that all expected years are present.

  3. Inspect Transfer Configuration:

    • Review the transfer configuration to ensure it is set to import data from all the desired years.
    • Check for any date filters or restrictions that might limit the import scope. Ensure that the settings align with your data requirements.

  4. Utilize BigQuery Audit Logs:

    • Access the BigQuery audit logs for comprehensive information about the import jobs.
    • Investigate these logs for any errors, warnings, or anomalies that might indicate issues with importing specific data ranges.

Additional Tips:

  • Re-Running the Transfer: If you suspect that some data is missing, consider re-running the transfer for the missing date ranges.
  • Cloud Monitoring: Implement Cloud Monitoring to track the transfer status. Set up alerts to notify you of any anomalies or issues during the data transfer process.
  • Data Consistency Checks: Perform data consistency checks by comparing the record counts and types in BigQuery against the expected data from Google Play reports.
3 Likes