When should I run daily ETL jobs for Firebase Analytics data exported to BigQuery?

Question

We use Firebase Analytics to collect events from our apps. We have enabled events export to BigQuery. Every day we run some ETL jobs to create more friendly analytics tables in BigQuery (e.g. sessions, purchases).

The question is when should we run these ETL jobs?

We know that Firebase Analytics creates in BigQuery 'events_intraday_' table which is changed to 'events_' after some hours after midnight. We also understand that some events might be reported later if client is not connected with the internet, but this is not the problem.

Our theory is that 'events_intraday_' table is some kind of temporary table and we should run ETL jobs when it changes to 'events_'. Unfortunately we could not find any documentation about it. Is this good solution?

Answer 1

From Announcing Realtime Exporting of your Analytics Data into BigQuery:

At the end of the day [1], this data will be moved into its permanent appevents_ home, and the old intraday table will be automatically cleaned up for you.

With:

[1] This is determined by looking at the developer's time zone.

So it looks like the daily table is created at midnight for your timezone.