BigQuery Client using Python | Timeout and Polling issues

Question

I am trying to execute a SQL statement on BQ Database by initializing the BQ Client. This has been running smoothly for a while but lately seeing an issue.

My code specifically fails when its trying to wait for the results from the DB

query_job = client.query(QUERY)  # API request to start the query    
db_rslt = query_job.result()     # Wait for the query to return results

Here is error message:

File "/opt/conda/default/lib/python3.7/site-packages/google/cloud/bigquery/job.py", line xxx, in result super(QueryJob, self).result(retry=retry, timeout=timeout) File "/opt/conda/default/lib/python3.7/site-packages/google/cloud/bigquery/job.py", line xxx, in result return super(_AsyncJob, self).result(timeout=timeout) File "/opt/conda/default/lib/python3.7/site-packages/google/api_core/future/polling.py", line xxx, in result self._blocking_poll(timeout=timeout, retry=retry, polling=polling) TypeError: _blocking_poll() got an unexpected keyword argument 'retry'

Added the timeout parameter to the result method but did not help

Answer 1

It depends on the version of your installed google-cloud-bigquery library, if you are using a 1.x version, make sure to set google-api-core to a version less than 1.34.0.

What is actually happening here is that when you install google-cloud-bigquery, a bunch of other google related libraries are installed automatically with it, google-api-core is one of these libraries. However, a higher version is being installed thus causing this problem.

In my case, setting my google-api-core==1.33.2 solves the issue. Other solution would be is to upgrade google-cloud-bigquery to a later version .

Answer 2

7 days ago Google released an update to google-api-core==1.34.0 for bug fixes. I guess this introduced some breaking changes if you are not using the later versions of google-cloud-bigquery. You may either:

1. Pin google-api-core to the previous working version (eg. 1.33.2)
2. Upgrade google-cloud-bigquery to the latest versions.