PUB/SUB To Bigquery Without Using DataFlow

Question

I would like to insert into bigquery tables data using pubsub. The data has been processed so I do not need dataflow. How can I do this? Thanks in advance

Answer 1

Cloud Pub/Sub is a queue service, imagine like a database.

You still need something between Cloud Pub/Sub and Bigquery which executes the jobs that are waiting in the queue. For this people often use DataFlow, but you can implement your own worker to read from Pub/Sub and write to BigQuery.

Answer 2

GCP has introduced new feature called BigQuery subscriptions iwhich allows the pubsub subscription to write to Bigquery directly without using dataflow jobs. You won't get billed any additional charges to write the data to bigquery.