Run a gsutil command in a Google Cloud Function

Question

I would like to run a gsutil command every x minutes as a cloud function. I tried the following:

# main.py
import os

def sync():
     line = "gsutil -m rsync -r gs://some_bucket/folder gs://other_bucket/other_folder"
     os.system(line)

While the Cloud Function gets triggered, the execution of the line does not work (or i.e. the files are not copied from one bucket to another). However, it does work fine when I run it locally in Pycharm or with cmd. What is the difference with cloud functions?

Answer 1

You can use Cloud Run for this. You have very few change to perform in your code.

Create a container with gsutil installed and python also, for example gcr.io/google.com/cloudsdktool/cloud-sdk as base image

Take care of the service account used when you deploy Cloud Run, grant the correct permission for accessing to your bucket

Let me know if you need more guidance