How to install newest library version in Google Cloud Functions from requirements.txt

Question

When a cloud function is deployed I assume Google performs some equivalent of:

pip install -r requirements.txt

Let's say the requirements.txt file contains:

google-cloud-pubsub
google-cloud-storage==1.26.0

Since the cloud storage version is specified, all existing and future scaled function instances will install that version.

A gcloud functions deploy ... would install 1.26.0 storage libraries on all instances.

However, let's say there is an instance already running with google-cloud-pubsub version 1.0.2 installed and the newest version is 1.3.0.

The pip command above would not load a newer version.

Instead there would be a Requirement already satisfied response. The existing instance would stay on version 1.0.2 while any newly scaled instances would pull 1.3.0. There would be a mismatch of library versions across instances of the same cloud function.

1. Am I understanding this process accurately? Does GCP do an equivalent of pip install -r requirements.txt?
2. Is there a way to force cloud functions to import the newest version of a library during deployment via requirements.txt or otherwise?

Answer 1

1. Am I understanding this process accurately? Does GCP do an equivalent of pip install -r requirements.txt?

You are correct. On deploy, Cloud Functions builds an image, installs your dependencies and your function into it, and uses that to serve your function.

2. Is there a way to force cloud functions to import the newest version of a library during deployment via requirements.txt or otherwise?

I'm not sure I'm following exactly, but you seem to think repeated installs would reuse the same image. They do not: each time your source changes and you deploy, it starts from scratch and performs the steps above.

This means that if you unpin your dependencies (i.e. no ==1.26.0) you will always get the latest compatible version of these projects every time you run a deployment.

NOTE: If your source files haven't changed, or if you are just modifying or adding environment variables, Cloud Functions will cache the previous build and will not re-evaluate your dependencies. Currently you must modify your source files to force Cloud Functions to rebuild your function.

Answer 2

2. Is there a way to force cloud functions to import the newest version of a library during deployment via requirements.txt or otherwise?

I tested this and found the opposite of https://stackoverflow.com/a/60405672/1803897

You have to specify the newer version in requirements.txt to force the use of the newer library. Else the existing library will be used.

https://cloud.google.com/functions/docs/concepts/exec#multiple_functions does not say that some files are not reused when making a new instance. They will copy in some some files, as they will surely not make a new complete install for every instance. That would take too much time. Regarding python libraries the same point is valid. It would take too much time to install all python libraries at every deploy.