How can I deploy a GAE service with a local npm dependency?

Question

I'm new to Google Cloud, and can't seem to figure out how to deploy a Google App Engine service that has a sibling local dependency. I have a project structure like this (my stack is TypeScript, nestJS, React):

-frontend
    app.yaml
    package.json
-backend
    app.yaml
    package.json
-common
    package.json
dispatch.yaml

The frontend package deploys static code to the default service, and the backend package deploys to an api service. The common package contains some code that I want to share between the front and backend. They include it in their package.json files like this:

dependencies: {
    common: "file:../common"
}

This structure works fine locally. The problem is that when I deploy the backend, npm install fails, because it can't find the common package. This make sense, since I understand that it's only going to upload the contents of backend to that service. But what is the right way to achieve this?

I suppose I could just deploy one service that contains all of the code, and have my top-level package.json delegate npm start to the backend's package.json. But that only works because my frontend is fully static, so I only have one package that needs npm start to get called. It seems like there should be a better way to handle this, because that approach would break down if I had two separate backends that each needed their own npm start.

I'm guessing I'm thinking about some aspect of this in a fundamentally wrong way, but I need help figuring out what that is.

Answer 1

The thing here is that you define 2 App Engine services, frontend and backend, that will be packaged independently and installed on different instances, they'll never co-exist on a single instance. So both will need to include the common package.

When you run gcloud app deploy, the folder containing the app.yaml file for that service is considered the root folder and files and folder up in the tree won't be deployed, as you mentioned.

I understand that from a development point of view it makes sense to have a single common package shared by both services, since it avoids duplicating code. One way to manage this is to use Cloud Build to create a build pipeline that will handle incorporating this common code into both services and deploy them separately. For example, something like this:

steps:
- name: ubuntu
  id: 'copy-file'
  args:
  - '-c'
  - |
        cp ./common/package.json frontend/ && cp ./common/package.json backend/
- name: 'gcr.io/cloud-builders/gcloud'
  args: ['app', 'deploy']
  dir: 'frontend/'
  timeout: '1600s'
  waitFor: ['copy-file']
- name: 'gcr.io/cloud-builders/gcloud'
  args: ['app', 'deploy']
  dir: 'backend/'
  timeout: '1600s'
  waitFor: ['copy-file']

The first step will copy the common package to both directories (you'll need to update your common dependency path in your package.json since it'll now be in the same directory). The next 2 steps will run in parallel (both wait for the first step to finish) and deploy each service separately (note the dir parameter).

You can then deploy your services by running the following command in the root directory:

gcloud builds submit

Note that this will always deploy both services.

If you'd rather like to be able to deploy one service and not the other, you could define 2 cloudbuilds files like so:

cloudbuild-frontend.yaml:

steps:
- name: ubuntu
  args:
  - '-c'
  - |
        cp ./common/package.json frontend/
- name: 'gcr.io/cloud-builders/gcloud'
  args: ['app', 'deploy']
  dir: 'frontend/'
  timeout: '1600s'

cloudbuild-backend.yaml:

steps:
- name: ubuntu
  args:
  - '-c'
  - |
        cp ./common/package.json backend/
- name: 'gcr.io/cloud-builders/gcloud'
  args: ['app', 'deploy']
  dir: 'backend/'
  timeout: '1600s'

You'd end up with a tree like this:

-frontend
    app.yaml
    package.json
-backend
    app.yaml
    package.json
-common
    package.json
cloudbuild-frontend.yaml
cloudbuild-backend.yaml
dispatch.yaml

You'd then be able to deploy one service or the other by running either gcloud builds submit --config=cloudbuild-frontend.yaml or gcloud builds submit --config=cloudbuild-backend.yaml