Google Cloud Pub/Sub to watch Gmail

Question

Introduction
I am looking to use the Pub/Sub API to monitor the inbox of an account in our organization. It is just a normal Gmail inbox, but it receives IT related notifications important to our business. Some of the emails are not that high of priority and can be checked by ourselves when the time comes, however other emails of high importance(such as our ISP performing maintenance) we would like to notify corporate team members in the office of the events that are happening.

Example
For example, our ISP alerts us(via email) that there is going to be Maintenance Work done on July 9th, 2022 from 12 AM - 6 AM, I would like to forward that email to our Warehouse Manager to make him aware of the maintenance that is happening.

Question
All of the documentation and other posts that I have found are typically using a database to hold the messages their "publisher" is pushing and are using their own application as the "subscriber" to the events being pushed. Do I need to create a database and application to perform the functionality that I desire? Am I overthinking this and making it more complicated than it needs to be and only one or so API call needs to be made

Answer 1

Here is an example using Node. Much of it comes from the Google documentation.

1. Create a PubSub topic.

2. Add a Push subscription to the topic. Make the endpoint of the subscription an HTTP cloud function.

Note: Give [email protected] Publish rights for Pub/Sub.

3. "Watch" the mailbox. When the mailbox receives an email, it will notify the topic and cause the Cloud Function to fire. You can email or set up other notifications via the cloud function.

Authorize and Watch:

const fs = require('fs').promises;
const path = require('path');
const process = require('process');
const {authenticate} = require('@google-cloud/local-auth');
const {google} = require('googleapis');

// If modifying these scopes, delete token.json.
const SCOPES = ['https://www.googleapis.com/auth/gmail.readonly'];
// The file token.json stores the user's access and refresh tokens, and is
// created automatically when the authorization flow completes for the first
// time.
const TOKEN_PATH = path.join(process.cwd(), 'token.json');
const CREDENTIALS_PATH = path.join(process.cwd(), 'credentials.json');

/**
 * Reads previously authorized credentials from the save file.
 *
 * @return {Promise<OAuth2Client|null>}
 */
async function loadSavedCredentialsIfExist() {
  try {
    const content = await fs.readFile(TOKEN_PATH);
    const credentials = JSON.parse(content);
    return google.auth.fromJSON(credentials);
  } catch (err) {
    return null;
  }
}

/**
 * Serializes credentials to a file compatible with GoogleAUth.fromJSON.
 *
 * @param {OAuth2Client} client
 * @return {Promise<void>}
 */
async function saveCredentials(client) {
  const content = await fs.readFile(CREDENTIALS_PATH);
  const keys = JSON.parse(content);
  const key = keys.installed || keys.web;
  const payload = JSON.stringify({
    type: 'authorized_user',
    client_id: key.client_id,
    client_secret: key.client_secret,
    refresh_token: client.credentials.refresh_token,
  });
  await fs.writeFile(TOKEN_PATH, payload);
}

/**
 * Load or request or authorization to call APIs.
 *
 */
async function authorize() {
  let client = await loadSavedCredentialsIfExist();
  if (client) {
    return client;
  }
  client = await authenticate({
    scopes: SCOPES,
    keyfilePath: CREDENTIALS_PATH,
  });
  if (client.credentials) {
    await saveCredentials(client);
  }
  return client;
}
// you can also use "me" instead of an email addy

/**
 * Watches a mailbox for changes
 *
 * @param {google.auth.OAuth2} auth An authorized OAuth2 client.
 */
async function watchUser(auth) {
    const gmail = google.gmail({version: 'v1', auth});
    const res = await gmail.users.watch ({
        userId: '[email protected]',
        topicName: 'projects/{project-id}/topics/your-gmail-topic',
        labelIds: ['INBOX']
    });

    console.log(res);
}

async function stopWatchUser(auth) {
    const gmail = google.gmail({version: 'v1', auth});
    const res = await gmail.users.stop ({
        userId: '[email protected]'
    });

    console.log(res);
}

authorize().then(watchUser).catch(console.error);

This approach does not look at the Subject or Body, just that an email to a specific address is received in a specific folder (label really). You could set up filters to forward emails to a specific folder based on Subject or other criteria.

Answer 2

If you just need to forward the email to a known address and you already know the sender's (ISP) email as well as the subject and/or keywords, you don't need to develop an application at all. You can just go into your inbox settings, create a filter, set the From address and Subject and finally set the forwarding address to your warehouse manager's email address.

---

If you're doing something more complicated like notifying people via a different channel or you have some custom logic to detect whether the email's contents are important to you or not then you'll have to write an application.

Normally you'd create a Watch request on the target inbox by using the Gmail API. Once you do that you will receive a historyId which corresponds to the current state of your inbox. You should store this id. Your application will also begin to receive notifications via Pub/Sub whenever there's a new change to your inbox. These notifications also contain history ids.

The benefit of having a database to store history ids is that every time you receive a new notification, you can pull the last known id from your DB and call the List History method/endpoint setting the startHistoryId as the id you just got from your DB. That will make sure you only retrieve the changes that have happened to your inbox since the last notification you processed. In other words you're only going to be processing new messages, which should be a smaller changeset than if you were to list all the messages every time. Without storing some kind of state you're going to be re-scanning messages that you've already seen before. Once you've processed all the new changes in the diff you can update the DB with the new history id.

Technically you don't need to do all of that. You could periodically poll the inbox calling List Messages every time but then you'd need to make other considerations like how often to poll and how many messages you want to list each time.