Flakey tests when testing firebase functions using Jest

Question

I'm testing Firebase functions using Jest and the emulator, though the tests are flakey presumably from a race condition. By flakey, I mean sometimes they pass and sometimes they don't, even on the same machine.

Tests and functions are written in TypeScript, then transpiled with babel.

Example test/function

Note: This is an example of just one of the flakey tests. Many other tests are flakey. A solution is preferably one that doesn't just solve this one case, but rather the general issue.

The test

import { onProfilesWrite } from '../src/profiles/on-write'
import { initializeAdminApp } from '@firebase/rules-unit-testing'

const admin = initializeAdminApp({ projectId: 'projectId' }).firestore()

const wrappedFunction = testEnvironment.wrap(onProfilesWrite)

const profilePath = `profiles/${uid}`

const customerProfile = {
    roles: ['customer'],
    slug: 'slug',
    image: 'image.png',
    fullName: 'John Smith',
}

const publisherRoles = ['customer', 'publisher']

const publisherProfile = {
    ...customerProfile,
    roles: publisherRoles,
}

const createChange = async (
    before: Record<string, unknown> | undefined,
    changes: Record<string, unknown>
) => {
    const publisherStatsRef = admin.doc(profilePath)
    if (before) await publisherStatsRef.set(before)

    const beforeSnapshot = await publisherStatsRef.get()
    await publisherStatsRef.set(changes, { merge: true })

    const afterSnapshot = await publisherStatsRef.get()

    return testEnvironment.makeChange(beforeSnapshot, afterSnapshot)
}

test('If user profile is created as a publisher, publisherDetails is created', async () => {
    const change = await createChange(undefined, publisherProfile)
    await wrappedFunction(change)
    const snapshot = await admin.doc(`profileDetails/${uid}`).get()
    const data = snapshot.data()
    expect(data).toBeTruthy()
    expect(data?.id).toBeTruthy()
    expect(data?.slug).toBe(publisherProfile.slug)
    expect(data?.profileImage).toBe(publisherProfile.image)
    expect(data?.publisherName).toBe(publisherProfile.fullName)
    expect(data?.music).toMatchObject([])
})

Run the test

firebase emulators:exec \"jest functions/__tests__ --detectOpenHandles\" --only firestore

Output

If user profile is created as a publisher, publisherDetails is created

    expect(received).toBeTruthy()

    Received: undefined

      46 |     const snapshot = await admin.doc(`profileDetails/${uid}`).get()
      47 |     const data = snapshot.data()
    > 48 |     expect(data).toBeTruthy()
         |                  ^
      49 |     expect(data?.id).toBeTruthy()
      50 |     expect(data?.slug).toBe(publisherProfile.slug)
      51 |     expect(data?.profileImage).toBe(publisherProfile.image)

The function

import * as functions from 'firebase-functions'

// initializes the admin app, then exports admin.firestore
import { firestore } from '../admin'

const database = firestore()

const createPublisherId = async (): Promise<string> => {
    let id = ''
    const MAX_NUMBER = 1000000
    while (id === '') {
        const temporaryId = String(Math.ceil(Math.random() * MAX_NUMBER))
        const snapshot = await firestore()
            .collection('publisherDetails')
            .where('sku', '==', temporaryId)
            .limit(1)
            .get()
        if (snapshot.empty) id = temporaryId
    }
    return id
}

export const createPublisherDetails = async (
    newData: firestore.DocumentData,
    uid: string
): Promise<void> => {
    const id = await createPublisherId()

    await database.doc(`publisherDetails/${uid}`).set(
        {
            id,
            slug: newData.slug,
            publisherName: newData.fullName,
            profileImage: newData.image,
            music: [],
        },
        { merge: true }
    )
}


export const onProfilesWrite = functions.firestore.document('profiles/{uid}').onWrite(
    async (change): Promise<void> => {
        const { id: uid } = change.after
        const oldData = change.before.data()
        const newData = change.after.data()

        if (
            newData?.roles?.includes('publisher') &&
            (typeof oldData === 'undefined' || !oldData.roles?.includes('publisher'))
        )
            await createPublisherDetails(newData, uid)
    }
)

Debug steps

• All promises are awaited in the cloud functions (as affirmed by an ESLint rule @typescript-eslint/no-floating-promises)
• Also converted the tests to Mocha (as suggested by the Firebase docs), same errors
• Converting async/await in tests to promise.then() syntax

Metadata

• OS: macOS 11.2, Ubuntu 18.04
• Jest: 26.6.3
• Firebase: 8.2.6
• Firebase tools: 9.3.0

As comments roll in, with either questions or suggestions, I'll continue to update this post.

Answer 1

Change your test portion to as follows :

test('If user profile is created as a publisher, publisherDetails is created', async () => {
  const change = await createChange(undefined, publisherProfile);
  await wrappedFunction(change);
  const documentObject = await admin.doc(`profileDetails/${uid}`);
  const snapshot = await documentObject.get();
  const data = snapshot.data();
  expect(data).toBeTruthy();
  expect(data?.id).toBeTruthy();
  expect(data?.slug).toBe(publisherProfile.slug);
  expect(data?.profileImage).toBe(publisherProfile.image);
  expect(data?.publisherName).toBe(publisherProfile.fullName);
  expect(data?.music).toMatchObject([]);
});

Reason being that in your test region, your use of await is a bit incorrect (function chaining on an object that is being waited for is a big no-no in the same calling line)