How to publish messages to Firebase Pubsub Emulator

Firebase Emulator Suite is great for testing your application locally, but all the joy is sucked out if you want to interact with the Pubsub emulator.

The issue is that there's no way to publish a message directly to the pubsub local emulator. Even if you use the stand-alone gcloud pubsub emulator you can't publish a message directly.

Using the emulator
To use the emulator, you must have an application built using the Google Cloud Client Libraries. The emulator does not support Cloud Console or gcloud pubsub commands.

<small class='text-muted'> (source: Google Pubsub Docs) </small>

So here's a workaround...

The heart of the matter is that you can only interact with pubsub through Firebase Functions.

<small class='text-muted'>(source: the horse's mouth)</small>

You can create a Firebase Function as a wrapper to interact with Pubsub.

In this tutorial we'll use two Firebase Functions:

1. pubsubTriggeredFunction — a function triggered by pubsub.
2. pubsubWrapper — the function we'll call using curl from the CLI to write a message to the local pubsub emulator.

If you are creating a new project run firebase init and make sure to select Functions and Emulators to be used.

On a next step you'll be asked to select which emulators you want to use and we need Functions and Pubsub for this tutorial.

When you are done initializing your project, your firebase.json file should have at least the following elements.

//firebase.json
{
	"functions": {
		//...
	},
	"emulators": {
		//...
		"functions": {
			"port": 5001
		},
		"pubsub": {
			"port": 8085
		},
		"ui": {
			"enabled": true
		}
	}
}

If you are using an existing project make sure you have the functions and pubsub emulators set up in firebase.json. This way Firebase Function emulator will connect to Firebase pubsub emulator.

Let's put in the placeholders for the functions we're going to use.

// src/index.ts
import * as functions from 'firebase-functions'

export const pubsubTriggeredFunction = functions.pubsub.topic('test-topic').onPublish((message, context) => {
	return null
})

export const pubsubHelper = functions.https.onRequest((request, response) => {
	response.end()
})

To use Pubsub in Firebase Functions we must first install the pubsub node client. To do so, run npm i @google-cloud/pubsub in the terminal.

The most basic implementation of the pubsubHelper function would look like this:

// src/index.ts
import * as functions from 'firebase-functions'
import { PubSub } from '@google-cloud/pubsub'

// ...

export const pubsubHelper = functions.https.onRequest(async (request, response) => {
	const pubsub = new PubSub()
	await pubsub.topic('test-topic').publishMessage({
		json: { msg: 'the emulator sends its regards' },
	})
	response.end()
})

Now we got an HTTP endpoint we can call that will post a message to the pubsub emulator. This code is just the bare bones and not too useful. For starters it assumes that the topic it posts a message to exists and beyond that it only posts a static message.

Let's draw the rest of the owl and explain what's going on.

The final version of the pubsubHelper function is:

export const pubsubHelper = functions.https.onRequest(async (request, response) => {
	// 1. make sure the function can't be used in production
	if (!process.env.PUBSUB_EMULATOR_HOST) {
		functions.logger.error('This function should only run locally in an emulator.')
		response.status(400).end()
	}

	const price = request.body.stockPrice

	const pubsub = new PubSub()

	// 2. make sure the test topic exists and
	// if it doesn't then create it.
	const [topics] = await pubsub.getTopics()

	// topic.name is of format 'projects/PROJECT_ID/topics/test-topic',
	const testTopic = topics.filter((topic) => topic.name.includes('test-topic'))?.[0]
	if (!testTopic) await pubsub.createTopic('test-topic')

	// 3. publish to test topic and get message ID
	const messageID = await pubsub.topic('test-topic').publishMessage({
		json: { price: price },
	})

	// 4. send back a helpful message
	response.status(201).send({ success: 'Published to pubsub test-topic -- message ID: ', messageID })
})

Pubsub emulator doesn't store data and topics across sessions, so we need to make sure the topic we want to publish to exists on each run.

With this code we expect that if we send a request with the 'stockPrice' attribute in the body, it will post the price to the 'test-topic'.

Let's build up the pubsub triggered function to show us the input when it's being executed.

export const pubsubTriggeredFunction = functions.pubsub.topic('test-topic').onPublish((message, context) => {
	console.log('Got a pubsub message')

	const data = message.data ? Buffer.from(message.data, 'base64').toString() : 'ERR'

	console.log({ data })
	console.log({ context })

	return null // returns nothing
})

With that in place we can now launch the emulator. The emulator should run Functions and Pubsub and have both functions up and running.

note: If you see an error that the pubsubTriggeredFunction was ignored, it means that the functions emulator doesn't see the pubsub emulator. Either pubsub emulator has not started or the Functions emulator can't see it.

Now you can build the project and start the emulators. In the functions folder run npm run build and then firebase emulators:start. You should see something like this

Note that both functions and pubsub emulators have started and both functions have been initialized.

You can now use your preferred method to make a call to the pubsubHelper HTTP endpoint, which is available on localhost:5001. For example you can make the following curl call:

curl http://localhost:5001/billing-killa/us-central1/pubsubHelper \
    -X POST \
    -H "Content-Type:application/json" \
    -d '{ "stockPrice": 100000 }'

# i functions: Finished "us-central1-pubsubTriggeredFunction" in ~1s

And if you look into the firebase functions emulator console you'll see that first pubsubHelper got executed and then pubsubTriggeredFunction got executed and printed out the required info. 🥳