How to Run JEST in Github Actions with Firestore
This post will guide you through the steps of running automated tests using Jest with GitHub Actions.
Real Case Scenario
We have an API built in NodeJS that was returning false responses or causing errors due to a lack of tests. This issue makes it difficult to debug or add new features to the codebase, since the behavior is not tested and cannot be guaranteed to work as expected.
So, the codebase needs to be covered with tests to ensure that endpoints are returning the expected results. These tests should be run automatically when creating a PR to ensure that all edge cases are covered and that no logical errors will occur in production.
It's not only limited to tests, but it's also a good practice to apply a few pipelines to the PR to ensure that all linting checks and test results are successfully passed.
In addition, the codebase uses Firestore as a NoSQL database to store all necessary data. This means that there are Firestore queries in the codebase that need to be handled in the tests. Mocking the queries is not a good practice since testing the query is a crucial part of verifying its functionality.
Using the development database directly is also not a good option since it can mutate the development database and interfere with all documents, which can break the staging environment and make it difficult to maintain.
Furthermore, another question arises regarding how to run it in GitHub actions if we are not going to use the development database instance.
Setting Up Project
I have already created a sample NodeJS API to focus solely on testing and GitHub actions. You can clone the repository as follows:
thepylotThe initial state of the project includes basic configurations and dependencies to get started. Next, we will use a Docker image to run the Firestore emulator locally. This also will be used in Github Workflows to start the emulator on-the-fly and use it as a test database.
Create a docker-compose-test.yml in root level of the project:
docker-compose-test.yml
version: "3.2"
services:
firestore_emulator:
image: mtlynch/firestore-emulator
environment:
- FIRESTORE_PROJECT_ID=my-blog-test
- PORT=8080
ports:
- 8080:8080
Now let’s run the docker-compose with following command:
docker-compose -f docker-compose-test.yml up -d
You will be able to see a container running the emulator, which is ready to use in your code.
Configuring Jest
Now that our emulator is up and running, it's time to configure Jest before starting the actual coding.
If you navigate to services/db.js, you will see the initialization of a Firestore instance based on the environment setup.
Environment variables are typically stored in .env files or populated through deployment platforms. For tests, it is not necessary to use actual environment variables since the emulator is running in a Docker container with dummy data. We don't need to push or use the .env file for a few test variables because Jest allows us to mock environment variables before running tests.
To mock testing environment variables, create a directory named .jest at the root level and add a new file inside it named setEnvVars.js.
.jest/setEnvVars.js
process.env.NODE_ENV = 'dev';
process.env.TEST_ENABLED = true;
process.env.FIRESTORE_EMULATOR_HOST = "localhost:8080"
process.env.FIRESTORE_PROJECT_ID = "my-blog-test"
TEST_ENABLED will separate the initialization of Firestore for testing environments, but not for actual staging or production, as defined in the file services/db.js.
We set the value of FIRESTORE_EMULATOR_HOST to localhost:8080 because we mapped it in the docker-compose file. This means that the emulator will be exposed to localhost on port 8080.
Finally, the FIRESTORE_PROJECT_ID points to the name of the Firestore project ID that we also set in the docker-compose file for the emulator.
The next step is to add a configuration file in root level named jest.config.js. This file will let Jest know that there are setup files that need to be populated.
jest.config.js
module.exports = {
setupFiles: ["<rootDir>/.jest/setEnvVars.js"]
};
Also, add another file named jest.setup.js to define max limit of timeout for tests. This will allow us to debug test without having timeout issue:
jest.setup.js
jest.setTimeout(50000);
Lastly, update your package.json scripts by adding the following line to run tests:
"test": "jest"
If you run npm run test now, it will attempt to run tests where none exist yet.
Adding tests
We have an /posts endpoint that retrieves data from Firestore and needs to be covered with tests. To begin, create a tests directory and add a new file named handler.test.js.
Then, let’s add our first test case:
tests/handler.test.js
const { handler } = require("../handler");
const { db } = require("../services/db");
const { faker } = require("@faker-js/faker");
const request = require('supertest');
function postsGenerator() {
let posts = []
const postTitles = Array.from({length: 5}, (_) => faker.random.word())
postTitles.forEach(title => {
posts.push(
{
title: title,
timestamp: new Date(),
}
);
})
return posts;
}
afterEach( async () => {
await db
.collection('posts')
.get()
.then((querySnapshot) => {
querySnapshot.forEach((doc) => {
doc.ref.delete();
});
});
}, 30000)
describe('GET /posts', () => {
it('GET /posts => retrieve posts', async () => {
const posts = postsGenerator()
const queryPromises = []
posts.forEach((createdPost) => {
queryPromises.push(db.collection("posts").doc().set(createdPost))
})
await Promise.all(queryPromises);
return await request(handler)
.get('/posts')
.expect('Content-Type', /json/)
.expect(200).then(response => {
expect(response.body.posts.length).toEqual(5)
})
});
});
Now let's break down the code to see what applies here:
const { handler } = require("../handler");
const { db } = require("../services/db");
const { faker } = require("@faker-js/faker");
const request = require('supertest');
To begin testing, we need to import the required packages:
handler- where the endpoint is declareddb- the initialized Firestore databasefaker- a fake data generator librarysupertest- a library that allows us to test endpoints using HTTP requests.
function postsGenerator() {
let posts = []
const postTitles = Array.from({length: 5}, (_) => faker.random.word())
postTitles.forEach(title => {
posts.push(
{
title: title,
timestamp: new Date(),
}
);
})
return posts;
}
The above function is a helper function that generates and returns sample posts using Faker.
afterEach( async () => {
await db
.collection('posts')
.get()
.then((querySnapshot) => {
querySnapshot.forEach((doc) => {
doc.ref.delete();
});
});
}, 30000)
In Jest, afterEach is a method that runs after each test case. It is used to clean up any resources that were created during the test and ensure that the environment is clean before the next test runs. In the given code, afterEach is used to delete all documents from the posts collection in Firestore after each test case is run, so that the tests do not interfere with each other.
Other methods, such as beforeEach or afterAll, can also be used to apply pre- and post-actions based on your needs.
describe('GET /posts', () => {
it('GET /posts => retrieve posts', async () => {
const posts = postsGenerator()
const queryPromises = []
posts.forEach((createdPost) => {
queryPromises.push(db.collection("posts").doc().set(createdPost))
})
await Promise.all(queryPromises);
return await request(handler)
.get('/posts')
.expect('Content-Type', /json/)
.expect(200).then(response => {
expect(response.body.posts.length).toEqual(5)
})
});
});
We’re verifying that the /posts endpoint retrieves the expected number of posts from Firestore.
It performs the following steps:
- Generates 5 sample post objects using a helper function
postsGenerator. - Iterates through the posts and pushes a Firestore
setmethod promise to an array of query promises. - Uses
Promise.allto wait for all the queries to finish executing. - Makes a GET request to the
/postsendpoint usingsupertest. - Expects the response to have a
Content-Typeof JSON and a status code of 200. - Expects the response body to have an array of posts with a length of 5.
Now run your tests with npm run test to verify that all the results have passed.
If you want to debug tests using VSCode, you can use the following launch.json file:
{
"version": "0.2.0",
"configurations": [
{
"type": "node",
"request": "launch",
"name": "Jest: current file",
"env": {
"NODE_ENV": "dev",
"TEST_ENABLED": "true",
"FIRESTORE_PROJECT_ID": "my-blog-test",
"FIRESTORE_EMULATOR_HOST": "localhost:8080"
},
"program": "${workspaceFolder}/node_modules/.bin/jest",
"console": "integratedTerminal",
"windows": {
"program": "${workspaceFolder}/node_modules/jest/bin/jest"
}
}
]
}
Adding Github Workflows
Now that we have tests, it's time to automatically run them when creating a PR and ensure that all tests pass before merging.
Create a new folder named .github and, inside it, another folder named workflows. All Github workflows must be stored in this specific directory.
Next, Create a new file called test.yml inside the workflows directory to hold the YAML file of the workflow steps:
.github/workflows/test.yml
name: Test
on:
pull_request:
jobs:
tests:
name: Running Tests
timeout-minutes: 10
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v1
- name: Start containers
run: docker-compose -f "docker-compose-test.yml" up -d --build
- name: Install node
uses: actions/setup-node@v1
with:
node-version: 16.x
- name: Install dependencies
run: npm install
- name: Run tests
run: npm run test
- name: Stop containers
if: always()
run: docker-compose -f "docker-compose-test.yml" down
The workflow will run automatically when a pull request is created, and the results will be displayed in the pull request page.
Since we are running an emulator inside the Docker container, it is very simple to run a workflow without additional configuration for Firestore. Everything comes in a container already.
Special shoutout to the author of https://github.com/mtlynch/firestore-emulator-docker for creating such a useful tool with great flexibility!
The source code and final version can be found in the repository below:
thepylotTroubleshooting Tips
- Before running tests see the logs of container to see is if Firestore is up and running.
- If tests are failing, try restarting the container to run fresh tests to avoid any unresolved query executions.
Conclusion
In this post, we have seen how to run automated tests using Jest with GitHub Actions. We also covered how to use Firestore as a NoSQL database to store all necessary data and how to run it in GitHub actions with a Firestore emulator.
After setting up the project, we configured Jest before starting the actual coding. We then added tests to our /posts endpoint, verified that they ran correctly, and debugged them if needed.
Finally, we added Github Workflows to automatically run tests when creating a pull request and ensure that all tests pass before merging.
We hope this post has helped you set up automated tests for your NodeJS API using Jest with GitHub Actions and Firestore.