End-to-End testing with Cypress

How the Postman Web Team tests web properties

Photo by Pankaj Patel on Unsplash

The Postman Marketing Tech team is a very small group of developers working on several properties including the Postman website. As a small team we are always looking for modular solutions that we can integrate quickly. We also love using the Postman App and use it where we can.

There are several parts the Postman website where dynamic data is served from our Postman Mock Servers. Our mock server handles many Collections (groups of requests), and for each request we write unit tests and monitor for stability. The experience of writing unit tests in Postman, imo, are not only quick but dare I say fun and effective within minutes. Writing tests in minutes in an intuitive tool is what we were looking for to begin testing our Postman website.

Writing Unit Tests in Postman 🚀

So what does our website codebase look like? Our codebase is fairly simple without a lot of bells and whistles. We use Gulp to process Embedded JavaScript templates (EJS) + Sass, sprinkle a little jQuery, and deploy into Amazon Web Services (AWS) with Bitbucket Pipelines.

Postman Web workflow

Because our site is fairly static and without a lot of complexity, we wanted a tool to just bolt on and simulate a user experience and test against that. Since we are testing as if we were a user, that means we want an end-to-end testing suite.

3 Common Types of Testing

• Unit Testing — small tests meant to verify a very specific piece of functionality
• Integration Testing — designed to test different pieces of functionality working together
• E2E Testing (End-to-End) — test entire flow from start to finish

Great so we know testing in Postman is dope, our codebase is fairly simplistic, and we want to test our site like a user would.

Commence the Googling!

After some major sleuthing and without going too in depth with all the available options, we landed between Puppeteer and Cypress. We created a couple of internal proof of concepts to see which would fit our needs and team methodologies. After setting up the configuration for both projects it became very apparent that Cypress had a closer alignment to what we were looking for with our needs.

Our needs:

• Easy to install with minimal configuration
• No need for dependencies/completely modular
• Testing in GUI plus headless testing for Command Line Continuous Integration
• Solid documentation
• A tool that can be used by non-technical individuals as well as seasoned developers
• Super easy to debug

We found that Cypress easily addressed everything we were looking for in a testing suite.

What is Cypress?

Cypress is “Fast, easy and reliable testing for anything that runs in a browser”. What you get with Cypress is a tool that makes it simple to set up, write, run, and debug tests. For those using Chrome, you get all your familiar developer tools to help in the process of debugging. There are no dependencies and no outright configuration needed which make this a perfectly modular tool. And before I forget, the documentation is extremely robust with videos, gifs, and short explanations which allow us to quickly build our test suite with very little friction.

Install Cypress.

$ npm install cypress — save-dev

Open the Cypress GUI.

$ ./node_modules/.bin/cypress open

Create your first test.

./cypress/integration/first-test.spec.js

Write your first test.

describe(‘First test’, function() {
  it(‘Testing true is true’, function() {
    expect(true).to.equal(true);
  });
});

Running our first test 👍:🎉

And just like that we are up and running. All of this can be discovered in this excellent 10 minute video on how to get started. The part that I like most about this flow is that we can get tests running so fast and can configure specifics later.

Configuring Cypress

There were a couple of things we noticed Cypress would do by default. So by updating the cypress.json file we could control a couple of options we wanted to omit.

First Cypress would record video and that’s not something we needed. We also wanted a nested test directory for reasons inside of /integration.

cypress.json

{
  “testFiles”: “tests/**/*”,
  “video”: false
}

Locally this was the only real configuration we needed, which was fantastic. We can open Cypress, watch our tests run, and see what the browser is doing. Watching those green check marks appear is so satisfying.

Continuous Integrations with Cypress

An aspect that was a little more tricky was completing Continuous Integrations with Bitbucket Pipelines to ultimately deploy to AWS. In working with CI environments, we need to do some configuration before we can deploy.

A Bitbucket Pipeline example for Cypress is provided along with many other CI Examples. So let’s briefly highlight some important aspects.

Our first step was to specify the Docker image environment Cypress will run in.

image: cypress/base:10

During the steps we call on our configured npm scripts.

“start-server”: “npm run dev”,
“cypress:run”: “./node_modules/.bin/cypress run”,
“test:ci”: “start-server-and-test start-server http://localhost:8000 cypress:run”

Using cypress:run will allow you to run in a headless mode in the command line perfect for executing tests in a CI environment. For CI, we test against a local server. To achieve this, we use the start-server-and-test tool referenced in the Cypress CI Docs. This allows us to have Gulp run all the necessary tasks, start a server on localhost:8000, run our tests on that local server, and finally shut down the server once the testing is completed.

The initial run will build as expected downloading Cypress for the first time and running our test suite. However, on subsequent builds Cypress will look for cached files and that’s where the Bitbucket definitions > caches are required.

definitions:
  caches:
    npm: $HOME/.npm
    cypress: $HOME/.cache/Cypress

And we call those definitions during our build processes.

- step:
  name: ‘CI:Test’
  caches:
    - npm
    - cypress
    - node

To recap:

1. Docker Image
2. Cypress Run + start-server-and-test + Gulp
3. Define/Call cache definitions
4. Profit 💰

And with all that we are running Cypress in all of our environments, and it has been a wonderful experience for over a month at the time of writing.

Takeaways

Cypress is a great tool and we give our gratitude to the folks over there working on it. Compared to other setups I’ve done in the past, this was by far the easiest to get up and working. I can’t say enough good things about this tool, and I look forward to writing tests like I only felt working with Postman Collections.

Is Cypress right for your project? That entirely depends on your setup and situation. There are a lot of amazing tools like Puppeteer, Selenium, Nightwatch.js, and so on. But I would say that taking 30 minutes and going through the Getting Started guide is well worth your time to see if it also fits your team’s needs. And if you enjoy Cypress be sure to head over to their GitHub and share some love.

What testing be like