Managing External Data Sources in End-to-End Testing for cross-platform React Native Application

Preface:

End-to-end (E2E) testing is a traditional testing methodology where the aim is to examine how an application works by assessing the flow from start to end. In Scenario-Based End-to-End Testing, the application is examined from all layers – frontend to backend along with its interfaces and the endpoints.

What and Why?

A well-written E2E test suite helps identify defects early and regularly in the development cycle and avoids the discovery of bugs in production. It builds up confidence in the implementation and pushes toward following an automatic release pipeline. With extensive Cypress on Rails experience, we endeavored to create a re-usable & extensible design for setting-up E2E testing CI/CD pipeline for React-Native Applications.

Setting-Up E2E Environment & Challenges

There are three Fundamental Phases to set-up the E2E environment on CI/CD pipelines.

1. Setting up the Execution Context
2. Managing the external data source required by tests
3. Writing, Executing & Reporting the test results

In our past article A definite guide to E2E Testing CI Setup for React-Native using Cavy-CLI, we briefly discussed the steps to configure the execution environment on CircleCI. The getting started guide configured Android & iOS Emulators and reported the results for tests written using Cavy.

The React-Native application consumes an API served by Backend microservices, making management of the tests' external data source difficult. There are two possible solutions to this data management problem:

1. Stub Network Requests: The network requests are proxied with stubbed data using APIs provided by test frameworks. This technique is easy to implement, but the drawback here is to keep stubbed data in sync with the backend changes. As the E2E test suite grows, the task to maintain stubbed data gets more tedious and often distracts the team from actually writing the E2E tests.
2. Cloned Deployment of Backend Microservice for E2E tests: This technique uses a separate API deployment for handling network calls during E2E tests on CI environment. The downside to this technique is an additional deployment cost. However, using a real API avoids the maintenance cost of updating stubbed API data.

What We Did:

We chose "Cloned Deployment of Backend Microservice for E2E tests." The challenges included:

• Each CI/CD execution run requires a separate backend deployment to perform network calls. Otherwise, using a single global deployment would result in data inconsistencies caused by the concurrent CI/CD runs' mutations.
• The deploy the backend API app resided in a different Github repo than the React-Native app repository.
• React-Native App had to be configured dynamically to use deployment URLs generated per CI/CD run.
• At the end of each CI/CD run, the deployed app was destroyed to save on the cost involved in hosting the service.

In our use case, the backend was implemented using React on Rails and was served on Heroku. We used the Heroku fork plugin to manually deploy the backend per E2E run. Here is a snippet of setup and fork job:

 setup-and-fork-heroku-app:
	 steps:
	 - run:
	   name: Install Heroku Fork plugIn
	   command: |
	     heroku plugins:install heroku-fork
	 - run:
	   name: Fork Web App to Mobile Test App
	   no_output_timeout: 30m
	   command: |
	     heroku fork --from source-app --to "${APP_NAME}" --confirm=confirm
	 - run:
	   name: Scale up Web Dyno
	   command: |
	     heroku ps:scale web=1:performance-m --app="${APP_NAME}"

To configure the network request URL dynamically in React-Native app, we used .env file to set the network Request URL. In CI/CD execution the create_env_file job was used to create .env file after a successful deployment.

create_env_file: &create_env_file
  run:
    name: Create ENV file
    command: |
      echo 'API_MODE = REVIEW' >> .env
      echo "REVIEW_APP_URL = http://${APP_NAME}.herokuapp.com/" >> .env
      echo "The content of ENV file is"
      cat .env

Finally, to destroy the app at the end of CI/CD execution, the delete_heroku_deployment job was created as follow:

delete_heroku_deployment: &delete_heroku_deployment
  run:
    name: Destroy Mobile Test App for PR
    when: always
    command: |-
      echo "Deleting: ${APP_NAME}"
      heroku destroy -a "$APP_NAME" --confirm="$APP_NAME" || true