What is a Testing Pipeline?
An Automated Testing Pipeline is a group of jobs or tasks that are executed in stages also called batches. Pipelines comprise -- Jobs that define what to run. For example, the compilation of code or test runs.
- Stages are defining when and how to run. For example, that test run after compilation of code.
- Jobs in a stage are executed in sequential/parallel by Runners if there are enough concurrent Runners.
- Succeeded, then the pipeline moves to the next step.
- Fails, the next stage will not get executed.
A software practice that can be automated with the combination of different tools related to testing.. Click to explore about, Application Automated Testing
Stages in Automated Testing Pipeline
As an example, imagine a pipeline consisting of four stages, administered in the following order -- Build A job called compile.
- Test - Two jobs called test and test2.
- Staging - A job called deploy-to-stage.
- Production - A job called deploy-to-prod.
- source code in Gitlab
- there is an automated mechanism triggered by a code change that
- builds an app package
- deploys an app on local or in the cloud (e.g., Heroku)
- runs automated tests and offer a run report
What is the process of building Testing Pipeline?
The steps for building testing pipeline are listed below:
Write the script and run locally
First of all, we will write test scripts to let our first manual test run automatically. Selenium is a well - known web automation tool. With the idea of a basic test model, we can quickly identify the three major components in the script -- Set up
- Run test case
- Tear down
ML pipeline is constructed to allow the flow of data from raw data format to some valuable information. Click to explore about, Machine Learning Pipeline Deployment
Setting up a runner with your testing environment
Gitlab runner can be installed in three ways to set up a runner with your testing environment.- Use a Docker
- Download a binary
- Use a repository for rpm/deb packages.
Set up an image with the testing environment
Build an image with the testing environment Test running requires a clean environment. To create a clean environment, we don't want to establish up a real machine each time and wait for hours to install all the necessary software. The concept of the container helps. Docker helps you build an image of your testing environment. The image includes all the software that needs to be pre-installed and run like a virtual machine on that container. You can create a new container with Docker and pull the same image every time, whenever you want to start over from your default environment. Forex - We want our image to pre-install the following to perform our test with the Selenium Python client -- Python
- Google Chrome
- Chrome driver
Setting Up GitLab CI for Automated Testing Pipeline
Create a job template for your product When commit/new codes are pushed to the repo, will look for directory and triggers and build according to the configured settings. GitLab Runner uses this file to manage the 'project's jobs which define how the project should be built.
image: joyzoursky/python-chromedriver:3.6
before_script:
- pip install -r requirements.txt
stages:
- test
- report
test_suite_1:
stage: test
script:
- python -u test_suite_1.py
test_suite_2:
stage: test
script:
- python -u test_suite_2.py
send_report:
stage: report
script:
- python send_report.py
Run and report periodically
Pipelines can be set up according to the user requirements. Tests can be made run on every build triggered on the pipeline or can be run periodically by setting up Cron Jobs and scheduling the time by cron syntax. Two different ways to integrate this into our system -- Running the tests every time a new code gets pushed to the project.
- Running the tests periodically.
The nightly run requirement depends on the team where they want to run all the tests (smoke & regressions) when there is no deployment. Test runs at night will yield useful test reports. Executing the tests on each change of code as part of the CD pipeline. The reason for this is that E2E tests are slow to run. We do not want these tests to slow down our pipeline as it will delay our process and cycle and will affect pull requests, merges, and deployments to different environments. We wanted a set of core End to End tests we can run regularly that let us know if anything is broken or off. So we can run these tests on a nightly basis via a Jenkins cron job.
Reporting Test results can also vary based on the teams, depending on their requirements such as intimating on the slack channel, generating HTML reports, saving test runs in DB and displaying them on dashboards. We can use Slack real-time messaging API to do the reporting so that we can receive notifications in the corresponding project channels.
- Slack Notification
- HTML Reporting
- Dashboard Reporting
- Build your application using the GitLab Runners
- Run unit tests and integration tests to check that your code is valid
- Look at the live preview of your development branch with Review Apps before merging it into stable
- Deploy to the multiple environments like staging and production, and support advanced features like canary deployments
- Monitor the performance and status of your application
Software testing is an important process that ensures customer satisfaction in the application. Click to explore about, AI in Software Testing
Automated Testing Pipeline Approach
Setting up an Automated Testing Pipeline makes QA life easier by finding the bugs in the application. Now, whenever code is updated on the production branch or code change in the project, this automated test is triggered and tests are done automatically and generate test reports to notify Developers and QA. Failure results will be pushed back to Slack channel to notify the developers. This makes the automated testing process becomes more manageable.To Adapt Automated Testing as a testing practice we recommend taking the following steps -- Understand the Role of Automated Testing in Agile Enterprise
- Explore Our Continuous Testing Solutions
