In this article we will describe the seven-steps to Adopt Continuous Testing:
- Define Tests Early, Even Before Writing Code!
Lack of clear requirements or the incorrect interpretation thereof leads to rework and delays. Techniques such as Behavior-Driven Development (BDD), Acceptance Test-Driven Development (ATDD), and Model-Based Testing using tools like cucumber/gherkin, and CA Agile Requirements Designer (ARD) allow business stakeholders, product managers, developers and testers alike to ensure that requirements are clearly documented and communicated, test cases clearly defined, and test scripts created ahead of time to enable swift testing of code.
- Optimize Tests and Coverage
Some organizations default to “running all the tests all the time” to ensure code coverage. This wastes resources and lengthens test cycles, without actually ensuring adequate code coverage. Save time, money and resources by testing only what you need to test.
Visual models provide different paths to be discovered and optimized such that only a handful of tests gives maximum coverage. It is also possible to import test cases, remove duplicates and distribute and allocate optimized test cases in existing tools such as Rally, Jira, HP ALM, JIRA and more.
To achieve “in-sprint” testing, shift testing left – so that tests can be run earlier in the development cycle. Developers test as they go; the CoE provides expertise, specialized systems, and services. Test automation spans UI, functional, performance, and security. Teams work together and are focused on the business value delivered to the customer. This needs developer-friendly techniques as well as a culture shift.
- Provide Complete Test Environments
The ability to provide test environments is critical to achieving continuous testing. Eliminate blocks and reduce wait times by providing complete test environments on demand with dev-friendly tools (like code, CI/CD integrations, supported open source). These environments should include:
- Virtual services – provide robust simulation for unavailable, inaccessible, or still-in-development services. Developers and testers can continue to work in parallel by using the virtual service to simulate the response from the actual service.
- Test data on demand – help ensure teams can perform comprehensive tests with production-like data.
- Ephemeral environments – ready on demand, decommissioned after use.
- Get the Right Test Data
The inability to get robust test data causes significant delays in many application release cycles. To precisely test new features and functionality, test data must be as close as possible to what the software will encounter in production. If the test data lacks particular real-world characteristics then the tests are improbable to find many probable issues or break the application where there are pathetic points.
Understanding data models in order to extract the right data is a special skill in and of itself. And production data is mainly the most realistic data to use for testing, data privacy rules also restrict its use.
Below, we’ll take a look at how CA Test Data Manager lets you copy production data and mask it to protect sensitive information while maintaining the characteristics that make production data desirable for testing: realistic and referentially intact data across rows. Test data can also be generated synthetically from scratch using TDM when production data simply cannot be used.
- Don’t Forget to Shift-Right!
Shift right, to use data from the development cycle to fine-tune tests, optimize test cycles, and build optimal regression suites. Shift right methodology consists of real and synthetic user monitoring, A/B testing, chaos engineering, canary deployments, etc.
For example, by shifting right, you can identify which features are being used in production and ensure your regression tests cover those features. In the same way, you can release a new feature to a small group of users (internal or external), understand the impact of that feature in production, and adjust as needed. Many of the nimblest companies like Facebook and Netflix rely heavily on shift-right testing. Gartner recently released a report on the shift-right testing, calling it a “must” for adopting DevOps practices.
- Use Data and Metrics to Continuously Improve
Develop cross-team association and incessantly improve with actionable analytics and feedback structure.
Continuous delivery and continuous testing are journeys. Make sure all your teams have KPIs and access to data that will inform additional optimization. Don’t just collect data that is easy to collect, collect and report on the data that will provide actual insights to continuous improvement. Look to release orchestration applications to deliver this insight and a “single source of truth” for all teams across the application delivery lifecycle.