Continuous Integration/Continuous Deployment (CI/CD) has changed the way we develop software, and testing has had to embrace automation to keep pace with demand. And these new test automation platforms are offering more than just a stamp of approval.
We once understood testing simply as Quality Assurance: manual testing that's a time-consuming, box-checking activity that is at odds with the automated, highly integrated approach to software development most teams now practice.
DevOps organizations have begun to develop a “Quality Assistance” attitude towards testing that keeps the development pipeline flowing through automation. What was once a bottleneck is now a facilitator of speed and quality. But the sophistication of these new tools continues to add value in post-production.
Today, customers and user bases of digital services and products now expect to receive a rich, faultless experience anywhere, on any platform, and at any time. Our tolerance for faulty software or a poor user experience has rapidly gone from low to zero.
And that's not the only set of demands we face. The arrival of CI/CD and DevOps teams has contributed to a huge increase in software releases. There's more to test, and typically we're never more than two weeks from making a release.
Taken together, this means QA has to facilitate pipelines that are releasing software faster and faster to end users who are increasingly impatient with downtime and poor performance. The value of ensuring consistently positive customer experience and site reliability has never been higher.
To actually understand production trends, you need to put yourself in the customer's shoes. What do they feel when accessing your application? How much time do they have to wait to complete a journey? What is their experience at a given point in time? Having this knowledge means having control.
Performing quality control by monitoring your production environment throughout the day allows you to measure application performance during peak hours. As well as noting any application downtime, you can keep a close eye on functional health and application performance.
Being able to call upon quick responses to failures means you really have to understand how the system performs in response to real user behavior. Seeing what your customers' preferences and engagements are when applying a different application flow or important design changes will give you valuable knowledge. Traditionally, we build confidence by executing as many tests (all kinds of tests, from performance tests to integration tests) as possible prior to the release, but this approach has many flaws. It provides no assurance that the product will not fail in production because all the tests have been done within dedicated test environments.
These are some of the testing strategies and types of testing that will help you to achieve effective CX monitoring:
In DevOps organizations, particular attention is paid to the ability to continuously shift right by testing in production. And when failures happen, the product team acts by deploying only small changes, released incrementally while the health of the application is closely monitored on a daily basis. It isn't just about performance monitoring either. New, intelligent test automation platforms allow us to do much more in terms of functional testing and CX than is being done today.
Above all, it is important to be able to react quickly throughout the testing process and to maintain control of live situations. Testing in production makes it possible to understand real user journeys and analyze real use cases that are extremely hard to predict.
In the end, the only way to prove that software is ready for production is to release it into production! Automation in testing is giving us the freedom to do so with confidence.