Shift-left Testing: Reimagining What "Testing Early" Means With the SDLC and Virtuoso

By

Hugo Farinha

,

Co-founder and CTO

2021-01-26

The term “shift-left” has been around for some time but it is worth reminding ourselves what it means in terms of the Software Development Life Cycle (SDLC).

Looking at a common SDLC and the different processes that some teams adopt, testing comes after the coding stage and before deployment  (Figure 1).  If we were to take a literal shift-left we could end up in a pickle.

By shift-left testing, someone may think it is to shift the SDLC test phase completely to the left, between design and code, which obviously would not work (see Figure 2). How can you test what doesn’t exist, or how do you test code that has not yet been written?

So, if shift-left testing doesn’t mean to shift the SDLC test phase completely to the left, what does shift-left actually mean? Shift-left seems to be a fancy term for starting doing tasks, or at least thinking about doing them (establishing a testing framework), earlier in the process. This is not meant to derail the current state of shift-left and the SDLC but to show possible improvements leveraging the latest technology allows. Since its inception shift-left has delivered countless advances and benefits.

Benefits to shift-left

Part of the impetus for the widespread adoption of left-shifted testing is its potential to offer a high ROI. The Systems Sciences Institute at IBM found that the cost of finding, or preventing, potential bugs before the implementation and testing phases saw savings of 650% and 1500% respectively, or $650 and $1500. And if those bugs found their way into the end product $10,000. That’s not to mention any negative effects on a company’s reputation. Left-shifted teams also work more collaboratively, with quicker release cycles, and produce higher-quality products.

The reimagined SDLC

We have witnessed the benefits of shift-left, but can we shift further left, can we actually test earlier? Tools like Virtuoso enable us to completely reimagine shift-left’s relationship with the SDLC and offer better quality products, shipped faster, and even greater returns on investment. Let’s see how.

SDLC reimagined

Shift-left testing during plan and design phases

As Forbes Magazine points out, one of the most common reasons software projects fail is poor requirements and the lack of collaboration between teams during the planning phase. There have been many studies across the industry which show what happens when teams don’t fully understand what the business needs are and the associated requirements. Shift-left attempts to alleviate this by ensuring QA teams are part of the planning stage and can establish coding standards, protocols, rules, and guidelines with corresponding acceptance criteria. This is, of course, a good thing but we can, at this point, leverage the latest technology and take an extra step left.

Virtuoso allows you to import requirements, generate the test structure, and ultimately generate executable tests before the design phase. This approach ensures that:

  • Resource allocation to testing gets more efficient
  • Missing or incomplete requirements can be detected before moving to the next phase 
  • Defects are discovered early before significant investment
  • Business requirements are aligned with the product

And our steady march to the left does not stop there. Thanks to Virtuoso’s very clever Natural Language Programming and game-changing Intelligent Object Identification (we will come to those shortly) you can create fully functioning tests with nothing more than a handwritten wireframe. It’s time for an example.

Below you can see a simple wireframe for a login. This one is not handwritten but it could be. Virtuoso users can build tests in plain English describing journeys through this application before a single line of code is written. For example, entering login credentials and clicking login (see figure3). All you need is the wireframe, Virtuoso’s ability to author tests in plain English, and object identification that behaves like a human. Virtuoso’s Intelligent Object Identification does not rely on XPaths or CSS but finds objects intelligently, like a human would, which enables tests to be authored before the coding stage. Test authoring has literally been shifted to the left.

login wireframe
Figure3: Example of a login wireframes on the left and Natural Language test automation on the right

Natural language test automation and intelligent object recognition: A quick guide 

Natural language test automation relies on intelligent technology that allows anyone, regardless of their technical skill, to automate tests in plain English. Tests are not dependent on element selectors. This means selectors do not need to be maintained and tests can be created before the application is built. Natural language test automation can be completed independently of the application and can be executed during each release to test if the application matches the requirements.
Intelligent Object Identification selects elements like a human, based upon a hint (e.g.: “field password”). Virtuoso then selects an element in the application to interact with, and, store and update it's references. This means that should the element’s position or properties change, the tests don't break. Tests are self-healing.

Shift-left testing during code phase

During this phase, the application is being produced and the fact that tests have already been created in the previous phase empowers testers to help developers more and at greater speed. Tests can be executed in the developer’s own environment meaning developers can have their code validated against requirements earlier. And the developers can repay the favour, informing testers of shortfalls in their tests ensuring smoother test execution throughout the cycle and, ultimately, that a higher-quality product is delivered faster.

When using a Continuous Integration (CI) automated process, code can be validated against the requirements the second it is sent to commit. As natural language automated tests have already been authored there is no test authoring post-commit. Test automation has taken another step to the left. Testing encompasses the coding stage with tests written before and executed both during and immediately after implementation.

Reducing end-to-end testing

With Virtuoso end-to-end testing can start earlier as the tests can be authored before the application is created. In addition, the reliance on end-to-end testing has been diminished as testing and development have worked together more efficiently, found bugs and regressions, and improved tests. But that is far from it. As Virtuoso finds elements like a human and does not rely on selectors, tests are self-healing.  If an element changes slightly Virtuoso will understand that change like a human would, ensuring that all tests are robust, low maintenance, and self-healing.

Maintenance in production 

Production systems are live systems, which means after deployment the system keeps evolving, databases grow in data, performance may suffer degradation over time and it may be too late before this is noticed. Virtuoso has built-in production monitoring ensuring that you only need one test automation tool for your entire SDLC.

Conclusion 

Shift-left has revolutionised the SDLC and brought countless benefits. However, third-generation test automation tools such as Virtuoso allow us to build on the benefits shift-left has delivered. We can reimagine the SDLC, shift test authoring left, execute tests earlier, and dramatically reduce test maintenance. We are in a new era of test automation and teams will need to embrace new technology and models if they are to keep pace with an ever-changing industry. Those who do will stand on the shoulders of giants and see their teams deliver even higher-quality products even faster.


Tags

Building Test Automation
for True Shift Left

How to shift your testing left and build web automation that keeps up with your development.

Download eBook

Subscribe to our Newsletter

@ Copyright 2021 SpotQA, Creators of Virtuoso – PRIVACY POLICY