Case Study

Building Test Automation for True Shift-Left

Published on
December 1, 2020
Building Test Automation for True Shift-Left

JATO adopted Virtuoso and completely shifted testing left by authoring tests before the application had been built enabling them to release weeks faster. Here's how.

JATO Dynamics

JATO Dynamics was founded in 1984 and provides the world's most timely, accurate and up-to-date information on vehicle specifications, pricing, sales and registrations. As part of a modernisation programme JATO Dynamics decided to redevelop their core vehicle intelligence platform as a web application. They had their own team developing the back end while a 3rd party software house developed the front end.

The Challenges

Developing and testing the new application presented several challenges. They were trying to build the platform from the ground up with no reusable or pre-existing user journeys. The back end and front end were being developed separately which created inevitable issues around handovers and feedback loops. The frequency and velocity of changes to the back end were affecting the UI and the mobile requirements were completely undefined. Furthermore, time was limited and there weren’t many automation skills or any prospect of increasing the size of the team.

The Solutions

By adopting Virtuoso and taking advantage of its easy to use and intuitive interface, JATO were able to start realising the benefits right away. User flows and wireframes allowed them to begin building automation immediately so that it could run in tandem with development. Testing was able to keep pace with the frequency of changes thanks to Virtuoso’s rapid test authoring.

There weren’t any defined requirements for testing across mobile, but Virtuoso was still able to execute all authored journeys across mobile devices, providing insights for future development. The shortage of automation skills wasn’t a problem either. Virtuoso is so easy to use that with only minimal training JATO were soon able to take ownership of their own automation. They were able to quickly build their own in-sprint automation with no requirements for frameworks, infrastructure or up-front costs. UI regressions were easily identified using Virtuoso’s snapshot feature.

From the beginning of the project JATO was able to achieve true shift-left as 100% of the user journeys were automated before the application was built. Having all of the test cases authored by Virtuoso’s NLP engine means that JATO has dynamic documentation and full version control of the application. They are also able to shift-right by executing future journeys in production.

The Results

How did JATO’s use of Virtuoso translate into numbers? Let’s have a closer look at the key metrics.

  • 0 hours frameworks setup or configuration
  • 1 Sprint to go from zero automation to full in sprint automation
  • 2 days to onboard the team and setup for success.
  • 42 Hours total authoring time, this includes any post deployment tidy up work
  • 100% Of in-scope user journey flows automated before the code was written
  • 80% Of the user flows executed successfully to the point of application failure needing only minor adjustment
  • 21 application snapshots delivering iterative automated behavioural insight and visual testing
  • 400+ Journey executions - End to end user flows to test functionality
  • 5 Minutes - Execution time for all 25 E-2-E user flows

In Summary

JATO successfully adopted Virtuoso on day 1 to shift testing left by automating full feature testing before any of the application had been built enabling them to release weeks faster.

They went from zero automation to in sprint automation in just two weeks and with only a couple of days onboarding and training, with no need to spend time on frameworks of infrastructure.

With Virtuoso, JATO has improved testing efficiency and coverage by utilising a fully collaborative Quality Assistance platform, that has made testing more accessible, faster and easier, whilst also giving them a blueprint for dynamic web testing success.

Subscribe to our Newsletter