Failure to automate - how to avoid the pitfalls and reap the rewards of smart automation

Looking one step ahead, smart automation promises faster release cycles, better test automation and smoother collaboration between all practitioners.  However, even though most companies believe that smart automation is useful, few companies can claim actual success.

 

Many organisations are still simply not sure what to automate, how complex the process will be, or how long it will take - so many attempts are destined to fail.

 

So how businesses turn the tide and make automation efforts a success? We believe that 2019 is the year for DevOps pros to make automation smarter.

 

Why is it failing

 

First and foremost, the complexity involved in automation strategies shouldn’t be underestimated. Automation in the digital era is simply more difficult than ever before.

 

Desktop and mobile browsers support increasingly more advanced capabilities. Services like FaceID, location capabilities and Virtual Reality (VR) /Augmented Reality (AR) are tough to automate and scale across multiple permutations. In addition, the available time within software iterations is continually shrinking - and so, for many, that means that there simply isn’t sufficient time to create automation that really works.

 

Test automation creation isn’t always structured or able to follow best practices and this leads to flaky and inconsistent test scenarios - something which is exacerbated by teams that don’t have the test automation skills (or time) to lend to the task.

 

So there’s a lot to contend with to make automation strategies a success.

 

A move to smart automation

 

To start to overcome automation roadblocks, it’s important to remember that not all test automation is created equal.

 

Let's expand.

 

At its most basic level, automation stands for the process of taking a set of requirements and creating code to implement them in an automated fashion. Traditional automation refers primarily to static test scenarios, such that whenever a change or occurs in the app, it requires code changes, refactoring and other time-consuming tasks. In addition, traditional automation is error prone since it is created manually, and so subject to human error (wrong object IDs, duplications, wrong prerequisites to the test scenario implemented, lab issues, etc.).

 

Smart automation, on the other hand, takes traditional automation to the next level by minimising the risk human error, as well as eliminating the need for test maintenance. Since smart automation often relies on record and playback, there is close to zero code that needs to be managed, maintained, and this is clearly a huge benefit to teams. Coders at every skill level can use these tools, the output (the tests themselves) are more reliable, robust and the flakiness that traditional code-based automation creates is reduced to minimum.

 

Getting it right

 

If automation is the ambition, organisations need to fundamentally determine what it is they want to achieve. Automating everything shouldn’t be the goal of continuous testing in DevOps. Instead, what’s important is automating what’s relevant from a risk mitigation / reduction perspective, and focusing on what provides value and feedback to the developer.

 

In today’s reality there should be a clear balance between smart automation and traditional automation - both are useful and have a part to play in the development process. If tests are flaky, manual or hard to automate due to time / complexity, then they should be pulled into the smart automation tools to boost test automation coverage and reduce the risks to quality issues in productions. Other scenarios, such as tests which are reliable and stable, can be handled traditionally within the DevOps pipeline.

 

So, a circumspect approach is important, but whatever the type of automation you’re employing, or your goals for doing it, it’s crucial that organisations have the right tools in place to succeed.

 

Using machine learning (ML) and artificial intelligence (AI) in smart record and playback test creation allows robust object identification and analysis. Smart test execution allows elasticity in scale - smart labs can self-heal itself when a platform either disconnects or busy - and smart analysis can shrink the time needed to analyse and debug issues - especially when dealing with big test data.

 

Support for highly complex scenarios especially in the mobile space requires more mature version of the smart automation tools, many of which are currently available, but with more tools being released to market all the time.

 

--

 

It is clear that automation is crucial, and smart automation is the ultimate ambition for many. Indeed, at Perfecto, we believe that smart automation will become the foundation of DevOps continuous testing across all verticals. Such automation will in most cases be heavily supported by AI capabilities that will allow test automation to be created almost end-to-end with minimal human involvement. The generated automation will be very reliable, tuned to the business process, tied to the analytics to match the right coverage and deliver fast feedback to the teams.

 

However, only the organisations which take strategic approach, with clear goals, and with the right tools in place, will make it a success.

 

Any company looking to complete a successful automation strategy should first assess their current state: looking at what tools are available today within the organization, what resources and skills are available, and what the teams’ future roadmap looks like. They should then drill down deeper still, looking into the pipeline to realize the percentage of reliable tests, the percentage of successful CI builds - and the covered vs. uncovered functional areas in the application.

 

This will allow them to build a pipeline of tests that are candidates for smart automation migration - and, equally importantly, decide which ones are not. And it’s only once this process is in motion, that the value of smart automation can truly be realised.