Interactive Visual Testing

From Visual Testing to INTERACTIVE Visual Testing

You cannot wait to test your site, after adding a new language. Testing if everything is alright. And it is, you are quite proud of your work and send your boss the news. He is more than happy to look at the newest feature of the companies’ site. Since he is on the road, he pulls out his smartphone and calls up the page.

What he sees does not fill him with pride. Quite the opposite happens, he calls you up. He asks you explicitly if you have tested the new application. You are irritated, on your desktop everything is looking nice and works well. Even the testing came out with thumbs up and ready to go live. So, what is he talking about? And then you know it, you minimize the screen and look at the outcome. There it is: overlapping elements. Titles are overwritten by the rating points. And not only in one listing point, no all.

What went wrong?

Visual Testing finds your UI problems

It seems that despite your functional tests are all passed, there were still UI problems that went undetected. You are missing Visual Testing, which basically means to ensure that your application appears to the user as you intended. This often implies checking the application on many different browsers, devices, and screen sizes. For certain, a manual execution generates the highest confidence in the correctness of your UI. On the other hand, it involves a lot of time and effort, that could be used somewhere else. Therefore, the combination of Visual Testing and an automated testing solution is needed.

Essentially, Visual Testing is a screenshot comparison

For automation techniques involving object detection based on object IDs or other properties this cannot be done directly. Since you are testing the code that displays the UI and not the UI itself. So many UI problems will still let a functional test case using code-based object detection pass. This shows in the popularity of tools as e.g. Applitool. Firstly, the tool takes screenshots, generated by a Functional Test. Secondly, it compares them with a determined master screenshot.

While this approach can uncover many problems in your UI, it also has its limitations. Especially for dynamic content, and it results that you basically have to maintain two test automation solutions and tools, one for Functional Testing and for Visual Testing.

To illustrate the problem with dynamic content let us look at the screenshot with the language drop-down again. As long, as you have just those four languages Visual Testing will execute properly. It ensures the correct display of the drop-down and the entries of the languages.

But now assume that we introduce even more languages to the site. Consequently, the drop-down list is a lot longer and causes a deviation in the Visual Testing. Although, the display of the drop-down list is perfectly fine. Then again, a wrong entry may not even be visible anymore since you need to scroll the display. By scrolling the display, Visual Testing is no longer checking this part.

Applying now the Interactive Visual Testing

With Interactive Visual Testing you are combining the Functional Testing and Visual Testing in one test. Instead of relying on the code that displays your UI, it interacts directly with the visuals of the UI itself. The core concept is to interact with the application in the same way as a human user would do. In other words, the user can use the application without any hurdles. As for the Visual Testing, a test failure will emerge, caused by the many occurring UI problems.

Applied in our two presented examples. Firstly, a smaller device displayed an overlapping text within TripAdvisor. Secondly, the French language not rendered correctly within the drop-down list. The Visual test does not find the results within the venue list. The reason being the overlapping text displayed on the screen. Furthermore, it fails also to select the French Language in the drop-down list in the second example. Accurately, Interactive Visual Testing would report a test failure.

Unlike the general screenshot comparison approach, Interactive Visual Testing is resilient towards new layouts or dynamic data and elements. Even, if you want to compare your test case against the “master” screenshot of your UI, it is still possible. Since a screenshot comparison remains a by-product of Interactive Visual Testing.

Demonstrating the strengths

Interactive Visual Testing has many other advantages in comparison to automation testing solutions using code-based object detection.

·         It is platform and technology independent:

Whether you are testing a webpage, a desktop application, or a mobile app, you can use the same approach to automate your tests. There is no need for an automation specialist anymore. Specialized in Selenium, Ranorex, Tosca, Espresso or any other tool in place to cover your full technology Portfolio.

·         Zero Intrusion:

Furthermore, you do not need to install any test runners, web drivers, or even add specific hooks in your source code. To make your application testable. All you need is the image of the UI and the capability to control the mouse, touch, and keyboard inputs. Available on most systems you can use the VNC or Remote Desktop protocols.

·         Simple handling of dynamic elements:

Dynamic UI-elements is the area where a code-based object detection struggles the most. Especially for data virtualization, responsive layouts or customizable dashboards where the UI consists mostly of dynamic elements, the limits are being reached. Whereas Interactive Visual Testing is not being affected in any way since there is no dependency on any IDs or properties which are generated dynamically.

·         Resilient against any changing object IDs or properties:

Within most automation projects, a major time-consuming task is the maintenance of a test automation solution. Broken tests mostly occur when the application is under test development. Here finding an object is impossible. Since the broken tests are caused by changed object IDs or properties.  Luckily, such technical changes, often have minor impact in the display of the UI. The elements are traceable with Interactive Visual Testing. Which results in less maintenance versus the code-based object detection.

How to make it work

In conclusion you will only be able to achieve the boss’s approval by applying a test automation solution. A solution that combines functional testing and Visual Testing in one. Not only you can mend at the very instant broken views, but also detect any further UI problems. Relying solemnly on the automated testing with Interactive Visual Testing.

Within the Interactive Visual Testing – equation belong also how to achieve robust and reliable object detection in automation. These points are presently topics as “Codified Expertise”, “Reverse OCR” and “No wait code”. Please stay tuned for the upcoming posts on Interactive Visual Testing.