Interactive Visual Testing is more than finding image and clicking on coordinate
Earlier, our last blog described a major issue first and foremost leaving you with a messed-up UI. Beginning, with overlaying titles and continuing with a broken language selection.
Therefore, you started to build up your automation with image-based object detection. First it starts quite smoothly, and you quickly build a repository of images that you can find on your UI-screen. Moreover, it then interacts with the mouse and keyboard at the coordinates these images where found.
Next up, you head to the language selection dropdown. That is when you see that things become more complicated. As you make the language selection dynamic. Subsequently, you rely on text recognition, to find the correct selection in your dropdown list. There another issue rises. The language name does not only appear within the dropdown list, but instead also in other parts of the screen. Furthermore, you notice that your OCR struggles with the different fonts and sizes when reading the whole screen. That means you need to detect the rectangle of the dropdown list and limit your text search to this area.
[/fusion_text][fusion_text columns=”” column_min_width=”” column_spacing=”” rule_style=”default” rule_size=”” rule_color=”” content_alignment_medium=”” content_alignment_small=”” content_alignment=”” hide_on_mobile=”small-visibility,medium-visibility,large-visibility” sticky_display=”normal,sticky” class=”” id=”” font_size=”” fusion_font_family_text_font=”” fusion_font_variant_text_font=”” line_height=”” letter_spacing=”” text_color=”” animation_type=”” animation_direction=”left” animation_speed=”0.3″ animation_offset=””]
Finally, you find a way to reliably detect the dropdown list and select a value from it. At that time, the developers push a new update which adds many more languages. What is more, the language selection dropdown now requires scrolling to select a value further down. Concluding, even more work for you to automate the scroll bar in the dropdown list.
What is codified expertise?
But that is where codified expertise comes in. Preceding automation challenges as this at hand, have been solved for other projects before. Being at ease to use the insight, generalize and finally reutilize it. At TestResults.io our library, named Base Model. It is “codified” with all our expertise and experience from former automation projects. This includes support for most UI elements that you will encounter.
In summary, codified expertise eliminates searches for images and interaction based on the coordinates where the image is found. But instead, you are focusing on the UI objects, such as, dropdowns, textboxes, buttons, tables to mention just to mention a few. Each of these elements offer you a range of methods to interact with. The language selection from our example would become a simple one-liner: “LanguageDropdown.SelectValue(“Français”);”. This saves the user of course a lot of time during automation. In addition, the whole automation is more reliable. Based on the fact, that many sources for flakiness are already now accounting in the Base Model.
Codified expertise within Interactive Visual Testing
The concept of codified expertise is in our opinion a cornerstone of Interactive Visual Testing. It is of course a key feature of TestResults.io, but it can (and should) be applied to other Testing Tool as wells. A tool that uses an object-oriented language (such as C# in TestResults.io) suits codified expertise naturally, but we have seen similar implementations of codified expertise in tools such as Eggplant that use a much simpler scripting language. In that case there was much more copy-paste instead of just instantiating a new object, but the main goal of reusing codified expertise was achieved as well.
Codified Expertise guarantees structured approach
For a successful and reliable test automation a good structure of the automation is important. For us this is part of codified expertise. The Base Model directly verifies every interaction with the subject under test. That again, is directly built into the interactions of the UI elements, thanks to our codified expertise. In other words, you typed a text within a textbox at the same time our Base Model verifies directly that now the textbox contains the correct text. Not only, are you ensured to continue with the next step, but also that the actual step is really completed. We will explain this in more detail within our upcoming blog on “No wait code”.
Hierarchy of models
The recommendation by using codified expertise in your test automation is to implement different hierarchical layers.
Beginning with the lowest level, which consists of the execution engine. The execution engine takes care of the basic interactions with the subject under test. Mainly finding text and images as well as using the mouse and keyboard. Succeeding with next layer on top, the Base Model. As mentioned before, it combines these simple interactions into functionalities on the UI elements. As described by selecting a value in a dropdown.
Followed by an additional layer on top, called the software model. The software model not only keeps the test case as simple as possible, but also avoids code duplication. What is more, it organizes the UI elements not only in individual screens, but on top allows to execute simple task with only one single instruction. E.g. entering a username and password in a sign-in mask and then clicking on the login button.
The top layer in the hierarchy is the test case itself. With this hierarchic structure the test cases remain simple and readable.
In conclusion, maintenance of the whole automation solution is also a lot easier, since not every single test case has to be changed. In other words, e.g. an additional verification code needs to be entered for the login. Instead of changing every single test case, you modify the software model with the new login method. Subsequently, every following test case uses the new login method.
Interaction with OS with Environment Model
Codified expertise does not stop at the general interaction with UI elements of the application under test. Often, we need to interact with the operating system (OS) when automating test cases for some application. Examples are down- or uploading a file to a webpage, changing a value in a configuration file, making logfiles accessible as test artifacts, or just simply installing the latest build of the application under test. For this we need a special software model that can interact with the OS. In TestResults.io this is provided as the Environment Model.
Codified Expertise allows quick and reliable Test Automation
With Codified Expertise you can rely on the combined experience of many years of testing of our engineers. This allows you to quickly develop reliable automated tests for your application.
Even your initial task of selecting the language from the dropdown is now just a single method call. The Base Model takes care of the rest. If you want to know how to reliably detect text on the screen, even with special characters such as the ç in Français read our next blog about “Reverse OCR”.
Leave a Reply