Features

Fully Integrated Behavior-Driven Development (BDD) and Testing Support

The Behavior-Driven Development approach – BDD – centers around stories written in an “ubiquitous language” that describe the expected behavior of an application. The use of the human-readable language Gherkin allows technical and non-technical project stakeholders to participate in the authoring of feature descriptions and therefore tests. Those descriptions serve as a base for the work of both developers (specification and feature descriptions) and testers (test steps).

Using this approach, the test logic is clearly separated from the test implementation – leading to a clean and future-proof test design.

The Squish IDE provides never before seen tooling support to create, record, maintain and debug Behavior Driven GUI Tests

BDD is a recognized and highly regarded development and testing approach made popular by a wide range of test frameworks such as Cucumber, Behave and others. Squish’s support for BDD is unique because it tightly combines and integrates the BDD approach with GUI test automation.

Squish is 100% compatible with the Gherkin language, the de-facto standard among all major BDD frameworks, allowing the reuse of existing Gherkin Feature Files in Squish without modification. The Squish IDE provides never before seen tooling support to create, record, maintain and debug Behavior Driven GUI Tests. Test steps can also be implemented in any of Squish’s supported script languages (currently Python, JavaScript, Ruby, Perl and TCL).

Squish also provides a tremendously valuable and handy reporting feature for behavior driven tests: Feature files in the Squish IDE are color-coded after a test run based on the results of the individual test steps.

Squish IDE: BDD test (feature file) with color-coded test run results

he ability to generate a very rich and powerful HTML report for BDD tests is also available. The new reporting has also been integrated in all major Squish ALM and CI integrations. Using of any of Squish’s test management and continuous integtration plugins, you will also benefit from the rich reporting features of Squish’s of BDD support.

Squish also allows BDD test cases and pure script test cases in a single test suite, for example sharing script code between script test cases and BDD test cases. We view this as gradual migration to BDD from existing, pure script-based tests – helping make the transition both easy and possible – all in a well supported environment.

BDD Tutorials

The video below demonstrates how to create your first BDD test case in the Squish IDE:

Tutorial: BDD GUI Test Recording

Tutorial: BDD Step Parameters

Tutorial: BDD Step Data Sharing

Smart Test Recording and Playback

Record test scripts with confidence, as Squish GUI Tester interacts with the application objects, captures user actions and displays the recorded script in your preferred scripting language. Run your recorded script, observing Squish intelligently waiting for objects, and progressing through your script as rapidly as the application allows.

• Automatic test script recording
• High-level interaction and object recognition (e.g. clicking a menu item) instead of low-level events
• Scripts independent of screen coordinates and resolutions
• Extend exiting scripts with Snippet recording
• Generate script code in Python, JavaScript, Perl, Ruby or Tcl

Test Verification & Validation: Verification Points and the Object Picker Tool

Confirm your scripts are producing the expected result at any point in your tests. Perform positive and negative testing, further validating each scripted test case, whether confirming data visible to the end user, verifying underlying objects and object properties contain the expected values or visually comparing results.

Create verification points during initial recording, while recording a Snippet, or manually scripting.

View object property details, by selecting an object using Squish Spy. Determine the valuable properties, viewing their current values, and create verification points to validate one or more properties throughout your tests.

Intelligently compare images, locating images using object properties, while adjusting image comparison modes (strict, pixel or histogram), applying positive and negative masks, adjusting thresholds allowing for a user-defined margin or error, as well as comparing and adapting each verification point for interface changes over time

• Automated Test Case validation using Verification Points
• Create verification points while recording, or when refactoring scripts
• Point-and-click or manual verification point creation
• Create global, local or scripted verification points
• Verify object properties, entire table contents and compare images
• Select the object(s) to verify using the Squish Spy’s object Picker

Learn how to add a property verification point to your test case with the following short tutorial:

Powerful and Intuitive Integrated Development Environment (IDE)

The Squish IDE, built on Eclipse, provides a feature-rich integrated development environment for GUI testing. Complete with test management, script debugging and an interactive application object spy.

Move easily from one task to another, using Squish’s customizable activity-based Perspectives, automatically placing the correct tools at your finger tips as you create and manage your test suite.

• Record and execute tests with a single click
• Review interactive Test Results describing errors and highlighting corresponding lines in the script
• Quickly organize your Test Suite, dragging and dropping Test Cases to the desired order of execution
• Create and maintain local and global scripts, verification points and test data from a single interface
• Inspect your application objects using the object Spy
• Set breakpoints, stepping through scripts, monitoring variables and recording Snippets, or additional actions
• Interact with application objects, methods and scripting language specific commands using Squish Console

Choose from a Variety of Non-proprietary Scripting Languages

A variety of non-proprietary scripting languages are available when working with Squish. Select from Python, JavaScript, Ruby, Perl or Tcl, and take advantage of the world of knowledge at your fingertips in each of the scripting languages.

Each scripting language offers its own advantages, whether based on your existing knowledge of one or more of the languages, or by addressing a key test scenario using an existing and proven approach already available in the scripting language.

• Create Test Cases using Python, JavaScript, Ruby, Perl or Tcl
• Leverage your existing scripting knowledge and strengths
• Discover and use existing modules and approaches in each non-proprietary scripting language.
• Incorporate your existing scripts in your Squish Test Suite
• Access a world of expert information for each scripting language

Drive your scripts with data from a variety of data sources. Incorporate logic in your scripts and iterate through extracted data, using the data in your scripts. Data-source connections can also be used to output information to your data source(s).

Data driven testing allows you to repeat test scenarios using various input values, enabling you to quickly test how your application handles mass data-entry, specific characters or data entry values.

Drive your tests using a variety of supported data sources including, but not limited to:

• TSV
• CSV
• TXT
• Microsoft Excel spreadsheets
• Databases
• and more

This video will show you how to implement a data-driven GUI test. Not sure how to rewrite your script to accommodate the new input data? No problem. The Squish IDE will refactor your code for you. Have a look:

Execute sets of scripts, or batches, across multiple destination environments. With each targeted batch, confirm a specific set of functionality, across one or more environment configurations. Create and execute a library of batch scripts, targeting portions of the application under test (AUT), or the entire AUT. Quickly validate batch results from the comprehensive report, summarizing the testing success as well as identifying and detailing any regressions.

• Execute sets of scripts or batches.
• Group related tests or dependencies into a single execution.
• Automate batch distribution to multiple destination environments.
• Validate requirements with requirement use case batches.
• Confirm key functionality across each supported environment configuration.
• Determine product readiness using the comprehensive batch reporting.
• Exercise targeted functionality with feature-specific batches.
• Regression test more in less time, unmanned, and against multiple environments.

Incorporate Squish GUI Tester into your application life cycle using our ALM, Continuous Integration, Build Integration, Test Management and Software Project Management integrations.

• Access and execute Squish tests from a 3rd-party interface.
• Incorporate Squish tests into your build process, identifying regressions before builds get to QA.
• Schedule routine and unmanned Squish test executions.
• Introduce a collaborative automated GUI regression testing environment.
• Associate Squish tests with defects, requirements and work items.

Squish Tool Integrations

The following add-ons and integrations are available for Qt Squish and provided by Qt:

The following add-ons and integrations are available for froglogic Squish and provided by 3rd party vendors:

• Build Integration/Test Management: CMake/CTest/CDash by Kitware
• Test Management: XStudio by XQual
• Test Management: SpiraTest by inflectra

Powerful GUI Object Map

Record test scripts with confidence, as Squish GUI Tester interacts with the application objects, capturing user actions and displaying recorded scripts in your preferred scripting languages. Captured objects details, or properties, auto-populate the GUI Object Map, storing objects hierarchically and containing only the required information to correctly identify your application objects going forward.

Scripts contain symbolic names, which refer to entries in the GUI Object Map, requiring modification to single location in the event of an object changing, instead of each instance of that object in the test suite. recorded script, observing Squish intelligently waiting for objects, and progressing through your script as rapidly as the application allows.

• Automatic test script recording
• High-level interaction and object recognition (e.g. clicking a menu item) instead of low-level events
• Scripts independent of screen coordinates and resolutions
• Extend exiting scripts with Snippet recording
• Generate script code in Python, JavaScript, Perl, Ruby or Tcl

With a comprehensive list of supported toolkits, across multiple platforms, Squish GUI Tester prevails as the single GUI testing solution for all your applications under test (AUT). Create and manage your automated GUI tests from a single IDE, as Squish hooks into your AUT, working with each toolkit or technology, natively understanding and interacting with each object.

• Seamlessly automate applications built using a combination of toolkits.
• Focus your efforts on testing, and let Squish handle toolkits interactions.
• Use the same Squish GUI Tester IDE regardless of toolkit.
• Test technologies natively, even when the AUT contains multiple toolkits.
• Simplify your GUI testing, with one solution, Squish GUI Tester, for all your applications.

What is Visual Verification?

Conceptionally visual verifications are a hybrid form of property verifications and screenshot verifications. Both types of checks are combined into a single complex algorithm to find the best possible match.

Unlike screenshot verifications, they do not just treat the rendered screen content as a single surface. Instead, the area is divided and compared per individual UI control. Rendered pixels, position, size and content properties will be compared against expected values.

What Visual Verification Points can do for you?

Visual verifications can check the on-screen display of dialogs, whole application windows or subsections. By default, all checks will be performed in a strict manner that does not allow for changes in state, layout and rendering.

However, an editor is implemented that allows modifications of individual checks to make them more tolerant.

How does it work?

One example where visual verifications are useful is when we need to verify that a dialog (or Web page) as a whole still shows the expected text and images (and no more) with the exact layout and UI theme differing between respective client platforms.

When a verification point has been added any screenshot that has been taken are saved along with object property values and geometries, as part of the expected results.

When the test script is replayed, whenever Squish reaches a visual verification point it takes a snapshot of the relevant widget and compares this with the corresponding snapshot that it originally took, and reports success or failure for the verification point depending on whether the snapshots are considered to be the same or not.

In addition, if a visual verification fails Squish saves the new (and different) snapshot for later inspection in the log. Which allows us to view the differences between the expected snapshot and the actual snapshot by right-clicking the failure message in the test log view.

Through wildcards for property values, tolerance ranges for geometries and correlation threshold for image data; Squish can show the differences between snapshots side-by-side and allowing for tuning of the comparison through changes of the check settings.

Flexible, Configurable Image-Based Lookups

In those cases for which Object-based recognition is not suitable, the Squish GUI Tester offers an Image-based recognition approach. Image-based testing is best used for applications with custom controls, applications built with unsupported toolkits for which object properties are not readily available, or 2D/3D graphics or plots, such as medical imaging data. The Squish GUI Tester is often the preferred test automation tool, owing to its support of both Object- and Image-based recognition methods, which can be used in a complimentary manner.

• Highly configurable Image-based lookups (per-pixel tolerance, image cross-correlation, multi-scale image lookups)
• Use of image groups to represent a single UI object (e.g., for those components that have differing appearances due to different rendering styles or cross-platform tests)
• Powerful Fuzzy Image Search

AI-powered Onscreen Text Recognition

Optical Character Recognition (OCR) is a technology that enables the digitization of scanned images with printed or handwritten text into machine-readable data that can later be used for electronic editing. Squish includes this technology as a compliment to its already powerful Object-based and Image-based recognition methods, offering it as a method of onscreen text recognition and verification.

Variability in a component’s visual appearance is particularly prominent for onscreen text when trying to create platform-independent tests, due to a wide assortment of fonts, font sizes, decorations and rendering modes. Thus, Image-based recognition methods, including Fuzzy Image Search, are generally unsuitable for locating text onscreen. OCR allows for efficient text handling in those scenarios where the same text is rendered with different parameters, making it look largely dissimilar in pixel-to-pixel comparison (i.e., due to varying letter widths, different kerning or shifting line break positions).

Challenges in Remote Device Testing

Testing remote systems brings unique challenges. Where the local and remote system are not geographically co-located, interactive script recording and debugging can become both cumbersome and time-consuming. Existing remote desktop solutions bring problems of their own: interoperability of the controlled and controlling platform, compatibility of the test framework and remote control application, setup and configuration issues, and others. Accessing certain embedded or mobile devices may not even be possible.

How It Works

The Squish GUI Tester relieves each of these pain points, through a fully-integrated, one-click remote control solution for virtually any target.

When working with a remote Application Under Test (AUT), Squish allows you to stream the display of the remote device locally on your machine, in real-time. When interactively recording tests, you can, from within the remote viewer, actually pick UI objects for inspection, use the Squish Spy tool for setting up a verification point, preview the current AUT state during a debug session, and more.

The data required for this functionality is embedded within regular Squish communication, meaning the required effort to get started with remote testing should be, in most cases, just a click away.

Remote control was designed to streamline test development of virtually any target, including desktop computers, mobile devices and even embedded platforms.

Tutorials & Documentation

We wrote a tutorial on testing an In-Vehicle Infotainment system running on an embedded board, using remote control to record and refactor a representative GUI test. Follow along here.

Check out our documentation for a complete list of Squish settings related to remote control.