The Eyes GitHub integration builds on the branching capabilities in Eyes, and on the GitHub pull request.
Eyes supports multiple branches, each of which can consist of multiple test baselinesDefines the sequence of images to which the sequence of images captured at checkpoints will be compared. A test can have multiple baselines, each of which is charachterized by the execution environment it ran on (OS, Browser, Viewport size). A baseline can have instances in multiple branches.. Eyes supports creating a new branch based on an existing branch, independent updates to the baselines in each branch, comparing branches to detect conflicts, and finally merging of the baselines in branches.
This article focuses on the Eyes GitHub integration and assumes that you are already acquainted with GitHub and a CI system that is integrated with GitHub.
Your CI needs to be set up to configure the Eyes SDK with information related to the GitHub pull-request. For detailed instructions on how to configure a variety of popular CI platforms see Configuring your CI for the Eyes GitHub integration.
The SDK uses the value of the environment variable APPLITOOLS_API_KEY to obtain the Applitools API key and the value APPLITOOLS_BATCH_ID to obtain the batch ID. Older versions of the SDK do not do this. We recommend updating your SDK. If this is not possible then add the following code to your test after the eyes instance variable has been instantiated and before the test is opened.
Once your system is set up as described above, you use your CI and GitHub in the usual way. A CI job is normally triggered by a push to your repository, this, in turn, will then run your visual UI tests.
The Eyes GitHub integration is based on the GitHub pull request. When there is an open pull request, Eyes adds the following functionality to your GitHub workflow:
Merging an Eyes source branch with a target branch results in adding new baselines to the target branch or updating its existing baselines according to baseline changes found in the source branch. Changes can include both updated baseline images as well as new, changed or removed annotations (e.g., an added ignore region). Eyes knows how to compare two baselines and determine if they can be merged automatically or if some sort of manual user resolution is required. For example, an ignore region added to a source baseline step that did not change in the target baseline can be automatically added to the target baseline, but if another ignore region was added to the same step in the target baseline, user intervention is required to determine whether to keep only one of the added regions, or to keep both of them.
As described above, the item on the GitHub pull-request page marked “scm/applitools” indicates the merge status of the Eyes baselines in the source branch.
When you click on the “Details” link on this item, the browser will switch to the Test Manager "Merge branch" page preloaded with the source and target branches derived from the pull-request. If there are no conflicts, then the Test Manager will first display a pop-up window stating this.
Press “Got it!” to remove the pop-up.
A full description of merging is not within the scope of this article; we will only briefly mention the main features available on the merge branches page.
The merge branches page shows all the baselines that have changed in the source branch with respect to the target branch. That is, either they don't exist in the target branch, or they exist in the target branch but have changed in the source branch. Note that baselines that have changed in the target branch and were not changed in the source branch will not be listed and are ignored in the merge process.
At the far left of each row, the status can show a value of “Conflict” indicating that the baseline needs resolving by the user.
You resolve conflicts by using the buttons at the right of each row, which allow you to choose if you want to replace the target baseline with the source baseline or to keep the target baseline unchanged. Choosing one of these will turn the “Conflict” status into a “Resolved” status. You can also compare the two baselines and manually edit the source baseline to include the changes from both branches.
An alternative to resolving each test individually is to use the checkbox at the start of each row to select rows and then use the buttons in the toolbar at the top left of the window to choose how to resolve the selected rows. The toolbar also has buttons that allow you to delete selected baselines in the source branch or to undo the resolution made for the selected baselines.
Once you have resolved all conflicts, the status of the scm/applitools item in the GitHub pull-request page will change to green (passed), and you can proceed to merge the branches, either from within test manager by clicking the Merge button, or via the GitHub pull-request page as described above.