{"id":6175,"date":"2021-09-02T11:55:02","date_gmt":"2021-09-02T09:55:02","guid":{"rendered":"https:\/\/florianbrinkmann.com\/en\/?p=6175"},"modified":"2021-09-02T11:55:27","modified_gmt":"2021-09-02T09:55:27","slug":"visual-regression-tests-jlineup","status":"publish","type":"post","link":"https:\/\/florianbrinkmann.com\/en\/visual-regression-tests-jlineup-6175\/","title":{"rendered":"Visual regression tests with JLineup"},"content":{"rendered":"

With visual regression tests, one can check two versions of a website for visual differences. Those tests are, for example, useful to check a website on a test server before and after deploying changes, to only move the changes to the live server when no differences are found.<\/p>\n\n\n\n

There are various solutions to run such tests, here I will show you JLineup<\/a>. JLineup can be used as a command-line tool or as a web service. I only tried the CLI version and am not going to describe the web service possibility.<\/p>\n\n\n\n\n\n\n\n

Installation of JLineup CLI<\/h2>\n\n\n\n

The GitHub repo contains a file with installation and usage instructions for the CLI version of JLineup<\/a>. Because it is a Java archive, the server needs to have at least Java 8 installed. On Ubuntu 20.04 we could install Java in form of OpenJDK with the following command:<\/p>\n\n\n

sudo apt-get install openjdk-14-jre <\/code><\/span>Code language:<\/span> Bash<\/span> (<\/span>bash<\/span>)<\/span><\/small><\/pre>\n\n\n
Installing the CLI itself happens with just downloading the .jar<\/code> file:<\/p>\n\n\n
wget https:\/\/repo1.maven.org\/maven2\/de\/otto\/jlineup-cli\/4.3.4\/jlineup-cli-4.3.4.jar -O jlineup.jar<\/code><\/span>Code language:<\/span> Bash<\/span> (<\/span>bash<\/span>)<\/span><\/small><\/pre>\n\n\n
To be able to run visual regression tests, we need a browser, too. I decided to use Firefox, JLineup also supports Chrome and Chromium. To install Firefox on Ubuntu, we run the following command:<\/p>\n\n\n
sudo apt-get<\/span> install --no-install-recommends firefox<\/code><\/span>Code language:<\/span> JavaScript<\/span> (<\/span>javascript<\/span>)<\/span><\/small><\/pre>\n\n\n
The --no-install-recommends<\/code> param prevents the installation of GNOME dependencies.<\/p>\n\n\n\n
Configuration<\/h2>\n\n\n\n
Configuration of JLineup is done in a JSON file, the tool by default looks for a lineup.json<\/code> in the current directory when no other file is specified via the --config<\/code> parameter.<\/p>\n\n\n\n
The tool comes with many configuration options<\/a> that are described in the repo in detail. For example, you can define multiple URLs to check, test in different viewport dimensions, and set a time in seconds that should be waited before taking the screenshot after loading the page.<\/p>\n\n\n\n
An easy example for testing a site looks like the following:<\/p>\n\n\n
{\n  \"urls\"<\/span>: {\n    \"https:\/\/time.gov\/\"<\/span>: {\n      \"paths\"<\/span>: [\n        \"\"<\/span>\n      ],\n      \"max-diff\"<\/span>: 0.0<\/span>,\n      \"devices\"<\/span> : [{\n        \"width\"<\/span> : 800<\/span>,\n        \"height\"<\/span> : 600<\/span>\n      }],\n      \"max-scroll-height\"<\/span>: 100000<\/span>,\n      \"wait-after-page-load\"<\/span>: 5<\/span>,\n      \"wait-after-scroll\"<\/span>: 0<\/span>\n    }\n  },\n  \"browser\"<\/span>: \"firefox-headless\"<\/span>\n}<\/code><\/span>Code language:<\/span> JSON \/ JSON with Comments<\/span> (<\/span>json<\/span>)<\/span><\/small><\/pre>\n\n\n
Using this configuration, JLineup checks the time.gov site with Firefox in headless mode and a window width of 800x600 pixels. The max height that is scrolled for taking screenshots is 100.000 pixels and after loading the page, the tool waits 5 seconds before taking a screenshot, after scrolling there is no pause.<\/p>\n\n\n\n
The defined window size also specifies the screenshot size, because JLineup does not take one screenshot of the whole site but multiple in the specified size. It visits the page, makes a screenshot, scrolls down, makes a screenshot, scrolls down, and so on.<\/p>\n\n\n\n
Run tests<\/h2>\n\n\n\n
To run a test with the configuration from above, we need the following command to run the \u00bbbefore\u00ab step:<\/p>\n\n\n
java<\/span> -jar<\/span> jlineup<\/span>.jar<\/span> --step<\/span> before<\/span><\/code><\/span>Code language:<\/span> CSS<\/span> (<\/span>css<\/span>)<\/span><\/small><\/pre>\n\n\n
The command line shows a log of what JLineup does, for example, \u00bbBrowsing to https:\/\/time.gov\/ with window size 800x600\u00ab.<\/p>\n\n\n\n
Let\u2019s imagine our changes were deployt now, and we run the \u00bbafter\u00ab step and comparison:<\/p>\n\n\n
java<\/span> -jar<\/span> jlineup<\/span>.jar<\/span> --step<\/span> after<\/span><\/code><\/span>Code language:<\/span> CSS<\/span> (<\/span>css<\/span>)<\/span><\/small><\/pre>\n\n\n
time.gov changes every second, so the command should finish with \u00bbJLineup finished. There was a difference between before and after. Return code is 1.\u00ab. In background, JLineup created a report<\/code> directory that contains, among other files, the screenshot files and a report in JSON and one in HTML format.<\/p>\n\n\n\n
The result in HTML format looks like the following example when an error is found:<\/p>\n\n\n\n