{"id":16307158,"url":"https://github.com/jessezach/RobotEyes","last_synced_at":"2025-10-25T14:31:01.554Z","repository":{"id":50258797,"uuid":"72233729","full_name":"jessezach/RobotEyes","owner":"jessezach","description":"Image comparison for Robot Framework","archived":false,"fork":false,"pushed_at":"2023-08-04T17:42:15.000Z","size":719,"stargazers_count":75,"open_issues_count":7,"forks_count":24,"subscribers_count":10,"default_branch":"master","last_synced_at":"2025-02-06T01:37:16.792Z","etag":null,"topics":["image-comparison","image-comparison-robot-framework","imagecomparison","imagemagick","imagemagick-wrapper","robot-framework","selenium","visualregression","visualtesting"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jessezach.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"jz-jess","patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"custom":null}},"created_at":"2016-10-28T18:52:05.000Z","updated_at":"2025-01-16T12:49:27.000Z","dependencies_parsed_at":"2024-06-21T07:10:29.189Z","dependency_job_id":"2492d448-1ac1-4878-a7b5-f06f3a392eb6","html_url":"https://github.com/jessezach/RobotEyes","commit_stats":null,"previous_names":["jz-jess/roboteyes"],"tags_count":16,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jessezach%2FRobotEyes","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jessezach%2FRobotEyes/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jessezach%2FRobotEyes/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jessezach%2FRobotEyes/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jessezach","download_url":"https://codeload.github.com/jessezach/RobotEyes/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":238155555,"owners_count":19425734,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["image-comparison","image-comparison-robot-framework","imagecomparison","imagemagick","imagemagick-wrapper","robot-framework","selenium","visualregression","visualtesting"],"created_at":"2024-10-10T21:13:37.702Z","updated_at":"2025-10-25T14:31:01.501Z","avatar_url":"https://github.com/jessezach.png","language":"Python","funding_links":["https://github.com/sponsors/jz-jess","https://paypal.me/jessezach"],"categories":[],"sub_categories":[],"readme":"# RobotEyes\n[![Downloads](https://pepy.tech/badge/robotframework-eyes)](https://pepy.tech/project/robotframework-eyes)\n[![Version](https://img.shields.io/pypi/v/robotframework-eyes.svg)](https://pypi.org/project/robotframework-eyes)\n\nVisual Regression Library for Robot Framework\n\nUses Imagemagick to Compare Images and create a diff image. Custom Report to view baseline, actual and diff images. View passed and failed tests. Blur regions (only for selenium) within a page to ignore comparison (helpful when there are dynamic elements like text etc in a page). Support SeleniumLibrary(tested) , Selenium2Library(tested) and AppiumLibrary(not tested).\n## Requirements\n- Install the `robotframework-eyes` library using `pip`: \n```\n    pip install robotframework-eyes\n```     \n- Install `Imagemagick` (for mac: `brew install imagemagick`, linux: `apt-get install imagemagick`) \u003cbr/\u003e\n\n-- **Important Imagemagick7: Make sure that you check the _Install Legacy Utilities (e.g. convert, compare)_ check mark in the installation process and that the directory to ImageMagick is in your PATH env variable. Please ensure that compare.exe is in your path env variable. If you still dont see diff images being generated, please downgrade to Imagemagick6** \n\n## Quick-reference Usage Guide\n- Import the Library into your Robot test. E.g: \u003cbr/\u003e    \n ```\n    Library    RobotEyes\n ```    \n- Call the `Open Eyes` keyword after opening the browser in your selenium test.\n- Use the `Capture Full Screen` and `Capture Element` keywords to capture images.\n- Call the `Compare Images` keyword at the end of the test to compare all the images captured in the respective test.\n- Once done running the tests, report with name `visualReport.html` should be generated at the root of the project\n- You can manually generate the report by running the below command. Eg:\u003cbr/\u003e\n```\n    reportgen --baseline=\u003cbaseline image directory\u003e --results=\u003coutput directory\u003e\n``` \n## Usage Guide\nThis guide contains the suggested steps to efficently integrate `RobotEyes` library into your Robot Framework test development workflow.\u003cbr/\u003e\nIt also serves as documentation to clarify how this library functions on a high level.\n\n## Keyword Documentation\n| Keyword                | Arguments                        | Comments                                                                                    |\n|------------------------|----------------------------------|---------------------------------------------------------------------------------------------|\n| Open Eyes              | lib, tolerance, template_id, cleanup                   | Ex `open eyes  lib=AppiumLibrary  tolerance=5  cleanup=all_passed`                                                |\n| Capture Full Screen    | tolerance, blur, radius, name, redact          | Ex `capture full screen  tolerance=5  name=homepage  blur=\u003carray of locators\u003e` radius=50(thickness of blur) |\n| Capture Element        | locator, tolerance, blur, radius, name, redact |                                                                                             |\n| Capture Mobile Element | locator, tolerance, blur, radius, name, redact |                                                                                             |\n| Scroll To Element      | locator                          | Ex `scroll to element  id=user`                                                             |\n| Compare Images         |                                  | Compares all the images captured in the test with their respective base image               |\n| Compare Two Images     | first, second, output, tolerance | Compares two images captured in the above steps. Takes image names, diff file name and tolerance as arguments Ex: Compare Two Images  img1  img2  diff  10|\n\n## Cleanup Options\nThis is only set when invoking open eyes\n- all_passed\n  - This will cleanup diff and actual folders that passed\n- diffs_passed\n  - Will only cleanup diffs that passed, leaving actuals in place\n- actuals_passed\n  - Will only cleanup actuals that passed, leaving diffs in place\n- None\n  - Won't do any image folder cleanups (default)\n\n### Running Tests ###\n`robot -d results -v images_dir:\u003cbaseline_images_directory\u003e tests`\u003cbr/\u003e\nIf baseline image directory does not exist, RobotEyes will create it.\nIf baseline image(s) does not exist, RobotEyes will move the captured image into the baseline directory.\nFor example, when running tests the first time all captured images will be moved to baseline directory passed by you (images_dir) \u003cbr/\u003e\n**Important** It is mandatory to pass baseline image directory, absence of which will throw an exception.\u003cbr/\u003e\n\n### Directory structure  \nThe `RobotEyes` library creates a `visual_images` directory which will contain two additional directories, named `actual` \u0026 `diff`, respectively.\u003cbr/\u003e\nThese directories are necessary for the library to function and are created by it at different stages of the test case (TC) development workflow.\u003cbr/\u003e\nThe resulting directory structure created in the project looks as follows:\n\n\u003cdiv\u003e\n  \u003cul class=\"ascii\"\u003e\n    \u003cli\u003evisual_images/\n      \u003cul\u003e\n        \u003cli\u003eactual/\n          \u003cul\u003e\n            \u003cli\u003ename_of_tc1/\n              \u003cul\u003e\n                \u003cli\u003eimg1.png\u003c/li\u003e\n                \u003cli\u003eimg1.png.txt\u003c/li\u003e\n              \u003c/ul\u003e\n            \u003c/li\u003e\n            \u003cli\u003ename_of_tc2/\n              \u003cul\u003e\n                \u003cli\u003eimg1.png\u003c/li\u003e\n                \u003cli\u003eimg1.png.txt\u003c/li\u003e\n              \u003c/ul\u003e\n            \u003c/li\u003e\n            \u003cli\u003ename_of_tc3/\n              \u003cul\u003e\n                \u003cli\u003eimg1.png\u003c/li\u003e\n                \u003cli\u003eimg1.png.txt\u003c/li\u003e\n              \u003c/ul\u003e \n            \u003c/li\u003e\n          \u003c/ul\u003e\n        \u003c/li\u003e\n        \u003cli\u003ediff/\n          \u003cul\u003e\n            \u003cli\u003ename_of_tc1/\n              \u003cul\u003e\n                \u003cli\u003eimg1.png\u003c/li\u003e\n              \u003c/ul\u003e\n            \u003c/li\u003e\n            \u003cli\u003ename_of_tc2/\n              \u003cul\u003e\n                \u003cli\u003eimg1.png\u003c/li\u003e\n              \u003c/ul\u003e\n            \u003c/li\u003e\n            \u003cli\u003ename_of_tc3/\n              \u003cul\u003e\n                \u003cli\u003eimg1.png\u003c/li\u003e\n              \u003c/ul\u003e\n            \u003c/li\u003e\n          \u003c/ul\u003e\n        \u003c/li\u003e\n      \u003c/ul\u003e\n    \u003c/li\u003e\n  \u003c/ul\u003e  \n\u003c/div\u003e\n\n### Generating the baseline images\nBaseline images will be generated when tests are run the first time. Subsequent test runs will trigger comparison of actual and baseline images.\n\nFor example:\n```robotframework\n*** Settings ***\nLibrary    SeleniumLibrary\nLibrary    RobotEyes\n\n*** Test Cases ***    \nSample visual regression test case  # Name of the example test case\n    Open Browser    https://www.google.com/    chrome\n    Maximize Browser Window\n    Open Eyes    SeleniumLibrary(AppiumLibrary)  5\n    Wait Until Element Is Visible    id=lst-ib\n    Capture Full Screen\n    Compare Images\n    Close Browser\n```\n### Comparing the images\nTo compare the images, the following needs to exist in the TC's code:\n- Library declaration:\n```robotframework\nLibrary    RobotEyes\n```\n- The `Open Eyes` keyword after the `Open Browser` keyword.\n- Any of the image capture keywords. E.g `Capture Full Screen`\n- The `Compare Images` keyword after capturing the desired images.\n\nFor Example:\n```robotframework\n*** Settings ***\nLibrary    SeleniumLibrary\nLibrary    RobotEyes\n\n*** Test Cases ***    \nSample visual regression test case  # Name of the example test case\n    Open Browser    https://www.google.com/    chrome\n    Maximize Browser Window\n    Open Eyes    SeleniumLibrary  5\n    Wait Until Element Is Visible    id=lst-ib\n    Capture Full Screen\n    Compare Images\n    Close Browser\n```\nAfter the comparison is completed (i.e. the `Compare Images` keyword in the TC is executed), a difference image will be generated and stored in the `diff` directory.\u003cbr/\u003e\nAlso, a text file will be created containing the result of the comparison between the RMSE (root mean squared error) of the `diff` image and the tolerance set by the user.\u003cbr/\u003e\nAfter that, the regular Robot Framework report will raise a failure if the comparison fails.\n\n### Another test example\n```robotframework\n*** Settings ***\nLibrary    SeleniumLibrary\nLibrary    RobotEyes\n\n*** Variables ***\n@{blur}    id=body    css=#SIvCob\n\n*** Test Cases ***    \nSample visual regression test case  # Name of the example test case\n    Open Browser    https://www.google.com/    chrome\n    Maximize Browser Window\n    Open Eyes    SeleniumLibrary  5\n    Wait Until Element Is Visible    id=lst-ib\n    # Below, the optional arguments are the tolerance to override global value, the regions to blur in the image and\n    # the thickness of the blur (radius of Gaussian blur applied to the regions) \n    Capture Full Screen    10(tolerance)    ${blur}    50\n    Capture Element    id=hplogo\n    Compare Images\n    Close Browser\n```\n\n### Non web/mobile image comparison tests\nYou can run plain non web/mobile image comparison tests as well. Here is an example:\n```robotframework\n*** Settings ***\nLibrary    RobotEyes\n\n\n*** Test Cases ***    \nPlain image comparison test case  # Name of the example test case\n    Open Eyes    lib=none  5\n    Compare Two Images   ref=oldsearchpage.png   actual=newsearchpage.png   output=diffimage.png  tolerance=5\n```\nYou need to place images to compare within two folders and provide their path while running the tests.\u003cbr/\u003e\n`robot -d results -v images_dir:\u003creference_directory\u003e -v actual_dir:\u003cactual_directory\u003e  Tests/nonwebtest.robot`\n\n**Important** Do not run non web tests and web/mobile tests together. This will result in errors during report creation.\n\n### Template Tests\nWhen writing a data driven Template Test, you need to provide a unique template_id in order to uniquely save images for each test.\n```robotframework\n*** Settings ***\nLibrary  SeleniumLibrary\nLibrary  RobotEyes\n\n*** Test Cases ***\nSample test\n    [Template]   Sample keyword\n    https://www.google.com/   0\n    https://www.google.com/   1\n    https://www.google.com/   2\n\n\n*** Keywords ***\nSample keyword\n    [Arguments]  ${url}  ${uid}\n    open browser  ${url}  chrome\n    open eyes  SeleniumLibrary   template_id=${uid}\n    sleep  3\n    capture element  id=hplogo\n    capture element  id=body  50\n    compare images\n    close browser\n```\n\n## Tolerance\nTolerance is the allowed dissimilarity between images. If comparison difference is more than tolerance, the test fails.\u003cbr/\u003e\nYou can pass tolerance globally to the `open eyes` keyword. Ex `Open Eyes  lib=SeleniumLibrary  tolerance=5`.\u003cbr/\u003e\nAdditionally you can override global tolerance by passing it to `Capture Element`, `Capture Fullscreen` keywords.\u003cbr/\u003e\nEx: \n```Capture Element  \u003clocator\u003e  tolerance=10  blur=${locators}```\u003cbr/\u003e\nTolerance should range between 1 to 100\u003cbr/\u003e\n\n## Blurring elements from image\nYou can also blur out unwanted elements (dynamic texts etc) from image to ignore them from comparison. This can help in getting more accurate test results. You can pass a list of locators or a single locator as argument to `Capture Element` and `Capture Full Screen` keywords.\u003cbr/\u003e\nEx: ```Capture Element  \u003clocator\u003e  blur=id=test```\n```\n    @{blur}    id=body    css=#SIvCob\n    Capture Element   \u003clocator\u003e  blur=${blur}\n    Capture Full Screen     blur=${blur}\n```\n## Redacting elements from image\nIf blurring elements does not serve your purpose, you can redact elements from images. Simply pass a list of locators that you want to redact as argument to the capture keywords.\nEx: ```Capture Element  \u003clocator\u003e  redact=id=test```\n```\n    @{redact}    id=body    css=#SIvCob\n    Capture Element   \u003clocator\u003e  redact=${redact}\n    Capture Full Screen     redact=${redact}\n```\n\n## Basic Report\n![Alt text](/basicreport.png \"Basic Report\")\n\nBasic report should be autogenerated after execution (not supported for pabot). Alternatively, you can generate report by running the following command.\u003c/br\u003e\n```\n    reportgen --baseline=\u003cbaseline image folder\u003e --results=\u003cresults folder\u003e\n```\n\n**Important: If you want to remotely view the report on Jenkins, you might need to update the CSP setting,\nRefer:** https://wiki.jenkins.io/display/JENKINS/Configuring+Content+Security+Policy#ConfiguringContentSecurityPolicy-HTMLPublisherPlugin\n\n## Interactive Report\nRobot Eyes generates a report automatically after all tests have been executed. However a more interactive and intuitive flask based report is available.\u003cbr/\u003e\n\nYou can view passed and failed tests and also use this feature to move acceptable actual images to baseline directory.\nRun eyes server like this. `eyes --baseline=\u003cbaseline image directory\u003e --results=\u003coutputdir\u003e(leave empty if output is at project root)`\n\n![Alt text](/overview.png \"Home screen\")\n![Alt text](/tests.png \"Tests List\")\n![Alt text](/images.png \"Images\")\n\nYou can move selected images in a testcase by selecting images and clicking on \"Baseline Images\" button.\u003c/br\u003e\nYou can also move all images of test cases by selecting the test cases you want to baseline and clicking on \"Baseline Images\" button.\u003c/br\u003e\n\n**Note: You need to have gevent library installed in the machine to be able to use eyes server.**\u003c/br\u003e\n\n## Pabot users\nVisual tests can be executed in parallel using pabot to increase the speed of execution. Generate the report using `reportgen --baseline=\u003cbaseline images folder\u003e --results=\u003cresults folder\u003e` after running the tests.\n\n## Contributors:\n[Adirala Shiva](https://github.com/adiralashiva8) Contributed in creating a robotmetrics inspired reporting for RobotEyes.\u003c/br\u003e\n[DiegoSanchezE](https://github.com/DiegoSanchezE) Added major improvements in the ReadMe.\u003c/br\u003e\n[Priya](https://www.linkedin.com/in/priyamarimuthu) Contributes by testing and finding bugs/improvements before every release.\u003c/br\u003e\n[Ciaran Doheny](ciaran@flipdish.ie) Actively testing and suggesting improvements.\n\n## Note\nIf you find this library useful, you can support me by doing the following:\u003cbr/\u003e \n    - Star the repository.\u003cbr/\u003e\n    - Make a donation via [Paypal](https://paypal.me/jessezach). You can request for features and I will prioritise them for you.\u003c/br\u003e\n\nFor any issue, feature request or clarification feel free to raise an issue in github or email me at iamjess988@gmail.com\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjessezach%2FRobotEyes","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjessezach%2FRobotEyes","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjessezach%2FRobotEyes/lists"}