{"id":15018742,"url":"https://github.com/manykarim/robotframework-doctestlibrary","last_synced_at":"2025-04-04T12:09:26.397Z","repository":{"id":37018957,"uuid":"338032152","full_name":"manykarim/robotframework-doctestlibrary","owner":"manykarim","description":"Robot Framework DocTest library. Simple Automated Visual Document Testing.","archived":false,"fork":false,"pushed_at":"2025-02-19T21:23:15.000Z","size":10443,"stargazers_count":50,"open_issues_count":29,"forks_count":22,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-03-28T11:09:09.511Z","etag":null,"topics":["ocr","pdf","robotframework","visual-testing"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/manykarim.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"code_of_conduct.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-02-11T13:09:57.000Z","updated_at":"2025-02-19T21:23:19.000Z","dependencies_parsed_at":"2024-10-17T04:44:19.896Z","dependency_job_id":"4679544b-bbc7-4800-bfc0-a7c9421b9347","html_url":"https://github.com/manykarim/robotframework-doctestlibrary","commit_stats":{"total_commits":303,"total_committers":9,"mean_commits":"33.666666666666664","dds":0.2178217821782178,"last_synced_commit":"c9b08b941a8047169f5826a87e7af1a6108a3364"},"previous_names":[],"tags_count":29,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/manykarim%2Frobotframework-doctestlibrary","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/manykarim%2Frobotframework-doctestlibrary/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/manykarim%2Frobotframework-doctestlibrary/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/manykarim%2Frobotframework-doctestlibrary/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/manykarim","download_url":"https://codeload.github.com/manykarim/robotframework-doctestlibrary/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247174453,"owners_count":20896078,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ocr","pdf","robotframework","visual-testing"],"created_at":"2024-09-24T19:52:23.805Z","updated_at":"2025-04-04T12:09:26.369Z","avatar_url":"https://github.com/manykarim.png","language":"Python","readme":"\n# robotframework-doctestlibrary\n----\n[Robot Framework](https://robotframework.org) DocTest library.  \nSimple Automated Visual Document Testing.\n\nSee **keyword documentation** for\n\n- [Visual Document Tests](https://manykarim.github.io/robotframework-doctestlibrary/VisualTest.html)\n- [Print Job Tests](https://manykarim.github.io/robotframework-doctestlibrary/PrintJobTest.html)\n- [Pdf Tests (very basic)](https://manykarim.github.io/robotframework-doctestlibrary/PdfTest.html)\n\n```RobotFramework\n*** Settings ***\nLibrary    DocTest.VisualTest\n\n*** Test Cases ***\nCompare two Images and highlight differences\n    Compare Images    Reference.jpg    Candidate.jpg\n```\n\n[![DocTest Library presentation at robocon.io 2021](https://img.youtube.com/vi/qmpwlQoJ-nE/0.jpg)](https://youtu.be/qmpwlQoJ-nE \"DocTest Library presentation at robocon.io 2021\")\n\n# Installation instructions\n\n`pip install --upgrade robotframework-doctestlibrary`\n\nOnly Python 3.X or newer is supported. Tested with Python 3.8/3.11/3.12\n\n## Install robotframework-doctestlibrary\n\n### Installation via `pip` from PyPI (recommended)\n\n* `pip install --upgrade robotframework-doctestlibrary`\n\n### Installation via `pip` from GitHub\n\n* `pip install git+https://github.com/manykarim/robotframework-doctestlibrary.git`  \n\nor\n\n* `git clone https://github.com/manykarim/robotframework-doctestlibrary.git`\n* `cd robotframework-doctestlibrary`\n* `pip install -e .`\n\n## Install dependencies\n\nInstall Tesseract, Ghostscript, GhostPCL, ImageMagick binaries and barcode libraries (libdmtx, zbar) on your system.\n\u003cbr\u003eHint: Since `0.2.0` Ghostscript, GhostPCL and ImageMagick are only needed for rendering `.ps` and `.pcl`files.\n\u003cbr\u003e Rendering and content parsing of `.pdf` is done via `MuPDF`\n\u003cbr\u003eIn the future there might be a separate pypi package for `.pcl` and `.ps` files to get rid of those dependencies.\n\nLinux\n```bash\napt-get install imagemagick tesseract-ocr ghostscript libdmtx0b libzbar0\n```\n\nWindows\n * https://github.com/UB-Mannheim/tesseract/wiki\n * https://ghostscript.com/releases/gsdnld.html\n * https://ghostscript.com/releases/gpcldnld.html\n * https://imagemagick.org/script/download.php\n\n\n## Some special instructions for Windows \n\n### Rename executable for GhostPCL to pcl6.exe (only needed for `.pcl` support)\nThe executable for GhostPCL `gpcl6win64.exe` needs to be renamed to `pcl6.exe`\n\nOtherwise it will not be possible to render .pcl files successfully for visual comparison.\n\n### Add tesseract, ghostscript and imagemagick to system path in windows (only needed for OCR, `.pcl` and `.ps` support)\n* C:\\Program Files\\ImageMagick-7.0.10-Q16-HDRI\n* C:\\Program Files\\Tesseract-OCR\n* C:\\Program Files\\gs\\gs9.53.1\\bin\n* C:\\Program Files\\gs\\ghostpcl-9.53.1-win64\n\n(The folder names and versions on your system might be different)\n\nThat means: When you open the CMD shell you can run the commands\n* `magick.exe`\n* `tesseract.exe`\n* `gswin64.exe`\n* `pcl6.exe`\n\nsuccessfully from any folder/location\n\n### Windows error message regarding pylibdmtx\n\n[How to solve ImportError for pylibdmtx](https://github.com/NaturalHistoryMuseum/pylibdmtx/#windows-error-message)\n\nIf you see an ugly `ImportError` when importing `pylibdmtx` on\nWindows you will most likely need the [Visual C++ Redistributable Packages for\nVisual Studio 2013](https://www.microsoft.com/en-US/download/details.aspx?id=40784). Install `vcredist_x64.exe` if using 64-bit Python, `vcredist_x86.exe` if using 32-bit Python.\n\n## ImageMagick\n\nThe library might return the error `File could not be converted by ImageMagick to OpenCV Image: \u003cpath to the file\u003e` when comparing PDF files.\nThis is due to ImageMagick permissions. Verify this as follows with the `sample.pdf` in the `testdata` directory:\n```bash\nconvert sample.pdf sample.jpg \nconvert-im6.q16: attempt to perform an operation not allowed by the security policy\n```\n\nSolution is to copy the `policy.xml` from the repository to the ImageMagick installation directory.\n\n## Docker\n\nYou can also use the [docker images](https://github.com/manykarim/robotframework-doctestlibrary/packages) or create your own Docker Image\n`docker build -t robotframework-doctest .`\nAfterwards you can, e.g., start the container and run the povided examples like this:\n* Windows\n  * `docker run -t -v \"%cd%\":/opt/test -w /opt/test robotframework-doctest robot atest/Compare.robot`\n* Linux\n  * `docker run -t -v $PWD:/opt/test -w /opt/test robotframework-doctest robot atest/Compare.robot`\n\n## Gitpod.io\n[![Open in Gitpod](https://gitpod.io/button/open-in-gitpod.svg)](https://gitpod.io/#https://github.com/manykarim/robotframework-doctestlibrary)  \nTry out the library using [Gitpod](https://gitpod.io/#https://github.com/manykarim/robotframework-doctestlibrary)\n\n# Examples\n\nHave a look at  \n* [Visual Comparison Tests](./atest/Compare.robot)\n* [PDF Content Tests](./atest/PdfContent.robot)\n* [Print Job Tests](./atest/PrintJobs.robot)\n\nfor more examples.\n\n### Testing with [Robot Framework](https://robotframework.org)\n```RobotFramework\n*** Settings ***\nLibrary    DocTest.VisualTest\n\n*** Test Cases ***\nCompare two Images and highlight differences\n    Compare Images    Reference.jpg    Candidate.jpg\n```\n\n### Use masks/placeholders to exclude parts from visual comparison\n\n```RobotFramework\n*** Settings ***\nLibrary    DocTest.VisualTest\n\n*** Test Cases ***\nCompare two Images and ignore parts by using masks\n    Compare Images    Reference.jpg    Candidate.jpg    placeholder_file=masks.json\n\nCompare two PDF Docments and ignore parts by using masks\n    Compare Images    Reference.jpg    Candidate.jpg    placeholder_file=masks.json\n\nCompare two Farm images with date pattern\n    Compare Images    Reference.jpg    Candidate.jpg    placeholder_file=testdata/pattern_mask.json\n\nCompare two Farm images with area mask as list\n    ${top_mask}    Create Dictionary    page=1    type=area    location=top    percent=10\n    ${bottom_mask}    Create Dictionary    page=all    type=area    location=bottom    percent=10\n    ${masks}    Create List    ${top_mask}    ${bottom_mask}\n    Compare Images    Reference.jpg    Candidate.jpg    mask=${masks}\n\nCompare two Farm images with area mask as string\n    Compare Images    Reference.jpg    Candidate.jpg    mask=top:10;bottom:10\n\n```\n#### Different Mask Types to Ignore Parts When Comparing\n##### Areas, Coordinates, Text Patterns\n```python\n[\n    {\n    \"page\": \"all\",\n    \"name\": \"Date Pattern\",\n    \"type\": \"pattern\",\n    \"pattern\": \".*[0-9]{2}-[a-zA-Z]{3}-[0-9]{4}.*\"\n    },\n    {\n    \"page\": \"1\",\n    \"name\": \"Top Border\",\n    \"type\": \"area\",\n    \"location\": \"top\",\n    \"percent\":  5\n    },\n    {\n    \"page\": \"1\",\n    \"name\": \"Left Border\",\n    \"type\": \"area\",\n    \"location\": \"left\",\n    \"percent\":  5\n    },\n    {\n    \"page\": 1,\n    \"name\": \"Top Rectangle\",\n    \"type\": \"coordinates\",\n    \"x\": 0,\n    \"y\": 0,\n    \"height\": 10,\n    \"width\": 210,\n    \"unit\": \"mm\"\n    }\n]\n```\n### Accept visual different by checking move distance or text content\n\n```RobotFramework\n*** Settings ***\nLibrary    DocTest.VisualTest\n\n*** Test Cases ***\nAccept if parts are moved up to 20 pixels by pure visual check\n    Compare Images    Reference.jpg    Candidate.jpg    move_tolerance=20\n\nAccept if parts are moved up to 20 pixels by reading PDF Data\n    Compare Images    Reference.pdf    Candidate.pdf    move_tolerance=20    get_pdf_content=${true}\n\nAccept differences if text content is the same via OCR\n    Compare Images    Reference.jpg    Candidate.jpg    check_text_content=${true}\n\nAccept differences if text content is the same from PDF Data\n    Compare Images    Reference.pdf    Candidate.pdf    check_text_content=${true}    get_pdf_content=${true}\n```\n\n#### Different options to detect moved parts/objects\n```RobotFramework\n*** Settings ***\nLibrary    DocTest.VisualTest   movement_detection=orb\n\n*** Test Cases ***\nAccept if parts are moved up to 20 pixels by pure visual check\n    Compare Images    Reference.jpg    Candidate.jpg    move_tolerance=20\n```\t\n\n```RobotFramework\n*** Settings ***\nLibrary    DocTest.VisualTest   movement_detection=template\n\n*** Test Cases ***\nAccept if parts are moved up to 20 pixels by pure visual check\n    Compare Images    Reference.jpg    Candidate.jpg    move_tolerance=20\n```\t\n\n```RobotFramework\n*** Settings ***\nLibrary    DocTest.VisualTest   movement_detection=classic\n\n*** Test Cases ***\nAccept if parts are moved up to 20 pixels by pure visual check\n    Compare Images    Reference.jpg    Candidate.jpg    move_tolerance=20\n```\t\n\n### Options for taking additional screenshots, screenshot format and render resolution\nTake additional screenshots or reference and candidate file.\n```RobotFramework\n*** Settings ***\nLibrary    DocTest.VisualTest   take_screenshots=${true}    screenshot_format=png\n```\nTake diff screenshots to highlight differences\n```RobotFramework\n*** Settings ***\nLibrary    DocTest.VisualTest   show_diff=${true}    DPI=300\n```\n\n### Experimental usage of Open CV East Text Detection to improve OCR\n\n```RobotFramework\n*** Settings ***\nLibrary    DocTest.VisualTest\n\n*** Test Cases ***\nCompare two Farm images with date pattern and east detection\n    Compare Images    Reference.jpg    Candidate.jpg    placeholder_file=masks.json    ocr_engine=east\n```\n\n### Check content of PDF files\n\n```RobotFramework\n*** Settings ***\nLibrary    DocTest.PdfTest\n\n*** Test Cases ***\nCheck if list of strings exists in PDF File\n    @{strings}=    Create List    First String    Second String\n    PDF Should Contain Strings    ${strings}    Candidate.pdf\n    \nCompare two PDF Files and only check text content\n    Compare Pdf Documents    Reference.pdf    Candidate.pdf    compare=text\n\nCompare two  PDF Files and only check text content and metadata\n    Compare Pdf Documents    Reference.pdf    Candidate.pdf    compare=text,metadata\n    \nCompare two  PDF Files and check all possible content\n    Compare Pdf Documents    Reference.pdf    Candidate.pdf\n```\n\n### Ignore Watermarks for Visual Comparisons\nStore the watermark in a separate B/W image or PDF.\n\u003cbr\u003e\nWatermark area needs to be filled with black color.\n\u003cbr\u003e\nWatermark content will be subtracted from Visual Comparison result.\n```RobotFramework\n*** Settings ***\nLibrary    DocTest.VisualTest\n\n*** Test Cases ***\nCompare two Images and ignore jpg watermark\n    Compare Images    Reference.jpg    Candidate.jpg    watermark_file=Watermark.jpg\n\nCompare two Images and ignore pdf watermark\n    Compare Images    Reference.pdf    Candidate.pdf    watermark_file=Watermark.pdf\n\nCompare two Images and ignore watermark folder\n    Compare Images    Reference.pdf    Candidate.pdf    watermark_file=${CURDIR}${/}watermarks\n```\n\nWatermarks can also be passed on Library import. This setting will apply to all Test Cases in Test Suite\n```RobotFramework\n*** Settings ***\nLibrary    DocTest.VisualTest   watermark_file=${CURDIR}${/}watermarks\n\n*** Test Cases ***\nCompare two Images and ignore watermarks\n    Compare Images    Reference.jpg    Candidate.jpg\n```\n\n### Get Text From Documents or Images\n\n```RobotFramework\n*** Settings ***\nLibrary    DocTest.VisualTest\n\n*** Test Cases ***\nGet Text Content And Compare\n    ${text}    Get Text From Document    Reference.pdf\n    List Should Contain Value    ${text}    Test String\n```\n\n### Get Barcodes From Documents or Images\n\n```RobotFramework\n*** Settings ***\nLibrary    DocTest.VisualTest\n\n*** Test Cases ***\nGet Text Content And Compare\n    ${text}    Get Barcodes From Document    reference.jpg\n    List Should Contain Value    ${text}    123456789\n```\n\n### Using pabot to run tests in parallel\n\nDocument Testing can be run in parallel using [pabot](https://pabot.org/).  \nHowever, you need to pass the additional arguments `--artifacts` and `--artifactsinsubfolders` to the `pabot` command, to move the screenshots to the correct subfolder.  \nOtherwise the screenshots will not be visible in the `log.html`\n\n```\npabot --testlevelsplit --processes 8 --artifacts png,jpg,pdf,xml --artifactsinsubfolders /path/to/your/tests/\n```\n\n### Visual Testing of Web Applications\n\nI experimented a bit and tried to use this library for Visual Testing of Web Applications.  \nPlease have a look at this pilot example [here](https://github.com/manykarim/robotframework-doctestlibrary/blob/main/atest/Browser.robot)\n\n# Development\n\nFeel free to create issues or pull requests.  \nI'm always happy for any feedback.\n\n## Core team\n\nIn order of appearance.\n\n  * Many Kasiriha\n  * April Wang\n\n## Contributors\n\nThis project is community driven and becomes a reality only through the work of all the people who contribute.\n","funding_links":[],"categories":["Libraries"],"sub_categories":["Visual Testing"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmanykarim%2Frobotframework-doctestlibrary","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmanykarim%2Frobotframework-doctestlibrary","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmanykarim%2Frobotframework-doctestlibrary/lists"}