{"id":17132858,"url":"https://github.com/thomasjbradley/markbot","last_synced_at":"2026-01-26T02:02:02.074Z","repository":{"id":3694020,"uuid":"50603025","full_name":"thomasjbradley/markbot","owner":"thomasjbradley","description":"An application that automatically tests and marks student code assignments in Algonquin College Graphic Design’s Web Dev courses.","archived":false,"fork":false,"pushed_at":"2022-12-08T17:31:18.000Z","size":3543,"stargazers_count":25,"open_issues_count":21,"forks_count":12,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-05-21T00:36:01.744Z","etag":null,"topics":["automation","css","grading","html","javascript","markbot","students","testing","unit-testing"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/thomasjbradley.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-01-28T18:14:49.000Z","updated_at":"2024-05-31T19:15:52.000Z","dependencies_parsed_at":"2023-01-11T16:33:06.907Z","dependency_job_id":null,"html_url":"https://github.com/thomasjbradley/markbot","commit_stats":null,"previous_names":[],"tags_count":111,"template":false,"template_full_name":null,"purl":"pkg:github/thomasjbradley/markbot","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thomasjbradley%2Fmarkbot","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thomasjbradley%2Fmarkbot/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thomasjbradley%2Fmarkbot/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thomasjbradley%2Fmarkbot/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/thomasjbradley","download_url":"https://codeload.github.com/thomasjbradley/markbot/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thomasjbradley%2Fmarkbot/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28764409,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-26T00:37:26.264Z","status":"online","status_checked_at":"2026-01-26T02:00:08.215Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automation","css","grading","html","javascript","markbot","students","testing","unit-testing"],"created_at":"2024-10-14T19:28:36.321Z","updated_at":"2026-01-26T02:02:02.053Z","avatar_url":"https://github.com/thomasjbradley.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ![](.readme/markbot-logo.png) Markbot\n\n*Your marvellously magnificent marking robot.*\n\n![](.readme/screenshot.png)\n\n**This is an application I use that automatically tests and marks student code assignments in Algonquin College Graphic Design’s Web Dev courses.**\n\nBuilt with Javascript, Node.js \u0026 Electron.\n\n---\n\n- [Why a desktop app?](#why-a-desktop-app)\n- [Use cases](#use-cases)\n- [Set up](#set-up)\n  - [Cheat prevention](#cheat-prevention)\n- [How the students use it](#how-the-students-use-it)\n- [Repo configuration with Markbot files](#repo-configuration-with-markbot-files)\n  - [Git \u0026 GitHub checks](#git--github-checks)\n  - [HTML file tests](#html-file-tests)\n  - [CSS file tests](#css-file-tests)\n  - [JS file tests](#javascript-file-tests)\n  - [Screenshot comparisons](#screenshot-comparisons)\n  - [Functionality tests](#functionality-tests)\n  - [Markdown tests](#markdown-tests)\n  - [YAML tests](#yaml-tests)\n  - [File \u0026 image tests](#file--image-tests)\n  - [Performance testing](#performance-testing)\n  - [Targeting all files](#targeting-all-files)\n    - [Unique information for all files](#unique-information-for-all-files)\n  - [Inheriting from templates](#inheriting-from-templates)\n  - [The Markbot ignore file](#the-markbot-ignore-file)\n- [Installation on student computers](#installation-on-student-computers)\n  - [Git](#git)\n  - [JDK](#jdk)\n  - [Student installation tutorial](#student-installation-tutorial)\n- [Building Markbot](#building-markbot)\n  - [Configure the Markbot application](#configure-the-markbot-application)\n    - [1. Environment variables](#1-environment-variables)\n    - [2. App config file](#2-app-config-file)\n    - [3. Passcode hashing \u0026 embedding](#3-passcode-hashing--embedding)\n    - [4. HTTPS certificate generation](#4-https-certificate-generation)\n    - [5. Markbot dependencies](#4-markbot-dependencies)\n      - [HTML validator](#html-validator)\n      - [CSS validator](#css-validator)\n        - [Compiling the CSS validator](#compiling-the-css-validator)\n      - [LanguageTool](#languagetool)\n        - [Add more words to the dictionary](#add-more-words-to-the-dictionary)\n      - [PDFBox](#pdfbox)\n  - [Running Markbot](#running-markbot)\n- [Debugging Markbot](#debugging-markbot)\n- [License \u0026 copyright](#license--copyright)\n\n---\n\n## Why a desktop app?\n\nThis is my second (and more successful) attempt. The first version was built on GitHub Pull Requests \u0026 Travis with MochaJS tests.\n\nUnfortunately version one had some problems: specifically volume. In classes of 25 or more students Travis would start to choke at the end when everybody was trying to finish their work on time. There would be upwards of 25 pull requests going into Travis per minute with many requested tests to complete. It was just too slow, sometimes tests would take 20–30 minutes to complete, or just stop.\n\nThe user experience wasn’t great for the students and also a little stressful for me, so I switched to a desktop app built with [Electron](http://electron.atom.io/) and (practically) the same Javascript tests.\n\nHaving the desktop app allows all the tests to be run locally and much more efficiently.\n\n---\n\n## Use cases\n\nStudents will fork assignment repositories from GitHub, make their changes, and drop it into Markbot. Markbot will run a battery of tests on the code and report back with the results, allowing the finalized work to be submitted \u0026 graded.\n\nThis is great for code assignments that are pass/fail—I use it in my courses. If they pass the tests then the system automatically sets their grade to complete.\n\nIt also works for non-pass/fail assignments but the grade submission component will only assign 1 point to their assignment—which I think makes sense because, I as the teacher, would then go in and do a complete assessment of their work.\n\n---\n\n## Set up\n\nThere’s a few things you’ll need to do to set the repo up properly for Markbot.\n\n1. Create a [build of the Markbot.app](#building-markbot) that points to your version of [Markbot Server](#markbot-server).\n2. Create a repo on GitHub with a `.markbot.yml` file inside—[see Markbot configuration below](#configuration-with-markbot-files).\n3. I usually put `.editorconfig` and `.gitignore` files in the repo to help the students not make simple mistakes. [See the Markbot repo template on GitHub.](https://github.com/thomasjbradley/markbot-template)\n4. I usually make sure the repo is set up with `gh-pages` so it is a live website for the students.\n5. If you’re using screenshot comparison be sure to use the “Develop” menu’s “Generate Reference Screenshots” for the most consistency.\n6. Make sure to run the “Develop” menu’s “Lock Requirements” before sending the repo to students or they will be marked as a cheater.\n7. Get the students to [download Markbot and it’s dependencies onto their computers](#installation-on-student-computers).\n8. Then students use Markbot while coding.\n\n### Cheat prevention\n\nThere is some cheat prevention build into Markbot—it’s not perfect but it’s annoying enough for students to work around that it’s just faster to do the homework.\n\nIn the “Develop” menu there’s an option named “Lock Requirements”—this should be done for every assignment or the student will get marked as a cheater.\n\nIt will hash the `.markbot.yml` file and the screenshots and put the hashes into a `.markbot.lock` file. If any of the HTML, CSS or JS files are marked as `locked: true` they will also be hashed into the lock file.\n\nIf Markbot detects any changes to these files the user will be marked as a cheater and given a grade of 0 with Markbot Server.\n\n**Lock the requirements right at the end to make sure everything is exactly how you want it to be.**\n\n---\n\n## How the students use it\n\nThe students will fork repos then drop into Markbot which will automate the marking.\n\n[**Check out this tutorial for the students to see how it works.**](http://learn-the-web.algonquindesign.ca/courses/web-dev-1/using-markbot/)\n\n---\n\n## Repo configuration with Markbot files\n\nAll the tests are set up and ready to go and inside the Markbot application. They can be configured from a `.markbot.yml` file.\n\n*Place the `.markbot.yml` file in the folder you want to test. Normally this is done when setting up a repository on GitHub that students would fork.*\n\nHere are the properties that you can use in the Markbot file for testing:\n\n- `repo` — used as an indicator in the app screen, for `liveWebsite`, \u0026 as part of the automatic grade submission\n- `canvasCourse` — used as part of the Canvas integration. **This option will be deprecated in the future but is currently used as a trigger for submissions to Progressinator.**\n- `naming` — will confirm every file \u0026 folder follows [our naming conventions](http://learn-the-web.algonquindesign.ca/topics/naming-paths-cheat-sheet/).\n- `namingIgnore` — an array of paths that allows certain filenames to bypass the naming conventions check.\n- `commits` — the minimum number of commits students need—will automatically subtract your commits.\n- `liveWebsite` — whether to make a `HEAD` request to the GitHub URL to check that it’s accessible or not. Requires the `repo` entry. If the repo isn’t set up with `gh-pages` or the student hasn’t synced any commits a 404 will be issued, failing the test.\n- `restrictFileTypes` — will disallow specific extensions and file names from existing in the folder—[see `app/checks/restricted-file-types` for a list](app/checks/restrict-file-types/).\n- `git` — [for more complex Git checks.](#git--github-checks)\n- `html` — [for testing HTML files.](#html-file-tests)\n- `css` — [for testing CSS files.](#css-file-tests)\n- `js` — [for testing Javascript files.](#javascript-file-tests)\n- `screenshots` — [for comparing visual differences with screenshots.](#screenshot-comparisons)\n- `functionality` — [for running Javascript live tests against the website.](#functionality-tests)\n- `md` — [for checking Markdown files.](#markdown-tests)\n- `yml` — [for checking YAML files.](#yaml-tests)\n- `files` — [for checking images \u0026 plain text files.](#file--image-tests)\n- `performance` — [for checking website performance against simulated networks.](#performance-testing)\n- `allFiles` — [for common requirements between all files of a specific type.](#targeting-all-files)\n\nHere’s a basic Markbot file:\n\n```yml\nrepo: markbot\ncanvasCourse: web-dev-1\n\nnaming: true\ncommits: 3\nliveWebsite: true\nrestrictFileTypes: true\n\n# Other tests, described below, would go here\n```\n\n### Git \u0026 GitHub checks\n\nUsing the `git` entry we can enforce some requirements on the status of the student’s Git repo.\n\n**This is a replacement for the simple `commits` entry.**\n\n```yml\ngit:\n  # The minimum number of commits students need—will automatically subtract your commits.\n  numCommits: 2\n\n  # Force the students to make sure all their files are committed.\n  allCommitted: true\n\n  # Force the students to make sure all their files are pushed \u0026 synced.\n  allSynced: true\n\n  # Confirm specific best practices on the last 5 Git commit messages.\n  # Won’t prevent students from handing assignments in will only show warnings.\n  # (Spelling \u0026 grammar, minimum word \u0026 character length, no trailing periods, attempts to enforce present-tense imperative verbs at the start)\n  bestPractices: true\n```\n\n### HTML file tests\n\nUse the `html` entry to test HTML files, it’s an array of objects, each representing a file to test.\n\n*The `path` option is the only one that’s required—leaving any of the others off will skip the test.*\n\n```yml\nhtml:\n    # The HTML file’s path\n  - path: 'index.html'\n\n    # Whether the code in this file should be locked or not, to help prevent students from changing the code\n    # I use it for assignments where I give them complete HTML \u0026 they just write the CSS or JS\n    # With locked on, there isn’t much point providing the rest of the options\n    locked: true\n\n    # Whether to validate it or not\n    valid: true\n\n    # Check its best practices \u0026 indentation (double quoting attributes, having a \u003ctitle\u003e, indented children, etc.)\n    # Can be further configured in the `htmlcs.json` file\n    # Will be skipped if validation isn’t also checked—the document must be valid first\n    bestPractices: true\n\n    # Check if the headings are in the proper order and that the document starts with an \u003ch1\u003e\n    # Will be skipped if validation isn’t also checked—the document must be valid first\n    outline: true\n\n    # Run accessibility tests on the HTML documents using AxeCore\n    # It’s usually more convenient to just inherit the accessibility template which includes more tests\n    accessibility: true\n\n    # Can be used to test for specific elements; each entry should be a valid CSS selector\n    # Will be skipped if validation isn’t also checked—the document must be valid first\n    # If given an array, the second argument can be a custom error message\n    has:\n      - 'header nav[role=\"navigation\"]'\n      - 'main'\n      - ['header nav li a[class][href*=\"index.html\"]', 'The navigation should be highlighted on this page']\n      # You can emit warnings that won’t prevent students from submitting work using an alternative syntax\n      - check: 'main' # This could also be `selector: 'main'`\n        message: 'The `main` tag should be included for accessibility reasons'\n        type: 'warning'\n      # You can specify a limit for how many times an element should be inside the HTML code\n      - check: '[role=\"banner\"]'\n        message: 'The “banner” role is used to define the header of the whole website so there should be only one per page'\n        limit: 1\n\n    # Can be used to test that specific selectors are not used in the HTML\n    # I would use this for ensuring that `\u003chr\u003e` tags aren’t used when borders should be or that `\u003cbr\u003e` tags aren’t used\n    # Will be skipped if validation isn’t also checked—the document must be valid first\n    # If given an array, the second argument can be a custom error message\n    hasNot:\n      - 'br'\n      - ['hr', 'The `hr` tag should not be used to create borders']\n      # Warnings too!\n      - selector: 'hr'\n        message: 'The `hr` tag should not be used to create borders'\n        type: 'warning'\n\n    # Regex searches on the file, for confirming specific content\n    # If given an array, the second argument can be a custom error message\n    search:\n      - 'Hello World!'\n      - ['Hello World!', 'Whoa, don’t be so grumpy, say “Hello”']\n      # And warnings!\n      - regex: 'Hello World!' # (Using `check` instead of `regex` is okay too, so everything can be consistent)\n        message: 'Whoa, don’t be so grumpy, say “Hello”'\n        type: 'warning'\n        # Limits work inside search to prevent content duplication\n        limit: 1\n\n    # Regex searches on the file, for confirming specific content isn’t found\n    # If given an array, the second argument can be a custom error message\n    searchNot:\n      - 'Thing-a-magic'\n      # Warnings too!\n      - check: 'Thing-a-magic'\n        type: 'warning'\n\n    # Confirm that the code file has less than or equal to this many lines\n    maxLines: 4\n```\n\n### CSS file tests\n\nUse the `css` entry to test CSS files, with many of the same options as the HTML.\n\n```yml\ncss:\n    # The CSS file’s path\n  - path: 'css/main.css'\n\n    # Whether the code in this file should be locked or not, to help prevent students from changing the code\n    # I use it for assignments where I give them complete CSS \u0026 they just write the HTML or JS\n    # With locked on, there isn’t much point providing the rest of the options\n    locked: true\n\n    # Whether to validate it or not\n    valid: true\n\n    # Check its best practices \u0026 indentation (spaces after colons, new lines after closing blocks, etc.)\n    # Can be further configured in the `stylelint.json` file\n    # Will be skipped if validation isn’t also checked—the document must be valid first\n    bestPractices: true\n\n    # Can be used to test for specific selectors, properties \u0026 values\n    #   [selector, property (optional), value (optional)]\n    # Will be skipped if validation isn’t also checked—the document must be valid first\n    has:\n      - ['.thing']\n      - ['.super', 'background-image']\n      - ['.thang', 'width', '50px']\n      # Starting the array with an `@` and a matching string will look inside media queries\n      - ['@38em', 'html', 'font-size', '110%']\n      # Adding another argument at the end will be a custom error message\n      - ['html', 'font-size', '100%', 'The `font-size` should always be `100%`']\n      - ['@38em', 'html', 'font-size', '110%', 'The `font-size` should always increase at `38em`']\n      # You can include all that same complexity with warnings\n      - check: ['@38em', 'html', 'font-size', '110%', 'The `font-size` should always increase at `38em`']\n        type: 'warning'\n      # Or a little cleaned-up version, can be used with or without warnings\n      - mediaQuery: '@38em'\n        selector: 'html'\n        property: 'font-size'\n        value: '110%'\n        message: 'The `font-size` should always increase at `38em`'\n        type: 'warning'\n\n    # Can be used to test that specific selectors do not contain certain properties\n    # I would use this for ensuring as little CSS duplication as possible, like forcing students to use multiple classes\n    #   [selector, [property, property, etc.]]\n    # Will be skipped if validation isn’t also checked—the document must be valid first\n    hasNot:\n      - ['.btn-ghost', ['display']]\n      - ['.btn-subtle', ['font-size', 'text-decoration']]\n      # Starting the array with an `@` and a matching string will look inside media queries\n      - ['@110em']\n      # Adding another argument at the end will be a custom error message\n      - ['.btn-subtle', ['font-size', 'text-decoration'], 'The `.btn-subtle` shouldn’t be used']\n      - ['@110em', '.btn-subtle', ['font-size', 'text-decoration'], 'The `.btn-subtle` shouldn’t be used']\n      # Also with warnings!\n      - check: ['.btn-ghost', ['display']]\n        type: 'warning'\n\n    # Regex searches on the file\n    # If given an array, the second argument can be a custom error message\n    search:\n      - '@keyframes'\n      - ['@viewport', 'The `@viewport` should be included for the best browser compatibility']\n      # Warnings—woot!\n      - check: '@keyframes'\n        type: 'warning'\n        # Limits work inside search to prevent content duplication\n        limit: 1\n\n    # Regex searches on the file for confirming certain things don’t exist\n    # If given an array, the second argument can be a custom error message\n    searchNot:\n      - ['@media.+\\(.*max-width', 'Media queries with `max-width` should not be used — use `min-width` instead']\n      - ['@media.+\\(.*px', 'Pixel units should not be used in media queries — use `em` instead']\n      - ['font-size\\s*:\\s*.+px', 'Pixel units should not be used for `font-size` — use `rem` instead']\n      # Using the object syntax and `type` there can also be warning messages\n      - check: 'font-size\\s*:\\s*.+px'\n        message: 'Pixel units should not be used for `font-size` — use `rem` instead'\n        type: 'warning'\n\n    # Confirm that the code file has less than or equal to this many lines\n    maxLines: 4\n```\n\n### Javascript file tests\n\nUse the `js` entry to test Javascript files.\n\n```yml\njs:\n    # The JS file’s path\n  - path: 'js/main.js'\n\n    # Whether the code in this file should be locked or not, to help prevent students from changing the code\n    # I use it for assignments where I give them complete JS \u0026 they just write the HTML or CSS\n    # With locked on, there isn’t much point providing the rest of the options\n    locked: true\n\n    # Whether to validate/lint it or not using a series of best practices\n    # Can be further configured in the `validation/eslint.json` file\n    valid: true\n\n    # Check its best practices \u0026 indentation (semicolons, spacing around brackets, etc.)\n    # Can be further configured in the `best-practices/eslint.json` file\n    # Will be skipped if validation isn’t also checked—the document must be valid before best practices are tested\n    bestPractices: true\n\n    # Regex searches on the file\n    search:\n      - 'querySelector'\n      - 'addEventListener'\n      # Using a slightly different syntax you can create warnings that don’t prevent the user from submitting\n      - check: 'querySelectorAll'\n        type: 'warning'\n        # Limits work inside search to prevent content duplication\n        limit: 1\n\n    # Regex searches on the file for confirming certain things don’t exist\n    # If given an array, the second argument can be a custom error message\n    searchNot:\n      - 'document.write\\('\n      - ['console.log\\(', 'The `console.log()` function should not be left in your code after you’ve finished debugging']\n      # Warnings work too!\n      - check: 'console.log\\('\n        message: 'The `console.log()` function should not be left in your code after you’ve finished debugging'\n        type: 'warning'\n\n    # Confirm that the code file has less than or equal to this many lines\n    maxLines: 4\n```\n\n### Screenshot comparisons\n\nMarkbot can be used to compare student work against reference screenshots included in the repository.\n\n```yml\nscreenshots:\n    # The path to the HTML file that will be screenshot\n  - path: 'index.html'\n\n    # An array of different screen widths for taking screenshots\n    sizes: [400, 650, 960]\n\n    # OR: use the alternative object syntax to control the allowed percentage for each screenshot\n    # This will bypass the default percentages (see below)\n    # The syntax is `screenshot-width: allowed-difference-percentage`\n    sizes:\n      400: 10\n      650: 10\n      960: 8\n\n    # If you’d like to capture the printed version of the website, send `print` as one of the sizes\n    sizes: ['print']\n\n    # If there are multiple screenshots of the same path the label is required to distinguish them (it’s used in the filenames)\n    # It can also be used to add a little extra information to the check list\n    label: 'Clicked'\n\n    # By default animations \u0026 transitions are generally disabled using the * selector and !important\n    # This will allow you to enable them again\n    # Be careful to wait for them to complete before taking a screenshot\n    # Something like: on('.elem', 'animationend', done);\n    allowAnimations: true\n\n    # Execute some Javascript before capturing the screenshots\n    # Happens immediately, before the first screenshot is taken and is not repeated for each size\n    # Has access to all the same functions as the `functionality` tests with a few small exceptions:\n    #   - `pass()` \u0026 `fail()` don’t exist\n    #   - `done()` must be called when the screenshot capturing should begin\n    before: |\n      on('.btn', 'transitionend', () =\u003e {\n        done();\n      });\n      activate('.btn');\n```\n\n*Markbot will look in the `screenshots` folder for images to compare against.*\n\nThe screenshots should be generated using Markbot itself for the most consistency—trigger the “Develop” menu ([See Environment variables](#environment-variables)) and press “Generate Reference Screenshots”.\n\nMarkbot will display differences to students highlighted in a black \u0026 white difference image. Difference percentages are calculated and **anything with a difference greater than 10%–13% is considered an error.**\n\nThe difference percentage is calculated based on the area of the image, following this formula:\n\n```\n\u003c 1000000px — 13%\n\u003e= 1000000px — 12%\n\u003e= 3000000px — 11%\n\u003e= 6000000px — 10%\n```\n\n*The difference percentage is a sliding scale because as the screenshot area increased the percentage must go down to maintain a similar level of strictness.*\n\n![](.readme/visual-diff.png)\n\n*An example of screenshot difference errors.*\n\nStudents can enlarge the difference screenshot into a split view window for clearer understanding.\n\n![](.readme/split-view.png)\n\n*Markbot split view screenshot comparison tool.*\n\n### Functionality tests\n\nMarkbot has the ability to run arbitrary Javascript code against an HTML file. This is great for running integration and functionality tests on student code to make sure their Javascript is doing the right thing.\n\nUse the `functionality` entry in the `.markbot.yml` file to add tests for HTML files.\n\nEach entry in the `functionality` list will perform the following actions:\n\n1. The `path` will be loaded into a hidden browser window\n2. When the website has finished loading the testing will start\n3. Markbut will run through every entry in the `tests` array\n4. Each Javascript test will be run inside a function that gets injected into the fully loaded page\n5. If the test calls the `pass()` function, then the test passes, otherwise it should call the `fail()` function with a string describing the problem\n\n**If a single test doesn’t pass the remainder of the tests will not execute.**\n\nHere’s an example from one of my assignments:\n\n```yml\nfunctionality:\n    # The path to the HTML file to load in a hidden browser window\n  - path: \"index.html\"\n\n    # The label can be used to help distinguish tests in the check list — it’s completely optional\n    # When you need a full refresh of the page, including another test with the same path makes sense\n    # But then they’re all listed the same in the check list\n    # Label gives you the ability to slightly distinguish the test names\n    label: \"Dinosaur link\"\n\n    # A replacement for individual tests\n    # Will just confirm that there are no Javascript errors\n    noErrors: true\n\n    # This Javascript code will be injected into the page before it loads\n    # The feature exists mainly so I can overwrite built-in features like `prompt()` to test user input\n    setup: |\n      window.prompt = function () {\n        return 'a';\n      }\n\n    # An array of Javascript code pieces to run against the live website\n    tests:\n      - |\n        let ball = $('.ball');\n        let currentColour = css(ball).backgroundColor;\n        $('input[type=\"color\"]').value = '#ffee66';\n        $('form').dispatchEvent(ev('change'));\n\n        if (currentColour == css(ball).backgroundColor) fail('The ball’s colour doesn’t change');\n\n        pass();\n```\n\nEach test entry will be embedded into a Javascript anonymous self-executing function with a try-catch block, like this:\n\n```js\n(function () {\n  'use strict';\n\n  try {\n    eval(\"(function(){'use strict';/* Your test code will be embedded here */}())\");\n  } catch (e) {\n    /* Show error messages in Markbot \u0026 console */\n  }\n}());\n```\n\n*Yes, eval is evil, etc. But it’s useful here to catch any syntax errors you may have in your code so they can be displayed in the debugging console.*\n\nAlso notice that your test code will be wrapped in a self-executing function, this allows you to use `return` to short-circuit functions when they `fail` or `pass`.\n\nYour injected code will have access to a few functions to simplify what you have to write:\n\n- **`pass()`** — Tell Markbot that this test has passed\n- **`fail(reason)`** — Tell Markbot that this test has failed\n  - The `reason` should be a string that will be shown to the user in Markbot’s error list\n- **`debug(...args)`** — For when writing the tests, to help you debug your test code\n  - What ever is passed into `debug()` will be written to the console\n- **`$(selector[, target = document])`** — Instead of having to write `document.querySelector()`\n  - The `target` parameter allows you to use `querySelector()` on elements other than `document`—but defaults to `document`\n  - If the `selector` isn’t found on the page the test will fail with an error message\n- **`$$(selector[, target = document])`** — Instead of having to write `document.querySelectorAll()`\n  - The `target` parameter allows you to use `querySelectorAll()` on elements other than `document`—but defaults to `document`\n  - If the `selector` isn’t found on the page the test will fail with an error message\n- **`css(element)`** — A shortcut to `getComputedStyle()`\n- **`bounds(element)`** — A shortcut to `getBoundingClientRect()`\n- **`offset(element)`** — Returns the complete offset for the element to the `top` and `left` of the page, using `getBoundingClientRect()` + `scrollY/X` for the calculation.\n  - Returns an object in the format:\n    ```js\n    {\n      left: 0,\n      top: 0,\n    }\n    ```\n- **`on(selector/element, eventname, callback[, timeoutlength = 2000])`** — This is instead of using `addEventListener`. The problem with `addEventListener` is timing.\n  - If the student’s code uses event delegation, but yours listens directly on the element, your listener will be fired first.\n  - Using `on()` will always bind to the `document` and listen for the event to bubble back upwards—guaranteeing that your listener gets called second.\n  - `selector/element` a pre-selected DOM element object or CSS selector to match for the target for your event.\n  - `eventname` is any standard event like `click`, `animationend`, etc.\n  - `callback` is a function that will be executed when the event is triggered. It will receive two arguments:\n    - `hasError` — (`bool`): whether or not the timeout was executed, meaning the event was never triggered. Set to `true` when there is an error and `false` otherwise.\n    - `ev` — the standard Javascript event object passed through.\n  - `timeoutlength` is an optional argument you can pass to control the maximum length the listener will wait to be called.\n- **`ev(eventString[, options])`** — Can be used to fire an event with `dispatchEvent()`\n  - It creates a `new Event`, `new MouseEvent` or `new KeyboardEvent`\n  - `options` has a default of: `{bubbles: true, cancelable: true}`\n  - If you provide an options argument it will be merged with the defaults\n- **`hover(selector/element, callback)`** — A specialized event dispatch that hovers the mouse over an element—regular JS events aren’t “trusted” and therefore won’t trigger the CSS `:hover` styles.\n  - Allows for testing to make sure student’s apply hover states to elements in CSS.\n  - `selector/element` a pre-selected DOM element object or the CSS selector of the target for your event.\n  - `callback` is a function that will be executed when the hover has triggered.\n- **`activate(selector/element, callback)`** — A specialized event dispatch that “activates” an element triggering the CSS `:active` styles.\n  - Allows for testing to make sure student’s apply active states to elements in CSS.\n  - `selector/element` a pre-selected DOM element object or the CSS selector of the target for your event.\n  - `callback` is a function that will be executed when the hover has triggered.\n- **`send(eventname[, options[, callback]])`** — sends trusted input events to the browser window. This is what `hover()` and `activate()` do internally.\n  - It’s essentially a wrapper around Electron’s `webContents.sendInputEvent()`—[See the Electron docs.](https://electron.atom.io/docs/api/web-contents/#contentssendinputeventevent)\n  - `eventname` is one of Electron’s allowed events: `mouseDown`, `mouseUp`, `mouseEnter`, `mouseLeave`, `contextMenu`, `mouseWheel`, `mouseMove`, `keyDown`, `keyUp`, `char`\n  - `options` allows you to set extra properties, like Electron’s `modifiers`. But most importantly it sets the `isTrusted` flag to `true` by default to allow `hover`, etc. activate the CSS changes.\n\n*Here’s an example of using `ev()` and `on()`:*\n\n```yaml\nfunctionality:\n  - path: 'index.html'\n    tests:\n      - |\n        let btn = $('.btn');\n        let btnFill = $('path:nth-child(2)', btn);\n\n        on('.btn path:nth-child(2)', 'transitionend', function (err, ev) {\n          if (err) fail('The transition on the button’s coloured setion never ends—check that is has a transition');\n          if (oldBtnFill == css(btnFill).fill) fail('The button doesn’t change colour');\n          pass();\n        });\n\n        btn.dispatchEvent(ev('click'));\n```\n\n*Here’s an example of using `hover()`:*\n\n```yaml\nfunctionality:\n  - path: 'index.html'\n    tests:\n      - |\n        let a = document.querySelector('a');\n        let oldBg = css(a).backgroundColor;\n\n        hover('a', function () {\n          if (css(a).backgroundColor == oldBg) fail('The hover colour doesn’t change on the link');\n          pass();\n        });\n```\n\n### Markdown tests\n\nMarkbot can check Markdown files looking for validation \u0026 best practices. It can also search inside the files for specific content.\n\nIf the Markdown files have YAML front matter that will also be validated with the same processor used by the YAML file checking.\n\n*The `path` option is the only one that’s required—leaving any of the others off will skip the test.*\n\n```yml\nmd:\n  # The file’s path\n  - path: \"README.md\"\n\n    # Check validation \u0026 best practices following a specific Markdown format\n    valid: true\n\n    # Regex searches on the file, for confirming specific content\n    # If given an array, the second argument can be a custom error message\n    search:\n      - 'Dinosaurs'\n      - ['T\\. Rex', 'Expected to see the T. Rex described']\n      # Warnings will work too!\n      # And so do limits!\n\n    # Regex searches on the file, for confirming specific content isn’t found\n    # If given an array, the second argument can be a custom error message\n    searchNot:\n      - 'Mammals'\n      # Warnings will work too!\n```\n\n### YAML tests\n\nMarkbot can check YAML files looking for validation \u0026 best practices. It can also search inside the files for specific content.\n\n*The `path` option is the only one that’s required—leaving any of the others off will skip the test.*\n\n```yml\nyml:\n  # The file’s path\n  - path: \"data.yml\"\n\n    # Check validation \u0026 best practices\n    valid: true\n\n    # Regex searches on the file, for confirming specific content\n    # If given an array, the second argument can be a custom error message\n    search:\n      - 'Mammals'\n      - ['Dimetrodon', 'Should have explained that the Dimetrodon isn’t a dinosaur']\n      # Warnings will work too!\n      # And so do limits!\n\n    # Regex searches on the file, for confirming specific content isn’t found\n    # If given an array, the second argument can be a custom error message\n    searchNot:\n      - 'Dinosaurs'\n      # Warnings will work too!\n```\n\n### File \u0026 image tests\n\nMarkbot can check random plain text files and images for specific features.\n\n- **Images (`.jpg`, `.png`):** compare dimensions, compare file size, check if metadata has been removed by smushing.\n- **Text files:** compare file size, check if it’s empty, search/not with regexes, check for smushing with removal of line-breaks.\n- **Favicons (`.ico`):** — Favicons can be checked but they won’t be checked for smushing or dimensions—Markbot always enforces `16` \u0026 `32` pixels sizes in favicons.\n- **SVG (`.svg`)** — SVG are treated as hybrids: dimensions can be checked even though they’re technically text files (Markbot will look at the `width`, `height`, and `viewBox` attributes) and smushing only checks for line breaks.\n\nUse the `files` entry to test images and text files, it’s an array of objects, each representing a file to test.\n\n*The `path` option is the only one that’s required—leaving any of the others off will skip the test.*\n\n```yml\nfiles:\n  # The file’s path\n  - path: \"images/mars-2.jpg\"\n\n    # Essentially checks to make sure the file doesn’t exist\n    # All other checks will be ignored\n    exists: false\n\n    # The maximum allowed file size represented in kilobytes (kB)\n    maxSize: 300\n\n    # For images only\n    # The maximum/minimum allowed width\n    maxWidth: 3000\n    minWidth: 320\n\n    # For images only\n    # The maximum/minimum allowed height\n    maxHeight: 1500\n    minHeight: 240\n\n    # Check if the image/file has been smushed\n    # For images it’ll look for extraneous metadata, something ImageOptim would remove\n    # For text files \u0026 SVGs it’ll just look for line breaks\n    smushed: true\n\n    # For text files only\n    # Regex searches on the file, for confirming specific content\n    # If given an array, the second argument can be a custom error message\n    search:\n      - '^Sitemap\\:.+sitemap\\.xml\\s+?$'\n      # Warnings will work too!\n      # And so do limits!\n\n    # For text files only\n    # Regex searches on the file, for confirming specific content isn’t found\n    # If given an array, the second argument can be a custom error message\n    searchNot:\n      - 'Allow:'\n      - ['Disallow\\:\\s*\\/', 'The disallow all directive (`Disallow: /`) should not be used']\n      # Warnings will work too!\n\n  # OR…\n  # Just pass a directory \u0026 rely on the `allFiles` directive described below\n  - directory: \"images\"\n```\n\n### Performance testing\n\nMarkbot can check the performance of a website on simulated networks—or without network throttling. Markbot will check specific performance statistics and compare them to a performance budget.\n\n*The `path` option is the only one that’s required.* If only the `path` is included the default performance budget will be used.\n\n```yml\nperformance:\n  # The path to an HTML file to load and test\n  - path: 'index.html'\n\n    # The network speed (see list below)\n    speed: 'WIFI'\n\n    budget:\n      # Milliseconds for maximum load time\n      maxLoadTime: 1000\n      # Maximum number of assets\n      maxRequests: 15\n      # Maximum page size of all assets in kilobytes (kB)\n      maxSize: 800\n      # Maximum number of fonts allowed on the page\n      maxFonts: 5\n```\n\n#### Simulated networks speeds\n\nMarkbot has a few simulated network speeds built in—you can see all the details of in the [app/networks.js](app/networks.js) file.\n\n- WIFI-FAST\n- WIFI-REGULAR (WIFI)\n- DSL\n- 4G-REGULAR (4G)\n- 3G-GOOD (3G)\n- 3G-REGULAR\n- 2G-GOOD (2G)\n- 2G-REGULAR\n- GPRS\n\n*The names in the brackets are shortcuts: using `speed: '4G'` is exactly the same as `speed: '4G-REGULAR'`.*\n\n#### Default performance budget\n\nHere’s the default performance budget that Markbot will use if you don’t specify your own. If you leave one of the performance budget options off, Markbot will add the missing properties from the default budget.\n\n```yml\nspeed: 'WIFI'\nbudget:\n  maxLoadTime: 1000\n  maxRequests: 15\n  maxSize: 800\n  maxFonts: 5\n```\n\n### Targeting all files\n\nIf you want the same values to work for all files of the same type in a project you can use the `allFiles` entry. With the all files entry we can set defaults that would be applied to all the selected files.\n\n```yml\nallFiles:\n  # Supports any of the entries that `html` supports\n  html:\n    valid: true\n    bestPractices: true\n    outline: true\n    performance: true\n    has:\n      - 'h1'\n    # There’s also a screenshots entry to take screenshots for all HTML files specified\n    # Creates a `screentshots.path` entry for each HTML file\n    screenshots: [320, 400, 608, 960, 1440]\n\n    # You can add the `except` entry to all types to prevent all these defaults from applying to specific files\n    except:\n      - 'test.html'\n\n  # Supports any of the entries that `css` supports\n  css:\n    valid: true\n\n  # Supports any of the entries that `js` supports\n  js:\n    valid: true\n\n  # Supports any of the entries that `md` supports\n  md:\n    valid: true\n\n  # Supports any of the entries that `yml` supports\n  yml:\n    valid: true\n\n  # The functionality tests will be applied to every HTML file\n  # Supports any of the entries that `functionality` supports\n  functionality:\n    noErrors: true\n\n  # Supports any of the entries that `files` supports\n  files:\n    maxWidth: 2500\n    maxHeight: 2500\n    maxSize: 300\n    smushed: true\n\n  # Supports any of the entries that `performance` supports\n  performance:\n    speed: '3G'\n\nhtml:\n  - path: index.html\n  - path: about.html\n    has:\n      - 'h2'\n\nfiles:\n  - path: 'images/dino.jpg'\n\nperformance:\n  - path: 'index.html'\n```\n\nWith this setup everything from the `allFiles-\u003ehtml` entry would be applied to all the HTML files listed below—to help alleviate duplication.\n\n*Adding `functionality` into the `allFiles` entry will test that functionality on every HTML file listed in the `html` entry.*\n\n#### Unique information for all files\n\nWith the `allFiles` entry, for HTML, we can check for uniqueness between the files: e.g. unique `\u003ctitle\u003e` tags or unique `meta description` tags.\n\n```yml\nallFiles:\n  html:\n    unique:\n      - 'title'\n      # Using an array, the second item will be used in the error message as the description\n      # I use this to make the error messages a little simpler to understand\n      # Without the second entry, the error message will just write the selector\n      - ['meta[name=\"description\"][content]', 'meta description']\n      - 'h1'\n```\n\nThe above set up would force all the HTML files to have different title tags, meta descriptions and `\u003ch1\u003e` tags.\n\n*The `unique` entry expects each item to be a valid CSS selector, similar to `has` \u0026 `hasNot`.*\n\nIf you want to check attribute content, select with the attribute selector. **Markbot will grab the last selected attribute** and compare its content. In the example above, Markbot is comparing the `content` attribute.\n\n### Inheriting from templates\n\nMarkbot has a bunch of templates inside the [templates folder](templates) that your Markbot files can inherit from, thereby getting all the requirements specified in that file. Your Markbot file is more powerful and will overwrite entries—but things like `has`, `search`, etc. will be merged together.\n\nThe templates are just standard Markbot files, with all the same properties.\n\nTo inherit from the built-in templates add an `inherit` property to your Markbot file—it’s a list of all the templates to use:\n\n```yml\ninherit:\n  - git-2\n  - html\n  - css\n  - responsive\n\nhtml:\n  - path: index.html\n```\n\nIn the above scenario, everything from the all those templates will be applied to your Markbot file (overwrites based on order) and therefore to the `index.html` file.\n\n### The Markbot ignore file\n\nYou can get Markbot to ignore files within your project directory when using the `allFiles` option. Include a `.markbotignore` file in the same location as your `.markbot.yml` and it’ll be loaded in.\n\n*It doesn’t support glob patterns only simple file and folder paths.*\n\nHere’s an example:\n\n```\npattern-library.html\npatterns/typography       # Since this is a directory everything within would be ignored\npatterns/grid\npatterns/brand\npatterns/icons\npatterns/modules\ncommon/grid.css\ncommon/type.css\ncommon/modules.css\n```\n\n---\n\n## Installation on student computers\n\nBefore getting Markbot working on student machines, these two things should be downloaded and installed on the user’s computer.\n\n### Git\n\nUse the Mac OS X Terminal and install the command line tools with `xcode-select --install`\n\nOn Windows, [install Git directly from the website](https://git-scm.com/download/win). *When installing, on the “Adjusting your PATH environment” screen, switch to “Use Git from the Windows Command Prompt”.*\n\n### JDK\n\nBecause Markbot shells out to two JAR files, the JDK must be available on the user’s computer.\n\n[**Download the JDK.**](http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html)\n\n### Student installation tutorial\n\nCheck out my lesson for the students on installing it on their computers.\n\n*Specifically starting at set 12:* [**Student Markbot installation lesson.**](http://learn-the-web.algonquindesign.ca/courses/web-dev-1/install-all-the-things/#step-12)\n\n**Or install each tool separately, look in the [docs](docs) folder for instructions.**\n\n---\n\n## Building Markbot\n\nMarkbot uses Javascript, [Node.js](https://nodejs.org/en/), and [Electron](http://electron.atom.io/) as its build platform.\n\nFirst clone Markbot to your computer using Git:\n\n```\ngit clone git@github.com:thomasjbradley/markbot.git\n```\n\nIf you’re using a Mac, I’d suggest installing Node.js with [Homebrew](http://brew.sh/).\n\n```\nbrew install node\ncd markbot\nnpm install\n```\n\nYou’ll also need [Wine](https://www.winehq.org/) to make the Windows version on your Mac:\n\n```\nbrew install wine\n```\n\n### Configure the Markbot application\n\nThere’s a few things you need to do to develop Markbot on your computer.\n\n1. [Create two environment variables on your computer.](#environment-variables)\n2. [Create your application config file.](#app-config-file)\n3. [Embed the hashed version of your password into your config file.](#passcode-hashing--embedding)\n4. [Download and install the dependencies.](#markbot-dependencies)\n\n#### 1. Environment variables\n\nStart by making two environment variables on your computer for the “Develop” menu and cheat locking.\n\n```\nMARKBOT_DEVELOP_MENU=\"on\"\nMARKBOT_LOCK_PASSCODE=\"some-long-password-thing\"\n```\n\n*These will allow you to enable the “Develop” menu on your computer and create locking hashes for screenshots, code files, and the `.markbot.yml` file itself.*\n\n[**To set persistent env vars on Mac OS X for command line and GUI, check out this answer on StackOverflow.**](https://stackoverflow.com/questions/135688/setting-environment-variables-in-os-x#answer-32405815)\n\n#### 2. App config file\n\nTo configure your installation of Markbot you’ll need to adjust the config file.\n\nRename `config.example.json` to just `config.json` and change the following options:\n\n- `ignoreCommitEmails` — (array) the list of email addresses to ignore when counting commits.\n\nYou should probably leave this setting alone unless you choose to host your own copy of Learn the Web’s Progressinator.\n\n- `progressinatorApi` — (string) the URL to the Progressinator instance hosted on Learn the Web.\n\n#### 3. Passcode hashing \u0026 embedding\n\nAfter you’ve created your `config.json` file *and* created the two environment variables, run the following command: `npm run hash-passcode`.\n\nThe `hash-passcode` script will generate a `secret` key and hash your password, embedding both into your `config.json`.\n\n*The hashed passcode isn’t really for security, it only uses HMAC-SHA512. The purpose is really to be sufficiently annoying that students will do their work instead of figuring out how to cheat Markbot.*\n\n**Don’t forget to copy this hashed passcode into Progressinator.**\n\n#### 4. HTTPS certificate generation\n\nMarkbot spins up a web server internally to help with performance testing and website loading. The web server uses HTTPS—though it is just a self-signed certificate. The private key and public certificate need to be generated with `openssl` before Markbot can work.\n\nIn your terminal run:\n\n```\nnpm run gen-https-cert\n```\n\nThe script will create the appropriate files and place them into the `app` directory.\n\n*While Markbot is running the tested website will be available at `https://127.0.0.1:PORT` with a randomly assigned port number.*\n\n#### 5. Markbot dependencies\n\nMarkbot has a few external dependencies that it shells out to internally:\n\n- Git\n- Nu HTML validator (JAR)\n- CSS validator (JAR)\n- LanguageTool (JAR)\n\nThe `vendor` folder should contain a bunch of JAR files for the HTML validator, the CSS validator and LanguageTool.\n\n##### HTML validator\n\nThe `vendor/html-validator` folder should contain the `vnu.jar`—the pre-built binary works well.\n\n[**Download the HTML validator release from GitHub.**](https://github.com/validator/validator)\n\n##### CSS validator\n\nThe pre-build JAR files seem to be out of date, so you’ll have to compile the JAR yourself.\n\n[**Download the CSS validator source from GitHub.**](https://github.com/w3c/css-validator)\n\n###### Compiling the CSS validator\n\nYou’ll need the JDK and `ant` to compile the validator. Install `ant` with Homebrew: `brew install ant`.\n\n1. Move into the directory and run `ant jar`.\n2. Move the `css-validator.jar` file into the `vendor/css-validator` folder.\n3. Move the newly created `lib` folder and all its contents into the `vendor/css-validator` folder.\n\n##### LanguageTool\n\nThe LanguageTool is used to check spelling and grammar of commit messages. It’s another Java JAR to install in the `vendor` folder.\n\n1. Go here and download the `.zip` package:\n\n    [**https://languagetool.org/**](https://languagetool.org/)\n\n    (Further down the page, look for: “Stand-alone for the Desktop”)\n\n2. Copy the following items into the `vendor/languagetool` folder:\n\n    - `languagetool-commandline.jar`\n    - `libs/`\n    - `META-INF/`\n    - `org/`\n\n###### Add more words to the dictionary\n\n1. Find the English dictionary, here:\n\n    `org/languagetool/resource/en/hunspell/ignore.txt`\n\n2. Copy all the text from `words-to-add.txt` (it’s inside the `vendor` folder) into the bottom of `ignore.txt`\n\n    **Make sure there are no blank lines between the words.**\n\n##### PDFBox\n\nThe PDFBox application is used primarily for checking screenshots of printed versions of websites—used to convert printed PDFs into PNGs for comparison. It’s another Java JAR to install in the `vendor` folder.\n\n1. Go here and download the “Command line tools” `pdfbox-app-0.0.0.jar` package:\n\n    [**https://pdfbox.apache.org/download.cgi**](https://pdfbox.apache.org/download.cgi)\n\n2. Copy the following items into the `vendor/pdfbox` folder:\n\n    - `pdfbox-app.jar` **Rename it without the version number.**\n\n### Running Markbot\n\nIf you want to run a test version, you can call `start`:\n\n```\nnpm start\n```\n\nTo build a final application for deployment to students, run `build`:\n\n```\nnpm run build\n```\n\n*This will create both the Mac OS X version and the Windows version.*\n\nOr optionally a single platform:\n\n```\nnpm run build-mac\nnpm run build-win\n```\n\nThat’s it, Markbot should be ready to go.\n\n**Upload the Markbot installer files to a location the students can download from.**\n\n---\n\n## Debugging Markbot\n\nMarkbot has access to the developer tools and web inspector as well as a few other tools hidden in the “Develop” menu.\n\n[**To enable the “Develop” menu make sure you have the correct environment variables.**](#environment-variables)\n\nThe “Develop” menu will only show when these two conditions are met:\n\n1. The `MARKBOT_DEVELOP_MENU` env var exists.\n2. The hashed version of the `MARKBOT_LOCK_PASSCODE` is the same as the hashed version in the `config.json` file.\n\n---\n\n## License \u0026 copyright\n\n© 2016–2017 Thomas J Bradley — [GPL](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthomasjbradley%2Fmarkbot","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthomasjbradley%2Fmarkbot","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthomasjbradley%2Fmarkbot/lists"}