{"id":21033152,"url":"https://github.com/josephuspaye/autocheck","last_synced_at":"2025-08-03T04:05:07.910Z","repository":{"id":38227738,"uuid":"255602269","full_name":"JosephusPaye/autocheck","owner":"JosephusPaye","description":"✅ Automatically run pattern matches against a set of files and executables, and generate a neat report of the results.","archived":false,"fork":false,"pushed_at":"2023-10-13T05:06:26.000Z","size":2319,"stargazers_count":4,"open_issues_count":20,"forks_count":3,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-07-03T08:46:48.258Z","etag":null,"topics":["createweekly","marking","nodejs"],"latest_commit_sha":null,"homepage":"","language":"Vue","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/JosephusPaye.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-04-14T12:25:03.000Z","updated_at":"2024-01-29T16:00:46.000Z","dependencies_parsed_at":"2024-11-19T12:52:27.734Z","dependency_job_id":"a84930dd-9ae7-402f-868c-40903a8ce436","html_url":"https://github.com/JosephusPaye/autocheck","commit_stats":null,"previous_names":[],"tags_count":9,"template":false,"template_full_name":null,"purl":"pkg:github/JosephusPaye/autocheck","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JosephusPaye%2Fautocheck","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JosephusPaye%2Fautocheck/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JosephusPaye%2Fautocheck/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JosephusPaye%2Fautocheck/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/JosephusPaye","download_url":"https://codeload.github.com/JosephusPaye/autocheck/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JosephusPaye%2Fautocheck/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":268492060,"owners_count":24258747,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-03T02:00:12.545Z","response_time":2577,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["createweekly","marking","nodejs"],"created_at":"2024-11-19T12:52:15.884Z","updated_at":"2025-08-03T04:05:07.812Z","avatar_url":"https://github.com/JosephusPaye.png","language":"Vue","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Autocheck\n\nAutocheck is a command-line tool that allows you to automatically check for patterns against a set of files and executables, and generate a report to quickly examine the results. It can be a very useful aid when marking programming assignments.\n\nAutocheck is not an auto-grader or auto-marker, and has no special understanding of the files it processes, or the commands it runs. Instead, it provides a simple framework for automating many of the manual checks that are involved with marking programming assignments. You use it by configuring a series of checks in a JSON file, and then running those checks against one or more target directories. The results are automatically consolidated into reports that are optimised for scanning, to save you time.\n\nWith Autocheck, you can automatically run checks that:\n\n- Look for specific files (text files, code files, PDFs, and images), and embed them in the report. Use this to check for READMEs, assignment cover sheets, etc.\n- Execute commands (with the option of using a file as stdin), and embed the output in the report. Use this to compile and run code. It even supports running commands in [Cygwin](https://cygwin.com/) on Windows.\n- Compare files or output from commands against an expected result, and show any differences in the report. Use this to compare the output of student code to some expected standard.\n- Search for the presence or absence of specific strings in specific files, optionally using a regex. It's smart enough to skip comments in source code. Use this to find keywords and special patterns.\n\n---\n\nThis project started as a part of [#CreateWeekly](https://twitter.com/JosephusPaye/status/1214853295023411200), my attempt to create something new publicly every week in 2020.\n\n## Installation\n\nAutocheck is a Node.js package, so you'll need [Node](https://nodejs.org/en/) and npm installed (npm comes with node).\n\nRun the following to install Autocheck:\n\n```\nnpm install -g autocheck\n```\n\n## Tutorial\n\nLearn how to use Autocheck to mark a programming assignment:\n\n\u003ca href=\"https://www.youtube.com/watch?v=lsQ4lIbZk-Y\" target=\"_blank\"\u003e\n  \u003cimg alt=\"Autocheck tutorial video thumbnail\" src=\"./tutorial-thumbnail.png\" width=\"380px\"\u003e\n\u003c/a\u003e\n\n## Usage\n\nCreate a JSON file with the checks you want to run, like the following example (see below for details on available checks):\n\n\u003cdetails\u003e\n\u003csummary\u003e Show example \u003c/summary\u003e\n\n```json\n[\n  {\n    \"type\": \"file\",\n    \"label\": \"Cover Sheet and Readme\",\n    \"patterns\": [\"**/*.pdf\", \"**/*.{jpg,jpeg,png}\", \"**/*.{txt,md}\"]\n  },\n  {\n    \"type\": \"file\",\n    \"label\": \"Code Files\",\n    \"patterns\": [\"**/Node.h\", \"**/Node.cpp\", \"**/LinkedList.h\", \"**/LinkedList.cpp\"]\n  },\n  {\n    \"type\": \"command\",\n    \"label\": \"Compiles\",\n    \"command\": \"make clean \u0026\u0026 make\",\n    \"runInCygwin\": true,\n    \"cygwinBin\": \"C:\\\\path\\\\to\\\\cygwin\\\\root\\\\bin\\\\\"\n  },\n  {\n    \"type\": \"command\",\n    \"label\": \"Runs\",\n    \"if\": \"Compiles\",\n    \"command\": \"./assignment1.exe\",\n    \"runInCygwin\": true,\n    \"cygwinBin\": \"C:\\\\path\\\\to\\\\cygwin\\\\root\\\\bin\\\\\"\n  },\n  {\n    \"type\": \"match\",\n    \"label\": \"Matches Expected Output\",\n    \"if\": \"Runs\",\n    \"expected\": \"file:C:\\\\path\\\\to\\\\file\\\\with\\\\expected-output.txt\",\n    \"actual\": \"output:Runs\"\n  },\n  {\n    \"type\": \"search\",\n    \"label\": \"Doesn't use `struct` or `friend`\",\n    \"filePatterns\": [\"**/Node.h\", \"**/Node.cpp\", \"**/LinkedList.h\", \"**/LinkedList.cpp\"],\n    \"patterns\": [\"friend\\\\s+\", \"struct\\\\s+\"],\n    \"matchAsRegex\": true,\n    \"skipComments\": true,\n    \"passWhen\": \"not-found\"\n  }\n]\n```\n\n\u003c/details\u003e\n\n### Run checks against a directory\n\nUse the following to run Autocheck against a target directory:\n\n```sh\nautocheck ./checks.json \"C:\\\\path\\\\to\\\\submissions\\\\JohnDoe\\\\\"\n```\n\nAutocheck will run the checks and save the report to an HTML file (`JohnDoe.html` in this case) in an `autocheck-reports` folder in the directory where it was run. Open the report file in a browser to view.\n\n### Run checks against multiple directories\n\nUse the following to run Autocheck against multiple target directories at the same time:\n\n```sh\nautocheck ./checks.json \"C:\\\\path\\\\to\\\\submissions\\\\\" --subfolders\n```\n\nThat will run Autocheck against every subfolder in `C:\\path\\to\\submissions\\`, e.g. `C:\\path\\to\\submissions\\JohnDoe\\`, `C:\\path\\to\\submissions\\JaneDoe\\`, etc.\n\n## Example\n\nGiven the following files in `C:\\code\\JosephusPaye\\SENG1120-Assignment-1`:\n\n\u003cdetails\u003e\n\u003csummary\u003eShow files\u003c/summary\u003e\n\n```\nC:\\code\\JosephusPaye\\SENG1120-Assignment-1\\\n|-- Assignment Cover Sheet.pdf\n|-- DeckOfCards.cpp\n|-- DeckOfCards.h\n|-- DeckOfCardsDemo.cpp\n|-- LinkedList.cpp\n|-- LinkedList.h\n|-- Makefile\n|-- Node.cpp\n|-- Node.h\n`-- readme.txt\n```\n\n\u003c/details\u003e\n\nAnd the following `checks.json` file:\n\n\u003cdetails\u003e\n\u003csummary\u003eShow checks.json\u003c/summary\u003e\n\n```json\n[\n  {\n    \"type\": \"file\",\n    \"label\": \"Cover Sheet and Readme\",\n    \"patterns\": [\"**/*.pdf\", \"**/*.{jpg,jpeg,png}\", \"**/*.{txt,md}\"]\n  },\n  {\n    \"type\": \"file\",\n    \"label\": \"Code Files\",\n    \"patterns\": [\"**/*.h\", \"**/*.cpp\", \"**/makefile\"]\n  },\n  {\n    \"type\": \"command\",\n    \"label\": \"Compiles\",\n    \"command\": \"make clean \u0026\u0026 make\",\n    \"runInCygwin\": true,\n    \"cygwinBin\": \"C:\\\\Users\\\\jpaye\\\\AppData\\\\Local\\\\scoop\\\\apps\\\\cygwin\\\\current\\\\root\\\\bin\\\\\"\n  },\n  {\n    \"type\": \"command\",\n    \"label\": \"Runs\",\n    \"if\": \"Compiles\",\n    \"command\": \"./test.exe my-random-seed\",\n    \"runInCygwin\": true,\n    \"cygwinBin\": \"C:\\\\Users\\\\jpaye\\\\AppData\\\\Local\\\\scoop\\\\apps\\\\cygwin\\\\current\\\\root\\\\bin\\\\\"\n  },\n  {\n    \"type\": \"match\",\n    \"label\": \"Matches Expected Output\",\n    \"if\": \"Runs\",\n    \"expected\": \"file:C:\\\\code\\\\JosephusPaye\\\\autocheck\\\\example\\\\expected.txt\",\n    \"actual\": \"output:Runs\"\n  }\n]\n```\n\n\u003c/details\u003e\n\nRunning Autocheck as follows:\n\n```sh\nautocheck checks.json \"C:\\\\code\\\\JosephusPaye\\\\SENG1120-Assignment-1\"\n```\n\nProduces the following output:\n\n\u003cdetails\u003e\n\u003csummary\u003eShow output\u003c/summary\u003e\n\n```\nC:\\code\\JosephusPaye\\autocheck\\example\n\u003e autocheck checks.json \"C:\\\\code\\\\JosephusPaye\\\\SENG1120-Assignment-1\"\nrunning checks from .\\checks.json\n\nchecking directory (1/1): C:\\code\\JosephusPaye\\SENG1120-Assignment-1\n  running check (1/5): Cover Sheet and Readme\n    check passed\n  running check (2/5): Code Files\n    check passed\n  running check (3/5): Compiles\n    check passed\n  running check (4/5): Runs\n    check passed\n  running check (5/5): Matches Expected Output\n    check failed\ngenerated report: .\\autocheck-reports\\SENG1120-Assignment-1.html\n\ndone\n```\n\n\u003c/details\u003e\n\nAnd [this report (screenshot)](https://github.com/JosephusPaye/autocheck/raw/master/screenshot.png).\n\n## Available Checks\n\n### Common options\n\nThe following options are common to all checks:\n\n| Option  | Type    | Presence | Description                                                                                                                         |\n| ------- | ------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------- |\n| `type`  | String  | Required | The type of check, one of `\"file\"`, `\"command\"`, `\"match\"`, or `\"search\"`                                                           |\n| `label` | String  | Required | The label of the check to show in the report                                                                                        |\n| `if`    | String  | Optional | The label of another check to wait for before running the check. If set, the check will only be run if the referenced check passes. |\n| `skip`  | Boolean | Optional | If set to `true`, the check will be skipped.                                                                                        |\n\n### File check\n\nUse this check to look for specific files in the target directory, and embed them in the report. Not every file type can be embedded in the report (see below for embeddable files).\n\nThis check supports the following configuration options (in additional to the common options described above):\n\n| Option     | Type            | Presence | Description                                                                                                                                                                                                                |\n| ---------- | --------------- | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `patterns` | List of Strings | Required | A list of file path patterns ([globs](\u003chttps://en.wikipedia.org/wiki/Glob_(programming)\u003e)), relative to the target directory. For example, `[\"**/*.{pdf,txt}\"]` will match all PDF and text files in the target directory. |\n\nFiles with the following extensions can be embedded. Files marked as **highlighted** will be syntax highlighted in the report:\n\n- `.pdf`\n- `.jpg` and `.jpeg`\n- `.png`\n- `.txt`\n- `md` (highlighted)\n- `c` (highlighted)\n- `cpp` (highlighted)\n- `cs` (highlighted)\n- `h` (highlighted)\n- `hpp` (highlighted)\n- `makefile` (highlighted)\n- `java` (highlighted)\n- `py` (highlighted)\n- `html` (highlighted)\n- `css` (highlighted)\n- `js` (highlighted)\n- `json` (highlighted)\n- `xml` (highlighted)\n- `svg` (highlighted)\n\n### Command check\n\nUse this check to run a command, capture its output, and embed it in the report. On Windows, checks can be run in [Cygwin](https://cygwin.com/).\n\nThis check supports the following configuration options:\n\n| Option        | Type    | Presence | Description                                                                                                                                                                           |\n| ------------- | ------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `command`     | String  | Required | The command to run. For example: `pwd \u0026\u0026 ls`                                                                                                                                          |\n| `directory`   | String  | Optional | The path to use as the command's working directory. Defaults to the target directory.                                                                                                 |\n| `input`       | String  | Optional | The path to a file to use as the command's standard input. Useful for commands that read from `stdin`.                                                                                |\n| `runInCygwin` | Boolean | Optional | Windows only. When `true`, the command will be ran in a Cygwin shell. Cygwin must be installed, and the path to the Cygwin `bin` folder must be provided with the `cygwinBin` option. |\n| `cygwinBin`   | String  | Optional | Windows only. The path to your Cygwin installation's `bin` folder. Used together with the `runInCygwin` option. E.g. `C:\\cygwin\\bin` or `C:\\cygwin\\root\\bin`.                         |\n\nSome notes:\n\n- Double quotes anywhere in a command should be escaped like so: `\\\"something\\\"`.\n- Commands that run longer than 10 seconds will be cancelled and marked as failed.\n- The output of commands (successful or failed) are automatically captured and can be compared to an expected output (see match check below).\n\n### Match check\n\nUse this check to compare two pieces of text for similarity. The texts can be provided as a string, read from a file, or read from a previous command check's output.\n\nThis check supports the following configuration options:\n\n| Option                 | Type    | Presence | Description                                                                                                                                                             |\n| ---------------------- | ------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `expected`             | String  | Required | The expected text. Could be a string (e.g. `\"abc\"`), file reference (e.g. `\"file:C:\\path\\to\\expected.txt\"`), or output capture (e.g. `\"output:LabelOfMyCommandCheck\"`). |\n| `actual`               | String  | Required | The actual text. Could be a string (e.g. `\"abc\"`), file reference (e.g. `\"file:C:\\path\\to\\expected.txt\"`), or output capture (e.g. `\"output:LabelOfMyCommandCheck\"`).   |\n| `ignoreTrailingSpaces` | Boolean | Optional | Ignore differences at the end of lines that are purely whitespace (spaces or tabs). Default `true`.                                                                     |\n\nSome notes:\n\n- Both pieces of text will be be trimmed and compared\n- If they're the same, the actual text will be shown in the report. Otherwise, a diff will be shown with the differences.\n\n### Search check\n\nUse this check to look for specific strings and keywords in text/code files, and embed the search results in the report. Note that only text files can be searched and embedded (see **File check** for embeddable files).\n\nThis check supports the following configuration options (in additional to the common options described above):\n\n| Option         | Type                      | Presence | Description                                                                                                                                                                                                                                                                                                                                                                |\n| -------------- | ------------------------- | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `filePatterns` | List\u0026nbsp;of\u0026nbsp;Strings | Required | A list of file path patterns ([globs](\u003chttps://en.wikipedia.org/wiki/Glob_(programming)\u003e)), relative to the target directory. For example, `[\"**/*.{h,cpp,txt}\"]` will match all `.h`, `.cpp`, and `.txt` files in the target directory.                                                                                                                                   |\n| `patterns`     | List of Strings           | Required | A list of patterns to search for, can be strings or regular expressions. For example, `[\"friend\", \"struct\"]` will match files with the string `friend` or `struct`, while `[\"if\\s*\\(\", \"while\\s*\\(\"]` will match the start of `if` and `while` statements.                                                                                                                 |\n| `matchCase`    | Boolean                   | Optional | For non-regex patterns, use case-sensitive matching. Default is `false`.                                                                                                                                                                                                                                                                                                   |\n| `matchAsRegex` | Boolean                   | Optional | Match the patterns as [JavaScript regular expressions](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_Expressions). The `/` and `/` delimiters are optional: patterns that don't start with `/` will be wrapped with `/\u003cpattern\u003e/i`, which means matching will be case-insensitive. All regex patterns, delimited or not, will be matched globally. |\n| `skipComments` | Boolean or Array          | Optional | Skip strings in comments when matching. Default is `false`. Strings are considered comments if they fall within the range of the given array of delimiters. For example, `[[\"/*\", \"*/\"], \"//\"]` will skip strings between `/*` and `*/`, as well as strings between `//` and a new line. Set to `true` to match the default C-style comments: `/*` + `*/` and `//`.        |\n| `passWhen`     | `found` or `not-found`    | Optional | Choose when to mark the check as passed. Setting to `found` will pass the check if a pattern is found, and fail it otherwise. Setting to `not-found` will fail the check if a pattern is found, and pass it otherwise.                                                                                                                                                     |\n\n## What's next\n\n- [x] Record a tutorial video\n- [x] Add screenshots to this README showing example results of each check\n- [x] Add colors to the CLI output\n- [ ] Warn when commands block for stdin if there's no `input` specified\n- [x] Add search check\n- ~~[ ] Add support for user-defined checks (bring your own checks)~~ - this need is largely served by command checks.\n- [x] Improve check navigation - add collapse/expand all checks button to header, ~~scroll to next (arrow down icon), and scroll to previous (arrow up icon) buttons, with smooth scrolling~~ - better served by \"Jump to...\" menu\n- [ ] Add AST aware checks, using something like [Clang's AST JSON dump](https://stackoverflow.com/a/59417943)\n\n## Licence\n\n[MIT](LICENCE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjosephuspaye%2Fautocheck","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjosephuspaye%2Fautocheck","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjosephuspaye%2Fautocheck/lists"}