{"id":22136467,"url":"https://github.com/kuredoro/scold","last_synced_at":"2025-03-24T10:16:20.734Z","repository":{"id":46244714,"uuid":"307137688","full_name":"kuredoro/scold","owner":"kuredoro","description":"Input/Output black box test automation tool.","archived":false,"fork":false,"pushed_at":"2023-02-25T10:00:59.000Z","size":1733,"stargazers_count":2,"open_issues_count":13,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-17T21:17:45.956Z","etag":null,"topics":["code-execution","code-runner","command-line","competitive-programming","cross-platform","go","output-checking","productivity","test-automation","testing","testing-tool","testing-tools","tool","validation","validation-engine","validator"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kuredoro.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.txt","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-10-25T16:10:06.000Z","updated_at":"2022-01-11T21:07:02.000Z","dependencies_parsed_at":"2024-06-19T00:07:46.668Z","dependency_job_id":"c4771e77-7063-4548-805a-8cf03b0f7c7e","html_url":"https://github.com/kuredoro/scold","commit_stats":null,"previous_names":["kuredoro/cptest","kureduro/cptest"],"tags_count":15,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kuredoro%2Fscold","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kuredoro%2Fscold/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kuredoro%2Fscold/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kuredoro%2Fscold/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kuredoro","download_url":"https://codeload.github.com/kuredoro/scold/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245249232,"owners_count":20584497,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["code-execution","code-runner","command-line","competitive-programming","cross-platform","go","output-checking","productivity","test-automation","testing","testing-tool","testing-tools","tool","validation","validation-engine","validator"],"created_at":"2024-12-01T19:22:37.423Z","updated_at":"2025-03-24T10:16:20.710Z","avatar_url":"https://github.com/kuredoro.png","language":"Go","readme":"## scold\n[![Coverage Status](https://coveralls.io/repos/github/kuredoro/scold/badge.svg?branch=main)](https://coveralls.io/github/kuredoro/scold?branch=main)\n[![GoReport](https://goreportcard.com/badge/github.com/kuredoro/scold)](https://goreportcard.com/report/github.com/kuredoro/scold)\n[![PkgGoDev](https://pkg.go.dev/badge/github.com/kuredoro/scold)](https://pkg.go.dev/github.com/kuredoro/scold)\n[![Actions Status](https://github.com/kuredoro/scold/workflows/build/badge.svg)](https://github.com/kuredoro/scold/actions)\n[![Release](https://img.shields.io/github/release/kuredoro/scold.svg?style=flat-square)](https://github.com/kuredoro/scold/releases/latest)\n\nA tool to speed up the testing of competitive programming code on multiple inputs. I.e., the purpose is to minimize the number of keypresses between finishing writing code and submitting it, knowing that the code passes on the provided test inputs.\n\n* [Install](#install)\n  * [Windows](#windows)\n  * [Linux](#linux)\n* [User guide](#user-guide)\n  * [Command line interface](#command-line-interface)\n  * [inputs\\.txt format](#inputstxt-format)\n  * [How are outputs compared?](#how-are-outputs-compared)\n  * [Verdicts](#verdicts)\n    * [OK: Test pass](#ok-test-pass)\n    * [WA: Wrong answer](#wa-wrong-answer)\n    * [RE: Runtime error](#re-runtime-error)\n    * [TL: Time limit exceeded](#tl-time-limit-exceeded)\n    * [IE: Internal error](#ie-internal-error)\n  * [Test suite configuration](#test-suite-configuration)\n    * [Specifying time limit](#specifying-time-limit)\n    * [Specifying floating point precision](#specifying-floating-point-precision)\n* [Building](#building)\n\n## Install\n\n### Windows\n\nOn Windows go to the releases page on the right and download the scold executable for your architecture. Rename it to `scold.exe`. Additionally, you can create a folder, add it to the PATH and put `scold.exe` in there to access scold from the console.\n\nYou can do all of this automatically by using `scoop`. scoop is a minimalistic install helper for Windows that allows you to install, update, and manage various command-line utilities and applications (python, go, node.js) from within the console in just one line. Get it from [scoop.sh](https://scoop.sh). If you're a fan of Linux, you will be pleased that you can install all of the Linux tools via scoop and enjoy your Linux habits on Windows. Also, if you need some app, just `scoop search` to see if you can have it without a hassle.\n\nWhen you installed scoop, run\n```\n\u003e scoop update\n```\n\nAnd then add the bucket with the install script and install scold.\n```\n\u003e scoop bucket add kuredoro https://github.com/kuredoro/scoop-bucket\n\u003e scoop install scold\n```\n\n### Linux\n\nYou can use AUR on ArchLinux to install scold. Via `yay` AUR helper it would be:\n```\n$ yay -S scold\n```\n\nDon't forget to star packages on AUR if you liked them (─‿‿─).\n\nFor other distros, you'll have to build it from the source, but don't worry, it's just one line. See [Building](#building).\n\n## User guide\n\n### Command-line interface\n\n```\nscold [options] EXECUTABLE [ARGS...]\n```\n\nscold requires an executable to run. Any arguments written after the executable are forwarded to it. This way, one can call `scold node index` to test a Node.js code. The options related to the scold are, therefore, specified before the executable.\n\nPossible arguments:\n\n* `-i`, `--inputs` -- specifies the path to the test suite. Default: `inputs.txt`.\n* `-j`, `--jobs` -- specifies the number of executables to run concurrently. Default: CPU count.\n* `--no-colors` -- disables colored output. Useful for environments that cannot render color, like Sublime Text console.\n* `--no-progress` -- disables progress bar. Since the progress bar requires some specific features of a console emulator, it might not work everywhere.\n\n### `inputs.txt` format\n\nThe format is simple:\n```\n===     (1)\nline 1  ┐\nline 2  │\n---     ├─ Test 1\nline 1  │\nline 2  ┘\n===     (1)\n===\nline 1  ┐\nline 2  │\n---     ├─ Test 2\nline 1  │\nline 2  ┘\n===     (1)\n```\n\nIn other words, your inputs are separated from outputs with `---`, and test cases are separated with `===`. The empty test cases (1) are ignored and allowed. All of the lines in the input and output sections will be newline terminated when parsed by scold.\n\nThe first test can specify a set of **test suite options** and follows a different format\n```\ntl = 10s\nprec = 8\n===\nline 1\nline 2\n---\nline 1\nline 2\n```\n\nSee [Test suite configuration](#test-suite-configuration).\n\n### How are outputs compared?\n\n**TL;DR:** The comparison routine is more or less equivalent to a program that reads the program's output from the `stdin` and compares it against correct values.\n\nThe key concepts in scold are the **lexeme** and the **lexeme type**. Lexeme is a string consisting of printable characters or a single newline. The program's output and the test's answer are parsed and turned into a sequence of lexemes while discarding all whitespaces, tabs, etc. between them.\n\nGiven this sequence, the **lexeme type** is deduced for each lexeme. The lexeme type specifies what type of data the lexeme holds. Currently, there are string, integer and floating-point number types. There is a specialization, or an \"is-a\", relation between the types. For example, every integer is a string, but not every string is an integer, hence integer specializes string type. In fact, specialization relation is the weak order relation: `\u003e=`, and it follows that: string `\u003e=` floating-point number `\u003e=` integer.\n\nFor each lexeme at the same position in both sequences (program's output and the answer) their **common type** is deduced by taking the least specialized type among the two lexemes. Then a comparison routine is invoked that performs highlighting of the mismatched parts depending on the deduced common type. For example, if a discrepancy is found in an integer, the whole integer should be highlighted, instead of individual characters, otherwise, it would be annoying and not logically correct.\n\nAdditional measures are taken to treat excessive newlines rationally. If a misplaced newline is encountered (meaning that the other lexeme is not a newline), the lexemes after this newline are skipped until a non-newline lexeme is encountered. This ensures that in case of an excessive newline, the comparison highlighting stays consistent and valid.\n\n### Verdicts\n\n#### `OK`: Test pass\n\nExample:\n```\n--- OK: Test 1 (0.123s)\n```\n\n`OK` verdict reflects the answer and the program's output being conceptually equal and that the program finished within the specified time frame.\n\nEven if the program has some `stderr` output, it won't be displayed.\n\n#### `WA`: Wrong answer\n\nExample:\n```\n--- WA:\tTest 1 (0.523s)\nInput:\n5\n5 4 3 2 1\n\nAnswer:\n5\n1 2 3 4 5\\n\n\nOutput:\n5\n1 2 4 3 5\n\nStderr:\nSome debug information...\n```\n\n`WA` verdict signalizes that the program's output somehow differs from the correct answer so much that the output cannot be any longer considered correct.\n\nThe discrepancies are highlighted by default and the behavior of the highlight is different depending on the nature of data the inconsistency is found in. If the lexemes that are different are\n\n* floating-point numbers -- the sign, the whole part or/and the fractional part up until the specified precision is highlighted.\n* integer numbers -- the sign or/and all of the digits are highlighted.\n* strings -- the individual characters are highlighted.\n\nMoreover, if a number is too long, it's treated as a string.\n\nThe newlines are not visible, but if one is missing or misplaced, it will be highlighted and rendered as text, in this example, as `\\n`. Since a misplaced newline can skew all of the lexemes that follow it, the extra newlines are ignored as if they have never been in the input while comparing further lexemes.\n\nNext, to aid debugging, the `stderr` is also being printed (if it has anything). This way, a printf-style debugging can not interfere with the output that needs to be compared. Further adjustments to the user's code can create a special logger that will output to `stderr` and that can be easily turned off before sending the code to the online judge.\n\nSome OJ add `ONLINE_JUDGE` preprocessor macro when compiling C/C++ code, like Codeforces or UVa. You can use `#ifdef ONLINE_JUDGE` to conditionally turn off debugging when sending to the OJ. Or do this in reverse: define a macro that will be present when compiling locally, like `g++ main.cpp -DLOCAL_BUILD` and use it instead. More ideas and for more languages can be found in this [Codeforces thread](https://codeforces.cc/blog/entry/14118).\n\n\n#### `RE`: Runtime error\n\nExample:\n```\n--- RE:\tTest 1 (0.005s)\nInput:\n1\n\nAnswer:\n1\\n\n\nExit code: 255\n\nOutput:\nSomething from stdout\n\nStderr:\nSomething from stderr\n```\n\nRuntime errors are detected by looking at the exit code. If it has a non-zero value -- it's a `RE`. When this happens, additional information is always printed: the exit code and the `stderr`.\n\n#### `TL`: Time limit exceeded\n\nExample:\n```\n--- TL:\tTest 1 (6.000s)\nInput:\nabcde\n\nAnswer:\nedcba\\n\n```\n\nWhen the program exceeds the default time limit or the one specified by the user (see [Specifying time limit](#specifying-time-limit)), the program is terminated and the test is failed. Nor the output, nor the `stderr` are shown.\n\n#### `IE`: Internal error\n\nExample (on Linux):\n```\n--- IE:\tTest 160 (0.169s)\nInput:\n55\n\nAnswer:\n55\\n\n\nError:\nexecutable: fork/exec /home/kuredoro/contest_code/a.out: too many open files\n```\n\nThe internal error is a failed test because scold could not perform what it was designed to do. The situations when IE pops out are extremely rare but sometimes can occur. In the example above, the problem is that the executable `a.out` was opened too many times simultaneously exceeding the limit Linux allows an executable to be opened at the same time (on the machine in question). The IE can also appear when scold panics itself, in which case it might be a potential bug. As always, read what the error says and, if anything, ask for help or file a bug on the [issue tracker](https://github.com/kuredoro/scold/issues).\n\n### Test suite configuration\n\nA set of key-value pairs can be specified at the very top of `inputs.txt`. For example:\n```\ntl = 10s\nprec = 8\n===\ninput 1\n---\noutput 1\n===\ninput 2\n---\noutput 2\n```\n\nA key-value pair is a line with an equality sign. The key and the value are located to the left and to the right of the sign, respectively. They both are space-trimmed. So, `\"  two words =   are  parsed\"` is parsed as: `key=\"two words\"` and `value=\"are  parsed\"`.\n\n#### Specifying time limit\n\nSyntax:\n```\ntl = \u003cdigits\u003e [ '.' \u003cdigits\u003e ] \u003cunit\u003e\nunit ::=  \"ns\" | \"us\" | \"µs\" | \"ms\" | \"s\" | \"m\" | \"h\"\n```\n\nExamples:\n```\ntl = 1ms\ntl = 6.66s\ntl = 0.01m\ntl = 0s\n```\n\nThe `tl` option specifies the time limit for the test suite, overriding the default value. The value for the time limit should contain a unit suffix and may contain a fractional part. \"us\" and \"µs\" both correspond to microseconds. If `tl` is specified to be `0`, then the time limit is considered to be infinite.\n\n#### Specifying floating point precision\n\nSyntax:\n```\nprec = \u003cdigits\u003e\n```\n\nExamples:\n```\nprec = 0\nprec = 12\n```\n\nThe `prec` option specifies how many digits after the decimal point should be considered when comparing floating-point lexemes. The value of 0 tells scold to ignore the fractional part.\n\n## Building\n\nTo build `scold` you'll need an installation of `go`. Installing it should be as simple as installing base-devel package (─‿‿─).\n\nE.g. on Arch:\n```\n$ sudo pacman -S go\n```\n\nThen clone the repository, and invoke `go build`.\n```\n$ git clone https://github.com/kuredoro/scold.git\n$ cd scold\n$ go build ./cmd/scold\n```\n\nFinally, move the executable to a folder accessible from PATH, like\n```\n$ sudo mv scold /usr/bin\n```\n\nYou're ready to go!\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkuredoro%2Fscold","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkuredoro%2Fscold","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkuredoro%2Fscold/lists"}