{"id":20210951,"url":"https://github.com/johnhany97/grader","last_synced_at":"2026-06-07T03:31:21.713Z","repository":{"id":74251308,"uuid":"176161319","full_name":"johnhany97/grader","owner":"johnhany97","description":"Automatic Programming Assignments Grader in Go","archived":false,"fork":false,"pushed_at":"2019-04-25T00:31:36.000Z","size":37235,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-01-13T21:34:21.013Z","etag":null,"topics":["assignments","go","grader","job-queue","programming"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/johnhany97.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-03-17T21:11:22.000Z","updated_at":"2021-11-17T09:13:53.000Z","dependencies_parsed_at":null,"dependency_job_id":"33a66477-6c68-4cc9-9e98-2699800d54b3","html_url":"https://github.com/johnhany97/grader","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/johnhany97%2Fgrader","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/johnhany97%2Fgrader/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/johnhany97%2Fgrader/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/johnhany97%2Fgrader/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/johnhany97","download_url":"https://codeload.github.com/johnhany97/grader/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241652921,"owners_count":19997574,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["assignments","go","grader","job-queue","programming"],"created_at":"2024-11-14T05:50:17.289Z","updated_at":"2026-06-07T03:31:21.679Z","avatar_url":"https://github.com/johnhany97.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Automatic Programming Assignments Grader\n\n```\nUsage of main.go:\n  -schema string\n        Marking scheme to follow when grading the assignment (required)\n```\n\n## Testing \u0026 Benchmarking\nTo run the test suite, simply execute the following command:\n```\ngo test -v -cover ./...\n```\nTo run the benchmarks, simply execute the following command:\n```\ngo test -v -run=\"none\" -bench=. -benchmem ./... -benchtime=\"5s\"\n```\n\n## Installation\nFirst off, you'll need to [Install Go](https://golang.org/doc/install) and make sure enviornment variables are set up properly. Following that, you'll need to [Install Docker](https://docs.docker.com/v17.09/engine/installation/). To be able to run Java Style checking test tasks, you'll need to install [Checkstyle](http://checkstyle.sourceforge.net/index.html) as well.\n\n\nRunning the following two commands install two Go-based projects that run the grading.\n```\ngo get -u github.com/docker-exec/dexec\ngo get -u github.com/johnhany97/grader\n```\nNote: We use a modified version of `dexec` which supports timing out executions. Changes should've been merged, however, ensure you have version `1.0.9-SNAPSHOT` of `dexec` installed before continuing.\n\nThat customized version could also be found at [https://github.com/johnhany97/dexec](https://github.com/johnhany97/dexec)\n\n## Supported languages\n\nFollowing is a list of supported languages with the required docker image.\n\n- `C`: `dexec/lang-c`\n- `Clojure`: `dexec/lang-clojure`\n- `CoffeeScript`: `dexec/lang-coffee`\n- `C++`: `dexec/lang-cpp`\n- `C#`: `dexec/lang-csharp`\n- `D`: `dexec/lang-d`\n- `Erlang`: `dexec/lang-erlang`\n- `F#`: `dexec/lang-fsharp`\n- `Go`: `dexec/lang-go`\n- `Groovy`: `dexec/lang-groovy`\n- `Haskell`: `dexec/lang-haskell`\n- `Java`:  `dexec/lang-java` AND `johnhany97/grader-junit`\n- `Lisp`:  `dexec/lang-lisp`\n- `Lua`: `dexec/lang-lua`\n- `JavaScript`: `dexec/lang-node`\n- `Nim`: `dexec/lang-nim`\n- `Objective C`: `dexec/lang-objc`\n- `OCaml`: `dexec/lang-ocaml`\n- `Perl 6`: `dexec/lang-perl6`\n- `Perl`: `dexec/lang-perl`\n- `PHP`: `dexec/lang-php`\n- `Python`: `dexec/lang-python`\n- `R`: `dexec/lang-r`\n- `Racket`: `dexec/lang-racket`\n- `Ruby`: `dexec/lang-ruby`\n- `Rust`: `dexec/lang-rust`\n- `Scala`: `dexec/lang-scala`\n- `Bash`: `dexec/lang-bash`\n\n## Example usage\nThe following commands are to be ran from the project's root directory.\n```\ngrader -schema=\"examples/inputOutput/javaSchema.json\"\n```\n```\ngrader -schema=\"examples/python/pySchema.json\"\n```\n```\ngrader -schema=\"examples/inputOutput/goSchema.json\"\n```\n\n## Schema Properties\n- `file` - Name of the file being graded, should contain it's extension - required\n- `language` - Name of the programming language in which the file being graded is written - optional\n- `className` - Class name of the file provided, usually is the name without the extension - required by junit and pyunit tests\n- `folder` - Folder within which the file provided is located - required\n- `tests` - Array of test tasks - required (even if just empty array)\n- `outfile` - Whether to store results in a file or just output it in the terminal - default = ''\n\n### Test Tasks\nCurrently supported are 4 different types of test tasks. They should confirm to the following format:\n```\n{\n  \"type\": \"TYPE_OF_TEST\", // io - junit - pyunit - output\n \n  // io --- start\n  \n  \"input\": [ \" \" ], // Array of strings where each string is a line in the input. They are joined by \\n\n  \"expectedOutput\": [ \" \" ], // Array of strings where each string is a line in the expected output. They are joined by \\n\n  \n  // io --- end\n \n  // junit/pyunit --- start\n  \n  \"unitTest\": \"\", // The test to be injected in the shell file, after being escaped\n  \n  // junit/pyunit --- end\n  \n  // output --- start\n  \n  \"expectedOutput\": [ \" \" ], // Array of strings where each string is a line in the expected output. They are joined by \\n\n  \n  // output --- end\n  \n}\n```\n\n## Example Schema\n```JSON\n{\n  \"file\": \"Solution.java\",\n  \"language\": \"java\",\n  \"className\": \"Solution\",\n  \"folder\": \"examples/inputOutput/\",\n  \"tests\": [\n    {\n      \"type\": \"javaStyle\"\n    },\n    {\n      \"type\": \"io\",\n      \"description\": \"Testing normal boundaries\",\n      \"input\": [\n        \"3\",\n        \"1\",\n        \"2\",\n        \"3\"\n      ],\n      \"expectedOutput\": [\n        \"1\",\n        \"2\",\n        \"3\"\n      ]\n    },\n    {\n      \"type\": \"io\",\n      \"input\": [\n        \"2\",\n        \"2\",\n        \"5\"\n      ],\n      \"expectedOutput\": [\n        \"2\",\n        \"5\"\n      ]\n    },\n    {\n      \"type\": \"junit\",\n      \"unitTest\": \"@Test\\n  public void adderWorksWithZero() {\\n    Solution s = new Solution();\\n    int actual = s.adder(0, 3);\\n    assertEquals(3, actual);\\n  }\"\n    }\n  ],\n  \"outfile\": \"\"\n}\n```\nOutput: (Order may vary as tasks are finished by the job worker)\n```JSON\n[  \n   {  \n      \"test\":{  \n         \"type\":\"javaStyle\",\n         \"description\":\"\",\n         \"input\":null,\n         \"expectedOutput\":null,\n         \"unitTest\":\"\"\n      },\n      \"description\":\"Results generated by Checkstyle. A static analyzer built for analyzing Java Programs.\",\n      \"stdOut\":\"Starting audit...\\n[WARN] /Users/john/projects/dissertation/grader/examples/inputOutput/Solution.java:1: Using the '.*' form of import should be avoided - java.util.*. [AvoidStarImport]\\n[WARN] /Users/john/projects/dissertation/grader/examples/inputOutput/Solution.java:4:3: Missing a Javadoc comment. [JavadocMethod]\\n[WARN] /Users/john/projects/dissertation/grader/examples/inputOutput/Solution.java:16: Comment has incorrect indentation level 0, expected is 2, indentation should be the same level as line 18. [CommentsIndentation]\\n[WARN] /Users/john/projects/dissertation/grader/examples/inputOutput/Solution.java:16: Line is longer than 100 characters (found 211). [LineLength]\\nAudit done.\",\n      \"stdErr\":\"\",\n      \"successful\":false,\n      \"similarity\":0,\n      \"timedOut\":false\n   },\n   {  \n      \"test\":{  \n         \"type\":\"io\",\n         \"description\":\"Testing normal boundaries\",\n         \"input\":[  \n            \"3\",\n            \"1\",\n            \"2\",\n            \"3\"\n         ],\n         \"expectedOutput\":[  \n            \"1\",\n            \"2\",\n            \"3\"\n         ],\n         \"unitTest\":\"\"\n      },\n      \"description\":\"Expected:\\n1\\n2\\n3\\nGot:\\n1\\n2\\n3\\nTest passed: true\",\n      \"stdOut\":\"1\\n2\\n3\",\n      \"stdErr\":\"\",\n      \"successful\":true,\n      \"similarity\":1,\n      \"timedOut\":false\n   },\n   {  \n      \"test\":{  \n         \"type\":\"io\",\n         \"description\":\"\",\n         \"input\":[  \n            \"2\",\n            \"2\",\n            \"5\"\n         ],\n         \"expectedOutput\":[  \n            \"2\",\n            \"5\"\n         ],\n         \"unitTest\":\"\"\n      },\n      \"description\":\"Expected:\\n2\\n5\\nGot:\\n2\\n5\\nTest passed: true\",\n      \"stdOut\":\"2\\n5\",\n      \"stdErr\":\"\",\n      \"successful\":true,\n      \"similarity\":1,\n      \"timedOut\":false\n   },\n   {  \n      \"test\":{  \n         \"type\":\"junit\",\n         \"description\":\"\",\n         \"input\":null,\n         \"expectedOutput\":null,\n         \"unitTest\":\"@Test\\n  public void adderWorksWithZero() {\\n    Solution s = new Solution();\\n    int actual = s.adder(0, 3);\\n    assertEquals(3, actual);\\n  }\"\n      },\n      \"description\":\"\",\n      \"stdOut\":\"JUnit version 4.10\\r\\n.\\r\\nTime: 0.009\\r\\n\\r\\nOK (1 test)\",\n      \"stdErr\":\"\",\n      \"successful\":true,\n      \"similarity\":0,\n      \"timedOut\":false\n   }\n]\n```\n\n## Test and Benchmark runs\nLast Test run:\n```\n=== RUN   TestExecute\n--- PASS: TestExecute (8.65s)\n=== RUN   TestExecuteWithInput\n--- PASS: TestExecuteWithInput (8.89s)\n=== RUN   TestExecuteJUnitTests\n--- PASS: TestExecuteJUnitTests (4.23s)\n=== RUN   TestExecutePyUnitTests\n--- PASS: TestExecutePyUnitTests (2.38s)\n=== RUN   TestExecuteJavaStyle\n--- PASS: TestExecuteJavaStyle (0.66s)\nPASS\ncoverage: 77.9% of statements\nok      github.com/johnhany97/grader/processors 24.823s coverage: 77.9% of statements\n=== RUN   TestRunTestInputOutputTestHandler\n--- PASS: TestRunTestInputOutputTestHandler (28.10s)\n    input_output_test.go:98: Successfully graded the solution for .java language\n    input_output_test.go:98: Successfully graded the solution for .py language\n    input_output_test.go:98: Successfully graded the solution for .cpp language\n    input_output_test.go:98: Successfully graded the solution for .cs language\n    input_output_test.go:98: Successfully graded the solution for .java language\n    input_output_test.go:98: Successfully graded the solution for .py language\n    input_output_test.go:98: Successfully graded the solution for .cpp language\n    input_output_test.go:98: Successfully graded the solution for .cs language\n    input_output_test.go:98: Successfully graded the solution for .java language\n    input_output_test.go:98: Successfully graded the solution for .py language\n    input_output_test.go:98: Successfully graded the solution for .cpp language\n    input_output_test.go:98: Successfully graded the solution for .cs language\n=== RUN   TestNewResultInputOutputTestHandler\n--- PASS: TestNewResultInputOutputTestHandler (0.00s)\n    input_output_test.go:187: Successfully obtained the expected result\n    input_output_test.go:187: Successfully obtained the expected result\n=== RUN   TestHandleErrInputOutputTestHandler\n--- PASS: TestHandleErrInputOutputTestHandler (0.00s)\n=== RUN   TestRunTestJavaStyleTestHandler\n--- PASS: TestRunTestJavaStyleTestHandler (0.62s)\n    java_style_test.go:49: Successfully style checked the solution for .java language\n=== RUN   TestNewResultJavaStyleTestHandler\n--- PASS: TestNewResultJavaStyleTestHandler (0.00s)\n    java_style_test.go:119: Successfully obtained the expected result\n    java_style_test.go:119: Successfully obtained the expected result\n=== RUN   TestHandleErrJavaStyleTestHandler\n--- PASS: TestHandleErrJavaStyleTestHandler (0.00s)\n=== RUN   TestRunTestJUnitTestHandler\n--- PASS: TestRunTestJUnitTestHandler (8.23s)\n    junit_test.go:65: Successfully graded the solution for .java language\n    junit_test.go:65: Successfully graded the solution for .java language\n=== RUN   TestNewResultJUnitTestHandler\n--- PASS: TestNewResultJUnitTestHandler (0.00s)\n    junit_test.go:138: Successfully obtained the expected result\n    junit_test.go:138: Successfully obtained the expected result\n=== RUN   TestHandleErrJUnitTestHandler\n--- PASS: TestHandleErrJUnitTestHandler (0.00s)\n=== RUN   TestRunTestOutputTestHandler\n--- PASS: TestRunTestOutputTestHandler (8.67s)\n    output_test.go:66: Successfully graded the solution for .java language\n    output_test.go:66: Successfully graded the solution for .py language\n    output_test.go:66: Successfully graded the solution for .cpp language\n    output_test.go:66: Successfully graded the solution for .cs language\n=== RUN   TestNewResultOutputTestHandler\n--- PASS: TestNewResultOutputTestHandler (0.00s)\n    output_test.go:145: Successfully obtained the expected result\n    output_test.go:145: Successfully obtained the expected result\n=== RUN   TestHandleErrOutputTestHandler\n--- PASS: TestHandleErrOutputTestHandler (0.00s)\n=== RUN   TestRunTestPyUnitTestHandler\n--- PASS: TestRunTestPyUnitTestHandler (3.36s)\n    pyunit_test.go:54: Successfully graded the solution for .py language\n    pyunit_test.go:54: Successfully graded the solution for .py language\n=== RUN   TestNewResultPyUnitTestHandler\n--- PASS: TestNewResultPyUnitTestHandler (0.00s)\n    pyunit_test.go:109: Successfully obtained the expected result\n    pyunit_test.go:109: Successfully obtained the expected result\n=== RUN   TestHandleErrPyUnitTestHandler\n--- PASS: TestHandleErrPyUnitTestHandler (0.00s)\nPASS\ncoverage: 93.3% of statements\nok      github.com/johnhany97/grader/test       48.986s coverage: 93.3% of statements\n```\n\n\nLast Benchmark Run:\n```\ngoos: darwin\ngoarch: amd64\npkg: github.com/johnhany97/grader/processors\nBenchmarkExecute-12                            2        2687419688 ns/op           18880 B/op        177 allocs/op\nBenchmarkExecuteWithInput-12                   2        2746329358 ns/op           19224 B/op        183 allocs/op\nBenchmarkExecuteJUnitTests-12                  2        4290938234 ns/op           12016 B/op        100 allocs/op\nBenchmarkExecutePyUnitTests-12                 5        1649888741 ns/op           18478 B/op        182 allocs/op\nBenchmarkExecuteJavaStyle-12                  10         622639507 ns/op           13876 B/op        115 allocs/op\nPASS\nok      github.com/johnhany97/grader/processors 53.897s\ngoos: darwin\ngoarch: amd64\npkg: github.com/johnhany97/grader/test\nBenchmarkRunTestInputOutputTestHandler-12              2        2855917014 ns/op           20500 B/op        197 allocs/op\nBenchmarkNewResultInputOutputTestHandler-12     20000000               430 ns/op              96 B/op          7 allocs/op\nBenchmarkHandleErrInputOutputTestHandler-12     200000000               45.9 ns/op             0 B/op          0 allocs/op\nBenchmarkRunTestJavaStyleTestHandler-12               10         664239069 ns/op           14096 B/op        117 allocs/op\nBenchmarkNewResultJavaStyleTestHandler-12       200000000               44.0 ns/op             0 B/op          0 allocs/op\nBenchmarkHandleErrJavaStyleTestHandler-12       200000000               46.9 ns/op             0 B/op          0 allocs/op\nBenchmarkRunTestJUnitTestHandler-12                    2        4260429266 ns/op           13772 B/op        105 allocs/op\nBenchmarkNewResultJUnitTestHandler-12           100000000               57.2 ns/op             0 B/op          0 allocs/op\nBenchmarkHandleErrJUnitTestHandler-12           200000000               37.6 ns/op             0 B/op          0 allocs/op\nBenchmarkRunTestOutputTestHandler-12                   2        2676039517 ns/op           20284 B/op        188 allocs/op\nBenchmarkNewResultOutputTestHandler-12          20000000               399 ns/op              92 B/op          6 allocs/op\nBenchmarkHandleErrOutputTestHandler-12          200000000               45.6 ns/op             0 B/op          0 allocs/op\nBenchmarkNewResultPyUnitTestHandler-12          100000000               56.4 ns/op             0 B/op          0 allocs/op\nBenchmarkHandleErrPyUnitTestHandler-12          200000000               36.7 ns/op             0 B/op          0 allocs/op\nPASS\nok      github.com/johnhany97/grader/test       141.407s\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjohnhany97%2Fgrader","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjohnhany97%2Fgrader","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjohnhany97%2Fgrader/lists"}