{"id":18989428,"url":"https://github.com/source-academy/grader","last_synced_at":"2025-09-20T02:06:07.628Z","repository":{"id":39302370,"uuid":"140687425","full_name":"source-academy/grader","owner":"source-academy","description":"Components for server-side auto-grading (Node.js, Elixir, AWS Lambda)","archived":false,"fork":false,"pushed_at":"2025-04-01T07:34:16.000Z","size":1628,"stargazers_count":2,"open_issues_count":5,"forks_count":8,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-04-16T22:09:13.996Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://sourceacademy.nus.edu.sg","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/source-academy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2018-07-12T08:52:34.000Z","updated_at":"2025-04-01T07:34:18.000Z","dependencies_parsed_at":"2024-04-15T10:54:21.978Z","dependency_job_id":null,"html_url":"https://github.com/source-academy/grader","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/source-academy%2Fgrader","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/source-academy%2Fgrader/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/source-academy%2Fgrader/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/source-academy%2Fgrader/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/source-academy","download_url":"https://codeload.github.com/source-academy/grader/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250228100,"owners_count":21395956,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-08T17:06:39.474Z","updated_at":"2025-09-20T02:06:02.527Z","avatar_url":"https://github.com/source-academy.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Source Academy Autograder Component\r\n\r\n[![Build_status](https://travis-ci.org/source-academy/grader.svg?branch=master)](https://travis-ci.org/source-academy/grader)\r\n[![Coverage](https://coveralls.io/repos/github/source-academy/grader/badge.svg?branch=master)](https://coveralls.io/github/source-academy/grader?branch=master)\r\n\r\nThe grader is a component of the [Cadet backend](https://github.com/source-academy/cadet). The grader,\r\n\r\n1. Receives a JSON format from the backend,\r\n2. For each test case program,\r\n\r\n    1. Concatenate all the program strings into a single combined program\r\n    2. Evaluates the single combined program in the [js-slang](https://github.com/source-academy/js-slang) interpreter\r\n    \r\n3. Returns a `Summary` JSON containing the results of the evaluation of the student code\r\n\r\n\r\n## Input JSON format\r\n\r\nThe input format consists of library, prepend, student, postpend, and testcases fields.\r\n\r\nExample input:\r\n```JSON\r\n{\r\n  \"library\": {\r\n    \"chapter\": 1,\r\n    \"external\": {\r\n      \"name\": \"NONE\",\r\n      \"symbols\": []\r\n    },\r\n    \"globals\": []\r\n  },\r\n  \"prependProgram\": \"// This line will be ignored\",\r\n  \"studentProgram\": \"const f = i =\u003e i === 0 ? 0 : i \u003c 3 ? 1 : f(i-1) + f(i-2);\",\r\n  \"postpendProgram\": \"// This line will also be ignored\",\r\n  \"testcases\": [\r\n    {\r\n      \"program\": \"f(1);\",\r\n      \"answer\": \"1\",\r\n      \"score\": 1\r\n    },\r\n    {\r\n      \"program\": \"f(3);\",\r\n      \"answer\": \"2\",\r\n      \"score\": 1\r\n    },\r\n    {\r\n      \"program\": \"f(5);\",\r\n      \"answer\": \"5\",\r\n      \"score\": 1\r\n    }\r\n  ]\r\n}\r\n```\r\n\r\nThe programs are programs written in [the source language](https://github.com/source-academy/js-slang). \r\n\r\nEach test case consists of the prepend, student, postpend and testcase program concatenated in that order.\r\n\r\nFor example, testcase 1 will look like:\r\n```javascript\r\n// This line will be ignored\r\nconst f = i =\u003e i === 0 ? 0 : i \u003c 3 ? 1 : f(i-1) + f(i-2);\r\n// This line will also be ignored\r\nf(1);\r\n```\r\n\r\n## Output `Summary` Format\r\nThe grader will produce a `Summary` for every input. The `Summary` will be an array of `Result`, with each testcase producing one `Result`. The format of the `Result` are as follows:\r\n### Example: Correct Answer\r\n\r\nThe student's code is correct.\r\n\r\nThe grader's corresponding `Summary` format will look like this:\r\n\r\n```json\r\n{   \r\n    \"totalScore\": 3,\r\n    \"results\": [\r\n        {\r\n            \"resultType\": \"pass\",\r\n            \"score\": 1\r\n        },\r\n        {\r\n            \"resultType\": \"pass\",\r\n            \"score\": 1\r\n        },\r\n        {\r\n            \"resultType\": \"pass\",\r\n            \"score\": 1\r\n        }\r\n    ]\r\n}\r\n```\r\n\r\nNote that the `resultType` pass represents any successful evaluation, i.e. no errors raised.\r\n\r\n### Example: Wrong Answer\r\n\r\nThe student's code is wrong.\r\n\r\nThe grader's corresponding `Summary` format will look like this:\r\n\r\n```json\r\n{\r\n    \"totalScore\": 0,\r\n    \"results\": [\r\n        {\r\n            \"resultType\": \"fail\",\r\n            \"expected\": \"1\",\r\n            \"actual\": \"2\"\r\n        },\r\n        {\r\n            \"resultType\": \"fail\",\r\n            \"expected\": \"2\",\r\n            \"actual\": \"3\"\r\n        },\r\n                {\r\n            \"resultType\": \"fail\",\r\n            \"expected\": \"5\",\r\n            \"actual\": \"8\"\r\n        }\r\n    ]\r\n}\r\n```\r\n\r\n### Example: Error in the code\r\nThe student's code has a syntax error.\r\n\r\nThe grader's corresponding `Summary` format will look like this:\r\n```JSON\r\n{\r\n  \"totalScore\": 0,\r\n  \"results\": [\r\n    {\r\n      \"resultType\": \"error\",\r\n      \"errors\": [\r\n        {\r\n          \"errorType\": \"syntax\",\r\n          \"line\": 1,\r\n          \"location\": \"student\",\r\n          \"errorLine\": \"const f = i =\u003e i === 0 ? 0 : i \u003c 3 ? 1 : f(i-1) + f(i-2)\",\r\n          \"errorExplanation\": \"Missing semicolon at the end of statement\"\r\n        }\r\n      ]\r\n    },\r\n    {\r\n      \"resultType\": \"error\",\r\n      \"errors\": [\r\n        {\r\n          \"errorType\": \"syntax\",\r\n          \"line\": 1,\r\n          \"location\": \"student\",\r\n          \"errorLine\": \"const f = i =\u003e i === 0 ? 0 : i \u003c 3 ? 1 : f(i-1) + f(i-2)\",\r\n          \"errorExplanation\": \"Missing semicolon at the end of statement\"\r\n        }\r\n      ]\r\n    },\r\n    {\r\n      \"resultType\": \"error\",\r\n      \"errors\": [\r\n        {\r\n          \"errorType\": \"syntax\",\r\n          \"line\": 1,\r\n          \"location\": \"student\",\r\n          \"errorLine\": \"const f = i =\u003e i === 0 ? 0 : i \u003c 3 ? 1 : f(i-1) + f(i-2)\",\r\n          \"errorExplanation\": \"Missing semicolon at the end of statement\"\r\n        }\r\n      ]\r\n    }\r\n  ]\r\n}\r\n```\r\n\r\nOther errors include timeout errors and runtime errors.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsource-academy%2Fgrader","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsource-academy%2Fgrader","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsource-academy%2Fgrader/lists"}