{"id":20037801,"url":"https://github.com/openedx/xqueue-watcher","last_synced_at":"2025-08-20T20:33:03.252Z","repository":{"id":16295785,"uuid":"19044441","full_name":"openedx/xqueue-watcher","owner":"openedx","description":null,"archived":false,"fork":false,"pushed_at":"2024-09-13T13:50:06.000Z","size":184,"stargazers_count":17,"open_issues_count":10,"forks_count":39,"subscribers_count":127,"default_branch":"master","last_synced_at":"2024-12-06T12:51:33.430Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/openedx.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.TXT","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS","dei":null,"publiccode":null,"codemeta":null}},"created_at":"2014-04-22T20:08:42.000Z","updated_at":"2024-09-13T13:50:08.000Z","dependencies_parsed_at":"2024-09-14T02:31:03.087Z","dependency_job_id":"91ac7100-ae99-4284-bd65-55516a646c29","html_url":"https://github.com/openedx/xqueue-watcher","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openedx%2Fxqueue-watcher","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openedx%2Fxqueue-watcher/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openedx%2Fxqueue-watcher/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openedx%2Fxqueue-watcher/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/openedx","download_url":"https://codeload.github.com/openedx/xqueue-watcher/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":230454430,"owners_count":18228392,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-13T10:22:30.296Z","updated_at":"2024-12-19T15:09:55.001Z","avatar_url":"https://github.com/openedx.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"⛔️ WARNING\n==========\n\nThis repository is under-maintained. We are not fixing bugs or developing new features for it. We hope to deprecate and replace it soon. For updates, [follow along on the DEPR ticket](https://github.com/openedx/public-engineering/issues/22)\n\nAlthough we have stopped integrating new contributions, we always appreciate security disclosures and patches sent to security@openedx.org\n\nxqueue_watcher\n==========\n\nThis is an implementation of a polling [XQueue](https://github.com/openedx/xqueue) client and grader.\n\nOverview\n========\n\nThere are several components in a working XQueue Watcher service:\n- **XQueue Watcher**: it polls an xqueue service continually for new submissions and grades them.\n- **Submissions Handler**: when the watcher finds any new submission, it will be passed to the handler for grading. It is a generic handler that can be configured to work with different submissions through individual submission graders.\n- **Individual Submission Grader**: each exercise or homework may specify its own \"grader\". This should map to a file on the server that usually specifies test cases or additional processing for the student submission.\n\nUsually your server will look like this:\n```\nroot/\n├── xqueue-watcher/\n│   ├── ... # xqueue-watcher repo, unchanged\n│   └── ...\n├── config/\n│   └── conf.d/\n│   │   └── my-course.json\n│   └── logging.json\n└── my-course/\n   ├── exercise1/\n   │   ├── grader.py  # - per-exercise grader\n   │   └── answer.py  # - if using JailedGrader\n   ├── ...\n   └── exercise2/\n       ├── grader.py\n       └── answer.py\n```\nRunning XQueue Watcher:\n======================\n\nUsually you can run XQueue Watcher without making any changes. You should keep course-specific files in another folder like shown above, so that you can update xqueue_watcher anytime.\n\nInstall the requirements before running `xqueue_watcher`\n```bash\ncd xqueue-watcher/\nmake requirements\n```\n\nNow you're ready to run it.\n```bash\npython -m xqueue_watcher -d [path to the config directory, eg ../config]\n```\n\nThe course configuration JSON file in `conf.d` should have the following structure:\n```json\n    {\n        \"test-123\": {\n            \"SERVER\": \"http://127.0.0.1:18040\",\n            \"CONNECTIONS\": 1,\n            \"AUTH\": [\"uname\", \"pwd\"],\n            \"HANDLERS\": [\n                {\n                    \"HANDLER\": \"xqueue_watcher.grader.Grader\",\n                    \"KWARGS\": {\n                        \"grader_root\": \"/path/to/course/graders/\",\n                    }\n                }\n            ]\n        }\n    }\n```\n\n* `test-123`: the name of the queue\n* `SERVER`: XQueue server address\n* `AUTH`: List containing [username, password] of XQueue Django user\n* `CONNECTIONS`: how many threads to spawn to watch the queue\n* `HANDLERS`: list of callables that will be called for each queue submission\n   * `HANDLER`: callable name, see below for Submissions Handler\n   * `KWARGS`: optional keyword arguments to apply during instantiation\n      * `grader_root`: path to the course directory, eg /path/to/my-course\n\n\u003e TODO: document logging.json\n\nSubmissions Handler\n===================\n\nWhen `xqueue_watcher` detects any new submission, it will be passed to the submission handler for grading. It will instantiate a new handler based on the name configured above, with submission information retrieved\nfrom XQueue. Base graders are defined in `xqueue_watcher`: Grader and JailedGrader (for Python, using CodeJail). If you don't use JailedGrader, you'd have to implement your own Grader by subclassing `xqueue_watcher.grader.Grader`\n\nThe payload received from XQueue will be a JSON object that usually looks like the JSON below. Note that \"grader\" is a required field in the \"grader_payload\" and must be configured accordingly in Studio.\n\n```json\n{\n    \"student_info\": {\n        \"random_seed\": 1,\n        \"submission_time\": \"20210109222647\",\n        \"anonymous_student_id\": \"6d07814a4ece5cdda54af1558a6dfec0\"\n    },\n    \"grader_payload\": \"\\n        {\\\"grader\\\": \\\"relative/path/to/grader.py\\\"}\\n      \",\n    \"student_response\": \"print \\\"hello\\\"\\r\\n      \"\n}\n```\n\n## Custom Handler\nTo implement a pull grader:\n\nSubclass `xqueue_watcher.grader.Grader` and override the `grade` method. Then add your grader to the config like `\"handler\": \"my_module.MyGrader\"`. The arguments for the `grade` method are:\n   * `grader_path`: absolute path to the grader defined for the current problem.\n   * `grader_config`: other configuration particular to the problem\n   * `student_response`: student-supplied code\n\nNote that `grader_path` is constructed by appending the relative path to the grader from `grader_payload` to the `grader_root` in the configuration JSON. If the handler cannot find a `grader.py` file, it would fail to grade the submission.\n\n## Grading Python submissions with JailedGrader\n\n`xqueue_watcher` provides a few utilities for grading python submissions, including JailedGrader for running python code in a safe environment and grading support utilities.\n\n### JailedGrader\nTo sandbox python, use [CodeJail](https://github.com/openedx/codejail). In your handler configuration, add:\n```json\n    \"HANDLER\": \"xqueue_watcher.jailedgrader.JailedGrader\",\n    \"CODEJAIL\": {\n        \"name\": \"python\",\n        \"python_bin\": \"/path/to/sandbox/python\",\n        \"user\": \"sandbox_username\"\n    }\n```\nThen, `codejail_python` will automatically be added to the kwargs for your handler. You can then import codejail.jail_code and run `jail_code(\"python\", code...)`. You can define multiple sandboxes and use them as in `jail_code(\"another-python-version\", ...)`\n\nTo use JailedGrader, you also need to provide an `answer.py` file on the same folder with the `grader.py` file. `answer.py` contains the correct/reference implementation of the solution to the problem. The grader will run both student submission and `answer.py` and compare the output with each other.\n\n### Grading Support utilities\nThere are several grading support utilities that make writing `grader.py` for python code easy. Check out\n`grader_support/gradelib.py` for the documentation.\n\n- `grader_support.gradelib.Grader`: a base class for creating a new submission grader. Not to be confused with `xqueue-watcher.grader.Grader`. You can add input checks, preprocessors and tests to a Grader object.\n- `grader_support.gradelib.Test`: a base class for creating tests for a submission. Usually a submission can be graded with one or a few tests. There are also few useful test functions and classes included, like `InvokeStudentFunctionTest` , `exec_wrapped_code`, etc.\n- Preprocessors: utilities to process the raw submission before grading it. `wrap_in_string` is useful for testing code that is not wrapped in a function.\n- Input checks: sanity checks before running a submission, eg check `required_string` or `prohibited_string`\n\nUsing the provided grader class, your `grader.py` would look something like this:\n```python\nfrom grader_support import gradelib\ngrader = gradelib.Grader()\n\n# invoke student function foo with parameter []\ngrader.add_test(gradelib.InvokeStudentFunctionTest('foo', []))\n```\n\nOr with a pre-processor:\n```python\nimport gradelib\n\ngrader = gradelib.Grader()\n\n# execute a raw student code \u0026 capture stdout\ngrader.add_preprocessor(gradelib.wrap_in_string)\ngrader.add_test(gradelib.ExecWrappedStudentCodeTest({}, \"basic test\"))\n```\n\nYou can also write your own test class, processor and input checks.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenedx%2Fxqueue-watcher","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopenedx%2Fxqueue-watcher","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenedx%2Fxqueue-watcher/lists"}