{"id":25177498,"url":"https://github.com/csivitu/code-executor","last_synced_at":"2025-05-07T01:50:04.697Z","repository":{"id":43120153,"uuid":"292100232","full_name":"csivitu/code-executor","owner":"csivitu","description":"A CLI/library to execute code against test cases in various languages and obtain relevant results. :rocket:","archived":false,"fork":false,"pushed_at":"2022-03-17T09:33:30.000Z","size":201,"stargazers_count":16,"open_issues_count":8,"forks_count":2,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-01-16T14:42:06.342Z","etag":null,"topics":["code-execution","code-executor","code-runner","hacktoberfest","npm-package","online-judge","typescript"],"latest_commit_sha":null,"homepage":"https://www.npmjs.com/package/code-executor","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/csivitu.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-09-01T20:25:56.000Z","updated_at":"2024-08-21T04:42:19.000Z","dependencies_parsed_at":"2022-09-17T12:01:24.112Z","dependency_job_id":null,"html_url":"https://github.com/csivitu/code-executor","commit_stats":null,"previous_names":[],"tags_count":12,"template":false,"template_full_name":"csivitu/Template","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csivitu%2Fcode-executor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csivitu%2Fcode-executor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csivitu%2Fcode-executor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csivitu%2Fcode-executor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/csivitu","download_url":"https://codeload.github.com/csivitu/code-executor/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":237962965,"owners_count":19394127,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["code-execution","code-executor","code-runner","hacktoberfest","npm-package","online-judge","typescript"],"created_at":"2025-02-09T14:25:32.822Z","updated_at":"2025-05-07T01:50:04.668Z","avatar_url":"https://github.com/csivitu.png","language":"TypeScript","readme":"[![csivit][csivitu-shield]][csivitu-url]\n\u003c!-- ALL-CONTRIBUTORS-BADGE:START - Do not remove or modify this section --\u003e\n[![All Contributors](https://img.shields.io/badge/all_contributors-3-orange.svg?style=flat-square)](#contributors-)\n\u003c!-- ALL-CONTRIBUTORS-BADGE:END --\u003e\n[![Issues][issues-shield]][issues-url]\n\n\u003c!-- PROJECT LOGO --\u003e\n\u003cbr /\u003e\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://github.com/csivitu/code-executor\"\u003e\n    \u003cimg src=\"https://raw.githubusercontent.com/csivitu/CSIWebsite2.0/master/images/favicon.png\" alt=\"Logo\" width=\"80\"\u003e\n  \u003c/a\u003e\n\n  \u003ch3 align=\"center\"\u003ecode-executor\u003c/h3\u003e\n\n  \u003cp align=\"center\"\u003e\n    A library to execute code against test cases in various languages and obtain relevant results.\n    \u003cbr /\u003e\n    \u003ca href=\"https://github.com/csivitu/code-executor\"\u003e\u003cstrong\u003eExplore the docs »\u003c/strong\u003e\u003c/a\u003e\n    \u003cbr /\u003e\n    \u003cbr /\u003e\n    \u003ca href=\"https://github.com/csivitu/code-executor\"\u003eView Demo\u003c/a\u003e\n    ·\n    \u003ca href=\"https://github.com/csivitu/code-executor/issues\"\u003eReport Bug\u003c/a\u003e\n    ·\n    \u003ca href=\"https://github.com/csivitu/code-executor/issues\"\u003eRequest Feature\u003c/a\u003e\n  \u003c/p\u003e\n\u003c/p\u003e\n\n\n\n\u003c!-- TABLE OF CONTENTS --\u003e\n## Table of Contents\n\n* [About the Project](#about-the-project)\n  * [Built With](#built-with)\n* [Getting Started](#getting-started)\n  * [Prerequisites](#prerequisites)\n  * [Installation](#installation)\n* [Usage](#usage)\n* [Roadmap](#roadmap)\n* [Contributing](#contributing)\n* [License](#license)\n* [Contributors](#contributors-)\n\n\n\n\u003c!-- ABOUT THE PROJECT --\u003e\n## About The Project\n\n\u003c!-- [![Product Name Screen Shot][product-screenshot]](https://example.com) --\u003e\n\n**code-executor** is a Node.js library built for the purpose of executing code in an isolated container against user defined test cases.\n\u003cbr /\u003e\n\n**code-executor** allows you to run arbitrary code in scalable, secure containers and returns metrics for each test case, such as the time taken, errors occured (if any), and the status (pass/fail).\n\u003cbr /\u003e\n\nThis library uses a master-worker structure to run programs, which makes it scalable across servers as long as they use the same `Redis` instance. Visit the [usage](#usage) section to learn more.\n\n\n### Built With\n\n* [Typescript](https://www.typescriptlang.org/)\n* [Node](https://nodejs.org/en/)\n* [Bull](https://optimalbits.github.io/bull/)\n* [Redis](https://redis.io/)\n\n\n\n\u003c!-- GETTING STARTED --\u003e\n## Getting Started\n\nTo get a local copy up and running follow these simple steps.\n\n### Prerequisites\n\n- [npm](https://www.npmjs.com/)\n- [redis](https://redis.io/)\n- [docker](https://www.docker.com/)\n\n### Installation\n \nYou can install **code-executor** using `npm`.\n\n```sh\nnpm install code-executor --save\n```\n\n\n\u003c!-- USAGE EXAMPLES --\u003e\n## Usage\n\n`code-executor` exports a default `CodeExecutor` class and a `Worker` class. The following documentation describes these classes in brief along with examples of how to use them in your project. For the purpose of documentation, we consider a simple Competitive Coding website backend which uses `code-executor` to run programs submitted by the users.\n\n\n### TL;DR\n\n1. In your backend, you can create a `CodeExecutor` object, which has a `runCode` method. This returns a promise which resolves when the program passed to it has finished executing. You can check out [this example](https://github.com/csivitu/code-executor/blob/master/examples/master.ts) to find out how to use a `CodeExecutor` object.\n\n2. Now, you can create a worker using the `code-executor` CLI. For that, you need to first install `code-executor` globally.\n\n```bash\nnpm i -g code-executor\n```\n\n3. You can spawn a worker using the `spawn-worker` command (`sw` in short), and optionally pass the `redis` URL (default: `redis://127.0.0.1:6379`), the name of the queue (default: `myExecutor`), and languages to build (default: all).\n\n```bash\ncode-executor sw\n```\n\n4. If you used some other redis URL or queue name in step 1, make sure you pass those to the CLI.\n\n```bash\ncode-executor sw --redis \u003credis-url\u003e --queue \u003cqueue-name\u003e\n```\n\n### Brief Description\n\n- As mentioned before, `code-executor` is built using the master-worker strategy. Therefore, there is a `master` which is responsible for assigning work to a set of `workers`. These workers respond to the master on completing their tasks. In the case of the Competitive Coding website, the backend of the website is the `master`, whereas you can run separate `Node.js` scripts for spawning workers (a CLI is coming soon!).\n\n- The backend assigns jobs to the workers through a `queue`, which is stored in the `Redis` instance and managed by the `bull` library. This abstracts process synchronization and ensures that no two workers are working on the same job. Once the workers finish executing the code, they respond to the master, and the backend can respond with success or failure, and other details returned by the worker.\n\n- `code-executor` is built in a way that it allows multiple masters and workers to run parallely while internally handling synchronization. This allows you to have `worker`s on different instances of a cluster as long as they use the same `Redis` instance. You can also scale your backend to have multiple `master`s.\n\n\n### CodeExecutor\n\nThe default export from the `code-executor` library is the `CodeExecutor` class. This is the `master` class, which can be run on the backend of your website. The purpose of this class is to assign jobs to the workers (through a `queue` as mentioned before, though this is abstracted so you need not worry about it). You can create an object of `CodeExecutor` and keep adding jobs to it as you keep getting submissions on your website. \n\u003cbr /\u003e\n\nYou can create a `CodeExecutor` object in the following manner. You must pass the name of your queue and the redis instance you want the queue to be placed on. These details will later be used by the `worker` to identify jobs and run them.\n\n\u003e Note: `job` refers to the task of execution of a single program which is passed using a queue to the workers.\n\n\n```js\nimport { CodeExecutor } from 'code-executor';\nconst codeExecutor = new CodeExecutor('myExecutor', 'redis://127.0.0.1:6379');\n```\n\n**OR**\n\n```js\nconst { CodeExecutor } = require('code-executor');\nconst codeExecutor = new CodeExecutor('myExecutor', 'redis://127.0.0.1:6379');\n```\n\nNow, say you received an a submission from a user, and you want to run their code against a set of test cases. You can use the following code inside your route handler.\n\n```js\nasync function routeHandler(code, language) {\n  const input = {\n    language: language,\n    code: code,\n    testCases: [\n      {\n        input: '',\n        output: 'hello\\n',\n      },\n    ],\n    timeout: 2,\n  };\n\n  // We re-use the codeExecutor object that was created before.\n  const results = await codeExecutor.runCode(input);\n\n  console.log(results);\n  return results;\n}\n```\n\nThat is all! `codeExecutor.runCode()` returns a promise which resolves when your code has been executed successfully by any of the `worker`s. You can also stop a `master` from interacting with a queue using `codeExecutor.stop()`.\n\n```js\ncodeExecutor.stop();\n```\n\n\n### Worker\n\nBy now, we know how to use the `CodeExecutor` class to assign jobs to workers. Now, we see how to use the `Worker` class to create workers that will run your code.\n\nThe easiest way to spawn workers is by using the CLI provided by the library. For this, you need to globally install `code-executor` using:\n\n```bash\nnpm install -g code-executor\n```\n\nOnce you install it globally, you can run `code-executor -h` to see the options available to you.\n\n```bash\n$ code-executor -h               \nUsage: code-executor [options] [command]\n\nOptions:\n  -r, --redis \u003credis\u003e     URL for the redis instance\n  -q, --queue \u003cqueue\u003e     name of the redis queue\n  -l, --langs \u003clangs...\u003e  list of languages to build\n  -h, --help              display help for command\n\nCommands:\n  spawn-worker|sw         spawn worker process\n  help [command]          display help for command\n```\n\nTo spawn a worker, run\n\n```bash\ncode-executor sw\n```\n\nHere's another small example to spawn a worker supporting Python and Bash.\n\n```bash\ncode-executor sw -l Python Bash\n```\n\nTo spawn multiple workers, you can run the previous command several times.\n\n\u003e Note: `code-executor` does not ensure that a worker will not take up a job if it doesn't support that language. Therefore, each worker you start should support all the languages you need to build. For example, if you need Python, Javascript and Bash, all the workers should be started with these languages. Since the workers build Docker containers, after 1 worker builds all its containers, the remaining workers will use the Docker cache and thus will be able to build much faster.\n\nYou can also use the API to build you own script to spawn workers. You can use the CLI in almost every use case, however, the API provides a lot more customization, as you can see below.\n\n\n```js\nimport { Worker } from 'code-executor';\nconst worker = new Worker('myExecutor', 'redis://127.0.0.1:6379');\n```\n\n**OR**\n\n```js\nconst { Worker } = require('code-executor');\nconst worker = new Worker('myExecutor', 'redis://127.0.0.1:6379');\n```\n\nYou can create a new `Worker` object and listen with the same `name` and `redis` string you passed to the master class. There is another optional parameter called `options`, which is an object that may consist of the following parameters:\n\n- `folderPath`, _string_: Will be discussed later.\n- `memory`, _number_: The amount of memory assigned to every Docker container spawned by this worker, in MB. The default is 0 (no limit).\n- `CPUs`, _number_: The number of CPUs assigned to every Docker container spawned by this worker. The default is 0.5.\n\nFor example, you could pass these values to the constructor.\n\n```js\nconst worker = new Worker('myExecutor', 'redis://127.0.0.1:6379', {\n  folderPath: '/tmp/myFolder',\n  memory: 100,\n  CPUs: 1,\n});\n```\n\nAn object of the `Worker` class has the following important functions:\n\n- `build(langs)`\n- `start()`\n- `pause()`\n- `resume()`\n\n\n### worker.build(langs)\n\n`worker.build()` is responsible for building docker images on the system. You can pass a list of languages to the function so that it builds images for just the specified languages.\n\n- You can also call `worker.build()` without an argument to build all languages supported by `code-executor`.\n\n```js\nasync function build() {\n  await worker.build(['Python', 'Bash']);\n  console.log('Python and Bash containers built successfully!');\n}\n```\n\n### worker.start()\n\nOn running `worker.start()`, the current worker starts listening on the `redis` queue. After this function is executed, whenever there is a new job on the queue that has not been taken by another worker (if any), this worker will take up the job and run the code.\n\n```js\nworker.start();\n```\n\n### worker.pause() and worker.resume()\n\nYou can pause the execution of a worker with the help of the `worker.pause()` function. Executing the `worker.resume()` function resumes processing jobs from the queue.\n\n```js\nworker.pause();\n\nworker.resume();\n```\n\nThe worker performs the following steps in order to execute a program:\n\n- First, the worker builds all the images on the server. If the image is already present, it uses the cache.\n- The worker listens on the queue for new jobs.\n- Whenever it gets a new job, it calls a `Runner` object to run the code.\n- The `Runner` object creates a folder with a random name in `/tmp/code-exec`. This can be changed with the help of the optional `folderPath` parameter passed to the constructor of the `Worker` class.\n- This newly-created folder is mounted inside a docker container to execute the code.\n- When execution is completed, or the time limit has exceeded, the `worker` responds to the master.\n\n\n\u003c!-- ROADMAP --\u003e\n## Roadmap\n\nSee the [open issues](https://github.com/csivitu/code-executor/issues) for a list of proposed features (and known issues).\n\n\n\n\u003c!-- CONTRIBUTING --\u003e\n## Contributing\n\nContributions are what make the open source community such an amazing place to be learn, inspire, and create. Any contributions you make are **greatly appreciated**.\n\n1. Fork the Project\n2. Create your Feature Branch (`git checkout -b feature/AmazingFeature`)\n3. Commit your Changes (`git commit -m 'feat: Add some AmazingFeature'`)\n4. Push to the Branch (`git push origin feature/AmazingFeature`)\n5. Open a Pull Request\n\nYou are requested to follow the contribution guidelines specified in [CONTRIBUTING.md](./CONTRIBUTING.md) while contributing to the project :smile:.\n\n\u003c!-- LICENSE --\u003e\n## License\n\nDistributed under the MIT License. See [`LICENSE`](./LICENSE) for more information.\n\n\n\n\n\u003c!-- MARKDOWN LINKS \u0026 IMAGES --\u003e\n\u003c!-- https://www.markdownguide.org/basic-syntax/#reference-style-links --\u003e\n[csivitu-shield]: https://img.shields.io/badge/csivitu-csivitu-blue\n[csivitu-url]: https://csivit.com\n[issues-shield]: https://img.shields.io/github/issues/csivitu/code-executor?style=flat-square\n[issues-url]: https://github.com/csivitu/code-executor/issues\n\n## Contributors ✨\n\nThanks goes to these wonderful people ([emoji key](https://allcontributors.org/docs/en/emoji-key)):\n\n\u003c!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section --\u003e\n\u003c!-- prettier-ignore-start --\u003e\n\u003c!-- markdownlint-disable --\u003e\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\"\u003e\u003ca href=\"https://github.com/roerohan\"\u003e\u003cimg src=\"https://avatars0.githubusercontent.com/u/42958812?v=4\" width=\"100px;\" alt=\"\"/\u003e\u003cbr /\u003e\u003csub\u003e\u003cb\u003eRohan Mukherjee\u003c/b\u003e\u003c/sub\u003e\u003c/a\u003e\u003cbr /\u003e\u003ca href=\"https://github.com/csivitu/code-executor/commits?author=roerohan\" title=\"Code\"\u003e💻\u003c/a\u003e \u003ca href=\"https://github.com/csivitu/code-executor/commits?author=roerohan\" title=\"Documentation\"\u003e📖\u003c/a\u003e\u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\u003ca href=\"https://github.com/ashikka\"\u003e\u003cimg src=\"https://avatars1.githubusercontent.com/u/58368421?v=4\" width=\"100px;\" alt=\"\"/\u003e\u003cbr /\u003e\u003csub\u003e\u003cb\u003eashikka\u003c/b\u003e\u003c/sub\u003e\u003c/a\u003e\u003cbr /\u003e\u003ca href=\"https://github.com/csivitu/code-executor/commits?author=ashikka\" title=\"Code\"\u003e💻\u003c/a\u003e \u003ca href=\"https://github.com/csivitu/code-executor/commits?author=ashikka\" title=\"Documentation\"\u003e📖\u003c/a\u003e\u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\u003ca href=\"https://alias-rahil.github.io/\"\u003e\u003cimg src=\"https://avatars2.githubusercontent.com/u/59060219?v=4\" width=\"100px;\" alt=\"\"/\u003e\u003cbr /\u003e\u003csub\u003e\u003cb\u003eRahil Kabani\u003c/b\u003e\u003c/sub\u003e\u003c/a\u003e\u003cbr /\u003e\u003ca href=\"https://github.com/csivitu/code-executor/commits?author=alias-rahil\" title=\"Code\"\u003e💻\u003c/a\u003e \u003ca href=\"https://github.com/csivitu/code-executor/commits?author=alias-rahil\" title=\"Documentation\"\u003e📖\u003c/a\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n\u003c!-- markdownlint-enable --\u003e\n\u003c!-- prettier-ignore-end --\u003e\n\u003c!-- ALL-CONTRIBUTORS-LIST:END --\u003e\n\nThis project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind welcome!\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcsivitu%2Fcode-executor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcsivitu%2Fcode-executor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcsivitu%2Fcode-executor/lists"}