{"id":13879291,"url":"https://github.com/rails-engine/script_core","last_synced_at":"2025-04-10T03:56:07.254Z","repository":{"id":48151368,"uuid":"124680700","full_name":"rails-engine/script_core","owner":"rails-engine","description":"A script engine powered by mruby sandboxie, It's a fork of Shopify's ESS.","archived":false,"fork":false,"pushed_at":"2023-09-20T14:56:08.000Z","size":3538,"stargazers_count":70,"open_issues_count":0,"forks_count":8,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-04-10T03:56:02.902Z","etag":null,"topics":["mruby","rails","saas","sandbox"],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rails-engine.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"MIT-LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-03-10T17:35:33.000Z","updated_at":"2023-07-28T18:10:47.000Z","dependencies_parsed_at":"2024-11-09T18:27:16.896Z","dependency_job_id":"084f0f14-4efa-420f-82cf-40924e8d7313","html_url":"https://github.com/rails-engine/script_core","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rails-engine%2Fscript_core","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rails-engine%2Fscript_core/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rails-engine%2Fscript_core/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rails-engine%2Fscript_core/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rails-engine","download_url":"https://codeload.github.com/rails-engine/script_core/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248155001,"owners_count":21056542,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["mruby","rails","saas","sandbox"],"created_at":"2024-08-06T08:02:16.387Z","updated_at":"2025-04-10T03:56:07.235Z","avatar_url":"https://github.com/rails-engine.png","language":"Ruby","readme":"ScriptCore\n====\n\nScriptCore is a fork of [Shopify's enterprise script service](https://github.com/Shopify/ess).\n\nThe enterprise script service (aka ESS) is a thin Ruby API layer that spawns a process, the `enterprise_script_engine`, to execute an untrusted Ruby script.\n\nThe `enterprise_script_engine` executable ingests the input from `stdin` as a msgpack encoded payload; then spawns an mruby-engine; uses seccomp to sandbox itself; feeds `library`, `input` and finally the Ruby scripts into the engine; returns the output as a msgpack encoded payload to `stdout` and finally exits.\n\n## Why fork?\n\nI want to make these changes:\n\n- Use latest mruby\n- Toolchain\n    - [x] Expose mruby build config to allow developer modify mruby-engine executable, e.g: add some gems\n    - [x] Expose `mrbc` to allow developer precompile mruby library that would inject to sandbox\n    - [x] Rake tasks for compiling mruby-engine \u0026 mruby library\n    - [ ] Watching and auto compiling mruby library when it change\n    - [ ] Capistrano recipe\n- Practice\n    - [x] Rails generator for mruby library\n    - [x] Find a good place for engines\n    - [ ] Find a good way to working with timezone on mruby side\n    - [ ] Find a good way to working with `BigDecimal` \u0026 `Date` (mruby doesn't have these) on mruby side\n\n## limitation\n\n- We enable `MRB_DISABLE_STDIO` flag when compiling mruby,\n  which means the sandbox will not support gems which dependent `mruby-io` or `stdio.h`,\n  the result is you can not do any HTTP request, read and write files in the sandbox,\n  you may consider preparing data on Ruby side and pass them to the sandbox.\n\n## Help wanted\n\nI'm not familiar with C/CPP, so I can't improve ESS (in `ext/enterprise_script_service`),\n\nCurrently there're too much warnings on compiling, hope some one could help to resolve them.\n\n## Demo\n\nClone the repository.\n\n```sh\n$ git clone https://github.com/rails-engine/script_core\n```\n\nChange directory\n\n```sh\n$ cd script_core\n```\n\nFetch submodules\n\n```sh\n$ git submodule update --init --recursive\n```\n\nRun bundler\n\n```sh\n$ bundle install\n```\n\nPreparing database\n\n```sh\n$ bin/rails db:migrate\n```\n\nBuild mruby engine \u0026 engine lib\n\n```sh\n$ bin/rails app:script_core:engine:build\n$ bin/rails app:script_core:engine:compile_lib \n```\n\nStart the Rails server\n\n```sh\n$ bin/rails s\n```\n\nOpen your browser, and visit `http://localhost:3000`\n\n## Installation\n\nAdd this line to your Gemfile:\n\n```ruby\ngem 'script_core'\n```\n\nOr you may want to include the gem directly from GitHub:\n\n```ruby\ngem 'script_core', github: 'rails-engine/script_core'\n```\n\nThen execute:\n\n```sh\n$ bundle\n```\n\n## Build your executable\n\nScriptCore already has a default executable, because of mruby's gem is compiled in binary, or you may want to build a mruby library, build your own engine is necessary.\n\nYou can check `spec/dummy/mruby` as reference.\n\n### Create a new engine\n\nRun the task in your app directory:\n\n```sh\n$ rails script_core:engine:new [engine_name]\n```\n\n`engine_name` is optional, by default it would be `mruby` that will generate `mruby` directory in your app root folder.\n\nThen execute:\n\n```sh\n$ rails script_core:engine:build [engine_name]\n```\n\nIt will build mruby executables.\n\n#### customizing `gembox`\n\nRemove `.example` extension for `engine.gembox.example`, customize it, then rebuild the engine.\n\n**Warning: because of `seccomp`, you may meet compatibility problems, especially for IO relates gems.**\n\n### Build lib for the engine\n\nWrite your own lib for mruby environment in `mruby/lib` directory.\n\n### Compile lib for the engine\n\nRun the task in your app directory:\n\n```sh\n$ rails script_core:engine:compile_lib [engine_name]\n```\n\n### Ignoring engine binaries\n\nBecause of engine binaries are platform dependent, it's good to compile in every deployment.\n\nSimply add `mruby/bin` to `.gitignore`.\n\n### Integrate to your app\n\nYou can wrap it for example:\n\n```ruby\nmodule ScriptEngine\n  class \u003c\u003c self\n    def engine\n      @engine ||= ScriptCore::Engine.new Rails.root.join(\"mruby/bin\")\n    end\n\n    def eval(string, input: nil, instruction_quota_start: nil, environment_variables: {})\n      sources = [\n        [\"user\", string],\n      ]\n\n      engine.eval sources, input: input,\n                  instruction_quota_start: instruction_quota_start,\n                  environment_variables: environment_variables\n    end\n  end\nend\n```\n\nThen use it:\n\n```ruby\nScriptEngine.eval \"@output = 'hello world'\"\n```\n\n## Tips\n\n- Add `/mruby/bin` into `.gitignore`\n- Don't do any IO in mruby side\n- Because of `seccomp`, it may have compatible issues with some mruby gems\n- mruby doesn't have `Date`, use `Time` instead\n- mruby doesn't have `BigDecimal`, you can use Shopify's `Decimal` instead\n- mruby is poor support timezone, you'd better handle it by yourself\n- mruby engine is fast, usually it only costs 3 - 5ms depends on complexity, but it consume a lot of memory (~300k at least per process)\n\n# More information about ESS\n\n## Data format\n\n### Input\n\nThe input is expected to be a msgpack `MAP` with three keys (Symbol): `library`, `sources`, `input`:\n\n - `library`: a msgpack `BIN` set of MRuby instructions that will be fed directly to the `mruby-engine`\n - `input`: a msgpack formated payload for the `sources` to digest\n - `sources`: a msgpack `ARRAY` of `ARRAY` with two elements each (tuples): `path`, `source`; the actual code to be executed by the mruby-engine\n\n### Output\n\nThe output is msgpack encoded as well; it is streamed to the consuming end though. Streamed items can be of different types.\nEach element streamed is in the format of an `ARRAY` of two elements, where the first is a `Symbol` describing the element type:\n\n * `measurement`: a msgpack `ARRAY` of two elements: a `Symbol` describing the measurement, and an `INT64` with the value in µs.\n * `output`: a msgpack `MAP` with two entries (keys are symbols):\n ** `extracted` with whatever the script put in `@output`, msgpack encoded; and\n ** `stdout` with a `STRING` containing whatever the script printed to \"stdout\".\n * `stat`: a `MAP` keyed with symbols mapping to their `INT64` values\n\n## Errors\n\nWhen the ESS fails to serve a request, it communicates the error back to the caller by returning a non-zero status code.\nIt can also report data about the error, in certain cases, over the pipe. In does so in returning a tuple, as an `ARRAY` with the type being the symbol `error` and the payload being a `MAP`. The content of the map will vary, but it always will have a `__type` symbol key that defines the other keys.\n\n## Build\n\nRun `./bin/rake` to build the project. This effectively runs the `spec` target, which builds all libraries, the ESS and native tests; then runs all tests (native and Ruby).\n\nTo rebuild the entire project (which is useful when switching from one OS to another), use `./bin/rake mrproper`.\n\n## Using it\n\nThe sample script `bin/sandbox` reads Ruby input from a file or stdin, executes it, and displays the results.\n\nYou can invoke ESS from your own Ruby code as follows:\n\n```ruby\nresult = ScriptCore.run(\n  input: {result: [26803196617, 0.475]}, # \u003c1\u003e\n  sources: [\n    [\"stdout\", \"@stdout_buffer = 'hello'\"],\n    [\"foo\", \"@output = @input[:result]\"], # \u003c2\u003e\n  ],\n  instructions: nil, # \u003c3\u003e\n  timeout: 10.0, # \u003c4\u003e\n  instruction_quota: 100000, # \u003c5\u003e\n  instruction_quota_start: 1, # \u003c6\u003e\n  memory_quota: 8 \u003c\u003c 20  # \u003c7\u003e\n)\nexpect(result.success?).to be(true)\nexpect(result.output).to eq([26803196617, 0.475])\nexpect(result.stdout).to eq(\"hello\")\n```\n\n- \u003c1\u003e invokes the ESS, with a map as the `input` (available as `@input` in the sources)\n- \u003c2\u003e two \"scripts\" to be executed, one sets the `@stdout_buffer` to a value, the second returns the value associated with the key `:result` of the map passed in in \u003c1\u003e\n- \u003c3\u003e some raw instructions that will be fed directly into MRuby; defaults to nil\n- \u003c4\u003e a 10 second time quota to spawn, init, inject, eval and finally output the result back; defaults to 1 second\n- \u003c5\u003e a 100k instruction limit that that the engine will execute; defaults to 100k\n- \u003c6\u003e starts counting the instructions at index 1 of the `sources` array\n- \u003c7\u003e creates an 8 megabyte memory pool in which the script will run\n\n## Where are things?\n\n### C++ sources\n\nConsists of our code base, plus `seccomp` and `msgpack` libraries, as well as the `mruby` stuff. All in `ext/enterprise_script_service`\n\nNote: lib `seccomp` is omitted on Darwin.\n\n### Ruby layer\n\nRuby code is in `lib/`\n\n### Tests\n\n- GoogleTest tests are in `tests/`, which also includes the Google Test library.\n- RSpec tests are in `spec/`\n\n## Other useful things\n\n- There is a `CMakeLists.txt` that's mainly there for CLion support; we don't use cmake to build any of this.\n- You can use vagrant to bootstrap a VM to test under Linux while on Darwin; this is useful when testing `seccomp`.\n\n### Clone git submodules\n\n`git submodule update --init --recursive`\n\n### Vagrant\n\n```\n$ vagrant up\n$ vagrant ssh\nvagrant@vagrant-ubuntu-bionic-64:~$ cd /vagrant\nvagrant@vagrant-ubuntu-bionic-64:/vagrant$ bundle install\nvagrant@vagrant-ubuntu-bionic-64:/vagrant$ git submodule init\nvagrant@vagrant-ubuntu-bionic-64:/vagrant$ git submodule update\nvagrant@vagrant-ubuntu-bionic-64:/vagrant$ bin/rake\n```\n\n## Contributing\n\nBug report or pull request are welcome.\n\n### Make a pull request\n\n1. Fork it\n2. Create your feature branch (`git checkout -b my-new-feature`)\n3. Commit your changes (`git commit -am 'Add some feature'`)\n4. Push to the branch (`git push origin my-new-feature`)\n5. Create new Pull Request\n\nPlease write unit test with your code if necessary.\n\n## License\n\nThe gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).\n","funding_links":[],"categories":["Ruby"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frails-engine%2Fscript_core","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frails-engine%2Fscript_core","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frails-engine%2Fscript_core/lists"}