{"id":15984667,"url":"https://github.com/aappddeevv/nodescala","last_synced_at":"2025-10-11T22:37:17.501Z","repository":{"id":97609877,"uuid":"270102807","full_name":"aappddeevv/nodescala","owner":"aappddeevv","description":"Nodejs and dotty/scalajs via graaljs based on Mike Hearn's work in kotlin.","archived":false,"fork":false,"pushed_at":"2020-06-21T13:41:51.000Z","size":39,"stargazers_count":6,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-02T11:36:26.243Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/aappddeevv.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-06-06T20:59:40.000Z","updated_at":"2023-09-20T18:55:43.000Z","dependencies_parsed_at":"2023-06-18T19:51:05.782Z","dependency_job_id":null,"html_url":"https://github.com/aappddeevv/nodescala","commit_stats":{"total_commits":7,"total_committers":1,"mean_commits":7.0,"dds":0.0,"last_synced_commit":"a635cf14f03fae6f9476b3edee4ff9e8a7c0b83b"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/aappddeevv/nodescala","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aappddeevv%2Fnodescala","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aappddeevv%2Fnodescala/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aappddeevv%2Fnodescala/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aappddeevv%2Fnodescala/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/aappddeevv","download_url":"https://codeload.github.com/aappddeevv/nodescala/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aappddeevv%2Fnodescala/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279009072,"owners_count":26084549,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-11T02:00:06.511Z","response_time":55,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-08T02:10:02.212Z","updated_at":"2025-10-11T22:37:17.493Z","avatar_url":"https://github.com/aappddeevv.png","language":"Scala","funding_links":[],"categories":[],"sub_categories":[],"readme":"# nodescala\n\nNotes on blending scala and javascript together via node and graaljs.\n\nThis repo is an concentration of blogs and another repo that does the same thing in kotlin. \n\nAll credit for clever code should go to Mike Hearn and others for figuring this out.\n\nYou can always call scala code directly from node quite easily and create\nand process js code from the jvm easily. If you want nodejs infrastructure, such as\nnode modules and other nodejs functions, you must use graaljs as the entry point.\nIf your scala jvm code calls into nodejs, it must synchronize on the nodejs\nthread. This repo describes some ways to handle nodejs interop using graaljs.\nThe online docs and unit tests for running js code from the jvm and vice-versa\nare good enough. It's only with the presence of nodejs that complicates interop.\n\nToplevel nodejs/graaljs docs are at https://github.com/graalvm/graaljs/blob/master/docs/user.\nSee the interop and multithread documents in particular.\n\n\n# Running Examples\n\nThe example demonstrate calling nodejs code from scala, after starting a scala\nprocess via nodejs. You can call out to scala quite easily and this scenario\nis described in another section.\n\nThe graal installation should be first on your path so that the graal node (aka graaljs) is \npicked up when you run node.\n\nFirst install the node files:\n\n* `npm i`\n\nThen run the examples via the mill build tool or a shell:\n\n* mill [-w] example1.runJS\n* node --experimental-worker --jvm --vm.cp $CP interop/resources/nodejs/boot.js -- example1.run\n\nThe CP env var can be created by running `mill example1.dumpPath` and exporting CP to the value it produces.\n\nIf you are changing the code and want to run the interop automatically, use the `-w` mill option\nto watch for source code changes.\n\nThe boot script expects `--` at the end of the node arguments and the start of\nthe jvm program arguments. The first argument must be the name of the main class or \n`-jar jarfile` with the `Main-Class` attribute set. \n\nWhen using\ndotty and the `@main` attribute, the main class is usually the name of the function\nprepended with the enclosing package. `@main` creates a sythesized class with \na proper \"main\" function. mill automatically adds a `Main-Class` attribute if you run `mill example1.jar`.\n\nTo run the examples, install mill (I did not include the mill self-installing script):\n\n* mill example1.runJS: Simple eval.\n* mill example2.runJS: Eval but with javascript promises.\n* mill scalajs.fastOpt: Create a scala.js module.\n* mill example3.runJS: Load the scalajs module and return primitives and promises.\n\nRunning the example through mill uses a \"boot\" script that allows the scala code to\ntake over running the application. When the scala program exits, the entire\napplication instances exits.  Hence the concept of \"boot.\"\n\nThere are other ways to run scala code while still entering in through nodejs.\nFor example, you may be running a HTTP server on the main node thread but \nalso  want to run a scala program as if its \"main\" had been called.\n\n# Running Examples Again!\n\nInstall dependencies via `npm i`.\n\nYou can run a JVM program that calls back into the nodejs thread using the \n`appstart.js` script. This script exports a function that can start\na new JVM thread that runs your \"main\" static function in a class.\n\nStart node aftering building a fat jar (fat jar makes this easy otherwise\nyou need a full classpath):\n\n```sh\n# get scala, interop and dependencies together in 1 jar\nmill example1.assembly\nnode --jvm --experimental-worker --vm.cp=out/example1/assembly/dest/out.jar\n```\n\nThen run each \"main\" class. You can start multiple `main`s but\nyou will need to create the right classpath or assembly/jars\nwith the programs in them. In this case, we just run this on\nexample 1, 2 and 3. Don't forget to run `mill scalajs.fastOpt` \nbefore running example3.\n\n```javascript\nconst start = require(\"./interop/resources/nodejs/appstarter\")\nstart(\"example1.run\")\n```\n\n# Running Examples Again Again!\n\nYou can call scala code methods directly from nodejs, you don't have to\nonly call `main` methods. But if you want to run multi-threaded code\nyou need to remeber to call `appstarter` to start the infrastructure\nregardless of whether you use the `start` function to run `main` functions\nthat call into nodejs.\n\nYour scala code can run on the nodejs thread, but that's not recommended\nso it is recommended that you immediately start a jvm thread and \ncall back into the nodejs thread if needed.\n\nIf you need merely call a scala function from javascript, you can do that\nas well and return a js promise. You can create a js promise directly,\nin scala code, on the nodejs thread. Inside the js promise, you can\nuse jvm threads as needed.\n\nYou can also return a jvm object that looks like a promise. You can\ncreate and manage the jvm object on any thread, so you should create\nthe promise-looking object as the return value from the js nodejs function\ncall and have the jvm code run on another thread. By making a jvm\nobject \"look\" like a promise, the js code can `await` or `then` on it.\nIn fact, similar to scala.js, as long as that object has a `then`\nmethod with the proper two-arg-function signature, your jvm\nobject is a js promise.\n\nExample 4 has a single function in open code. Build and load the\nassembly as above, then run the code on the nodejs thread.\n\n```javascript\n// don't start the interop infrastructure\n\u003e Packages.example4.example4$package.process(\"hah\", \"blah\")\nargs: hah, blah\n42\n```\nNode the scala byte code manipulation requires us to find the \nfunction using mangled JVM names.\n\nNone of this code required calling back into the nodejs\nthread from a separate jvm thread, so we could call this\nfunction directly without starting the infrastructure. Also,\nthe values were simple primitives so they could be translated\nby the graal engine directly without going through the `Value`\nclass.\n\nThe individual functions in example4 show some\nasync examples that you would encounter when using\nthe jvm for fulfilling async requests initiated in nodejs.\n\nExample 4 source code also has translations of the interop tests from\nhttps://github.com/graalvm/graaljs/blob/master/graal-js/src/com.oracle.truffle.js.test/src/com/oracle/truffle/js/test/interop/AsyncInteropTest.java\n\n# Interop Concept\n\n* Nodejs is single threaded. You should only access it when on that thread.\n* To use nodejs with jvm you *must* use graaljs (node on the jvm). You cannot just use a js context from java.\n  * Only the graaljs entry point has the nodejs infrastructure builtin.\n* Start from graaljs. \n* Start a js worker. These are like threads in js land.\n  * The js worker watches a queue of requests coming from the jvm side.\n* Call into the JVM with the scala interop worker thread. That thread is the \"main\" thread for your scala program.\n* When you need to call back into node/javascript either ensure you are on that thread using the nodejs\nexecutor. To eval js code use a queue to push messages back and forth to the js worker to run \"eval\" and return the result.\n\nSee the resources above to understand the details.\n\nIs this hard? \n\nNo, the key is to not block the event loop in nodejs on the java side. If you want to run cool multi-threaded computations on the jvm, you still need a way to coordinate between threads and this is the way to coordinate with the nodejs thread.\n\nSome issues that need to be resolved:\n\n* A way to catch unhandled promise failures. nodejs only posts these to `process.on(\"unhandledReject\", ...)`\nevents. We would need a global handler that calls back into the jvm side so that when js code evals fail, we\ncan handle those.\n* Some syntax and sugar and javascript promises and scala futures or any other scala effects. The async test code\nin graaljs has some good starting points.\n  * See: https://thecodebarbarian.com/unhandled-promise-rejections-in-node.js.html\n* Performance tests.\n\n# mill\n\nI'm still learning mill but the following example build.sc files were helpful as examples:\n\n* https://gitlab.com/hmf/srllab/-/blob/master/build.sc\n\n# Global objects in JS namespace\n\nSee the graaljs documentation for their APIs: https://github.com/graalvm/graaljs/blob/master/docs/user/JavaScriptCompatibility.md\n\n* Polyglot: Add and remove data from special \"maps\" that are shared between languages.\n* Graal: Graal version\n* Java: Access JVM types (and hence values).\n* Packages: Package root to access classes.\n\n# Running graajs (aka node)\n\nSome helpful things:\n\n* --vm.cp yourclasspath\n\t* NODE_JVM_CLASSPATH can also hold the classpath for the node process\n \t* Have not seen NODE_JVM_CLASSPATH before, is that real?\n* Need --jvm to run on the jvm and have jvm available\n  * Note I ran it without --jvm and jvm still seems accessible!?!?!\n* --experimental-workers to turn on workers\n* --experimental-sourcemaps to turn on sourcemap, especially if using scala.js/typescript/babel\n\nLots of VM options to tune graal.\n\n# Resources\n\n* Mike Hearn:\n  * https://blog.plan99.net/vertical-architecture-734495f129c4: Mike's article on how it works.\n  * https://github.com/mikehearn/nodejvm: Code for Mike's work.\n  * https://mikehearn.github.io/nodejvm/: Doc site. Start here.\n* Laurynas Lubys: Early versions of interop. Both vidoes are about the same\n  * https://www.youtube.com/watch?v=3SRjPHnWHa0\u0026t=460s\n  * https://www.youtube.com/watch?v=o4rRWckkUyE\u0026t=456s\n* A question on this topic that helps formulate the solution space:\n  * https://www.gitmemory.com/issue/graalvm/graaljs/12/493957872\n* General interop:\n  * https://technology.amis.nl/2019/11/05/node-js-application-running-on-graalvm-interoperating-with-java-python-r-and-more/\n* graaljs async interop tests, steal some of this code: https://github.com/graalvm/graaljs/blob/master/graal-js/src/com.oracle.truffle.js.test/src/com/oracle/truffle/js/test/interop/AsyncInteropTest.java\n* Key jvm docs for jvm polyglot. Open these during development:\n  * https://www.graalvm.org/sdk/javadoc/index.html?org/graalvm/polyglot/Context.html\n  * https://www.graalvm.org/sdk/javadoc/org/graalvm/polyglot/Value.html\n  * https://github.com/graalvm/graaljs/blob/master/docs/user/JavaInterop.md\n  * https://github.com/graalvm/graaljs/blob/master/docs/user/JavaScriptCompatibility.md\n  * multithreading: https://github.com/graalvm/graaljs/blob/master/docs/user/Multithreading.md\n  * https://github.com/graalvm/graaljs/blob/master/graal-js/src/com.oracle.truffle.js.test.threading/src/com/oracle/truffle/js/test/threading/JavaAsyncTaskScheduler.java\n  * worker threads and jvm interop: https://github.com/graalvm/graaljs/blob/master/graal-nodejs/test/graal/unit/worker.js\n\nNote on mill and dotty:\n\n* https://appddeevvmeanderings.blogspot.com/2020/03/mill-for-dotty.html\n\nThis repo translates Mike's work to dotty and all credit for anything clever\ngoes to everyone else.\n\n# JS Source on Classpath\n\nIf you place a js file inside a jar, you can access it directly assuming you used\n`node --jvm --vm.cp=yourjar.jar`:\n\n```javascript\nconst cl =  Packages.java.lang.ClassLoader=.getSystemClassLoader()\nconst source = Buffer.from(cl.getResource(\"nodejs/yoursource.js\").getContent().readAllBytes()).toString(\"utf-8\")\n```\n\n# License\n\nApache, just like Mike Hearn's since there is code from his repo.\n\nNote that alot of his code is actually commented out because it was kotlin code. \nMost of the original pure java code is still being used.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faappddeevv%2Fnodescala","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faappddeevv%2Fnodescala","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faappddeevv%2Fnodescala/lists"}