{"id":15315018,"url":"https://github.com/himself65/talking-about-module-under-nodejs-and-deno","last_synced_at":"2026-02-26T20:10:57.300Z","repository":{"id":101158457,"uuid":"216733499","full_name":"himself65/talking-about-module-under-nodejs-and-deno","owner":"himself65","description":"[WIP] 浅谈 Node.js 和 Deno 下的 Module loader","archived":false,"fork":false,"pushed_at":"2019-10-27T08:23:14.000Z","size":679,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-02-22T03:28:35.538Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/himself65.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-10-22T05:53:52.000Z","updated_at":"2019-10-27T08:23:15.000Z","dependencies_parsed_at":null,"dependency_job_id":"8293939c-9a40-43f7-a656-2f6011c5fdba","html_url":"https://github.com/himself65/talking-about-module-under-nodejs-and-deno","commit_stats":{"total_commits":7,"total_committers":1,"mean_commits":7.0,"dds":0.0,"last_synced_commit":"dc37a252a3594402cfae20e4e579be80d6d2354a"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/himself65/talking-about-module-under-nodejs-and-deno","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/himself65%2Ftalking-about-module-under-nodejs-and-deno","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/himself65%2Ftalking-about-module-under-nodejs-and-deno/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/himself65%2Ftalking-about-module-under-nodejs-and-deno/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/himself65%2Ftalking-about-module-under-nodejs-and-deno/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/himself65","download_url":"https://codeload.github.com/himself65/talking-about-module-under-nodejs-and-deno/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/himself65%2Ftalking-about-module-under-nodejs-and-deno/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29870750,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-26T18:42:30.764Z","status":"ssl_error","status_checked_at":"2026-02-26T18:41:47.936Z","response_time":89,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-01T08:48:43.092Z","updated_at":"2026-02-26T20:10:57.284Z","avatar_url":"https://github.com/himself65.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# 浅谈Node.js 下的 Module loader\n\n## 前言\n\n题目的 idea 来自 [这个问题](https://www.zhihu.com/question/349550048/answer/864295558)\n\n## Node.js 下的 Module Loader\n\n在 Node.js 中，每个文件都视作一个单独的模块，所以我们可以这么写\n\n```js\n// index.js\nconst module1 = require('./module1')\n\nconsole.log(module1.foo(0))\n\nconsole.log(module1.goo(0)(0))\n```\n\n```js\n// module1/index.js\nexports.foo = a =\u003e a + 114514\nexports.goo = b =\u003e a =\u003e foo\n```\n\n在 Node.js 源代码中，我们可以找到 `require` 函数的实现，其中 `id` 是 一个非空 string\n\n```js\n// Loads a module at the given file path. Returns that module's\n// `exports` property.\nModule.prototype.require = function(id) {\n  validateString(id, 'id');\n  if (id === '') {\n    throw new ERR_INVALID_ARG_VALUE('id', id,\n                                    'must be a non-empty string');\n  }\n  requireDepth++;\n  try {\n    return Module._load(id, this, /* isMain */ false);\n  } finally {\n    requireDepth--;\n  }\n};\n```\n\n我们可以看到，最顶层会有一个 `requreDepth` 来控制调用深度，具体什么用，我们可以接下来看到\n\n然后 `require` 直接调用 `Module._load`\n\n顺带一提，`Module` 是一个类（JS意义上的类）\n\n```js\nfunction Module(id = '', parent) {\n  this.id = id;\n  this.path = path.dirname(id);\n  this.exports = {};\n  this.parent = parent;\n  updateChildren(parent, this, false);\n  this.filename = null;\n  this.loaded = false;\n  this.children = [];\n}\n```\n\n继续看 `_load`\n\n```js\n// Check the cache for the requested file.\n// 1. If a module already exists in the cache: return its exports object.\n// 2. If the module is native: call\n//    `NativeModule.prototype.compileForPublicLoader()` and return the exports.\n// 3. Otherwise, create a new module for the file and save it to the cache.\n//    Then have it load  the file contents before returning its exports\n//    object.\nModule._load = function(request, parent, isMain) {\n    // 一大堆代码，暂时不贴上来\n}\n```\n\n我们可以从注释中看到，他会先查看缓存中是否有模块，或者是否为 Native 模块，最后才会加载文件\n\n三个参数，第一个是地址，第二个是父地址，第三个是判断是否主文件的boolean\n\n其中，加载 Native 会一直调用到 `src/node_binding.cc`\n\n```cpp\nvoid GetInternalBinding(const FunctionCallbackInfo\u003cValue\u003e\u0026 args) {\n  Environment* env = Environment::GetCurrent(args);\n\n  CHECK(args[0]-\u003eIsString());\n\n  // 模块名称\n  Local\u003cString\u003e module = args[0].As\u003cString\u003e();\n\n  // 转到 UTF-8 String\n  node::Utf8Value module_v(env-\u003eisolate(), module);\n  Local\u003cObject\u003e exports;\n\n  node_module* mod = get_internal_module(*module_v);\n  if (mod != nullptr) {\n    // 能找到内部的 module\n    exports = InitModule(env, mod, module);\n  } else if (!strcmp(*module_v, \"constants\")) {\n    exports = Object::New(env-\u003eisolate());\n    CHECK(\n        exports-\u003eSetPrototype(env-\u003econtext(), Null(env-\u003eisolate())).FromJust());\n    DefineConstants(env-\u003eisolate(), exports);\n  } else if (!strcmp(*module_v, \"natives\")) {\n    exports = native_module::NativeModuleEnv::GetSourceObject(env-\u003econtext());\n    // Legacy feature: process.binding('natives').config contains stringified\n    // config.gypi\n    CHECK(exports\n              -\u003eSet(env-\u003econtext(),\n                    env-\u003econfig_string(),\n                    native_module::NativeModuleEnv::GetConfigString(\n                        env-\u003eisolate()))\n              .FromJust());\n  } else {\n    return ThrowIfNoSuchModule(env, *module_v);\n  }\n\n  args.GetReturnValue().Set(exports);\n}\n```\n\n我们打一个断点看看\n\n![breakpoint](./breakpoint.png)\n\n```json\n{\n  \"id\": \".\",\n  \"exports\": {},\n  \"parent\": null,\n  \"filename\": \"C:\\\\Users\\\\76128\\\\Desktop\\\\article\\\\examples\\\\02\\\\src\\\\index.js\",\n  \"loaded\": false,\n  \"children\": [],\n  \"paths\": [\n    \"C:\\\\Users\\\\76128\\\\Desktop\\\\article\\\\examples\\\\02\\\\src\\\\node_modules\",\n    \"C:\\\\Users\\\\76128\\\\Desktop\\\\article\\\\examples\\\\02\\\\node_modules\",\n    \"C:\\\\Users\\\\76128\\\\Desktop\\\\article\\\\examples\\\\node_modules\",\n    \"C:\\\\Users\\\\76128\\\\Desktop\\\\article\\\\node_modules\",\n    \"C:\\\\Users\\\\76128\\\\Desktop\\\\node_modules\",\n    \"C:\\\\Users\\\\76128\\\\node_modules\",\n    \"C:\\\\Users\\\\node_modules\",\n    \"C:\\\\node_modules\"\n  ]\n}\n```\n\n其中的 Module 实例，`paths` 保存了所有能找到的 `node_modules` 黑洞\n\n然后我们断点看到 `NativeModule._source` 里面已经保存了所有的内建代码，所以直接返回 `NativeModule.require(filename)` （如图）\n\n```js\n  const mod = loadNativeModule(filename, request, experimentalModules);\n  if (mod \u0026\u0026 mod.canBeRequiredByUsers) return mod.exports;\n```\n\n![02](./02.png)\n\n---\n\n我们写一个互相调用的 example\n\n```js\nlet M\nM = require('./module')\n\nconsole.log(M)\n```\n\n```js\n// module/index.js\nmodule.exports = {\n  ...require('./1'),\n  ...require('./2')\n}\n```\n\n```js\n// module/1.js\nrequire('./2')\n\nmodule.exports = {\n  foo: 1\n}\n```\n\n```js\n// module/2.js\nrequire('./1')\n\nmodule.exports = {\n  goo: 2\n}\n```\n\n第一次加载，没有缓存，他会调用到 `Module.prototype.load`\n\n```js\n// Given a file name, pass it to the proper extension handler.\nModule.prototype.load = function(filename) {\n  debug('load %j for module %j', filename, this.id);\n\n  assert(!this.loaded);\n  this.filename = filename;\n  this.paths = Module._nodeModulePaths(path.dirname(filename));\n\n  const extension = findLongestRegisteredExtension(filename);\n  // allow .mjs to be overridden\n  if (filename.endsWith('.mjs') \u0026\u0026 !Module._extensions['.mjs']) {\n    throw new ERR_REQUIRE_ESM(filename);\n  }\n  Module._extensions[extension](this, filename);\n  this.loaded = true;\n\n  if (experimentalModules) {\n    const ESMLoader = asyncESM.ESMLoader;\n    const url = `${pathToFileURL(filename)}`;\n    const module = ESMLoader.moduleMap.get(url);\n    // Create module entry at load time to snapshot exports correctly\n    const exports = this.exports;\n    // Called from cjs translator\n    if (module !== undefined \u0026\u0026 module.module !== undefined) {\n      if (module.module.getStatus() \u003e= kInstantiated)\n        module.module.setExport('default', exports);\n    } else {\n      // Preemptively cache\n      // We use a function to defer promise creation for async hooks.\n      ESMLoader.moduleMap.set(\n        url,\n        // Module job creation will start promises.\n        // We make it a function to lazily trigger those promises\n        // for async hooks compatibility.\n        () =\u003e new ModuleJob(ESMLoader, url, () =\u003e\n          new ModuleWrap(url, undefined, ['default'], function() {\n            this.setExport('default', exports);\n          })\n        , false /* isMain */, false /* inspectBrk */)\n      );\n    }\n  }\n};\n```\n\n默认情况下允许读取 `.js`, `.mjs`, `.node`, `.json`\n\n然后 `.js` 文件调用 `.js` 相关的handle\n\n```js\nModule._extensions['.js'] = function(module, filename) {\n  // ...省略多余判断\n  const content = fs.readFileSync(filename, 'utf8');\n  module._compile(content, filename);\n};\n```\n\n而 `_compiler` 则做了一通检查，然后调用 [`vm.runInThisContext`](https://nodejs.org/dist/latest-v12.x/docs/api/vm.html)，本文不阐述非 module 部分\n\n我们来看 `.mjs` 是如何处理的，首先，moduleJS与commonJS最大的区别就是\n\nmjs是**静态加载模块**\n\n如何理解呢？\n\n```js\n// index.mjs\nconsole.log('this is main script')\n\nimport { foo } from './module/index.mjs'\n\nconsole.log(foo)\n```\n\n```js\n// module/index.mjs\nexport let foo = 1\n\nconsole.log('this is module')\n```\n\n然后我们运行 `node.js` 的实验功能\n\n```bash\n\u003e node --experimental-modules ./src/index.mjs\n(node:9932) ExperimentalWarning: The ESM module loader is experimental.\nthis is module\nthis is main script\n1\n```\n\n如果我们用相同的代码，用commonJS编写\n\n```js\nconsole.log('this is main script')\n\nconst { foo } = require('./module')\n\nconsole.log(foo)\n```\n\n```js\nexports.foo = 1\n\nconsole.log('this is module')\n```\n\n```bash\n\u003e node ./src/index.js\nthis is main script\nthis is module\n1\n```\n\nmjs的处理方式与cjs方式不同，在 `lib/internal/modules/esm` 文件夹中未另一套方案\n\n目前，`node.js` 目测是模拟了一套esm的静态加载流程\n\n启动脚本为：\n\n```js\n'use strict';\n\nconst {\n  prepareMainThreadExecution\n} = require('internal/bootstrap/pre_execution');\n\nprepareMainThreadExecution(true);\n\nconst CJSModule = require('internal/modules/cjs/loader').Module;\n\nmarkBootstrapComplete();\n\n// Note: this loads the module through the ESM loader if\n// --experimental-loader is provided or --experimental-modules is on\n// and the module is determined to be an ES module\nCJSModule.runMain(process.argv[1]);\n```\n\n```js\nasync import(specifier, parent) {\n    const job = await this.getModuleJob(specifier, parent);\n    const { module } = await job.run();\n    return module.getNamespace();\n}\n```\n\n其中，也有缓存机制\n\n```js\n// Tracks the state of the loader-level module cache\nclass ModuleMap extends SafeMap {\n  get(url) {\n    validateString(url, 'url');\n    return super.get(url);\n  }\n  set(url, job) {\n    validateString(url, 'url');\n    if (job instanceof ModuleJob !== true \u0026\u0026\n        typeof job !== 'function') {\n      throw new ERR_INVALID_ARG_TYPE('job', 'ModuleJob', job);\n    }\n    debug(`Storing ${url} in ModuleMap`);\n    return super.set(url, job);\n  }\n  has(url) {\n    validateString(url, 'url');\n    return super.has(url);\n  }\n}\n```\n\n其中 `SafeMap`，就是一个普通的 `Map`，node.js为了防止global变量被修改，所有东西都经过二次封装\n\n```js\n// lib/internal/per_context/primordials.js\nprimordials.SafeMap = makeSafe(\n  Map,\n  class SafeMap extends Map {}\n);\n```\n\n---\n\n## Deno 下的 Module Loader\n\n`Deno` 中有一个非常重要的地方，就是权限（Permisson）\n\n我们控制台输入 `deno --help`，可以看到权限选项\n\n```bash\n    -A, --allow-all\n            Allow all permissions\n\n        --allow-env\n            Allow environment access\n\n        --allow-hrtime\n            Allow high resolution time measurement\n\n        --allow-net=\u003callow-net\u003e\n            Allow network access\n\n        --allow-read=\u003callow-read\u003e\n            Allow file system read access\n\n        --allow-run\n            Allow running subprocesses\n\n        --allow-write=\u003callow-write\u003e\n            Allow file system write access\n```\n\n那么，权限是如何实现的呢？我们找了半天，一路向下看到这个地方\n\n```ts\n// cli/js/util.ts\nexport function createResolvable\u003cT\u003e(): Resolvable\u003cT\u003e {\n  let methods: ResolvableMethods\u003cT\u003e;\n  const promise = new Promise\u003cT\u003e(\n    (resolve, reject): void =\u003e {\n      methods = { resolve, reject };\n    }\n  );\n  // TypeScript doesn't know that the Promise callback occurs synchronously\n  // therefore use of not null assertion (`!`)\n  return Object.assign(promise, methods!) as Resolvable\u003cT\u003e;\n}\n\n// cli/js/dispatch_json.ts\nexport async function sendAsync(\n  opId: number,\n  args: object = {},\n  zeroCopy?: Uint8Array\n): Promise\u003cOk\u003e {\n  const promiseId = nextPromiseId();\n  args = Object.assign(args, { promiseId });\n  const promise = util.createResolvable\u003cOk\u003e();\n\n  const argsUi8 = encode(args);\n  const buf = core.dispatch(opId, argsUi8, zeroCopy);\n  if (buf) {\n    // Sync result.\n    const res = decode(buf);\n    promise.resolve(res);\n  } else {\n    // Async result.\n    promiseTable.set(promiseId, promise);\n  }\n\n  const res = await promise;\n  return unwrapResponse(res);\n}\n```\n\n`core.dispatch` 提供了一个native函数调用方法，ry曾说过设计 `node.js` 最大失误就是把各种方法直接绑定到 `v8` 中。\n\n```ts\nconst buf = core.dispatch(opId, argsUi8, zeroCopy);\n```\n\n但是，在 `deno` 中，各种 `native` 方法都绑定到了 `deno.isolate` 中\n\n```rust\n// core/isolate.rs\n/// Defines the how Deno.core.dispatch() acts.\n/// Called whenever Deno.core.dispatch() is called in JavaScript. zero_copy_buf\n/// corresponds to the second argument of Deno.core.dispatch().\n///\n/// Requires runtime to explicitly ask for op ids before using any of the ops.\npub fn register_op\u003cF\u003e(\u0026mut self, name: \u0026str, op: F) -\u003e OpId\nwhere\n  F: Fn(\u0026[u8], Option\u003cPinnedBuf\u003e) -\u003e CoreOp + Send + Sync + 'static,\n{\n  self.op_registry.register(name, op)\n}\n```\n\n```rust\n// core/ops.rs\n/// This function returns None only if op with given id doesn't exist in registry.\npub fn call(\n  \u0026self,\n  op_id: OpId,\n  control: \u0026[u8],\n  zero_copy_buf: Option\u003cPinnedBuf\u003e,\n) -\u003e Option\u003cCoreOp\u003e {\n  // Op with id 0 has special meaning - it's a special op that is always\n  // provided to retrieve op id map. The map consists of name to `OpId`\n  // mappings.\n  if op_id == 0 {\n    return Some(Op::Sync(self.json_map()));\n  }\n\n  let d = match self.dispatchers.get(op_id as usize) {\n    Some(handler) =\u003e \u0026*handler,\n    None =\u003e return None,\n  };\n\n  Some(d(control, zero_copy_buf))\n}\n```\n\n![cli/worker.ts](register_op.png)\n\n回到正题，我们来看 Module Loader 部分\n\n试着写一个错误代码然后看一下栈信息\n\n```ts\n// examples/05/deno.ts\n// import了一个不存在的东西\nimport * as Foo from 'https://himself65.com/foo.ts'\n```\n\n```bash\n\u003e deno examples/05/deno.ts\nCompile file:///C:/Users/76128/Desktop/article/examples/05/deno.ts\nDownload https://himself65.com/foo.ts\nerror: Uncaught HttpOther: https://himself65.com/foo.ts: error trying to connect: 不知道这样的主机。 (os error 11001)\n► $deno$/dispatch_json.ts:40:11\n    at DenoError ($deno$/errors.ts:20:5)\n    at unwrapResponse ($deno$/dispatch_json.ts:40:11)\n    at sendAsync ($deno$/dispatch_json.ts:91:10)\n```\n\n我们可以看见，import 一个外网URL 其实也是通过了 `dispatch`\n\n并通过 `fetch` 函数来进行下载远程 module\n\n```ts\n/** Fetch a resource from the network. */\nexport async function fetch(\n  input: domTypes.Request | string,\n  init?: domTypes.RequestInit\n): Promise\u003cResponse\u003e {\n  // ...\n}\n```\n\n然后正常的遵循 `cjs` 规范，也用到了缓存，在控制台通过 `-r` 进行重新下载缓存\n\n```bash\n                  -r, --reload=\u003cCACHE_BLACKLIST\u003e\n                        Reload source code cache (recompile TypeScript)\n                      --reload\n                        Reload everything\n                      --reload=https://deno.land/std\n                        Reload all standard modules\n                      --reload=https://deno.land/std/fs/utils.ts,https://deno.land/std/fmt/colors.ts\n                        Reloads specific modules\n```\n\n其他的调用顺序：`Module` --\u003e `ModuleSpecifier`，懒得写了，自己去读，一个``\n\n`ModuleSpecifier` 实际是一个带了 `Url` 的 new type pattern\n\n```rs\npub struct ModuleSpecifier(Url);\n```\n\n从个人粗略的阅读来看，就是套了个 `sandbox` 的 `Node.js` （因为好多地方实在是太像了\n\n有兴趣的可以阅读 `core` 文件夹下的所有代码\n\n去玩巫师3了（逃\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhimself65%2Ftalking-about-module-under-nodejs-and-deno","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhimself65%2Ftalking-about-module-under-nodejs-and-deno","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhimself65%2Ftalking-about-module-under-nodejs-and-deno/lists"}