{"id":30105733,"url":"https://github.com/moshen/wasmagic","last_synced_at":"2025-08-10T00:19:11.021Z","repository":{"id":40574921,"uuid":"394505426","full_name":"moshen/wasmagic","owner":"moshen","description":"A WebAssembly compiled version of libmagic with a simple API for Node. WASMagic provides accurate filetype detection with zero prod dependencies","archived":false,"fork":false,"pushed_at":"2025-02-09T17:08:10.000Z","size":305,"stargazers_count":47,"open_issues_count":1,"forks_count":7,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-08-09T18:53:54.431Z","etag":null,"topics":["hacktoberfest","nodejs","wasm"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-2-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/moshen.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS","dei":null}},"created_at":"2021-08-10T02:48:12.000Z","updated_at":"2025-07-28T23:30:11.000Z","dependencies_parsed_at":"2024-04-15T03:29:52.448Z","dependency_job_id":"73c4962e-8ba3-43ee-aacb-ba5458f02d46","html_url":"https://github.com/moshen/wasmagic","commit_stats":{"total_commits":36,"total_committers":3,"mean_commits":12.0,"dds":0.05555555555555558,"last_synced_commit":"6b529d947d877c432dfb0419f282824c9faca924"},"previous_names":[],"tags_count":35,"template":false,"template_full_name":null,"purl":"pkg:github/moshen/wasmagic","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moshen%2Fwasmagic","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moshen%2Fwasmagic/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moshen%2Fwasmagic/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moshen%2Fwasmagic/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/moshen","download_url":"https://codeload.github.com/moshen/wasmagic/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moshen%2Fwasmagic/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":269648768,"owners_count":24453489,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-09T02:00:10.424Z","response_time":111,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["hacktoberfest","nodejs","wasm"],"created_at":"2025-08-10T00:19:09.571Z","updated_at":"2025-08-10T00:19:10.989Z","avatar_url":"https://github.com/moshen.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# WASMagic\n\n[![NPM](https://img.shields.io/npm/v/wasmagic)](https://www.npmjs.com/package/wasmagic)\n[![Build status](https://img.shields.io/github/actions/workflow/status/moshen/wasmagic/ci.yml?branch=master)](https://github.com/moshen/wasmagic/actions/workflows/ci.yml)\n\n## About\n\nA [WebAssembly](https://webassembly.org/) compiled version of\n[libmagic](https://www.darwinsys.com/file/) with a simple API for Node, Bun, or\nDeno. WASMagic provides accurate filetype detection with zero\nprod dependencies.\n\n## Usage\n\n[Install `wasmagic` from npm](https://www.npmjs.com/package/wasmagic):\n\n```bash\nnpm install wasmagic\n```\n\nDetect the mime of something in Node, or Bun:\n```javascript\nimport { WASMagic } from \"wasmagic\";\n\nconst magic = await WASMagic.create();\n\nconst pngFile = Buffer.from(\"89504E470D0A1A0A0000000D49484452\", \"hex\");\nconsole.log(magic.detect(pngFile));\n// outputs: image/png\n```\n\nCommonJS version in Node, or Bun:\n```javascript\nconst { WASMagic } = require(\"wasmagic\");\n\nasync function main() {\n  const magic = await WASMagic.create();\n  const pngFile = Buffer.from(\"89504E470D0A1A0A0000000D49484452\", \"hex\");\n  console.log(magic.detect(pngFile));\n}\n\nmain().catch((err) =\u003e console.error(err));\n// outputs: image/png\n```\n\nDeno:\n```javascript\nimport { WASMagic } from \"npm:wasmagic\";\nconst magic = await WASMagic.create();\nconst pngFile = Uint8Array.from([\n  0x89,  0x50,  0x4e,  0x47,  0x0d,  0x0a,  0x1a,  0x0a,\n  0x00,  0x00,  0x00,  0x0d,  0x49,  0x48,  0x44,  0x52\n]);\nconsole.log(magic.getMime(pngFile));\n// outputs: image/png\n```\n\nRun with permissions:\n```shell\ndeno run \\\n  --allow-read=\"myscript.js,$PWD/node_modules/.deno/wasmagic@0.2.0/node_modules/wasmagic/dist/libmagic-wrapper.wasm\" \\\n  myscript.js\n```\n\nWASMagic *should* also work in modern browsers by using a packaging utility like\n[Rollup](https://rollupjs.org/) or [Webpack](https://webpack.js.org/), however\nthis isn't tested, and requires downloading the WASM payload which is currently\n1.6MB.\n\n### Options\n\nThe `WASMagic.create()` method takes an optional options object with the type\n`WASMagicOptions`. These options can be used to customize file type detection by\neither augmenting the default magic file or replacing it completely.\n\nFor examples on using these options, [please look at the\ntests](src/test/integration/index.ts).\n\n#### `WASMagicOptions`\n\n```typescript\ntype WASMagicOptions = {\n  flags?: WASMagicFlags;\n  loadDefaultMagicfile?: boolean;\n  magicFiles?: Uint8Array[];\n  stdio?: (stdioName: \"stdout\" | \"stderr\", text: string) =\u003e void;\n};\n```\n\nDefault options:\n\n```typescript\n{\n  flags: WASMagicFlags.MIME_TYPE,\n  loadDefaultMagicfile: true,\n  magicFiles: [],\n  stdio: (_stdioName: \"stdout\" | \"stderr\", _text: string) =\u003e {},\n}\n```\n\n##### `flags`\n\n`flags` control `libmagic` behavior. To use the flags, import the `enum` from\nthe module, and pass the desired combination of flags:\n\n```javascript\nimport { WASMagic, WASMagicFlags } from \"wasmagic\";\n\nconst magic = await WASMagic.create({\n  flags: WASMagicFlags.MIME_TYPE | WASMagicFlags.MIME_ENCODING,\n});\n\nconst pngFile = Buffer.from(\"89504E470D0A1A0A0000000D49484452\", \"hex\");\nconsole.log(magic.detect(pngFile));\n// outputs: image/png; charset=binary\n```\n\n[Please refer to the code for the flag definitions](src/index.ts#L8)\n\n**Default**: `WASMagicFlags.MIME_TYPE`\n\n##### `loadDefaultMagicfile`\n\n`loadDefaultMagicFile` is a `boolean` dictates whether or not to load the\nbundled magic file. The bundled magic file is the default `magic.mgc` file\ngenerated in the `libmagic` build.\n\n**Default**: `true`\n\n##### `magicFiles`\n\n`magicFiles` is an `Array` of `Uint8Array`s representing magic definition files.\nThis option can be used to tell `libmagic` to use custom file type detection.\n\n[See the test `foobarfiletype` magic file definition as an example custom file\ntype magic definition](src/test/integration/foobar_magic).\n\nAs these are passed as `Uint8Array`s, custom definitions can be loaded from\na file, or embedded directly in your code. For example:\n\n```javascript\nimport { WASMagic } from \"wasmagic\";\n\nconst foobarMagic = Buffer.from(\n  `\n0  string  FOOBARFILETYPE  Made up filetype\n!:mime foobarfiletype\n`,\n);\n\nconst magic = await WASMagic.create({\n  magicFiles: [foobarMagic],\n});\n\nconsole.log(\n  magic.detect(\n    Buffer.from(\n      `FOOBARFILETYPE\n\nSome made up stuff\n`,\n    ),\n  ),\n);\n\n// outputs: foobarfiletype\n```\n\n**Default**: `[]`\n\n##### `stdio`\n\n`stdio` is a `function` defined to override stdout / stderr output from\n`libmagic`. This option might not be particularly useful for normal usage, but\nvery useful for getting warnings about custom magic files and debugging the\ndevelopment of this module.\n\n**Default**: `(_stdioName: \"stdout\" | \"stderr\", _text: string) =\u003e {}` (No\noutput)\n\n### Examples\n\n- [Async / Worker threads](examples/worker/)\n- [Stream mime type detection](examples/stream-detection/)\n\n### Performance considerations\n\n#### WASMagic instantiation\n\nYou should instantiate as few copies of `WASMagic` as you can get away with for\nyour use case. Each instantiation loads the magic database, which is around 8MB.\nOne instance per process / worker thread should be enough as the main api\n(`WASMagic.detect`) is synchronous.\n\nIf you want to offload processing to another thread (and in production workloads\nyou probably should be), take a look at the [Async / Worker\nthreads](examples/worker/) example.\n\n---\n\nIf you aren't passing your instantiated dependencies down in your application,\nand you aren't using Javascript modules (or Typescript), and are trying to use\nthis as a global, try something like the following for a CommonJS style module:\n\n```javascript\nconst { WASMagic } = require(\"wasmagic\");\nlet magicGlobal = null;\nconst magicPromise = WASMagic.create().then((instance) =\u003e {\n  magicGlobal = instance;\n  return magicGlobal;\n});\n\nasync function main() {\n  const magic = magicGlobal || (await magicPromise);\n  const pngFile = Buffer.from(\"89504E470D0A1A0A0000000D49484452\", \"hex\");\n  console.log(magic.detect(pngFile));\n}\n\nmain().catch((err) =\u003e console.error(err));\n// outputs: image/png\n```\n\n#### Large files\n\nWASMagic will copy the entire provided buffer into the wasm heap for processing.\nThis can cause significant performance penalties if the files being processed\nare exceptionally large (ex. video files, image files).\n\nTo ensure that your application remains performant, either load only the head\nof the file you want to detect, or slice the head off of a file buffer:\n\n```javascript\nimport { WASMagic } from \"wasmagic\";\nimport * as fs from \"fs/promises\";\n\nconst magic = await WASMagic.create();\nconst file = await fs.open(\"largeFile.mp4\");\n\n// Only read the first 1024 bytes of our large file\nconst { bytesRead, buffer } = await file.read({ buffer: Buffer.alloc(1024) });\n\n// We're assuming that largeFile.mp4 is \u003e= 1024 bytes in size and our buffer\n// will only have the first 1024 bytes of largeFile.mp4 in it\nconsole.log(magic.detect(buffer));\nawait file.close();\n\n// outputs: video/mp4\n```\n\nAlternatively, if you are streaming a large file, look at the [stream mime type\ndetection example](examples/stream-detection/). When you're dealing with\nstreams you can attempt detection of the mime type when the first chunks are\nloaded.\n\nHowever, this strategy falls apart depending on the type of file that you are\ntrying to detect and the accuracy you are looking for. For example; pre 2007\nMicrosoft Office documents will only be accurately detected if the entire file\nis available to be parsed. If you use the above strategy to check the head of\nthe file, you will get `application/CDFV2` instead of `application/msword`.\n\nTo make sure you're getting accurate results for the files that you're trying\nto detect, be sure to test example files.\n\n### Detected filetypes\n\nWASMagic includes the default magic file which enables detection of any file\ntype [detected by\nlibmagic](https://github.com/file/file/tree/master/magic/Magdir), which (at time\nof writing) is over 1500 mime types. For comparison; the\n[file-type](https://www.npmjs.com/package/file-type) library supports 138 types.\n\nSpecifically, WASMagic will accurately detect all types of Microsoft Office\nfiles, as well as many plain text file formats where file-type does not.\n\n## Development\n\nInstall Node and dependencies:\n\n```bash\nnvm install \u0026\u0026 nvm use\nnpm ci\n```\n\n---\n\nBuilding requires the [Emscripten](https://emscripten.org/) sdk, autoconf,\nautomake, and libtool.\n\n---\n\nThe easiest way to build is to use the bundled Docker builder image based on the\nofficial [Emscripten](https://hub.docker.com/r/emscripten/emsdk) image. Simply\nrun:\n\n```bash\nmake clean docker-builder-run test\n```\n\n---\n\nIf you would like to build natively on your Mac, and have\n[Homebrew](https://brew.sh/) installed, install these additional packages:\n\n```bash\nbrew install autoconf automake coreutils emscripten libtool\n```\n\nThen:\n\n```bash\nmake\n```\n\nWill build and test the module.\n\n## Why WASMagic?\n\nLike many open source projects, WASMagic was created to solve my problems:\n\n- I need to detect a diverse set of file types\n  - Recent versions of libmagic work for all the file types I need\n- I need to detect the mime type of a `Buffer` in Node\n  - Incurring the performance penalty of disk I/O to detect a file I already had\n    in memory was unacceptable\n\n### Why existing libraries didn't meet these needs\n\n- Non libmagic based libraries do not properly detect the filetypes I need. This\n  includes the popular [file-type](https://www.npmjs.com/package/file-type)\n  library.\n- All libmagic Node libraries [I found](https://www.npmjs.com/search?q=libmagic)\n  are using a _very_ old or broken version of libmagic. At time of writing, these\n  were the libraries available:\n  - [libmagic](https://www.npmjs.com/package/libmagic), 4 years old\n  - [mmmagic](https://www.npmjs.com/package/mmmagic), 3 years old\n  - [mime-magic](https://www.npmjs.com/package/mime-magic), 9 years old\n  - [@npcz/magic](https://www.npmjs.com/package/@npcz/magic), 6 months old, but\n    uses a broken version of libmagic. Misidentifies `.mp3` files\n- In the case of [@npcz/magic](https://www.npmjs.com/package/@npcz/magic), it's\n  API required reading the mime of a file on disk\n\n### Why WASM instead of using `libmagic` natively?\n\nWhen benchmarking against native modules like `mmmagic` I found my WASM based\nproof of concept to be faster. It also seems prudent to run libmagic within the\nWASM sandbox, as it bypasses security concerns about libmagic itself.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmoshen%2Fwasmagic","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmoshen%2Fwasmagic","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmoshen%2Fwasmagic/lists"}