{"id":26976862,"url":"https://github.com/seangarner/node-stream-mmmagic","last_synced_at":"2025-04-03T12:17:15.783Z","repository":{"id":22909123,"uuid":"26257849","full_name":"seangarner/node-stream-mmmagic","owner":"seangarner","description":"Node module to sniff the start of a stream to detect the file type and encoding","archived":false,"fork":false,"pushed_at":"2024-06-17T00:26:48.000Z","size":66,"stargazers_count":29,"open_issues_count":2,"forks_count":10,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-08T03:15:33.683Z","etag":null,"topics":["encoding","filetype","javascript","magic","mime","nodejs","stream"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"mozilla/global-sprint","license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/seangarner.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-11-06T06:52:59.000Z","updated_at":"2021-11-25T16:29:48.000Z","dependencies_parsed_at":"2022-08-21T06:40:35.661Z","dependency_job_id":null,"html_url":"https://github.com/seangarner/node-stream-mmmagic","commit_stats":null,"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seangarner%2Fnode-stream-mmmagic","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seangarner%2Fnode-stream-mmmagic/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seangarner%2Fnode-stream-mmmagic/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seangarner%2Fnode-stream-mmmagic/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/seangarner","download_url":"https://codeload.github.com/seangarner/node-stream-mmmagic/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246998229,"owners_count":20866696,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["encoding","filetype","javascript","magic","mime","nodejs","stream"],"created_at":"2025-04-03T12:17:15.028Z","updated_at":"2025-04-03T12:17:15.766Z","avatar_url":"https://github.com/seangarner.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# node-stream-mmmagic\n[![Build Status](https://travis-ci.org/seangarner/node-stream-mmmagic.svg?branch=master)](https://travis-ci.org/seangarner/node-stream-mmmagic)\n\nNode module to sniff the start of a stream (non-destructively) to detect the file type and encoding\nwhen you don't have the luxury of being able to restart the stream again.\n\nIt does so by using [buffer-peek-stream](https://github.com/seangarner/node-buffer-peek-stream) to\nget the first 16KB of the stream then send that to mmmagic (which uses libmagic).  Before it's\nfinished the peek stream will unshift the bytes it's received back onto the origin stream thereby\nmaking it appear as if the origin stream was new.\n\n```bash\nnpm install stream-mmmagic\n```\n\n### Use\n```js\nconst magic = require('stream-mmmagic');\nconst input = fs.createReadStream('somefile.csv');\n\nconst [mime, output] = await magic.promise(input);\nconsole.log('TYPE:', mime.type);\nconsole.log('ENCODING:', mime.encoding);\noutput.pipe(process.stdout);\n\n//- TYPE: text/plain\n//- ENCODING: us-ascii\n//- \u003cthe file content\u003e\n```\n\n\n## Use (Callbacks)\n```js\nvar magic = require('stream-mmmagic');\n\nvar input = fs.createReadStream('somefile.csv');\n\nmagic(input, function (err, mime, output) {\n  if (err) throw err;\n\n  console.log('TYPE:', mime.type);\n  console.log('ENCODING:', mime.encoding);\n\n  // will print the *whole* file\n  output.pipe(process.stdout);\n});\n\n//- TYPE: text/plain\n//- ENCODING: us-ascii\n//- \u003cthe file content\u003e\n```\n\n### `options.magicFile` Custom Magic File\nA magic file is bundled with the mmmagic npm module but if you want to use your own then set the path to the file on\nthe `magicFile` option.\n\n```js\nconst magicFile = '/usr/share/magic';\nmagic(input, {magicFile}, callback);\n```\n\n### `options.splitMime` Original Mime String\nUse `{splitMime: false}` option to get back the original mime string instead of a split object.\n```js\nconst [mime] = magic.promise(input, {splitMime: false});\nconsole.log(mime);\n//- text/plain; charset=us-ascii\n```\n\n### `options.peekBytes` Control Bytes Used for Analysis\nAs the input stream starts to get data the first 16KB is buffered and sent to libmagic for analysis to get file type and\nencoding.  1KB is more than enough for detecting file type with a standard `magicFile` but the reliabilty of getting the\ncorrect encoding is increased the more bytes are buffered.  The tradeoff is performance and memory use.\n\nSet `peekBytes` to the number of bytes you want buffered and sent to libmagic.  For best results do not set below 256\nbytes.\n\n```js\n// somefile.txt is a utf8 file where the first doublebyte char is after the first 1KB of the file\nconst input = fs.createReadStream('somefile.txt');\n\nconst [{encoding}, output] = magic.promise(input, {peekBytes: 1024});\nconsole.log(encoding);\n// not detected as utf8 because the first doublebyte char wasn't until later in the stream\n//- us-ascii\n\nconst [{encoding}, output] = magic.promise(input, {peekBytes: 16384});\nconsole.log(encoding);\n// now we're peeking 16KB into the file libmagic gets that first doublebyte char and knows it's utf8\n//- charset=utf8\n```\n\n## LICENSE\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fseangarner%2Fnode-stream-mmmagic","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fseangarner%2Fnode-stream-mmmagic","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fseangarner%2Fnode-stream-mmmagic/lists"}