{"id":20230819,"url":"https://github.com/sharkdp/content_inspector","last_synced_at":"2025-04-10T18:05:17.783Z","repository":{"id":60775504,"uuid":"135914804","full_name":"sharkdp/content_inspector","owner":"sharkdp","description":"Fast inspection of binary buffers to guess/determine the type of content","archived":false,"fork":false,"pushed_at":"2018-11-04T16:26:29.000Z","size":23,"stargazers_count":32,"open_issues_count":0,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-05-08T22:46:30.515Z","etag":null,"topics":["binary","encoding","rust","text"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sharkdp.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE-APACHE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-06-03T15:10:01.000Z","updated_at":"2023-11-22T06:34:16.000Z","dependencies_parsed_at":"2022-10-04T17:15:21.772Z","dependency_job_id":null,"html_url":"https://github.com/sharkdp/content_inspector","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sharkdp%2Fcontent_inspector","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sharkdp%2Fcontent_inspector/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sharkdp%2Fcontent_inspector/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sharkdp%2Fcontent_inspector/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sharkdp","download_url":"https://codeload.github.com/sharkdp/content_inspector/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248265813,"owners_count":21075045,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["binary","encoding","rust","text"],"created_at":"2024-11-14T07:43:50.707Z","updated_at":"2025-04-10T18:05:17.765Z","avatar_url":"https://github.com/sharkdp.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# content_inspector\n\n[![Crates.io](https://img.shields.io/crates/v/content_inspector.svg)](https://crates.io/crates/content_inspector)\n[![Documentation](https://docs.rs/content_inspector/badge.svg)](https://docs.rs/content_inspector)\n\nA simple library for *fast* inspection of binary buffers to guess the type of content.\n\nThis is mainly intended to quickly determine whether a given buffer contains \"binary\"\nor \"text\" data. Programs like `grep` or `git diff` use similar mechanisms to decide whether\nto treat some files as \"binary data\" or not.\n\nThe analysis is based on a very simple heuristic: Searching for NULL bytes\n(indicating \"binary\" content) and the detection of special [byte order\nmarks](https://en.wikipedia.org/wiki/Byte_order_mark) (indicating a particular kind of textual\nencoding). Note that **this analysis can fail**. For example, even if unlikely, UTF-8-encoded\ntext can legally contain NULL bytes. Conversely, some particular binary formats (like binary\n[PGM](https://en.wikipedia.org/wiki/Netpbm_format)) may not contain NULL bytes. Also, for\nperformance reasons, only the first 1024 bytes are checked for the NULL-byte (if no BOM was\ndetected).\n\nIf this library reports a certain type of encoding (say `UTF_16LE`), there is **no guarantee** that\nthe binary buffer can actually be decoded as UTF-16LE.\n\n## Usage\n\n```rust\nuse content_inspector::{ContentType, inspect};\n\nassert_eq!(ContentType::UTF_8, inspect(b\"Hello\"));\nassert_eq!(ContentType::BINARY, inspect(b\"\\xFF\\xE0\\x00\\x10\\x4A\\x46\\x49\\x46\\x00\"));\n\nassert!(inspect(b\"Hello\").is_text());\n```\n\n## CLI example\n\nThis crate also comes with a small example command-line program (see [`examples/inspect.rs`](examples/inspect.rs)) that demonstrates the usage:\n```bash\n\u003e inspect\nUSAGE: inspect FILE [FILE...]\n\n\u003e inspect testdata/*\ntestdata/create_text_files.py: UTF-8\ntestdata/file_sources.md: UTF-8\ntestdata/test.jpg: binary\ntestdata/test.pdf: binary\ntestdata/test.png: binary\ntestdata/text_UTF-16BE-BOM.txt: UTF-16BE\ntestdata/text_UTF-16LE-BOM.txt: UTF-16LE\ntestdata/text_UTF-32BE-BOM.txt: UTF-32BE\ntestdata/text_UTF-32LE-BOM.txt: UTF-32LE\ntestdata/text_UTF-8-BOM.txt: UTF-8-BOM\ntestdata/text_UTF-8.txt: UTF-8\n```\n\nIf you only want to detect whether something is a binary or text file, this is about a factor of 250 faster than `file --mime ...`.\n\n## License\n\nLicensed under either of\n\n * Apache License, Version 2.0, ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)\n * MIT license ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT)\n\nat your option.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsharkdp%2Fcontent_inspector","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsharkdp%2Fcontent_inspector","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsharkdp%2Fcontent_inspector/lists"}