{"id":16746973,"url":"https://github.com/selfup/scnnr","last_synced_at":"2025-04-10T13:50:49.861Z","repository":{"id":57558465,"uuid":"182177245","full_name":"selfup/scnnr","owner":"selfup","description":"Scan keywords in certain file types really fast! Output is pipe friendly. Also scan for filenames/file sizes. Includes regex support.","archived":false,"fork":false,"pushed_at":"2025-03-06T18:17:46.000Z","size":232,"stargazers_count":11,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-24T12:39:01.455Z","etag":null,"topics":["cli","command-line","command-line-tool","filesystem-tool","finder","golang","security-tools","unix-like"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/selfup.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-04-19T00:50:27.000Z","updated_at":"2025-03-04T13:46:47.000Z","dependencies_parsed_at":"2023-02-18T02:00:25.368Z","dependency_job_id":null,"html_url":"https://github.com/selfup/scnnr","commit_stats":null,"previous_names":[],"tags_count":9,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/selfup%2Fscnnr","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/selfup%2Fscnnr/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/selfup%2Fscnnr/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/selfup%2Fscnnr/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/selfup","download_url":"https://codeload.github.com/selfup/scnnr/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248228270,"owners_count":21068654,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cli","command-line","command-line-tool","filesystem-tool","finder","golang","security-tools","unix-like"],"created_at":"2024-10-13T02:08:48.025Z","updated_at":"2025-04-10T13:50:49.852Z","avatar_url":"https://github.com/selfup.png","language":"Go","readme":"\u003c!-- START doctoc generated TOC please keep comment here to allow auto update --\u003e\n\u003c!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE --\u003e\n\n- [Scnnr](#scnnr)\n    - [Help](#help)\n    - [No Keywords](#no-keywords)\n    - [Single Keyword](#single-keyword)\n    - [Multiple Keywords and Multiple File Extensions](#multiple-keywords-and-multiple-file-extensions)\n- [File Name Finder (NameFinder) (fnf)](#file-name-finder-namefinder-fnf)\n- [File Fingerprint Finder (FingerprintFinder) (fff)](#file-fingerprint-finder-fingerprintfinder-fff)\n- [File Size Finder (SizeFinder) (fsf)](#file-size-finder-sizefinder-fsf)\n- [Back to Scnnr](#back-to-scnnr)\n    - [Using the package github.com/selfup/scnnr/pkg](#using-the-package-githubcomselfupscnnrpkg)\n  - [Regex](#regex)\n    - [Using Regex Patterns](#using-regex-patterns)\n    - [Using the package github.com/selfup/scnnr/pkg](#using-the-package-githubcomselfupscnnrpkg-1)\n- [Install](#install)\n    - [If you have Go](#if-you-have-go)\n    - [If you do not have Go](#if-you-do-not-have-go)\n      - [Release Binaries](#release-binaries)\n      - [Direct Download Link](#direct-download-link)\n      - [cURL](#curl)\n      - [wget](#wget)\n    - [Docker](#docker)\n- [Performance (scn)](#performance-scn)\n\n\u003c!-- END doctoc generated TOC please keep comment here to allow auto update --\u003e\n\n# Scnnr\n\nScans files (by extension) in a given directory for a keyword. Can be any file, or can be just `.js` or `.js,.html,.jsx`.\n\nPrints out a `\\n` delimited string of each file (filepath in artifact) containing one of the keywords.\n\nMax file descriptors is set to 1024 (linux default) in scnnr scn mode.\n\nHas 3 additional modes: FileSizeFinder (find a file above a certain size), FileNameFinder (fuzzy find files with certain keywords in the filename), and FileFingerprintFinder (find files based on their SHA2-256 hash)\n\n`scn` mode is the default\n\n_Caveat: will not throw an error if a file cannot be read due to permissions. It is assumed that you know what files you can read/have access to. This allows for a clean/parseable output_\n\n### Help\n\nCall scnnr with the `-h` flag:\n\n```\n$ scnnr -h\n  -d string\n        OPTIONAL Scnnr MODE and OPTIONAL FingerrintFinder MODE\n            directory where scnnr will scan\n            default is current directory and all child directories (default \".\")\n  -e string\n        OPTIONAL Scnnr MODE\n            a comma delimited list of file extensions to scan\n            if none are given all files will be searched\n  -f string\n        REQUIRED NameFinder MODE\n            fuzzy find the filename(s) contain(s) - can be comma delimited: Example 'wow' or 'wow,omg,lol'\n  -k string\n        OPTIONAL Scnnr MODE and REQUIRED FingerprintFinder\n            scnnr: this is a comma delimited list of characters to look for in a file\n                if no keywords are given - all file paths of given file extensions will be returned\n                if keywords are given - only filepaths of matches will be returned\n            FingerprintFinder: this is a comma delimited list of SHA2-256 hashes to find files by\n  -m string\n        OPTIONAL\n            mode that scnnr will run in\n    \n            options are:\n                (scn) for scnnr (default)\n                (fnf) for File/NameFinder\n                (fsf) for File/SizeFinder\n                (fff) for File/FingerprintFinder (uses SHA2-256)\n    \n            ex: scnnr -d $HOME/Documents -k password,token,authorization\n            ex: scnnr -m fsf -d E:/ -s 100MB\n            ex: scnnr -m fnf -p /tmp,$HOME/Documents -f DEFCON\n            ex: scnnr -m fff -d $HOME/Documents -k de4f51f97fa690026e225798ff294cd182b93847aaa46fe1e32b848eb9e985bd\n    \n         (default \"scn\")\n  -p string\n        REQUIRED NameFinder MODE\n            any absolute path - can be comma delimited: Example: $HOME or '/tmp,/usr'\n  -r    OPTIONAL Scnnr MODE\n            if you want to use the regex engine or not\n            defaults to false and will not use the regex engine for scans unless set to a truthy value\n            truthy values are: 1, t, T, true, True, TRUE\n            falsy values are: 0, f, F, false, False, FALSE\n  -s string\n        REQUIRED SizeFinder MODE\n            size: 1MB,10MB,100MB,1GB,10GB,100GB,1TB\n```\n\n### No Keywords\n\nIf you just want to scan for file paths\n\n```bash\n$ scnnr -e .md -d .\nREADME.md\n```\n\nYou can also use equal signs for the flags:\n\n```bash\n$ scnnr -e=.md -d=.\nREADME.md\n```\n\nDo not provide any keywords and scnnr will return all given filepaths matching given extensions.\n\nIf no extensions are given, all filepaths will be returned in the scanned directory. This will walk all dirs!\n\n### Single Keyword\n\nScan this repo for markdown files with the keyword `cache=` in them.\n\n_With quotes_\n\n```bash\n$ scnnr -e \".md\" -d \".\" -k \"cache=\"\nREADME.md\n```\n\n_Without quotes (if no need to escape anything)_\n\n```bash\nscnnr -e .md -d . -k cache=\n```\n\n### Multiple Keywords and Multiple File Extensions\n\n```bash\n$ scnnr -d . -e .md,.go -k fileData,cache\nREADME.md\ncmd/scanner.go\n```\n\n# File Name Finder (NameFinder) (fnf)\n\n```\n  -p string\n        REQUIRED NameFinder MODE\n            any absolute path - can be comma delimited: Example: $HOME or '/tmp,/usr'\n  -f string\n        REQUIRED NameFinder MODE\n            fuzzy find the filename(s) contain(s) - can be comma delimited: Example 'wow' or 'wow,omg,lol'\n```\n\nExample use to search `/tmp`, `/etc`, `/usr`, and `$HOME/Documents` for filenames that contain:\n\n1. DEFCON\n1. Tax\n1. Return\n1. Finance\n\n```bash\nscnnr -m fnf -f DEFCON,Finance,Tax,Return -p /tmp,/usr,/etc,$HOME/Documents\n```\n\n# File Fingerprint Finder (FingerprintFinder) (fff)\n\n```\n  -d string\n        OPTIONAL Scnnr MODE and OPTIONAL FingerrintFinder MODE\n            directory where scnnr will scan\n            default is current directory and all child directories (default \".\")\n  -k string\n        OPTIONAL Scnnr MODE and REQUIRED FingerprintFinder\n            scnnr: this is a comma delimited list of characters to look for in a file\n                if no keywords are given - all file paths of given file extensions will be returned\n                if keywords are given - only filepaths of matches will be returned\n            FingerprintFinder: this is a comma delimited list of SHA2-256 hashes to find files by\n```\n\nExample use to find a file with a known hash:\n\n```bash\n$ known_hash=\"de4f51f97fa690026e225798ff294cd182b93847aaa46fe1e32b848eb9e985bd\"\n$ go run main.go -m fff -d $HOME/Documents -k $known_hash\n/home/selfup/Documents//dotfiles/mac/.bash_profile\n```\n\nPlease refer to the release notes for more details:\n\nGitHub: https://github.com/selfup/scnnr/releases/tag/v1.1.8\n\nGitlab: https://gitlab.com/selfup/scnnr/-/releases/v1.1.8\n\n# File Size Finder (SizeFinder) (fsf)\n\n```\n  -s string\n        REQUIRED SizeFinder MODE\n            size: 1MB,10MB,100MB,1GB,10GB,100GB,1TB\n```\n\nExample use to find any file over 100MB in E:/LotsOfStuff\n\n```bash\nscnnr -m fsf -s 100MB -d E:/LotsOfStuff\n```\n\n# Back to Scnnr\n\n### Using the package github.com/selfup/scnnr/pkg\n\n```go\nimport (\n  scnnr \"github.com/selfup/scnnr/pkg\"\n)\n\ndirectory := \"./artifact\"\nkeywords := []string{\"something\",\"something else\", \"another thing\"}\npatterns := []string{\".js\",\".go\",\".md\"}\n\nscanner := scnnr.Scanner{\n  Directory:      directory,\n  FileExtensions: patterns,\n  Keywords:       keywords,\n}\n\nerr := scanner.Scan()\n\nif err != nil {\n  log.Fatal(err)\n}\n```\n\n## Regex\n\n### Using Regex Patterns\n\n`scnnr -e \".js\" -d \"artifact\" -k \"cons?\" -r T \u003e .results`\n\nAccording to the godoc for `flag.BoolVar` you can use a few things for boolean flag values:\n\n`t, T, 1, true, True, TRUE`\n\n```\nscnnr $ time scnnr -r 1 -d artifact -e .js,.ts,.md -k 'cons*,let?,var?, impor*, expor*' \u003e .results\n\nreal    0m0.748s\nuser    0m2.398s\nsys     0m0.311s\n```\n\n### Using the package github.com/selfup/scnnr/pkg\n\n```go\nimport (\n  scnnr \"github.com/selfup/scnnr/pkg\"\n)\n\nrgx := true\ndirectory := \"./artifact\"\nkeywords := []string{\"const PASSW*\",\"password?\", \"export PASS?\"}\nextensions := []string{\".js\",\".ts\"}\n\nscanner := scnnr.Scanner{\n  Regex:          rgx,\n  Directory:      directory,\n  FileExtensions: extensions,\n  Keywords:       keywords,\n}\n\nerr := scanner.Scan()\n\nif err != nil {\n  log.Fatal(err)\n}\n```\n\n# Install\n\n### If you have Go\n\nGitHub repo:\n\n```bash\ngo install github.com/selfup/scnnr@latest\n```\n\nGitLab repo:\n\n```bash\ngo install gitlab.com/selfup/scnnr@latest\n```\n\n### If you do not have Go\n\n#### Release Binaries\n\nI have a [GitLab Release Repo](https://gitlab.com/selfup/scnnr) that builds the needed artifacts using [GitLabCI](https://docs.gitlab.com/ee/ci/quick_start/)\n\n#### Direct Download Link\n\nhttps://gitlab.com/selfup/scnnr/-/jobs/artifacts/master/download?job=release\n\n#### cURL\n\n```bash\ncurl -L https://gitlab.com/selfup/scnnr/-/jobs/artifacts/master/download?job=release \u003e artifacts.zip\n```\n\n#### wget\n\n```bash\nwget https://gitlab.com/selfup/scnnr/-/jobs/artifacts/master/download?job=release -O artifacts.zip\n```\n\n_the sha2-256 sum is provided in the artifact zip of the `scnnr_bins.zip`_\n\n_you can also verify the provided sum matches the output in CI (output for verification)_\n\n1. Unzip `artifacts.zip`\n1. Unzip `scnnr_bins.zip`\n\nFrom here pick your arch (mac/windows/linux) and appropriate binary and move to needed path! Mac and Linux builds have both intel and arm builds.\n\n```\nscnnr_bins/linux/intel:\nscnnr\n\nscnnr_bins/linux/arm:\nscnnr\n\nscnnr_bins/mac/intel:\nscnnr\n\nscnnr_bins/mac/arm:\nscnnr\n\nscnnr_bins/windows:\nscnnr.exe\n```\n\n### Docker\n\n1. Clone repo: `git clone https://github.com/selfup/scnnr`\n1. `cd` into repo\n\n   - Shell: `./scripts/dind.build.sh`\n   - Powershell: `./scripts/dind.build.ps1`\n\n1. Unzip `scnnr_bins.zip`\n\nFrom here pick your arch (mac/windows/linux) and appropriate binary and move to needed path!\n\n```\nscnnr_bins/linux/intel:\nscnnr\n\nscnnr_bins/linux/arm:\nscnnr\n\nscnnr_bins/mac/intel:\nscnnr\n\nscnnr_bins/mac/arm:\nscnnr\n\nscnnr_bins/windows:\nscnnr.exe\n```\n\n## Performance (scn)\n\nUse of goroutines, buffers, streams, mutexes, and simple checks.\n\nMemory in the following example never went above 5.5MB for the entire program.\n\nNo matches on 33k files after `npm i` for a JavaScript project as the `artifact`:\n\n```\n$ time scnnr -d artifact -e .kt -k cache\n\nreal    0m0.121s\nuser    0m0.053s\nsys     0m0.076s\n```\n\n33k files, two file types, one keyword, and 567 matches:\n\n```\n$ time scnnr -d artifact -e .md,.js -k cache \u003e .results\n\nreal    0m0.232s\nuser    0m0.574s\nsys     0m0.210s\n$ ls -lahg .results\n-rw-r--r-- 1 selfup 33K Jul 21 00:55 .results\n```\n\n33k files, two file types, 5 keywords, and 360 matches:\n\n```\n$ time scnnr -d artifact -e .js,.md -k stuff,things,wow,lol,omg \u003e .results\n\nreal    0m0.266s\nuser    0m0.813s\nsys     0m0.174s\n$ ls -lahg .results\n-rw-r--r-- 1 selfup 22K Jul 21 00:53 .results\n```\n\n33k files, 4 file types, 5 common keywords, and 18866 matches:\n\nResults are piped into a file to reduce noise.\n\nThe amount of file paths results in 1.2MB of text data..\n\n```\n$ time scnnr -d artifact -e .js,.ts,.md,.css -k const,let,var,import,export \u003e .results\n\nreal    0m0.344s\nuser    0m0.755s\nsys     0m0.351s\n$ ls -lahg .results\n-rw-r--r-- 1 selfup 1.2M Jul 21 00:57 .results\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fselfup%2Fscnnr","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fselfup%2Fscnnr","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fselfup%2Fscnnr/lists"}