{"id":24434937,"url":"https://github.com/bodo-run/yek","last_synced_at":"2025-05-13T22:02:57.948Z","repository":{"id":272112357,"uuid":"915358848","full_name":"bodo-run/yek","owner":"bodo-run","description":"A fast Rust based tool to serialize text-based files in a repository or directory for LLM consumption","archived":false,"fork":false,"pushed_at":"2025-04-21T21:27:37.000Z","size":673,"stargazers_count":2008,"open_issues_count":25,"forks_count":50,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-04-30T14:12:49.674Z","etag":null,"topics":["ai","cli","large-language-models","llm","rust-lang","serialization"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bodo-run.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-01-11T16:35:26.000Z","updated_at":"2025-04-30T08:06:29.000Z","dependencies_parsed_at":"2025-01-31T15:26:56.596Z","dependency_job_id":"5e2df350-1e68-46ef-9c93-81d97228ab4a","html_url":"https://github.com/bodo-run/yek","commit_stats":null,"previous_names":["mohsen1/yek","bodo-run/yek"],"tags_count":41,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bodo-run%2Fyek","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bodo-run%2Fyek/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bodo-run%2Fyek/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bodo-run%2Fyek/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bodo-run","download_url":"https://codeload.github.com/bodo-run/yek/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254036809,"owners_count":22003652,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","cli","large-language-models","llm","rust-lang","serialization"],"created_at":"2025-01-20T17:01:35.340Z","updated_at":"2025-05-13T22:02:57.894Z","avatar_url":"https://github.com/bodo-run.png","language":"Rust","readme":"# `yek`\n\nA [fast](#performance) Rust based tool to serialize text-based files in a repository or directory for LLM consumption.[^1]\n\nBy default:\n\n- Uses `.gitignore` rules to skip unwanted files.\n- Uses the Git history to infer what files are more important.\n- Infers additional ignore patterns (binary, large, etc.).\n- Automatically detects if output is being piped and streams content instead of writing to files.\n- Supports processing multiple directories in a single command.\n- Supports glob patterns and individual file selection.\n- Configurable via a `yek.yaml` file.\n\nYek \u003ca href=\"https://fa.wikipedia.org/wiki/۱\"\u003eيک\u003c/a\u003e means \"One\" in Farsi/Persian.\n\nConsider having a simple repo like this:\n\n```\n.\n├── README.md\n├── src\n│   ├── main.rs\n│   └── utils.rs\n└── tests\n    └── test.rs\n```\n\nRunning `yek` in this directory will produce a single file and write it to the temp directory with the following content:\n\n```txt\n\u003e\u003e\u003e\u003e README.md\n... content ...\n\u003e\u003e\u003e\u003e tests/test.rs\n... content ...\n\u003e\u003e\u003e\u003e src/utils.rs\n... content ...\n\u003e\u003e\u003e\u003e src/main.rs\n... content ...\n```\n\n\u003e [!NOTE]  \n\u003e `yek` will prioritize more important files to come last in the output. This is useful for LLM consumption since LLMs tend to pay more attention to content that appears later in the context.\n\n## Installation\n\nChoose the installation method for your platform:\n\n### Unix-like Systems (macOS, Linux)\n\n\u003c!-- UNIX_INSTALLATION_BEGIN --\u003e\n\n```bash\ncurl -fsSL https://bodo.run/yek.sh | bash\n```\n\n\u003c!-- UNIX_INSTALLATION_END --\u003e\n\nFor Windows (PowerShell):\n\n\u003c!-- WINDOWS_INSTALLATION_BEGIN --\u003e\n\n```powershell\nirm https://bodo.run/yek.ps1 | iex\n```\n\n\u003c!-- WINDOWS_INSTALLATION_END --\u003e\n\n\u003cdetails\u003e\n\u003csummary style=\"cursor: pointer;\"\u003eBuild from Source\u003c/summary\u003e\n\n```bash\ngit clone https://github.com/bodo-run/yek\ncd yek\ncargo install --path .\n```\n\n\u003c/details\u003e\n\n## Usage\n\n`yek` has sensible defaults, you can simply run `yek` in a directory to serialize the entire repository. It will serialize all files in the repository and write them into a temporary file. The path to the file will be printed to the console.\n\n### Examples\n\nProcess current directory and write to temp directory:\n\n```bash\nyek\n```\n\nPipe output to clipboard (macOS):\n\n```bash\nyek src/ | pbcopy\n```\n\nCap the max output size to 128K tokens:\n\n```bash\nyek --tokens 128k\n```\n\n\u003e [!NOTE]\n\u003e `yek` will remove any files that won't fit in the capped context size. It will try to fit in more important files\n\n```bash\nyek --max-size 100KB --output-dir /tmp/yek src/\n```\n\nProcess multiple directories:\n\n```bash\nyek src/ tests/\n```\n\nPorcess multiple files\n\n```bash\nyek file1.txt file2.txt file3.txt\n```\n\nUse glob patterns:\n\n```bash\nyek \"src/**/*.ts\"\n```\n\n```bash\nyek \"src/main.rs\" \"tests/*.rs\" \"docs/README.md\"\n```\n\n\u003e [!NOTE]\n\u003e When using glob patterns, make sure to quote them to prevent shell expansion.\n\n### CLI Reference\n\n```bash\nyek --help\nUsage: yek [OPTIONS] [input-dirs]...\n\nArguments:\n  [input-dirs]...\n\nOptions:\n      --no-config\n      --config-file \u003cCONFIG_FILE\u003e\n      --max-size \u003cMAX_SIZE\u003e                       [default: 10MB]\n      --tokens \u003cTOKENS\u003e\n      --json\n      --debug\n      --output-dir [\u003cOUTPUT_DIR\u003e]\n      --output-template \u003cOUTPUT_TEMPLATE\u003e         [default: \"\u003e\u003e\u003e\u003e FILE_PATH\\nFILE_CONTENT\"]\n      --ignore-patterns \u003cIGNORE_PATTERNS\u003e...\n      --unignore-patterns \u003cUNIGNORE_PATTERNS\u003e...\n  -h, --help                                      Print help\n```\n\n## Configuration File\n\nYou can place a file called `yek.yaml` at your project root or pass a custom path via `--config`. The configuration file allows you to:\n\n1. Add custom ignore patterns\n2. Define file priority rules for processing order\n3. Add additional binary file extensions to ignore (extends the built-in list)\n4. Configure Git-based priority boost\n5. Define output directory\n6. Define output template\n\n### Example `yek.yaml`\n\nYou can also use `yek.toml` or `yek.json` instead of `yek.yaml`.\n\nThis is optional, you can configure the `yek.yaml` file at the root of your project.\n\n```yaml\n# Add patterns to ignore (in addition to .gitignore)\nignore_patterns:\n  - \"ai-promots/**\"\n  - \"__generated__/**\"\n\n# Configure Git-based priority boost (optional)\ngit_boost_max: 50 # Maximum score boost based on Git history (default: 100)\n\n# Define priority rules for processing order\n# Higher scores are processed first\npriority_rules:\n  - score: 100\n    pattern: \"^src/lib/\"\n  - score: 90\n    pattern: \"^src/\"\n  - score: 80\n    pattern: \"^docs/\"\n\n# Add additional binary file extensions to ignore\n# These extend the built-in list (.jpg, .png, .exe, etc.)\nbinary_extensions:\n  - \".blend\" # Blender files\n  - \".fbx\" # 3D model files\n  - \".max\" # 3ds Max files\n  - \".psd\" # Photoshop files\n\n# Define output directory\noutput_dir: /tmp/yek\n\n# Define output template.\n# FILE_PATH and FILE_CONTENT are expected to be present in the template.\noutput_template: \"{{{FILE_PATH}}}\\n\\nFILE_CONTENT\"\n```\n\nAll configuration keys are optional. By default:\n\n- No extra ignore patterns, only the ones from `.gitignore` are used.\n- All files have equal priority (score: 1)\n- Git-based priority boost maximum is 100\n- Common binary file extensions are ignored (.jpg, .png, .exe, etc. - see source for full list)\n\n## Performance\n\n`yek` is fast. It's written in Rust and does many things in parallel to speed up processing.\n\nHere is a benchmark comparing it to [Repomix](https://github.com/yamadashy/repomix) serializing the [Next.js](https://github.com/vercel/next.js) project:\n\n```bash\ntime yek\nExecuted in    5.19 secs    fish           external\n   usr time    2.85 secs   54.00 micros    2.85 secs\n   sys time    6.31 secs  629.00 micros    6.31 secs\n```\n\n```bash\ntime repomix\nExecuted in   22.24 mins    fish           external\n   usr time   21.99 mins    0.18 millis   21.99 mins\n   sys time    0.23 mins    1.72 millis    0.23 mins\n```\n\n`yek` is **230x faster** than `repomix`.\n\n## Roadmap\n\nSee [proposed features](https://github.com/bodo-run/yek/issues?q=type:%22Feature%22). I am open to accepting new feature requests. Please write a detailed proposal to discuss new features.\n\n## Alternatives\n\n- [Repomix](https://github.com/yamadashy/repomix): A tool to serialize a repository into a single file in a similar way to `yek`.\n- [Aider](https://aider.chat): A full IDE like experience for coding using AI\n\n## License\n\n[MIT](LICENSE)\n\n[^1]: `yek` is not \"blazingly\" fast. It's just fast, as fast as your computer can be.\n","funding_links":[],"categories":["Rust","HarmonyOS","A01_文本生成_文本对话","\u003ca name=\"text-processing\"\u003e\u003c/a\u003eText processing"],"sub_categories":["Windows Manager","大语言对话模型及数据"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbodo-run%2Fyek","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbodo-run%2Fyek","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbodo-run%2Fyek/lists"}