{"id":26189101,"url":"https://github.com/johnlindquist/file-forge","last_synced_at":"2025-04-15T01:34:32.090Z","repository":{"id":274536176,"uuid":"923226879","full_name":"johnlindquist/file-forge","owner":"johnlindquist","description":"GitHub and local directory ingestion CLI tool","archived":false,"fork":false,"pushed_at":"2025-04-09T23:56:56.000Z","size":835,"stargazers_count":15,"open_issues_count":12,"forks_count":3,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-10T00:32:08.297Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/johnlindquist.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-01-27T21:12:54.000Z","updated_at":"2025-04-09T23:56:23.000Z","dependencies_parsed_at":"2025-02-06T16:57:35.166Z","dependency_job_id":"4891912a-d86a-4029-9329-0871d6d11ee1","html_url":"https://github.com/johnlindquist/file-forge","commit_stats":null,"previous_names":["johnlindquist/ghi","johnlindquist/file-forge"],"tags_count":107,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/johnlindquist%2Ffile-forge","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/johnlindquist%2Ffile-forge/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/johnlindquist%2Ffile-forge/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/johnlindquist%2Ffile-forge/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/johnlindquist","download_url":"https://codeload.github.com/johnlindquist/file-forge/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248989253,"owners_count":21194558,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-03-12T00:30:31.997Z","updated_at":"2025-04-15T01:34:32.081Z","avatar_url":"https://github.com/johnlindquist.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# File Forge\n\n**File Forge** is a powerful command‑line tool for deep analysis of codebases. It scans your project (or GitHub repository) and generates comprehensive markdown reports that include a summary, a visual directory structure, file contents, and even dependency graphs. These reports are designed to feed AI reasoning models and support a variety of advanced use cases.\n\n\u003e **Note:** File Forge works both with GitHub URLs (by cloning/updating a cached repository) and local directories. It also supports filtering files by patterns, searching for specific content, and advanced output options like piping, clipboard copy, and XML wrapping.\n\n## Features\n\n- **Comprehensive Analysis**: Automatically generate detailed reports with a summary, directory structure, and full file content.\n- **Smart Filtering**: Use glob patterns with `--include` and `--exclude` to select exactly which files to analyze.\n- **Content Search**: Find files by name using `--find` (OR behavior) or require specific strings in file content using `--require` (AND behavior).\n- **Git Integration**: Analyze specific branches (`--branch`) or commits (`--commit`) for GitHub-hosted repositories.\n- **Dependency Graphs**: Generate visual dependency graphs using the `--graph` flag.\n- **Flexible Output Options**: \n  - Pipe output to stdout with `--pipe`\n  - Open results in your default editor with `--open`\n  - Copy output to clipboard using `--clipboard`\n  - Generate Markdown output with `--markdown` (default output is XML)\n  - Control whitespace/indentation with `--whitespace`\n  - Preview output in terminal without saving using `--dry-run` / `-D`\n- **Advanced Modes**:\n  - **Bulk Mode**: Append AI processing instructions with `--bulk`\n  - **Debug/Verbose**: Enable additional logging with `--debug` or `--verbose`\n  - **AI Templates**: Apply prompt templates for AI processing with `--template`\n- **Token Limit:** Prevents processing excessively large projects (over ~200k tokens) by default. Use `--allow-large` to override this check for very large codebases.\n- **Configuration File**: Define default options and reusable named command sets in `ffg.config.jsonc`.\n\n## Installation\n\nInstall **File Forge** globally with your favorite package manager:\n\n```bash\n# Using pnpm:\npnpm add -g @johnlindquist/file-forge\n\n# Or using npm:\nnpm install -g @johnlindquist/file-forge\n```\n\n## Usage Examples\n\n### Analyze Current Directory\n```bash\n# Analyze all files in current directory\nffg\n\n# Analyze specific directory/files in current directory\nffg src\nffg \"src/**/*.ts\"\n```\n\n### Analyze a GitHub Repository\n```bash\nffg https://github.com/owner/repo --branch develop\n```\n\n### Analyze a Local Directory\n```bash\nffg /path/to/local/project\n```\n\n### Filter Files by Pattern\n\nUse glob patterns to precisely control which files are included or excluded.\n\n```bash\n# Include only TypeScript files in the root of the current directory\nffg --include \"*.ts\"\n\n# Include TypeScript files recursively in all subdirectories\nffg --include \"**/*.ts\"\n\n# Exclude node_modules and test files recursively\nffg --exclude \"**/*test.*,node_modules/**\"\n\n# Combine include and exclude for specific filtering in the 'src' directory\nffg src --include \"**/*.ts\" --exclude \"**/*.spec.ts,**/__tests__/**\"\n```\n\n**Note:**\n- A pattern like `*.ts` matches files only in the immediate directory being scanned.\n- Use `**/*` to match files recursively through all subdirectories (e.g., `**/*.ts`).\n- You can provide multiple patterns separated by commas (e.g., `\"*.ts,*.js\"`) or by using multiple `--include` / `--exclude` flags (e.g., `--include \"*.ts\" --include \"*.js\"`).\n- You can also include specific files using their absolute paths alongside glob patterns (e.g., `ffg --include \"**/*.ts,/Users/me/Documents/important_config.json\"`).\n\n### Search for Specific Content\n- **Find files containing ANY of the terms:**\n  ```bash\n  ffg src --find \"console,debug\"\n  ```\n- **Require files to have ALL of the terms:**\n  ```bash\n  ffg src --require \"console,log\"\n  ```\n\n### Generate a Dependency Graph\n```bash\nffg --graph src/index.js\n```\n\n### Advanced Options\n- **Pipe output:**\n  ```bash\n  ffg src --pipe\n  ```\n- **Copy to clipboard:**\n  ```bash\n  ffg src --clipboard\n  ```\n- **Generate Markdown output (default is XML):**\n  ```bash\n  ffg src --markdown\n  ```\n- **Enable whitespace/indentation in output:**\n  ```bash\n  ffg src --whitespace\n  ```\n- **Preview output without saving (Dry Run):**\n  ```bash\n  ffg src --dry-run\n  # Alias\n  ffg src -D\n  ```\n- **Allow large projects (override token limit):**\n  ```bash\n  ffg /path/to/very/large/project --allow-large\n  ```\n- **Bulk Analysis Mode:**\n  ```bash\n  ffg src --bulk\n  ```\n- **Enable Debug/Verbose Output:**\n  ```bash\n  ffg src --debug\n  ffg src --verbose\n  ```\n- **Command Traceability:**\n  The original command used to generate the output is now included in the XML output for better traceability.\n  ```xml\n  \u003cproject\u003e\n    \u003csource\u003e/path/to/source\u003c/source\u003e\n    \u003ctimestamp\u003e20240324-123456\u003c/timestamp\u003e\n    \u003ccommand\u003effg src --verbose\u003c/command\u003e\n  \u003c/project\u003e\n  ```\n- **Using AI Prompt Templates:**\n  ```bash\n  # List all available templates\n  ffg --list-templates\n\n  # Apply a specific template\n  ffg src --template refactor\n\n  # Combine with other options\n  ffg src --template test --include \"*.js\" --exclude \"*.test.js\"\n  ```\n\n## Configuration File (`ffg.config.jsonc`)\n\nFile Forge supports a configuration file named `ffg.config.jsonc` (or `ffg.config.json`) in the root of your project (the directory where you run `ffg`). This allows you to define default options and create reusable, named command configurations.\n\n**Features:**\n\n-   **Default Command:** Specify a `defaultCommand` object whose flags will be applied automatically whenever `ffg` is run without the `--use` flag.\n-   **Named Commands:** Define multiple named command configurations under the `commands` object. Invoke a specific named command using the `--use \u003ccommand-name\u003e` flag.\n-   **JSONC Support:** Use comments in your `ffg.config.jsonc` file for better documentation.\n-   **Precedence:** Command-line flags always take precedence over configuration file settings. If using `--use \u003cname\u003e`, the named command's settings override `defaultCommand` settings.\n-   **Array Merging:** For array flags like `--include`, `--exclude`, `--find`, `--require`, and `--extension`, values provided on the command line are *merged* with (added to) the values defined in the applied configuration (either `defaultCommand` or the named command used with `--use`).\n-   **Saving Commands (`--save` / `--save-as`):** You can easily save the current set of command-line flags to your `ffg.config.jsonc` file:\n    -   `--save`: Saves the current flags (excluding transient flags like `--pipe`, `--debug`, etc.) as the new `defaultCommand`, overwriting any existing default.\n    -   `--save-as \u003cname\u003e`: Saves the current flags under a new named command (or overwrites an existing one with the same name) in the `commands` section.\n\n**Example `ffg.config.jsonc`:**\n\n```jsonc\n{\n  // Default settings applied when no --use flag is given\n  \"defaultCommand\": {\n    \"exclude\": [\"node_modules/**\", \"dist/**\", \"*.log\"],\n    \"ignore\": true, // Respect .gitignore by default\n    \"skipArtifacts\": true\n  },\n  // Reusable named command configurations\n  \"commands\": {\n    \"ts-analysis\": {\n      \"include\": [\"src/**/*.ts\"],\n      \"exclude\": [\"**/*.test.ts\", \"**/*.spec.ts\"], // Merged with defaultCommand exclude on use\n      \"require\": [\"import\", \"export\"],\n      \"verbose\": true\n    },\n    \"docs-only\": {\n      \"include\": [\"**/*.md\", \"docs/**\"],\n      \"markdown\": true\n    },\n    \"find-todos\": {\n      // Inherits defaults from defaultCommand if not overridden\n      \"find\": [\"TODO\", \"FIXME\"],\n      \"include\": [\"src/**\", \"scripts/**\"]\n    }\n  }\n}\n```\n\n**Usage with Config:**\n\n```bash\n# Uses defaultCommand settings (exclude node_modules, etc.)\nffg .\n\n# Uses the 'ts-analysis' named command settings\nffg --use ts-analysis\n\n# Uses 'ts-analysis' but overrides verbose and adds another include pattern\nffg --use ts-analysis --verbose=false --include \"tests/**/*.ts\"\n# Resulting flags: include=[\"src/**/*.ts\", \"tests/**/*.ts\"], exclude=[...], require=[...], verbose=false\n\n# Uses 'find-todos' command\nffg --use find-todos\n\n# Save current flags as the new default command\nffg . --include \"**/*.ts\" --exclude \"node_modules/**\" --verbose --save\n# This will update ffg.config.jsonc's \"defaultCommand\"\n\n# Save current flags as a named command 'test-setup'\nffg . --require \"test\" --exclude \"dist/**\" --save-as test-setup\n# This will add or overwrite \"test-setup\" in ffg.config.jsonc's \"commands\"\n```\n\n## AI Prompt Templates\n\nFile Forge includes a set of prompt templates that can be applied to your analysis results. These templates are designed to guide AI models (like GPT-4 or Claude) in performing specific tasks on your code.\n\n### Available Template Categories\n\n- **Documentation \u0026 Explanation**\n  - `explain`: Summarize what a code file does in plain language\n  - `document`: Insert explanatory comments into the code\n\n- **Refactoring \u0026 Improvement**\n  - `refactor`: Improve code clarity and maintainability without changing behavior\n  - `optimize`: Improve code efficiency without changing behavior\n  - `fix`: Find potential bugs or issues and fix them\n\n- **Code Generation**\n  - `test`: Generate unit tests for the given code\n\n### Using Templates\n\n1. List all available templates:\n   ```bash\n   ffg --list-templates\n   ```\n\n2. Apply a template to your analysis:\n   ```bash\n   ffg src --template refactor\n   ```\n\n3. The template will be included in the output, with your code analysis inserted in the appropriate place.\n\n4. When you view the output in an editor or copy it to the clipboard, you can then paste it to an AI assistant to get the desired result.\n\n### Customizing Templates\n\nYou can create your own templates or override the built-in ones by creating individual template files in your File Forge configuration directory:\n\n- **macOS:** `~/Library/Preferences/@johnlindquist/file-forge/templates/`\n- **Linux:** `~/.config/@johnlindquist/file-forge/templates/`\n- **Windows:** `%APPDATA%/@johnlindquist/file-forge/templates/`\n\nEach template is a Markdown (`.md`) file with front-matter containing metadata and a body containing the template content. Here's an example template `custom-explain.md`:\n\n```markdown\n---\nname: custom-explain\ncategory: documentation\ndescription: My custom explanation template\n---\n\n**Goal:** Explain this code in simple terms.\n\n**Context:**  \n{{ code }}\n\n\u003cinstructions\u003e\n- Explain what this code does in simple language\n- Focus on the main functionality\n\u003c/instructions\u003e\n\n\u003ctask\u003e\nProvide a clear explanation of the code for a non-technical audience.\n\u003c/task\u003e\n```\n\nWhen you run File Forge, it will automatically load and merge your custom templates with the built-in ones.\n\n## Viewing Help\n\nFor a complete list of options and examples, run:\n```bash\nffg --help\n```\n\n## Configuration \u0026 Development\n\nAnalysis results and configuration are stored in:\n- **macOS:** `~/Library/Preferences/@johnlindquist/file-forge/config/`\n- **Linux:** `~/.config/@johnlindquist/file-forge/config/`\n- **Windows:** `%APPDATA%/@johnlindquist/file-forge/config/`\n\nTo contribute or run File Forge locally:\n```bash\ngit clone https://github.com/johnlindquist/file-forge.git\ncd @johnlindquist/file-forge\npnpm install\npnpm build\n```\n\n## License\n\nThis project is licensed under the MIT License – see the LICENSE file for details.\n\n## Test Optimization\n\nThe test suite has been optimized to run faster using several strategies:\n\n### Parallel Test Execution\n\nTests can now run concurrently, leveraging multiple CPU cores for faster test execution. This is enabled in `vitest.config.ts` by removing the `sequence: { concurrent: false }` option.\n\n### Direct Function Testing\n\nInstead of spawning a new process for each test (which is slow), many tests now use the direct function testing approach through `utils/directTestRunner.ts`. This utility allows tests to call the main application functions directly while capturing their output.\n\nBenefits:\n- Much faster test execution (typically 5-10x faster)\n- Same test coverage and assertions\n- No process spawning overhead\n\nTo migrate an existing test to use direct function testing:\n\n1. Run the test analysis to see candidates for optimization:\n   ```\n   pnpm test:optimize:analyze\n   ```\n\n2. Migrate a specific test file:\n   ```\n   pnpm test:optimize test/your-test-file.test.ts\n   ```\n\n3. Verify the test still passes and check the performance improvement.\n\n### Batch Migration\n\nTo optimize multiple tests at once, you can use the batch processing options:\n\n1. Analyze and run a dry-run batch (no changes applied):\n   ```\n   pnpm test:optimize:batch:dry\n   ```\n\n2. Fast batch migration (skips performance measurement):\n   ```\n   pnpm test:optimize:batch:fast\n   ```\n\n3. Full batch migration with performance measurements:\n   ```\n   pnpm test:optimize:batch\n   ```\n\nAdditional options:\n- Use `--force` to re-optimize already migrated files\n- Use `--dry-run` to see what would be changed without making actual changes\n- Use `--skip-measure` to skip performance measurements (faster)\n\n### Best Practices\n\n- Use `beforeAll` instead of `beforeEach` when the setup only needs to be done once for all tests in a describe block\n- For expensive file operations, consider mocking the filesystem\n- Use the `waitForFile` helper with appropriate timeouts for file-based tests\n- Run optimized test files with `pnpm test:direct` to verify they still work\n\n### Additional Tools\n\n- `scripts/optimize-tests.js`: Helper script to analyze and optimize test files\n- `utils/directTestRunner.ts`: Direct function execution utility\n- `test/helpers/fileWaiter.ts`: Optimized file waiting utility with adaptive polling\n\n### Inspiration\n\n- https://gitingest.com/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjohnlindquist%2Ffile-forge","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjohnlindquist%2Ffile-forge","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjohnlindquist%2Ffile-forge/lists"}