{"id":25038991,"url":"https://github.com/edu-ap/knowledge-synthesizer","last_synced_at":"2026-03-04T06:31:42.315Z","repository":{"id":276033572,"uuid":"927992721","full_name":"edu-ap/knowledge-synthesizer","owner":"edu-ap","description":"A Python tool that integrates with Daniel Miessler's Fabric project, allowing you to apply its powerful AI patterns to your local content in bulk.","archived":false,"fork":false,"pushed_at":"2025-03-20T18:14:48.000Z","size":43,"stargazers_count":2,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-01T15:22:32.155Z","etag":null,"topics":["ai","cli","fabric","gpt","knowledge-management","nlp","openai","python"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/edu-ap.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-02-05T21:59:06.000Z","updated_at":"2025-04-11T19:49:31.000Z","dependencies_parsed_at":null,"dependency_job_id":"f941a6a2-2ed6-4b72-b2fa-39ad7490c6de","html_url":"https://github.com/edu-ap/knowledge-synthesizer","commit_stats":null,"previous_names":["edu-ap/knowledge-synthesizer"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/edu-ap%2Fknowledge-synthesizer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/edu-ap%2Fknowledge-synthesizer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/edu-ap%2Fknowledge-synthesizer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/edu-ap%2Fknowledge-synthesizer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/edu-ap","download_url":"https://codeload.github.com/edu-ap/knowledge-synthesizer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251895741,"owners_count":21661355,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","cli","fabric","gpt","knowledge-management","nlp","openai","python"],"created_at":"2025-02-06T02:07:46.614Z","updated_at":"2026-03-04T06:31:37.267Z","avatar_url":"https://github.com/edu-ap.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Knowledge Synthesizer\n\n[![CI](https://github.com/edu-ap/knowledge-synthesizer/actions/workflows/ci.yml/badge.svg)](https://github.com/edu-ap/knowledge-synthesizer/actions/workflows/ci.yml)\n[![codecov](https://codecov.io/gh/edu-ap/knowledge-synthesizer/branch/main/graph/badge.svg)](https://codecov.io/gh/edu-ap/knowledge-synthesizer)\n[![Python 3.9+](https://img.shields.io/badge/python-3.9+-blue.svg)](https://www.python.org/downloads/)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)\n[![security: bandit](https://img.shields.io/badge/security-bandit-yellow.svg)](https://github.com/PyCQA/bandit)\n[![Checked with mypy](https://www.mypy-lang.org/static/mypy_badge.svg)](https://mypy-lang.org/)\n\nA Python tool that integrates with Daniel Miessler's groundbreaking [Fabric](https://github.com/danielmiessler/fabric) project, allowing you to apply its powerful AI patterns to your local content.\n\n## About Fabric\n\nFabric, created by Daniel Miessler, is a revolutionary framework that provides a collection of carefully crafted prompts (patterns) for extracting deep insights from content using AI. These patterns represent years of expertise in prompt engineering and knowledge synthesis, making them invaluable for anyone working with AI.\n\nThis tool builds upon Fabric's foundation by:\n\n1. Directly accessing Fabric's latest patterns from GitHub\n2. Making them easily applicable to local content through Python\n3. Preserving the original patterns while adding automation capabilities\n\n## Why Python Integration?\n\nWhile Fabric's patterns are powerful on their own, this Python integration offers several advantages:\n\n- **Batch Processing**: Apply patterns to multiple files automatically\n- **Local Content**: Process your documents without copying/pasting into chat interfaces\n- **Customization**: Easily modify output formats and processing workflows\n- **Version Control**: Track changes to synthesized content over time\n- **Automation**: Integrate with existing tools and workflows\n- **Cost Control**: Monitor and manage [API usage](https://platform.openai.com/usage) efficiently\n\n## Features\n\n- Direct integration with [Fabric's pattern repository](https://github.com/danielmiessler/fabric)\n- Support for multiple [OpenAI models](https://platform.openai.com/docs/models) (GPT-4, GPT-4 Turbo, GPT-3.5)\n- Interactive pattern selection from Fabric's collection\n- Configurable input/output formats\n- [Rate limiting](https://platform.openai.com/docs/guides/rate-limits) and error handling\n- Automated [GitHub-based](https://docs.github.com/en/rest) pattern updates\n\n## Architecture\n\n### Project Structure\n\nThe codebase is organized into the following structure:\n\n```mermaid\ngraph TD\n    A[knowledge-synthesizer/] --\u003e B[src/]\n    A --\u003e C[tests/]\n    A --\u003e D[docs/]\n    B --\u003e E[knowledge_synthesizer/]\n    E --\u003e F[cli.py\u003cbr/\u003eCommand-line interface]\n    E --\u003e G[synthesizer.py\u003cbr/\u003eCore functionality]\n    E --\u003e H[__init__.py\u003cbr/\u003ePackage initialization]\n    C --\u003e I[test_cli.py]\n    C --\u003e J[test_synthesizer.py]\n    C --\u003e K[conftest.py]\n```\n\n### Data Flow\n\nThis diagram shows how data flows through the system:\n\n```mermaid\nsequenceDiagram\n    participant User\n    participant CLI\n    participant Synthesizer\n    participant GitHub\n    participant Cache\n    participant OpenAI\n\n    User-\u003e\u003eCLI: Run command\u003cbr/\u003e(knowledge-synthesizer file.md)\n    CLI-\u003e\u003eSynthesizer: Initialize\n    \n    alt First Run\n        Synthesizer-\u003e\u003eGitHub: Fetch Fabric patterns\n        GitHub--\u003e\u003eSynthesizer: Return patterns\n        Synthesizer-\u003e\u003eCache: Store patterns\n    else Cache Available\n        Synthesizer-\u003e\u003eCache: Check for patterns\n        Cache--\u003e\u003eSynthesizer: Return cached patterns\n    end\n\n    Synthesizer-\u003e\u003eUser: Display available patterns\n    User-\u003e\u003eSynthesizer: Select patterns\n    \n    loop For Each Pattern\n        Synthesizer-\u003e\u003eOpenAI: Send content + pattern\n        OpenAI--\u003e\u003eSynthesizer: Return analysis\n    end\n    \n    Synthesizer-\u003e\u003eUser: Save results to file\n```\n\n### API Integration\n\nHow the tool interacts with external services:\n\n```mermaid\nflowchart LR\n    A[Your Content] --\u003e B[Knowledge Synthesizer]\n    B --\u003e C{Cache Check}\n    C --\u003e|Cache Miss| D[GitHub API]\n    D --\u003e|Fetch Patterns| E[Pattern Cache]\n    C --\u003e|Cache Hit| E\n    E --\u003e F[OpenAI API]\n    F --\u003e|Process Content| G[Results]\n    G --\u003e|Save| H[Output Files]\n\n    style B fill:#f9f,stroke:#333,stroke-width:2px\n    style D fill:#bbf,stroke:#333,stroke-width:2px\n    style F fill:#bfb,stroke:#333,stroke-width:2px\n```\n\n### Environment Setup\n\nSteps to get started:\n\n```mermaid\ngraph LR\n    A[Prerequisites] --\u003e|Install| B[Python]\n    A --\u003e|Install| C[Git]\n    A --\u003e|Install| D[Xcode CLI]\n    \n    B \u0026 C \u0026 D --\u003e E[Setup Project]\n    \n    E --\u003e|Create| F[Virtual Environment]\n    E --\u003e|Clone| G[Repository]\n    \n    F \u0026 G --\u003e H[Install Dependencies]\n    \n    H --\u003e I[Configure API Key]\n    I --\u003e J[Ready to Use]\n\n    style J fill:#9f9,stroke:#333,stroke-width:2px\n```\n\n## Installation\n\n### Prerequisites for macOS Users\n\nIf you're new to coding on macOS, follow these steps first:\n\n1. **Install Command Line Tools**:\n   - Open Terminal (press `Cmd + Space`, type \"Terminal\", press Enter)\n   - Run this command: `xcode-select --install`\n   - Click \"Install\" when prompted\n   - Wait for the installation to complete\n\n2. **Install [Python](https://www.python.org/downloads/)**:\n   - Visit [Python's website](https://www.python.org/downloads/)\n   - Download the latest Python installer for macOS\n   - Open the downloaded `.pkg` file and follow the installer\n   - Verify installation by opening Terminal and running: `python3 --version`\n\n3. **Install [Git](https://git-scm.com/)**:\n   - If not installed, visit [Git's website](https://git-scm.com/download/mac)\n   - Download and install Git for macOS\n   - Verify installation: `git --version`\n\n### Installation Steps\n\nOpen Terminal and run these commands (you can copy/paste them):\n\n```bash\n# Clone the repository\ngit clone https://github.com/edu-ap/knowledge-synthesizer.git\n\n# Navigate to the project directory\ncd knowledge-synthesizer\n\n# Create a virtual environment\npython3 -m venv .venv\n\n# Activate the virtual environment\nsource .venv/bin/activate\n\n# Your prompt should now start with (.venv)\n# Install dependencies and the package\npip install -r requirements.txt\npip install -e .\n```\n\n### Setting Up Your API Key\n\n1. Get an OpenAI API key:\n   - Visit [OpenAI's website](https://platform.openai.com/)\n   - Sign up or log in\n   - Go to API settings\n   - Create a new API key\n\n2. Set up your key (choose one method):\n\n   **Option 1 - Using a .env file (recommended for beginners):**\n   ```bash\n   # Create .env file and open it in TextEdit\n   touch .env\n   open -a TextEdit .env\n   ```\n   - In TextEdit, add this line (replace with your actual key):\n   ```\n   OPENAI_API_KEY=your-key-here\n   ```\n   - Save and close TextEdit\n\n   **Option 2 - Using Terminal:**\n   ```bash\n   # Export as environment variable\n   export OPENAI_API_KEY=your-key-here\n   ```\n\n### Verifying Installation\n\nRun this command to verify everything is working:\n```bash\nknowledge-synthesizer --version\n```\n\nYou should see the version number displayed.\n\n## Usage\n\nAfter installation, you can run the Knowledge Synthesizer using these commands:\n\n```bash\n# Basic usage - process a markdown file\nknowledge-synthesizer your-file.md\n\n# Process all markdown files in a folder\nknowledge-synthesizer your-folder\n\n# See what would happen without making changes\nknowledge-synthesizer your-file.md --dry-run\n```\n\n### Common Commands Explained\n\n```bash\n# Process a single markdown file\nknowledge-synthesizer document.md\n\n# Process all markdown files in a directory\nknowledge-synthesizer docs/\n\n# Process files recursively (including subfolders)\nknowledge-synthesizer docs/ -r --pattern \"*.txt\"\n\n# Preview without making changes (safe to try!)\nknowledge-synthesizer docs/ --dry-run\n\n# Save each pattern's output separately\nknowledge-synthesizer docs/ --separate\n\n# Force regeneration of existing outputs\nknowledge-synthesizer docs/ --force\n\n# Use a custom output directory and suffix\nknowledge-synthesizer docs/ --output \"analysis\" --suffix \"_insights\"\n```\n\n### Troubleshooting Common Issues\n\n1. **Command not found**:\n   ```bash\n   # Make sure you're in the virtual environment\n   source .venv/bin/activate\n   ```\n\n2. **Permission denied**:\n   ```bash\n   # Make sure you can execute the script\n   chmod +x run_tests.sh\n   ```\n\n3. **API key not working**:\n   - Check if your .env file is in the correct directory\n   - Make sure there are no spaces around the = sign\n   - Try printing the key: `echo $OPENAI_API_KEY`\n\n## Configuration\n\n1. Set your OpenAI API key:\n   - Export as environment variable: `export OPENAI_API_KEY=your-key`\n   - Or create a `.env` file with: `OPENAI_API_KEY=your-key`\n\n2. The tool will prompt you to:\n   - Select an OpenAI model\n   - Choose which Fabric patterns to apply\n   - Confirm processing options\n\n## Pattern Caching\n\nTo optimize performance and reduce GitHub API calls, the tool caches Fabric patterns locally:\n\n- Patterns are cached in `~/.knowledge_synthesizer/cache/`\n- Cache expires after 24 hours\n- Use `--skip-cache` to force downloading fresh patterns\n- Cache is automatically created and managed\n\n## Output Formats\n\nThe tool supports two output formats:\n\n1. **Combined Output** (default):\n   - All pattern results are saved in a single markdown file\n   - Each pattern's output is preceded by a level-1 heading\n   - Files are named: `{input_file}{suffix}.md`\n\n2. **Separate Files** (with `--separate`):\n   - Each pattern's output is saved in its own file\n   - Files are named: `{input_file}_{pattern_name}.md`\n   - Includes source file reference in each output\n\n## Examples\n\n```bash\n# Process a single markdown file\nknowledge-synthesizer document.md\n\n# Process all markdown files in a directory\nknowledge-synthesizer docs/\n\n# Process files recursively with a specific pattern\nknowledge-synthesizer docs/ -r --pattern \"*.txt\"\n\n# Preview what would be processed without making changes\nknowledge-synthesizer docs/ --dry-run\n\n# Save each pattern's output separately\nknowledge-synthesizer docs/ --separate\n\n# Force regeneration of all outputs\nknowledge-synthesizer docs/ --force\n\n# Custom output directory and suffix\nknowledge-synthesizer docs/ --output \"analysis\" --suffix \"_insights\"\n```\n\n## Development\n\nFor development setup and contribution guidelines, please see [CONTRIBUTING.md](CONTRIBUTING.md).\n\n### Making Code Changes\n\nAfter making changes to the Python code, you need to reinstall the package for the changes to take effect:\n\n```bash\npip install -e .  # Don't forget the dot (.) at the end!\n```\n\nThe `-e .` means:\n- `-e`: Install in [\"editable\" or \"development\" mode](https://pip.pypa.io/en/stable/cli/pip_install/#editable-installs)\n- `.`: Install from the current directory\n\nThis reinstallation is necessary when you:\n- Make changes to Python code files (`.py` files)\n- Add new Python files\n- Change package dependencies in `pyproject.toml`\n- Modify package structure\n\nYou don't need to reinstall when you:\n- Change content files (like `.md` files)\n- Change configuration files (like `.env`)\n- Change documentation\n\nThe editable mode creates a special link to your source code instead of copying it, making it easier to test changes during development.\n\n### Running Tests\n\nThe project uses [pytest](https://docs.pytest.org/) for testing. Here's how to run tests on macOS:\n\n```bash\n# First, make sure you're in the project directory\ncd path/to/knowledge-synthesizer\n\n# Activate the virtual environment if not active\nsource .venv/bin/activate\n\n# Run all tests with the test runner script\n./run_tests.sh\n\n# If you get a permission error, make it executable:\nchmod +x run_tests.sh\n./run_tests.sh\n\n# Or run tests directly with pytest\npytest\n\n# Run tests with coverage report\npytest --cov=src/knowledge_synthesizer --cov-report=html\n\n# View the coverage report in your browser\nopen htmlcov/index.html\n```\n\n### Directory Structure\n\nWhen running the Knowledge Synthesizer, several directories will be created:\n\n1. **Output Directories** (in your current folder):\n   - `synthesis/`: Where your processed files go\n   - Example: If you process `document.md`, you'll get `synthesis/document_synthesis.md`\n\n2. **Cache Directory** (in your home folder):\n   - `~/.knowledge_synthesizer/cache/`: Stores downloaded patterns\n   - To find it: `open ~/.knowledge_synthesizer`\n   - Cache files expire after 24 hours\n   - Safe to delete if you need space\n\n3. **Development Directories** (hidden, in project folder):\n   - `.pytest_cache/`: [Test cache](https://docs.pytest.org/en/stable/cache.html)\n   - `htmlcov/`: [Test coverage reports](https://coverage.readthedocs.io/)\n   - `.venv/`: [Python virtual environment](https://docs.python.org/3/library/venv.html)\n   - `__pycache__/`: [Python cache](https://docs.python.org/3/tutorial/modules.html#compiled-python-files)\n   - `*.egg-info/`: [Package metadata](https://setuptools.pypa.io/en/latest/userguide/development_mode.html)\n   - All these are managed automatically and safe to delete\n\nTo see hidden folders in Finder:\n- Press `Cmd + Shift + .` in Finder\n- Or use Terminal: `ls -la`\n\n## Credits\n\nThis tool would not be possible without Daniel Miessler's Fabric project. All patterns used in this tool are sourced directly from the [Fabric repository](https://github.com/danielmiessler/fabric) and are subject to its license terms. We encourage users to:\n\n1. Star and support the original [Fabric project](https://github.com/danielmiessler/fabric)\n2. Follow Daniel Miessler's work on [Twitter/X](https://twitter.com/DanielMiessler)\n3. Consider contributing to Fabric's pattern collection\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details. ","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fedu-ap%2Fknowledge-synthesizer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fedu-ap%2Fknowledge-synthesizer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fedu-ap%2Fknowledge-synthesizer/lists"}