https://github.com/bloodworks-io/phlox
Open source, local first AI medical scribe for desktop and web.
https://github.com/bloodworks-io/phlox
llamacpp macos medical ollama rag scribe whisper
Last synced: about 1 month ago
JSON representation
Open source, local first AI medical scribe for desktop and web.
- Host: GitHub
- URL: https://github.com/bloodworks-io/phlox
- Owner: bloodworks-io
- License: mit
- Created: 2024-09-13T13:10:16.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2026-05-30T13:06:31.000Z (about 1 month ago)
- Last Synced: 2026-05-30T15:06:04.026Z (about 1 month ago)
- Topics: llamacpp, macos, medical, ollama, rag, scribe, whisper
- Language: JavaScript
- Homepage: https://phlox.bloodworks.io
- Size: 8.74 MB
- Stars: 82
- Watchers: 3
- Forks: 17
- Open Issues: 16
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
> [!WARNING]
> Phlox is an experimental project. For full details on limitations and risks, please read the **[Usage Warning](#usage-warning-️)** section carefully before proceeding.
[](https://github.com/bloodworks-io/phlox/actions/workflows/coverage.yml)
[](https://snyk.io/test/github/bloodworks-io/phlox/badge.svg)
[](https://coveralls.io/github/bloodworks-io/phlox?branch=main)
[](https://github.com/bloodworks-io/phlox/actions/workflows/github-code-scanning/codeql)
[](https://github.com/astral-sh/ruff)
[](https://opensource.org/licenses/MIT)
[](https://github.com/bloodworks-io/phlox/issues)
Phlox is an open-source patient management system integrating AI-powered medical transcription, clinical note generation, an AI chatbot interface, and agentic tool-calling capabilities. It's designed to run locally, utilizing local models for inference and transcription.
## Key Features ✨
- **🔒 100% Local & Private:** Runs entirely on your machine with no third-party services - all data stays local. Free and open source forever; the security of your data is in your hands.
- **🖥️ Desktop App:** Native Apple Silicon application with bundled LLM and transcription servers - no external dependencies required.
- **🎤 AI Medical Transcription & Summarization:** Convert patient encounters to structured clinical notes using customizable templates.
- **📝 Flexible Template System:** Structure clinical notes to your preferences, with versioning and automated template generation from example notes.
- **✅ Task Manager:** Parse clinical plans into actionable task lists with AI-generated summaries.
- **✉️ Correspondence Generation:** One-click generation of patient letters based on clinical notes, with support for both ambient listening and dictate mode.
- **🤖 AI-chat/RAG:** Reference tool to query medical guidelines, literature, and documentation backed by a local knowledge base (ChromaDB).
- **💡 Adaptive Refinement:** Outputs improve the more you use it; Phlox learns from your previous notes.
- **🛠️ Agentic Tool-Calling System:** Built-in tools including PubMed search, Wikipedia lookup, patient note search, transcript search, outstanding job management, and note creation — with interleaved thinking for complex multi-step queries.
- **🔌 MCP Server Support:** Connect external tool servers via the Model Context Protocol (SSE transport). Tools are dynamically loaded and can be toggled per-server, with optional sensitive data filtering to prevent PHI from being sent to external servers.
- **📄 Vision-Enhanced Document Processing:** Hybrid PDF processing — uses vision models directly when available, falling back to OCR. Supports image uploads for clinical document analysis.
- **📋 Agent Dashboard:** Central hub with chat-driven clinic management, built-in todo panel, outstanding jobs view, and clinic summary by date.
## Stack 🛠️
- **Frontend:** Chakra UI (React/[Vite](https://github.com/vitejs/vite))
- **Backend:** FastAPI (Python/[uv](https://github.com/astral-sh/uv))
- **Database:** [SQLCipher](https://github.com/sqlcipher/sqlcipher)
- **Desktop Wrapper:** [Tauri](https://github.com/tauri-apps/tauri) (Rust)
- **LLM Backend:** Ollama, OpenAI-compatible endpoints, or bundled [llama.cpp ](https://github.com/ggml-org/llama.cpp) server
- **Transcription:** Whisper-compatible endpoints or bundled [whisper.cpp](https://github.com/ggml-org/whisper.cpp) server
- **RAG:** [ChromaDB](https://github.com/chroma-core/chroma)
## Quick Start 🚀
### Desktop App (Apple Silicon)
Pre-built Apple Silicon binaries are available from [GitHub Releases](https://github.com/bloodworks-io/phlox/releases).
**Note:** The desktop app provides transcription and correspondence features only. For extended reference tools (Chat, RAG, PDF Upload), use the Docker/Podman deployment below.
### Docker/Podman (Extended Reference Tools)
Pre-built images are available from [GitHub Container Registry](https://github.com/bloodworks-io/phlox/pkgs/container/phlox):
```bash
docker pull ghcr.io/bloodworks-io/phlox:latest
```
Or build from source:
1. **Prerequisites:** Podman/Docker, Ollama/OpenAI-compatible endpoint, Whisper endpoint.
2. **Hardware Requirements:** For reasonable performance, a GPU (CUDA, ROCm) or Apple M-Series chip is strongly recommended. Without these, especially with larger models, the system will run extremely slowly.
3. **Clone:** `git clone https://github.com/bloodworks-io/phlox.git && cd phlox`
4. **Build:** `docker build -t phlox:latest .`
5. **Environment:** Create `.env` in `phlox/` (see example in documentation).
6. **Run:** `docker-compose up` (Production) or `docker-compose -f docker-compose.dev.yml up` (Development).
7. **Access:** http://localhost:5000
**For detailed setup, feature explanations, and important warnings, please see the [Documentation](./docs/README.md).**
## Deployment Options
### Docker/Podman (Extended Reference Tools)
The complete Phlox experience with all features:
- Medical transcription and clinical notes
- Correspondence generation
- AI Chat interface
- RAG/document knowledge base
- Agent Dashboard with agentic tool-calling and MCP server support
### Desktop App (Streamlined)
Native desktop application for Apple Silicon:
- Medical transcription and clinical notes
- Correspondence generation
- Bundled llama.cpp and whisper servers - no external dependencies
- All data stored locally. Nothing leaves your machine.
*Additional platforms and full feature parity coming in future releases.*
## Roadmap 🗺️
Here's what's coming next for Phlox:
- [x] Use structured JSON outputs for managing LLM responses
- [x] Add support for OpenAI-compatible endpoints
- [x] Tauri desktop app with local inference (llama.cpp + whisper bundled)
- [x] MCP server support for custom tools and agentic workflows
- [ ] Advanced template version control
- [ ] Meeting and multi-disciplinary meeting scribing
## Usage Warning ⚠️
Phlox is an experimental project intended for educational and personal use. **It is not a certified medical device and should NOT be used for clinical decision-making.**
Phlox is **not** suitable for production deployment in the form provided in this repo. If you intend to use it in a clinical setting, you are responsible for ensuring compliance with local applicable regulations (HIPAA, GDPR, TGA, etc.)
AI outputs can be unreliable. Always verify AI-generated content and use professional clinical judgment. The application displays a disclaimer on startup with full details.
**Security note:** The Docker deployment binds to `0.0.0.0` by default and has no authentication by default. You MUST place it behind a reverse proxy with an authentication layer like [Authelia](https://github.com/authelia/authelia). Proxy authentication (Traefik, Caddy, Nginx) and rate limiting are supported via environment variables. The desktop app requires a passphrase to unlock the encrypted database.
## License 📄
[MIT License](LICENSE)
## Contributing 🤝
[Contributing Guidelines](.github/CONTRIBUTING.md)