{"id":29704294,"url":"https://github.com/mindscope-world/fastapi-bitnet-inference","last_synced_at":"2026-05-19T04:06:06.451Z","repository":{"id":290921988,"uuid":"968700118","full_name":"mindscope-world/fastapi-bitnet-inference","owner":"mindscope-world","description":"BitNet Inference Web UI: A modern web interface for running Microsoft's BitNet models efficiently on CPU. This project provides a user-friendly way to download, manage, and run inference with 1-bit quantized language models.","archived":false,"fork":false,"pushed_at":"2025-05-01T10:08:30.000Z","size":2484,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-07-31T11:51:57.878Z","etag":null,"topics":["bitnet","fastapi","llms"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mindscope-world.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-04-18T15:10:14.000Z","updated_at":"2025-05-01T10:10:58.000Z","dependencies_parsed_at":"2025-05-01T11:19:50.285Z","dependency_job_id":"55087f9e-3f45-4ac1-9159-43323c6f8a0b","html_url":"https://github.com/mindscope-world/fastapi-bitnet-inference","commit_stats":null,"previous_names":["mindscope-world/fastapi-bitnet-inference"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/mindscope-world/fastapi-bitnet-inference","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mindscope-world%2Ffastapi-bitnet-inference","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mindscope-world%2Ffastapi-bitnet-inference/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mindscope-world%2Ffastapi-bitnet-inference/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mindscope-world%2Ffastapi-bitnet-inference/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mindscope-world","download_url":"https://codeload.github.com/mindscope-world/fastapi-bitnet-inference/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mindscope-world%2Ffastapi-bitnet-inference/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33201543,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-18T09:27:30.708Z","status":"online","status_checked_at":"2026-05-19T02:00:06.763Z","response_time":58,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bitnet","fastapi","llms"],"created_at":"2025-07-23T14:09:59.460Z","updated_at":"2026-05-19T04:06:06.413Z","avatar_url":"https://github.com/mindscope-world.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# BitNet Inference Web UI 🧠\n\nA modern web interface for running Microsoft's BitNet models efficiently on CPU. This project provides a user-friendly way to download, manage, and run inference with 1-bit quantized language models.\n\n![BitNet Inference UI](/app/static/imgs/bit-inference5.png)\n\n## 🌟 Features\n\n- **Easy Model Management**\n  - One-click downloads from Hugging Face\n  - Direct model uploads (GGUF format)\n  - Real-time download progress tracking\n  - Popular models quick access\n\n- **Efficient Inference**\n  - CPU-optimized inference\n  - Support for 1-bit quantized models\n  - Conversation mode\n  - Adjustable parameters (temperature, max tokens)\n\n- **Modern UI/UX**\n  - Clean, responsive interface\n  - Dark/Light theme support\n  - Real-time status updates\n  - System logs viewer\n\n- **Technical Features**\n  - FastAPI backend\n  - Async model downloads\n  - Automatic fallback mechanisms\n  - Progress monitoring system\n![BitNet Inference UI](/app/static/imgs/bit-inference1.png)\n\n## 🚀 Getting Started\n\n### Prerequisites\n\n- Python 3.8 or higher\n- pip package manager\n- CPU with AVX2 support (recommended)\n\n### Installation\n\n1. Clone the repository:\n```bash\ngit clone https://github.com/mindscope-world/bitnet-inference.git\ncd bitnet-inference\n```\n\n2. Install dependencies:\n```bash\npip install -r requirements.txt\n```\n\n3. Run the application:\n```bash\npython app.py\n```\n\nThe web interface will be available at `http://localhost:8000`\n\n## 💻 Usage\n\n### Downloading Models\n\n1. Navigate to the \"Download Model\" tab\n2. Enter a model name or HuggingFace path (e.g., `microsoft/BitNet-b1.58-2B-4T`)\n3. Click \"Download Model\"\n4. Monitor the download progress in real-time\n\n### Running Inference\n\n1. Ensure a model is loaded\n2. Enter your prompt in the text area\n3. Adjust generation parameters if needed:\n   - Temperature (0.1 - 1.5)\n   - Max Tokens (10 - 2048)\n   - Conversation Mode (on/off)\n4. Click \"Generate\"\n![BitNet Inference UI](/app/static/imgs/bit-inference6.png)\n\n### Model Compatibility\n\nThe application supports various BitNet models, including:\n- BitNet-b1.58-2B-4T\n- bitnet_b1_58-large\n- bitnet_b1_58-3B\n\n## 🛠️ Technical Details\n\n### Architecture\n\n```\nbitnet-inference/\n├── app/\n│   ├── static/\n│   │   ├── imgs/\n│   │   ├── css/\n│   │   └── js/\n│   ├── templates/\n│   └── models/\n├── app.py\n├── setup_env.py\n├── simple_model_server.py\n└── requirements.txt\n```\n\n### Key Components\n\n- **FastAPI Backend**: Handles model management and inference requests\n- **Async Downloads**: Non-blocking model downloads with progress tracking\n- **Fallback System**: Automatic switching between optimized and standard inference\n- **Theme System**: Dynamic theme switching with system preference detection\n\n## 🤝 Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.\n\n### Contributors\n\n- [@mindscope-world](https://github.com/mindscope-world) - Project Lead \u0026 Main Developer\n\n## 📝 License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n## 🙏 Acknowledgments\n\n- [Microsoft BitNet](https://github.com/microsoft/BitNet) - For the original BitNet implementation\n- [FastAPI](https://fastapi.tiangolo.com/) - For the excellent web framework\n- [Hugging Face](https://huggingface.co/) - For model hosting and transformers library\n\n## 📞 Support\n\nFor support, please open an issue in the GitHub repository or contact [@mindscope-world](https://github.com/mindscope-world).\n\n## 🔮 Future Plans\n\n- [ ] Add batch processing support\n- [ ] Implement model fine-tuning interface\n- [ ] Add more visualization options\n- [ ] Support for custom quantization\n- [ ] API documentation interface\n- [ ] Docker deployment support\n\n---\n\nMade with ❤️ by [@mindscope-world](https://github.com/mindscope-world)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmindscope-world%2Ffastapi-bitnet-inference","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmindscope-world%2Ffastapi-bitnet-inference","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmindscope-world%2Ffastapi-bitnet-inference/lists"}