{"id":18831699,"url":"https://github.com/yuting1214/fastapichat","last_synced_at":"2026-01-22T08:33:25.044Z","repository":{"id":245599411,"uuid":"817531987","full_name":"yuting1214/FastAPIChat","owner":"yuting1214","description":"FastAPIChat is a powerful application for evaluating large language model based chatbots.","archived":false,"fork":false,"pushed_at":"2024-08-22T18:24:09.000Z","size":2146,"stargazers_count":3,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-09-03T01:37:01.353Z","etag":null,"topics":["chatbot","fastapi","gradio","langchain","llm","openrouter","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yuting1214.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-06-19T23:33:22.000Z","updated_at":"2025-06-12T11:47:09.000Z","dependencies_parsed_at":"2024-08-22T20:15:27.586Z","dependency_job_id":"2ea45b92-436c-44db-8b73-f15b1fe401aa","html_url":"https://github.com/yuting1214/FastAPIChat","commit_stats":null,"previous_names":["yuting1214/fastapichat"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/yuting1214/FastAPIChat","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yuting1214%2FFastAPIChat","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yuting1214%2FFastAPIChat/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yuting1214%2FFastAPIChat/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yuting1214%2FFastAPIChat/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yuting1214","download_url":"https://codeload.github.com/yuting1214/FastAPIChat/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yuting1214%2FFastAPIChat/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28659518,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-22T01:17:37.254Z","status":"online","status_checked_at":"2026-01-22T02:00:07.137Z","response_time":144,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chatbot","fastapi","gradio","langchain","llm","openrouter","python"],"created_at":"2024-11-08T01:55:45.981Z","updated_at":"2026-01-22T08:33:25.027Z","avatar_url":"https://github.com/yuting1214.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://github.com/yuting1214/FastAPIChat\"\u003e\n    \u003cimg src=\"frontend/login/static/favicon.ico\" height=\"50\"\u003e\n  \u003c/a\u003e\n  \u003ch1 align=\"center\"\u003e\n    \u003ca href=\"https://github.com/yuting1214/FastAPIChat\"\u003eFastAPIChat\u003c/a\u003e\n  \u003c/h1\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n\u003ca href=\"https://railway.app/template/_-qAbG?referralCode=jk_FgY\"\u003e\n  \u003cimg src=\"https://railway.app/button.svg\" alt=\"Deploy on Railway\" height=\"30\"\u003e\n\u003c/a\u003e \n\u003c/p\u003e\n\n\nFastAPIChat is a powerful application for testing and evaluating large language model based chatbots.\n- FastAPIChat powers [LLM Arena](https://gradioapp-production.up.railway.app/arena/), serving over 10+ LLMs, for diverse and dynamic chatbot interactions\n- FastAPIChat enables seamlessly rate [LLM](https://gradioapp-production.up.railway.app/chatbot/) generated responses, ensuring continuous improvement based on user feedback.\n\n![Demo of FastAPIChat](https://github.com/yuting1214/FastAPIChat/blob/main/materials/demo.gif)\n\n## News\n- [2024/06] 🔥 We released FastAPIChat v1.0.0\n\n\u003cdetails\u003e\n\u003csummary\u003eMore\u003c/summary\u003e\n\n- [2024/06] We released FastAPIChat v1.0.0 with ChatBot and ChatBot Arena\n\n\u003c/details\u003e\n\n\u003ca href=\"https://chat.lmsys.org\"\u003e\u003cimg src=\"assets/demo_narrow.gif\" width=\"70%\"\u003e\u003c/a\u003e\n\n## 🎉 Key Features and Integrations\nKey features:\n\n* 💬 LLM Chats(10+ LLMs)\n* ⚔️ LLM Arena\n* 🗳️ Chat Feedback collection\n* ⚡ FastAPI API Documentation and Authentication\n\nFastAPIChat integrates seamlessly with Python programs and libraries and includes out-of-the-box integrations for:\n\n- [LangChain](https://docs.chainlit.io/integrations/langchain)\n- [OpenRouter](https://openrouter.ai/)\n- [Gradio](https://www.gradio.app/)\n\n## Quick-Start 🚀\n\nNote: To begin, you'll need an API key from [OpenRouter](https://openrouter.ai/keys).\n\n### Development Mode\n\n* Note: In Dev mode, the default config with backend is **SQLite**.\n\n1. Clone Repo:\n\n```\ngit clone https://github.com/yuting1214/FastAPIChat.git\n```\n\n2. Configure Virtual Environment (Recommended):\n\u003cdetails\u003e\n  \n   * Create Virtual Environment\n   \n     ```\n     # macOS/Linux\n     # You may need to run `sudo apt-get install python3-venv` first on Debian-based OSs\n     python3 -m venv .venv\n     \n     # Windows\n     # You can also use `py -3 -m venv .venv`\n     python -m venv .venv\n     ```\n\n   * Activate the virtual environment:\n     ```\n     # macOS/Linux\n     source .venv/bin/activate\n     \n     # Windows\n     .venv\\Scripts\\activate\n     ```\n\u003c/details\u003e\n     \n3. Install Dependencies:\n```\npip install -r requirements.txt\n```\n\n4. Update the .env file with\n\n```\nUSER_NAME=\u003cAPI_Doc_Username\u003e\nPASSWORD=min=\u003cAPI_Doc_PASSWORD\u003e\nOPENROUTER_API_KEY=\u003cYour_API_KEY\u003e\n```\n\n5. Run the application:\n\n```\npython -m backend.app.main --mode dev \n```\n\n### Production Mode\n\n1. Add extra envs in with your environment configurations:\n\n```\nDB_ENGINE=\u003cYour_DB_config\u003e\nDB_USERNAME=\u003cYour_DB_config\u003e\nDB_PASS=\u003cYour_DB_config\u003e\nDB_HOST=\u003cYour_DB_config\u003e\nDB_PORT=\u003cYour_DB_config\u003e\nDB_NAME=\u003cYour_DB_config\u003e\nAPI_BASE_URL=\u003cYour_Host_URL\u003e\n```\n\n2. Run the application:\n\n```\npython -m backend.app.main --mode prod\n```\n\n* Alternatively, use Dockerfile for deployment.\n\n## App Settings 📋\n\n### Managing Quota\n\n* Control quotas by adjusting limits in `backend/app/core/constants.py`:\n\n```\nTEXT_API_QUOTA_LIMIT = \u003cint\u003e\n```\n\n### Adding New LLM Models (OpenRouter)\n\n* To add a new LLM model, specify its details in backend/data/llm_models:\n\n```\n    {\n        'llm_model_name': 'Mixtral 8x22B',\n        'llm_vendor': 'Mistral AI',\n        'llm_type': 'text',\n        'api_provider': 'OpenRouter',\n        'api_endpoint': 'mistralai/mixtral-8x22b-instruct',\n    }\n```\n\n## Features 🌟\n\n1. **Utilize Gradio and FastAPI Integration**: Seamlessly combine Gradio for the frontend display and FastAPI for robust backend functionality, offering a smooth user experience.\n\n2. **Well-Crafted API Management**: Our meticulously designed API ensures efficient data transportation and effective management of API quota usage, providing users with a reliable and hassle-free experience.\n\n3. **Integration of Leading Language Models**: Integrate more than 10+ popular language models from various vendors, including OpenAI, Mistral, Meta and Anthropic, enabling users to leverage cutting-edge AI capabilities for text generation.\n\n4. **Comprehensive Testing and Documentation**: Benefit from comprehensive testing suites and detailed documentation, empowering developers to understand and contribute to the project with ease, ensuring robustness and maintainability.\nThird-Party Services: Integrate with third-party APIs for enhanced functionality.\n\n5. **Third-Party Services Integration**: Seamlessly integrates with third-party APIs for enhanced functionality.\n\n## Project Structure 📁\n\n* Project Details: [Heptabase](https://app.heptabase.com/w/80fcc9a0476f3a3ac30ac895c36eef51ede0bc4aa090cb7be1c6c0ed507cfda9)\n\n\u003cdetails\u003e\n\u003csummary\u003eMore Project Details\u003c/summary\u003e\n\n```\nFastAPIChat/\n├── backend/                      # Backend directory for the FastAPI application\n│   ├── app/                      # Main application directory\n│   │   ├── __init__.py           # Initialization file for the app package\n│   │   ├── api/                  # Directory for API related code\n│   │   │   ├── __init__.py       # Initialization file for the API package\n│   │   │   ├── v1/               # Version 1 of the API\n│   │   │   │   ├── __init__.py   # Initialization file for the v1 API package\n│   │   │   │   ├── endpoints/    # Directory for API endpoint definitions\n│   │   │   │   │   ├── __init__.py          # Initialization file for endpoints package\n│   │   │   │   │   ├── text_generation.py   # Endpoints for text generation\n│   │   │   │   │   ├── llm_management.py    # Endpoints for LLM model management\n│   │   │   │   │   ├── api_usage.py         # Endpoints for API usage management\n│   │   │   │   │   ├── api_calldetail.py    # Endpoints for API Detail management\n│   │   │   │   │   ├── message.py           # Endpoints for messasge management\n│   │   │   │   │   ├── chat.py              # Endpoints for chat management\n│   │   │   │   │   ├── rating.py            # Endpoints for rating management\n│   │   │   │   │   ├── user.py              # Endpoints for user management\n│   │   │   │   │   ├── quota.py             # Endpoints for quota management\n│   │   │   │   │   ├── doc.py               # Endpoints for OpenAPI doc management\n│   │   ├── dependencies/         # Directory for dependency management\n│   │   │   ├── __init__.py       # Initialization file for dependencies package\n│   │   │   ├── database.py       # Database connection and session management\n│   │   │   ├── rate_limiter.py   # Rate limiting logic\n│   │   ├── core/                 # Core application logic\n│   │   │   ├── __init__.py       # Initialization file for core package\n│   │   │   ├── constant.py       # Constant settings\n│   │   │   ├── config.py         # Configuration settings\n│   │   │   ├── init_setting.py   # Init settings with user's input\n│   │   ├── models/               # Directory for SQLAlchemy models\n│   │   │   ├── __init__.py       # Initialization file for models package\n│   │   │   ├── user.py           # User model\n│   │   │   ├── quota.py          # Quota model\n│   │   │   ├── message.py        # Message model\n│   │   │   ├── api_usage.py      # API usage model\n│   │   │   ├── llm.py            # LLM model\n│   │   │   ├── chat.py           # Chat model\n│   │   │   ├── rating.py         # rating model\n│   │   ├── schemas/              # Directory for Pydantic schemas\n│   │   │   ├── __init__.py       # Initialization file for schemas package\n│   │   │   ├── user.py           # Schemas for user data\n│   │   │   ├── quota.py          # Schemas for quota data\n│   │   │   ├── message.py        # Schemas for message data\n│   │   │   ├── api_usage.py      # Schemas for API usage data\n│   │   │   ├── llm.py            # Schemas for LLM model data\n│   │   │   ├── llm_message.py    # Schemas for LLM generated output data\n│   │   │   ├── chat.py           # Schemas for chat data\n│   │   │   ├── rating.py         # Schemas for rating data\n│   │   ├── crud/                 # Directory for CRUD operations\n│   │   │   ├── __init__.py       # Initialization file for crud package\n│   │   │   ├── user.py           # CRUD for user management\n│   │   │   ├── quota.py          # CRUD for quota management\n│   │   │   ├── message.py        # CRUD for message management\n│   │   │   ├── api_usage.py      # CRUD for API usage management\n│   │   │   ├── llm.py            # CRUD for LLM model management\n│   │   │   ├── llm_message.py    # CRUD for LLM generated output management\n│   │   │   ├── chat.py           # CRUD for chat management\n│   │   │   ├── rating.py         # CRUD for rating management\n│   │   ├── main.py               # Main FastAPI application file\n│   ├── data/                     # Directory for data when initiating db\n│   │   ├── __init__.py           # Initialization file for data package\n│   │   ├── llm_models.py         # LLM models infomation\n│   ├── security/                  # New directory for authentication and authorization\n│   │   ├── __init__.py            # Initialization file for security package\n│   │   ├── authentication.py      # Authentication logic\n│   │   ├── authorization.py       # Authorization logic\n│   ├── tests/                    # Directory for test files\n│   │   ├── __init__.py           # Initialization file for tests package\n│   │   ├── test_text_generation.py  # Tests for text generation endpoints\n├── frontend/                     # Frontend directory for the Gradio app\n│   ├── __init__.py               # Initialization file for frontend package\n│   ├── gradio/                   # Main gradio UI directory\n│   │   ├── __init__.py           # Initialization file for the gradio package\n│   │   ├── text/                 # Directory for ChatBot UI related code\n│   │   │   ├── __init__.py       # Initialization file for the text package\n│   │   │   ├── event_listeners.py #      \n│   │   │   ├── text_generation.py # \n│   │   │   ├── text_generation_arena.py               \n│   ├── login/                     # Main login UI directory\n│   │   ├── __init__.py           # Initialization file for the gradio package\n│   │   ├── static/               # Directory for static files\n│   │   │   ├── style.css         # CSS for login UI               \n│   │   ├── templates/            # Directory for HTML templates\n│   │   │   ├── base.html         # HTML base template\n│   │   │   ├── login.html        # HTML login template   \n├── llm/\n│   ├── __init__.py               # Makes llm a Python package\n│   ├── llm_text_chain.py         # Module for LLM text generation integration\n│   └── prompt/                   # Folder for prompt handling\n│       ├── __init__.py           # Initializes the prompt package\n│       ├── base_text_templates.py# Stores base prompt templates for text generation\n│       └── examples/             # Directory for few-shot examples used by the chain\n│       └── deprecated/           # Directory for deprecated prompts\n│   └── vendors/                  # Directory for vendor-specific LLM configurations\n│       ├── __init__.py           # Makes vendors a Python package\n│       └── openrouter.py         # Configurations and usage for OpenRouter as LLM provider\n├── .env                          # Environment variables file\n├── .gitignore                    # Git ignore file\n├── requirements.txt              # Python dependencies file\n├── Dockerfile                    # Docker configuration\n├── README.md                     # Project README file\n```\n\u003c/details\u003e\n\n## Future Direction 📝\n\n1. Integraion of local LLM: Integrate with [Ollama](https://github.com/ollama/ollama) to enable users to run LLMs locally and battle with LLMs from API.\n\n2. Integraion of RAG LLM: Enable users to compare different configuration of RAG LLM in LLM Arena.\n\n3. Integration of Redis or other NoSQL Databases: Incorporate Redis or other NoSQL databases to efficiently track and store user history data, enabling personalized experiences and insights for users.\n\n4. History Message Management: Allow user to fetch back the previous messages from database and continue the chat.\n\n5. Rate Limiter: Integrate rate limiter for API protection.\n\n## Contributing 🤝\nWe welcome contributions from the community! Whether it's bug fixes, feature enhancements, or documentation improvements, feel free to open a pull request.\n\n---\n\nDesigned with :heart: by [Mark Chen](https://github.com/yuting1214)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyuting1214%2Ffastapichat","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyuting1214%2Ffastapichat","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyuting1214%2Ffastapichat/lists"}