{"id":47922464,"url":"https://github.com/truefoundry/agentic-ai-deployment-cookbook","last_synced_at":"2026-04-04T06:18:04.167Z","repository":{"id":318952346,"uuid":"1075961586","full_name":"truefoundry/agentic-ai-deployment-cookbook","owner":"truefoundry","description":"A cookbook for deploying custom multi-agent systems 1) behind FastAPI endpoint 2) as a custom MCP server using FastMCP. This project demonstrates integration with the TrueFoundry AI Gateway for production readiness.","archived":false,"fork":false,"pushed_at":"2025-12-10T13:14:37.000Z","size":663,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-02-13T08:05:11.875Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/truefoundry.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-10-14T08:12:30.000Z","updated_at":"2025-12-11T18:47:03.000Z","dependencies_parsed_at":"2025-10-17T11:29:10.243Z","dependency_job_id":"0c28c7b6-26a8-4e35-9e91-039c482c753c","html_url":"https://github.com/truefoundry/agentic-ai-deployment-cookbook","commit_stats":null,"previous_names":["truefoundry/agentic-ai-deployment-cookbook"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/truefoundry/agentic-ai-deployment-cookbook","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/truefoundry%2Fagentic-ai-deployment-cookbook","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/truefoundry%2Fagentic-ai-deployment-cookbook/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/truefoundry%2Fagentic-ai-deployment-cookbook/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/truefoundry%2Fagentic-ai-deployment-cookbook/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/truefoundry","download_url":"https://codeload.github.com/truefoundry/agentic-ai-deployment-cookbook/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/truefoundry%2Fagentic-ai-deployment-cookbook/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31389691,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-04T04:26:24.776Z","status":"ssl_error","status_checked_at":"2026-04-04T04:23:34.147Z","response_time":60,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-04-04T06:18:00.963Z","updated_at":"2026-04-04T06:18:03.281Z","avatar_url":"https://github.com/truefoundry.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Agentic AI on TrueFoundry: Deployment Cookbook\n\nThis repository provides a minimal, end-to-end template for deploying a powerful, tool-powered **Agentic AI system**. We demonstrate wrapping an agent workflow (built with **CrewAI** or **LangGraph**) with scalable deployment via the **TrueFoundry AI Gateway** through:\n\n1) a custom agent on a **FastAPI** endpoint as well as\n2) a custom **Model Context Protocol (MCP) server** with **FastMCP**\n\n\u003e **Note on Security:** We demonstrate a scenario **without MCP server authentication** for simplicity. For production deployments, **TrueFoundry** offers robust security features. Refer to the [TrueFoundry MCP Gateway Authentication Guide](https://docs.truefoundry.com/gateway/mcp-gateway-auth-security) for implementation details.\n\n---\n\n## Cookbook Steps\n\nThis guide follows the process of developing a single-purpose agent and wrapping it for enterprise deployment.\n\n### 1. Getting Started\n\n#### 1a. Install and Login to TrueFoundry\n\nBefore you begin, you need to install the TrueFoundry SDK and authenticate with the platform.\n\n**Step 1: Install TrueFoundry**\n```bash\npip install -U \"truefoundry\"\n```\n\n**Step 2: Login to TrueFoundry** (via the tenant provided to you)\n```bash\ntfy login --host \"https://demo.truefoundry.cloud\"\n```\n\n\u003e **Important**: Always make sure to use the correct tenant provided to you (for example, `https://demo.truefoundry.cloud`) as specified above.\n\n#### 1b. Agent Flow Design (Jupyter Notebook)\n\nThe agent system is designed to perform **real-time research and format the output for human consumption**.\n\nWe are building a **Two-Agent Research and Report Generator** that uses the `TavilySearchTool` to find up-to-date information and then formats the final output into a clean Markdown string (we are no longer saving to a file in this simplified version).\n\n| Agent | Role | Output |\n| :--- | :--- | :--- |\n| **Researcher** | Conducts web search and synthesizes findings. | A detailed, cited summary of the topic. |\n| **Writer** | Takes the summary and formats it into a final Markdown report. | The final Markdown string. |\n\nReview the design and test the flow interactively in the notebook:\n- **[`With LangGraph`](./notebooks/research_report_generation_langgraph.ipynb)**\n- **[`With CrewAI`](./notebooks/research_report_generation_crewai.ipynb)**\n\n#### 1c. TrueFoundry Integration Code\n\nYou must configure your LLM framework to correctly route requests through the **TrueFoundry AI Gateway**.\n\n**Option A: Using CrewAI**\n```python\n# Create an LLM instance with TrueFoundry AI Gateway\nfrom crewai import LLM\n\ntruefoundry_llm = LLM(\n    api_key=\"your_truefoundry_api_key\",\n    base_url=\"your_truefoundry_gateway_base_url\",\n    model=\"openai/openai/gpt-5\",  # Format: \u003cprovider\u003e/\u003cyour-truefoundry-model-id\u003e\n)\n# This format allows you to call any model (Anthropic, Gemini, etc.)\n# configured on your TrueFoundry Gateway.\n```\n\n**Option B: Using LangChain (If building with LangGraph)**\n\n```python\nfrom langchain_openai import ChatOpenAI\n\nllm = ChatOpenAI(\n    api_key=\"your_truefoundry_api_key\",\n    base_url=\"your_truefoundry_gateway_base_url/v1\",\n    model=\"openai/gpt-5\"  # Format: \u003cyour-truefoundry-model-id\u003e\n)\n# This format allows you to call any model (Anthropic, Gemini, etc.)\n# configured on your TrueFoundry Gateway.\n```\n\n#### 1d. Python Environment Management (`uv`)\n\nWe use `uv` for ultra-fast, reproducible, and robust dependency management. Your repository uses a **[`pyproject.toml`](./pyproject.toml)** file to declare direct dependencies and a **[`uv.lock`](./uv.lock)** file to lock the entire environment.\n\nTo set up your project locally:\n\n| Command | Purpose |\n| :--- | :--- |\n| `uv venv` | Creates an isolated Python virtual environment, typically named `.venv`. |\n| `uv add \u003cpackage\u003e` | Installs a package and **adds it** to the `dependencies` list in `pyproject.toml` as well as updates `uv.lock`. |\n| `uv sync` | **Installs all dependencies** listed in `pyproject.toml` using the exact versions specified in the `uv.lock` file, ensuring perfect reproducibility. |\n| `uv run \u003cscript\u003e` | Executes a script within the project's virtual environment. |\n\n#### 1e. Python Script for Agent Code\n\nThe final agent logic is isolated into a runnable Python script, which exposes the core `run_agent(query)` function used by the FastAPI endpoint as well as the MCP server.\n- **[`With LangGraph`](./src/agents/langgraph/server.py)**\n- **[`With CrewAI`](./src/agents/crewai/server.py)**\n\n### 2. Deploy Agent Behind FastAPI Endpoint\n\nFor traditional RESTful communication, such as serving a **web UI** or integrating with a **synchronous microservice**, you should expose your agent logic through a standard FastAPI endpoint. This section details the complete configuration for your REST service.\n\n#### 2a. The FastAPI Endpoint Code\n\nThis script defines the application, input structure, and the core routing logic.\n- **[`With LangGraph`](./src/agents/langgraph/server.py)**\n- **[`With CrewAI`](./src/agents/crewai/server.py)**\n\n#### 2b. Core Component Explanation\n\n| Component | Code / Logic | Purpose in Deployment |\n| :--- | :--- | :--- |\n| **Agent Import** | `from src.{framework}.agent import run_agent` | **The single key difference** between agent frameworks. This imports the core asynchronous function. |\n| **`app = FastAPI(...)`** | Initialization | Creates the **main ASGI application** that the Uvicorn server will host. |\n| **`@app.get(\"/health\")`** | Health Probe | A synchronous endpoint required by **TrueFoundry** for container health checks (Liveness). |\n| **`UserInput(BaseModel)`** | Input Schema | Defines the expected request body (`{\"user_input\": \"...\"}`), providing **validation** and clear API documentation. |\n| **`@app.post(\"/chat\")`** | Main Endpoint | Defines the primary entry point for external web/REST calls. |\n| **`await run_agent_endpoint(...)`** | Execution Bridge | Correctly **awaits** the asynchronous agent function. This ensures the long-running workflow does not block the main server's event loop, maintaining service responsiveness. |\n\n***\n\n#### 2c. Execution Command\n\nTo run the standalone FastAPI agent endpoint for web consumption:\n\n```bash\n##Example with LangGraph\nuv run uvicorn src.agents.langgraph.server:app --host 0.0.0.0 --port 8000\n```\n\n### 3. Deploy a Custom MCP Server with FastMCP\n\n#### 3a. Create the MCP Server\n\nThe **`agent_as_mcp_server.py`** file uses the **FastMCP** framework to convert the single agent function (`conduct_research_and_report`) into a network-accessible tool.\n- **[`With LangGraph`](./src/agents/langgraph/agent_as_mcp_server.py)**\n- **[`With CrewAI`](./src/agents/crewai/agent_as_mcp_server.py)**\n\n#### 3b. FastMCP Overview\n\n* **Initialization:** The `FastMCP` object handles all protocol details.\n    ```python\n    mcp = FastMCP(\"research_report_generation\", stateless_http=True)\n    ```\n* **Tool Definition:** We define a single tool that wraps the entire process.\n    ```python\n    @mcp.tool()\n    def conduct_research_and_report(query: str) -\u003e str:\n        # ... calls run_agent(query) ...\n        # ... returns final Markdown report ...\n    ```\n\n#### 3c. Key FastMCP Components\n\n| Component | Purpose | TrueFoundry Context |\n| :--- | :--- | :--- |\n| **`@mcp.tool()`** | Exposes the single capability: `conduct_research_and_report(query)`. | This is the function executed by an LLM client (e.g., Cursor, Claude). |\n| **`@mcp.custom_route(\"/health\")`** | Creates a standard HTTP endpoint for health checks. | **Crucial for Deployment.** This can be used as the **Liveness Probe** to monitor the service health. |\n| **Startup Configuration** | Sets the host and port for the internal Uvicorn server. | Must listen on `host=\"0.0.0.0\"` and `port=8000` for containerization within TrueFoundry. |\n\nThe `agent_as_mcp_server.py` file uses **FastMCP** to wrap the agent logic into a deployable service and adds necessary deployment configuration.\n\n#### 3d. Execution Command\n\nTo run the standalone FastMCP server endpoint for web consumption:\n\n```bash\n##Example with LangGraph\nuv run src.agents.langgraph.agent_as_mcp_server.py\n```\n***\n\n#### 3e. Local Testing with MCP Inspector\n\nBefore deployment, verify the server's functionality using the **MCP Inspector** CLI tool. This tool allows you to confirm the server's capabilities and test tools directly via HTTP.\n\n1.  **Start the MCP Server:** Run your server script in one terminal window (if not already up):\n    ```bash\n    uv run src.agents.langgraph.agent_as_mcp_server.py\n    ```\n2.  **Test the Server (in a new terminal):** Use `npx` to inspect the capabilities and call the tool.\n    ```bash\n    # Inspect the capabilities (tools, resources, prompts)\n    npx @modelcontextprotocol/inspector\n    ```\n\n### 4. Deployment on TrueFoundry (Video Guide)\n\n\u003e **Important**: Sign up only on **https://demo.truefoundry.cloud/** (or the tenant provided to you) and do **NOT** use any other tenants/URLs.\n\n---\n\nWith the code complete, the service is ready for containerization and deployment.\n\nThe final deployment step involves using the TrueFoundry platform to containerize your code and 1) deploy your agent behind FastAPI endpoint and 2) deploy your custom MCP server.\n\n#### 4a. Deploy your agent behind FastAPI endpoint on TrueFoundry (~3mins watchtime)\n\nWatch this guide to see the full deployment process of Agent FastAPI endpoint in action.\n\n[![Watch the Deployment Guide](https://img.youtube.com/vi/Ps4h1tn10H4/0.jpg)](https://youtu.be/Ps4h1tn10H4)\n\n#### 4b. Deploy your custom MCP server on TrueFoundry (~3mins watchtime)\n\nWatch this guide to see the full deployment process of MCP server in action.\n\n[![Watch the Deployment Guide](https://img.youtube.com/vi/wL1MbcwUq7Q/0.jpg)](https://youtu.be/wL1MbcwUq7Q)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftruefoundry%2Fagentic-ai-deployment-cookbook","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftruefoundry%2Fagentic-ai-deployment-cookbook","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftruefoundry%2Fagentic-ai-deployment-cookbook/lists"}