{"id":16283256,"url":"https://github.com/zhudotexe/fastlmi","last_synced_at":"2025-07-22T02:31:46.742Z","repository":{"id":163666472,"uuid":"637988874","full_name":"zhudotexe/fastlmi","owner":"zhudotexe","description":"A fast framework (in both performance and development time) for creating language model interfaces - a more generic term for tools built for AIs like ChatGPT plugins.","archived":false,"fork":false,"pushed_at":"2023-05-11T16:51:24.000Z","size":251,"stargazers_count":7,"open_issues_count":0,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-07-01T20:16:07.230Z","etag":null,"topics":["asyncio","chatgpt","chatgpt-plugins","fastapi","framework","lmi","openai","openapi","pydantic","python","python3","starlette","uvicorn"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zhudotexe.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"github":"zhudotexe","ko_fi":"zhuexe"}},"created_at":"2023-05-08T21:02:02.000Z","updated_at":"2023-09-21T22:33:05.000Z","dependencies_parsed_at":null,"dependency_job_id":"56d2d2db-66da-4b0c-84cf-c8d8da47e0da","html_url":"https://github.com/zhudotexe/fastlmi","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/zhudotexe/fastlmi","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhudotexe%2Ffastlmi","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhudotexe%2Ffastlmi/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhudotexe%2Ffastlmi/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhudotexe%2Ffastlmi/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zhudotexe","download_url":"https://codeload.github.com/zhudotexe/fastlmi/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhudotexe%2Ffastlmi/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266412289,"owners_count":23924479,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-22T02:00:09.085Z","response_time":66,"last_error":null,"robots_txt_status":null,"robots_txt_updated_at":null,"robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["asyncio","chatgpt","chatgpt-plugins","fastapi","framework","lmi","openai","openapi","pydantic","python","python3","starlette","uvicorn"],"created_at":"2024-10-10T19:12:57.267Z","updated_at":"2025-07-22T02:31:46.721Z","avatar_url":"https://github.com/zhudotexe.png","language":"Python","funding_links":["https://github.com/sponsors/zhudotexe","https://ko-fi.com/zhuexe"],"categories":[],"sub_categories":[],"readme":"# FastLMI\n\nFastLMI is a modern, fast (both in performance and development time) framework for creating LMIs, based on the beloved\n[FastAPI](https://github.com/tiangolo/fastapi) library.\n\n## What is an LMI?\n\n*and why not just call it an API?*\n\nLMI stands for \"**Language Model Interface**\" -- it's a catch-all term for tools given to AI agents that can interact\nwith them, and how we define those tools (interfaces). We aren't providing an application programming interface;\nrather than providing instructions (in the form of docs) for human developers to write apps, LMIs come bundled with\ninstructions for AI agents to interface with them.\n\nFastLMI is more than just a library for making ChatGPT plugins -- we believe the LMI ecosystem has the potential to be\nand do much more without relying on a single authority to curate and provide \"good\" plugins. FastLMI is designed to be\necosystem-agnostic with adapters for popular ecosystems, such as ChatGPT/OpenAI plugins.\n\n## Cite Us\n\nDoing academic research on language models and their abilities to use tools using the FastLMI library? Cite us with\nthis BibTex entry!\n\n```bibtex\n@software{Zhu_FastLMI_2023,\n    author = {Zhu, Andrew},\n    doi = {10.5281/zenodo.7925999},\n    month = may,\n    title = {{FastLMI}},\n    url = {https://github.com/zhudotexe/fastlmi},\n    version = {0.2.0},\n    year = {2023}\n}\n```\n\n## Requirements\n\nPython 3.8+\n\n## Installation\n\n```shell\n$ pip install fastlmi\n```\n\nJust as with FastAPI, you will need an ASGI server to run the app, such as [Uvicorn](https://www.uvicorn.org/).\n\n```shell\n$ pip install uvicorn\n# or pip install \"uvicorn[standard]\" for Cython-based extras \n```\n\n## Example (OpenAI/ChatGPT Plugin)\n\n\u003e NOTE: As of v0.2.0 FastLMI includes the AI Plugin (OpenAI, LangChain) interface as a default. This may change to an\n\u003e extension-based system in the future as the library develops.\n\nTo show off just how easy it is to create a plugin, let's make a ChatGPT plugin that gives it the ability to roll dice\nin the d20 format (AIs playing D\u0026D, anyone?).\n\n### Example Requirements\n\nFirst, you'll need to install the [`d20`](https://github.com/avrae/d20) library:\n\n```shell\n$ pip install d20\n```\n\n### Create it\n\nThen, create a `main.py` file.\n\n```python\nimport d20  # pip install d20\nfrom fastlmi import FastLMI, utils\nfrom pydantic import BaseModel\n\napp = FastLMI(\n    title=\"Dice Roller\",\n    name_for_model=\"DiceRoller\",\n    description=\"A simple plugin to roll dice.\",\n    description_for_model=(\n        \"DiceRoller can roll dice in XdY format and do math.\\n\"\n        \"Some dice examples are:\\n\"\n        \"4d6kh3 :: highest 3 of 4 6-sided dice\\n\"\n        \"2d6ro\u003c3 :: roll 2d6s, then reroll any 1s or 2s once\\n\"\n        \"8d6mi2 :: roll 8d6s, with each die having a minimum roll of 2\\n\"\n        \"(1d4 + 1, 3, 2d6kl1)kh1 :: the highest of 1d4+1, 3, and the lower of 2 d6s\\n\"\n        \"Normal math operations are also supported.\"\n    ),\n    contact_email=\"foo@example.com\",\n    legal_url=\"https://example.com/legal\",\n)\n# use this when developing localhost plugins to allow the browser to make the local request\nutils.cors_allow_openai(app)\n\n\nclass DiceRequest(BaseModel):\n    dice: str  # the dice to roll\n\n\nclass DiceResponse(BaseModel):\n    result: str  # the resulting dice string\n    total: int  # the total numerical result of the roll (rounded down to nearest integer)\n\n\n@app.post(\"/roll\")\ndef roll(dice: DiceRequest) -\u003e DiceResponse:\n    \"\"\"Roll the given dice and return a detailed result.\"\"\"\n    result = d20.roll(dice.dice)\n    return DiceResponse(result=result.result, total=result.total)\n```\n\n_(this example script is also available at `examples/dice_roller.py`!)_\n\n### Run it\n\n... and run it with:\n\n```shell\n$ uvicorn main:app\n\nINFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)\nINFO:     Started server process [53532]\nINFO:     Waiting for application startup.\nINFO:     Application startup complete.\n```\n\n### Register it\n\nFinally, we need to tell ChatGPT about the new plugin.\nIn the ChatGPT interface, select the \"Plugins\" model, then head to the Plugin Store -\u003e Develop your own plugin.\n\nHere, type in the address of your plugin. By default, it's `localhost:8000`.\n\n![](assets/oai_manifest.png \"Enter your website domain\")\n\nClick \"Find manifest file,\" and you should see your plugin appear. FastLMI automatically handles generating all the\nplugin metadata needed by OpenAI!\n\n![](assets/oai_found_plugin.png \"Found plugin\")\n\n### Chat away\n\nTo use your new plugin, select it from the list of plugins when starting a new chat:\n\n![](assets/oai_select_plugins.png \"Select plugins\")\n\nand start chatting. Congratulations! 🎉 You've just created a brand-new ChatGPT plugin - and we're excited to see what\nelse you'll make!\n\n![](assets/oai_dice_roller.png \"Conversation with ChatGPT using Dice Roller\")\n\n## Example (LangChain Tool)\n\nLangChain supports\n[interfacing with AI plugins](https://python.langchain.com/en/latest/modules/agents/tools/examples/chatgpt_plugins.html)!\nIf you've followed the steps above (at least up through \"Run it\"), you can also expose your new LMI to a LangChain\nagent.\n\nThis example assumes that you\nhave [LangChain](https://python.langchain.com/en/latest/getting_started/getting_started.html) installed, and that your\nLMI is running at `http://localhost:8000`.\n\n```python\nfrom langchain.agents import AgentType\nfrom langchain.agents import initialize_agent, load_tools\nfrom langchain.chat_models import ChatOpenAI\nfrom langchain.tools import AIPluginTool\n\ntool = AIPluginTool.from_plugin_url(\"http://localhost:8000/.well-known/ai-plugin.json\")\nllm = ChatOpenAI(temperature=0)\ntools = load_tools([\"requests_all\"])\ntools += [tool]\n\nagent_chain = initialize_agent(tools, llm, agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION, verbose=True)\nagent_chain.run(\"Can you roll me stats for a D\u0026D character?\")\n```\n\n## Authentication\n\nAs of v0.2.0, FastLMI has built-in support for service-level auth, where the AI agent or LMI driver will send an\nauthorization token you decide as a header with each request.\n\nEnabling authentication is easy! First, define the authentication scheme you wish to use - this will be a subclass of\n`LMIAuth`. For example, to provide service-level auth, you can define the `LMIServiceAuth` scheme:\n\n```python\nfrom fastlmi import Depends, FastLMI\nfrom fastlmi.auth import LMIServiceAuth\n\nauth = LMIServiceAuth(\n    access_tokens=[\"your_secret_token_here\"],\n    verification_tokens={\"openai\": \"verification_token_generated_in_the_ChatGPT_UI\"}\n)\n```\n\nThis auth scheme allows defining a set of allowed access tokens (if one wanted to, for example, have a different\ntoken for each plugin service). Then, when you define your app, all you have to do is add 2 parameters:\n\n```python\napp = FastLMI(..., auth=auth, dependencies=[Depends(auth)])\n```\n\nTada! 🎉 Your LMI now tells consumers that it uses the `service_http` auth scheme, and will validate that each request\nto one of your defined routes provides a valid Bearer token.\n\nA complete example is in `examples/ai_plugin_auth.py`. You can read more about OpenAI's service-level\nauth [here](https://platform.openai.com/docs/plugins/authentication/service-level).\n\n### Route-Level Auth\n\nIf you wanted to only require that certain routes use auth, you can also define the auth dependency on a route level:\n\n```diff\n- app = FastLMI(..., auth=auth, dependencies=[Depends(auth)])\n+ app = FastLMI(..., auth=auth)\n...\n- @app.post(\"/hello\")\n+ @app.post(\"/hello\", dependencies=[Depends(auth)])\ndef hello():\n    ...\n```\n\n## Read More\n\nBeing based on FastAPI, FastLMI can take full advantage of its superpowers. Check out\nthe [FastAPI documentation](https://fastapi.tiangolo.com/) for more!\n\n## Todo\n\n- scopes\n- script to check for missing docs, over limits, etc\n- logging\n- configure ecosystems\n\n\u003c!--\nFor developers:\n\n## Build and Publish\n\n`fastlmi` uses Hatchling to build.\n\nMake sure to bump the version in pyproject.toml before publishing, then update CITATION.cff + README.md with the latest\ncitation from zenodo after the release is indexed.\n\n```shell\nrm dist/*\npython -m build\npython -m twine upload dist/*\n```\n--\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzhudotexe%2Ffastlmi","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzhudotexe%2Ffastlmi","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzhudotexe%2Ffastlmi/lists"}