{"id":13554364,"url":"https://github.com/eth-sri/lmql","last_synced_at":"2025-05-14T02:07:37.124Z","repository":{"id":107608959,"uuid":"570020780","full_name":"eth-sri/lmql","owner":"eth-sri","description":"A language for constraint-guided and efficient LLM programming.","archived":false,"fork":false,"pushed_at":"2024-06-03T13:02:24.000Z","size":189619,"stargazers_count":3924,"open_issues_count":120,"forks_count":207,"subscribers_count":21,"default_branch":"main","last_synced_at":"2025-05-09T12:06:41.286Z","etag":null,"topics":["chatgpt","huggingface","language-model","programming-language"],"latest_commit_sha":null,"homepage":"https://lmql.ai","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/eth-sri.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-11-24T06:43:00.000Z","updated_at":"2025-05-09T07:38:29.000Z","dependencies_parsed_at":null,"dependency_job_id":"68962d8c-978f-4241-b87c-49a3c09e9534","html_url":"https://github.com/eth-sri/lmql","commit_stats":null,"previous_names":[],"tags_count":19,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eth-sri%2Flmql","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eth-sri%2Flmql/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eth-sri%2Flmql/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eth-sri%2Flmql/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/eth-sri","download_url":"https://codeload.github.com/eth-sri/lmql/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254052870,"owners_count":22006717,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chatgpt","huggingface","language-model","programming-language"],"created_at":"2024-08-01T12:02:45.988Z","updated_at":"2025-05-14T02:07:32.110Z","avatar_url":"https://github.com/eth-sri.png","language":"Python","funding_links":[],"categories":["Uncategorized","Python","NLP","Prompt Generators","Projects","A01_文本生成_文本对话","Learning","其他LLM框架","\u003ca id=\"tools\"\u003e\u003c/a\u003e🛠️ Tools","Model Serving Frameworks","Other LLM Frameworks","Developer \u0026 Coding Assistants","Libraries","SDK, Libraries, Frameworks","Tools and Code"],"sub_categories":["Uncategorized","Developer tools","大语言对话模型及数据","Repositories","文章","Bleeding Edge ⚗️","LangManus","Videos Playlists","IDE Integrations","Python library, sdk or frameworks","Prompt Management and Testing"],"readme":"\u003cdiv align=\"center\"\u003e\n  \u003ca href=\"https://lmql.ai\"\u003e\n    \u003cimg src=\"https://raw.githubusercontent.com/eth-sri/lmql/web/lmql.svg\" alt=\"Logo\" width=\"80\" height=\"80\"\u003e\n  \u003c/a\u003e\n\n  \u003ch3 align=\"center\"\u003eLMQL\u003c/h3\u003e\n\n  \u003cp align=\"center\"\u003e\n    A programming language for large language models.\n    \u003cbr /\u003e\n    \u003ca href=\"https://lmql.ai/docs\"\u003e\u003cstrong\u003eDocumentation »\u003c/strong\u003e\u003c/a\u003e\n    \u003cbr /\u003e\n    \u003cbr /\u003e\n    \u003ca href=\"https://lmql.ai\"\u003eExplore Examples\u003c/a\u003e\n    ·\n    \u003ca href=\"https://lmql.ai/playground\"\u003ePlayground IDE\u003c/a\u003e\n    ·\n    \u003ca href=\"https://github.com/eth-sri/lmql/issues\"\u003eReport Bug\u003c/a\u003e\n    \u003cbr/\u003e\n    \u003cbr/\u003e\n    \u003ca href=\"https://discord.gg/7eJP4fcyNT\"\u003e\u003cimg src=\"https://img.shields.io/discord/1091288833997410414?style=plastic\u0026logo=discord\u0026color=blueviolet\u0026logoColor=white\" height=18/\u003e\u003c/a\u003e\n    \u003ca href=\"https://badge.fury.io/py/Lmql\"\u003e\u003cimg src=\"https://badge.fury.io/py/Lmql.svg?cacheSeconds=3600\" alt=\"PyPI version\" height=18\u003e\u003c/a\u003e\n  \u003c/p\u003e\n\u003c/div\u003e\n\n\nLMQL is a programming language for large language models (LLMs) based on a *superset of Python*. LMQL offers a novel way of interweaving traditional programming with the ability to call LLMs in your code. It goes beyond traditional templating languages by integrating LLM interaction natively at the level of your program code. \n\n\u003e \u003cimg width=\"200pt\" alt=\"image\" src=\"https://github.com/eth-sri/lmql/assets/17903049/bfa7ecf5-847b-47d0-9160-f8596aa47749\"\u003e \u003cbr/\u003eHelp us shape the next major version of LMQL by filling out the LMQL developer survey: https://forms.gle/pGvAicNpUhS1rAkK9\n\n## Explore LMQL\n\nAn LMQL program reads like standard Python, but top-level strings are interpreted as query strings: They are passed to an LLM, where template variables like `[GREETINGS]` are automatically completed by the model:\n\n```python\n\"Greet LMQL:[GREETINGS]\\n\" where stops_at(GREETINGS, \".\") and not \"\\n\" in GREETINGS\n\nif \"Hi there\" in GREETINGS:\n    \"Can you reformulate your greeting in the speech of \\\n     victorian-era English: [VIC_GREETINGS]\\n\" where stops_at(VIC_GREETINGS, \".\")\n\n\"Analyse what part of this response makes it typically victorian:\\n\"\n\nfor i in range(4):\n    \"-[THOUGHT]\\n\" where stops_at(THOUGHT, \".\")\n\n\"To summarize:[SUMMARY]\"\n```\n\nProgram Output:\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/eth-sri/lmql/assets/17903049/243176f1-dfd4-4129-a59e-ca3dee068295\"/\u003e\n  \u003cbr/\u003e\n\u003c/div\u003e\n\nLMQL allows you to express programs that contain both, traditional algorithmic logic, and LLM calls. \nAt any point during execution, you can prompt an LLM on program variables in combination with standard natural language prompting, to leverage model reasoning capabilities in the context of your program.\n\nTo better control LLM behavior, you can use the `where` keyword to specify constraints and data types of the generated text. This enables guidance of the model's reasoning process, and constraining of intermediate outputs using an [expressive constraint language](https://lmql.ai/docs/language/constraints.html).\n\nBeyond this linear form of scripting, LMQL also supports a number of decoding algorithms to execute your program, such as `argmax`, `sample` or even advanced branching decoders like [beam search and `best_k`](https://lmql.ai/docs/language/decoding.html). \n\nLearn more about LMQL by exploring thne **[Example Showcase](https://lmql.ai)**, by running your own programs in our **[browser-based Playground IDE](https://lmql.ai/playground)** or by reading the **[documentation](https://lmql.ai/docs)**.\n\n## Feature Overview\n\nLMQL is designed to make working with language models like OpenAI and 🤗 Transformers more efficient and powerful through its advanced functionality, including multi-variable templates, conditional distributions, constraints, datatypes and control flow.\n\n- [X] **Python Syntax**: Write your queries using [familiar Python syntax](https://lmql.ai/docs/language/overview.html), fully integrated with your Python environment (classes, variable captures, etc.)\n- [X] **Rich Control-Flow**: LMQL offers full Python support, enabling powerful [control flow and logic](https://lmql.ai/docs/language/scripted-prompting.html) in your prompting logic.\n- [X] **Advanced Decoding**: Take advantage of advanced decoding techniques like [beam search, best_k, and more](https://lmql.ai/docs/language/decoding.html).\n- [X] **Powerful Constraints Via Logit Masking**: Apply [constraints to model output](https://lmql.ai/docs/language/constraints.html), e.g. to specify token length, character-level constraints, datatype and stopping phrases to get more control of model behavior.\n- [X] **Optimizing Runtime:** LMQL leverages speculative execution to enable faster inference, constraint short-circuiting, more efficient token use and [tree-based caching](https://lmql.ai/blog/release-0.0.6.html).\n- [X] **Sync and Async API**: Execute hundreds of queries in parallel with LMQL's [asynchronous API](https://lmql.ai/docs/lib/python.html), which enables cross-query batching.\n- [X] **Multi-Model Support**: Seamlessly use LMQL with [OpenAI API, Azure OpenAI, and 🤗 Transformers models](https://lmql.ai/docs/models/).\n- [X] **Extensive Applications**: Use LMQL to implement advanced applications like [schema-safe JSON decoding](https://github.com/microsoft/guidance#guaranteeing-valid-syntax-json-example-notebook), [algorithmic prompting](https://twitter.com/lbeurerkellner/status/1648076868807950337), [interactive chat interfaces](https://twitter.com/lmqllang/status/1645776209702182917), and [inline tool use](https://lmql.ai/#kv).\n- [X] **Library Integration**: Easily employ LMQL in your existing stack leveraging [LangChain](https://lmql.ai/docs/lib/integrations/langchain.html) or [LlamaIndex](https://lmql.ai/docs/lib/integrations/llama_index.html).\n- [X] **Flexible Tooling**: Enjoy an interactive development experience with [LMQL's Interactive Playground IDE](https://lmql.ai/playground), and [Visual Studio Code Extension](https://marketplace.visualstudio.com/items?itemName=lmql-team.lmql).\n- [X] **Output Streaming**: Stream model output easily via [WebSocket, REST endpoint, or Server-Sent Event streaming](https://github.com/eth-sri/lmql/blob/main/src/lmql/output/).\n\n## Getting Started\n\nTo install the latest version of LMQL run the following command with Python ==3.10 installed.\n\n```\npip install lmql\n```\n\n**Local GPU Support:** If you want to run models on a local GPU, make sure to install LMQL in an environment with a GPU-enabled installation of PyTorch \u003e= 1.11 (cf. https://pytorch.org/get-started/locally/) and install via `pip install lmql[hf]`.\n\n## Running LMQL Programs\n\nAfter installation, you can launch the LMQL playground IDE with the following command:\n\n```\nlmql playground\n```\n\n\u003e Using the LMQL playground requires an installation of Node.js. If you are in a conda-managed environment you can install node.js via `conda install nodejs=14.20 -c conda-forge`. Otherwise, please see the official Node.js website https://nodejs.org/en/download/ for instructions how to install it on your system.\n\nThis launches a browser-based playground IDE, including a showcase of many exemplary LMQL programs. If the IDE does not launch automatically, go to `http://localhost:3000`.\n\nAlternatively, `lmql run` can be used to execute local `.lmql` files. Note that when using local HuggingFace Transformers models in the Playground IDE or via `lmql run`, you have to first launch an instance of the LMQL Inference API for the corresponding model via the command `lmql serve-model`.\n\n### Configuring OpenAI API Credentials\n\nIf you want to use OpenAI models, you have to configure your API credentials. To do so you can either define the `OPENAI_API_KEY` environment variable or create a file `api.env` in the active working directory, with the following contents:\n\n```\nopenai-org: \u003corg identifier\u003e\nopenai-secret: \u003capi secret\u003e\n```\n\nFor system-wide configuration, you can also create an `api.env` file at `$HOME/.lmql/api.env` or at the project root of your LMQL distribution (e.g. `src/` in a development copy).\n\nAlternatively, you can use LMQL-specific env variables `LMQL_OPENAI_SECRET` and `LMQL_OPENAI_ORG`.\n\n## Installing the Latest Development Version\n\nTo install the latest (bleeding-edge) version of LMQL, you can also run the following command:\n\n```\npip install git+https://github.com/eth-sri/lmql\n```\n\nThis will install the `lmql` package directly from the `main` branch of this repository. We do not continously test the `main` version, so it may be less stable than the latest PyPI release.\n\n# Contributing\n\nLMQL is a community-centric project. If you are interested in contributing to LMQL, please see the [contributing guidelines](./CONTRIBUTING.md) for more information, and reach out to us via [Discord](https://discord.gg/7eJP4fcyNT). We are looking forward to your contributions!\n\n## Setting Up a Development Environment\n\nTo setup a `conda` environment for local LMQL development with GPU support, run the following commands:\n\n```\n# prepare conda environment\nconda env create -f scripts/conda/requirements.yml -n lmql\nconda activate lmql\n\n# registers the `lmql` command in the current shell\nsource scripts/activate-dev.sh\n```\n\n\u003e **Operating System**: The GPU-enabled version of LMQL was tested to work on Ubuntu 22.04 with CUDA 12.0 and Windows 10 via WSL2 and CUDA 11.7. The no-GPU version (see below) was tested to work on Ubuntu 22.04 and macOS 13.2 Ventura or Windows 10 via WSL2.\n\n### Development without GPU\n\nThis section outlines how to setup an LMQL development environment without local GPU support. Note that LMQL without local GPU support only supports the use of API-integrated models like `openai/text-davinci-003`. Please see the OpenAI API documentation (https://platform.openai.com/docs/models/gpt-3-5) to learn more about the set of available models.\n\nTo setup a `conda` environment for LMQL with no GPU support, run the following commands:\n\n```\n# prepare conda environment\nconda env create -f scripts/conda/requirements-no-gpu.yml -n lmql-no-gpu\nconda activate lmql-no-gpu\n\n# registers the `lmql` command in the current shell\nsource scripts/activate-dev.sh\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feth-sri%2Flmql","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Feth-sri%2Flmql","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feth-sri%2Flmql/lists"}