{"id":25531623,"url":"https://github.com/normal-computing/outlines","last_synced_at":"2026-01-18T04:30:18.452Z","repository":{"id":166459221,"uuid":"615403340","full_name":"dottxt-ai/outlines","owner":"dottxt-ai","description":"Structured Text Generation","archived":false,"fork":false,"pushed_at":"2024-10-27T13:32:08.000Z","size":6654,"stargazers_count":9001,"open_issues_count":211,"forks_count":452,"subscribers_count":46,"default_branch":"main","last_synced_at":"2024-10-29T11:29:58.391Z","etag":null,"topics":["cfg","generative-ai","json","llms","prompt-engineering","regex","structured-generation","symbolic-ai"],"latest_commit_sha":null,"homepage":"https://dottxt-ai.github.io/outlines/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dottxt-ai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-03-17T16:01:18.000Z","updated_at":"2024-10-29T10:11:57.000Z","dependencies_parsed_at":"2023-09-29T02:08:44.479Z","dependency_job_id":"7b963c36-31e2-4527-a684-b1e1cc444295","html_url":"https://github.com/dottxt-ai/outlines","commit_stats":null,"previous_names":["normal-computing/outlines","outlines-dev/outlines","dottxt-ai/outlines"],"tags_count":48,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dottxt-ai%2Foutlines","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dottxt-ai%2Foutlines/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dottxt-ai%2Foutlines/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dottxt-ai%2Foutlines/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dottxt-ai","download_url":"https://codeload.github.com/dottxt-ai/outlines/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239758849,"owners_count":19692037,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cfg","generative-ai","json","llms","prompt-engineering","regex","structured-generation","symbolic-ai"],"created_at":"2025-02-20T01:01:50.742Z","updated_at":"2025-02-20T01:01:51.808Z","avatar_url":"https://github.com/dottxt-ai.png","language":"Python","funding_links":[],"categories":["Prompting libraries \u0026 tools (in alphabetical order)","Other LLM Frameworks","Prompting libraries \u0026 tools","其他LLM框架","Repos","LLM Applications"],"sub_categories":["Videos Playlists","文章"],"readme":"\u003cdiv align=\"center\" style=\"margin-bottom: 1em;\"\u003e\n\n\u003cimg src=\"./docs/assets/images/logo.png\" alt=\"Outlines Logo\" width=500\u003e\u003c/img\u003e\n\n\n 🗒️ *Make LLMs speak the language of every application.* 🗒️\n\nMade with ❤👷️ by the team at [.txt](https://dottxt.co).\n\n[![Documentation][documentation-badge]][documentation]\n[![Contributors][contributors-badge]][contributors]\n[![Downloads][downloads-badge]][pypistats]\n[![Discord][discord-badge]][discord]\n\n[Youtube channel][youtube-dottxt] | [.txt blog][blog-dottxt] | [Twitter][dottxt-twitter]\n\n\n\u003c/div\u003e\n\n\n``` bash\npip install outlines\n```\n\nFirst time here? Go to our [setup guide](https://dottxt-ai.github.io/outlines/latest/welcome/)\n\n## Features\n\n- [x] 🤖 [Multiple model integrations](https://dottxt-ai.github.io/outlines/latest/installation): OpenAI, transformers, llama.cpp, exllama2, mamba\n- [x] 🖍️ Simple and powerful prompting primitives based on the [Jinja templating engine](https://jinja.palletsprojects.com/)\n- [x] 🚄 [Multiple choices](#multiple-choices), [type constraints](#type-constraint) and dynamic stopping\n- [x] ⚡ Fast [regex-structured generation](#efficient-regex-structured-generation)\n- [x] 🔥 Fast [JSON generation](#efficient-json-generation-following-a-pydantic-model) following a JSON schema or a Pydantic model\n- [x] 📝 [Grammar-structured generation](#using-context-free-grammars-to-guide-generation)\n- [x] 🐍 Interleave completions with loops, conditionals, and custom Python functions\n- [x] 💾 Caching of generations\n- [x] 🗂️ Batch inference\n- [x] 🎲 Sample with the greedy, multinomial and beam search algorithms (and more to come!)\n- [x] 🚀 [Serve with vLLM](https://dottxt-ai.github.io/outlines/latest/reference/serve/vllm), with official Docker image, [`outlinesdev/outlines`](https://hub.docker.com/r/outlinesdev/outlines)!\n\n\nOutlines  has new releases and features coming every week. Make sure to ⭐ star and 👀 watch this repository, follow [@dottxtai][dottxt-twitter] to stay up to date!\n\n## Why should I use structured generation?\n\n* It doesn't add any overhead during inference (cost-free)\n* It allows Open Source models to beat closed source models ([Mistral](https://x.com/dottxtai/status/1797692104023363765), [GPT-4](https://x.com/dottxtai/status/1798443290913853770))\n* [It speeds up inference](http://blog.dottxt.co/coalescence.html)\n* [It improves the performance of base models (GSM8K)](http://blog.dottxt.co/performance-gsm8k.html)\n* [It improves the performance of finetuned models (CoNNL)](https://predibase.com/blog/lorax-outlines-better-json-extraction-with-structured-generation-and-lora)\n* [It improves model efficiency (less examples needed)](https://huggingface.co/blog/evaluation-structured-outputs)\n\n## .txt company\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"./docs/assets/images/dottxt.png\" alt=\"Outlines Logo\" width=100\u003e\u003c/img\u003e\n\u003c/div\u003e\n\nWe started a company to keep pushing the boundaries of structured generation. Learn more about [.txt](https://twitter.com/dottxtai), and  [give our .json API a try](https://h1xbpbfsf0w.typeform.com/to/ZgBCvJHF) if you need a hosted solution ✨\n\n## Structured generation\n\nThe first step towards reliability of systems that include large language models\nis to ensure that there is a well-defined interface between their output and\nuser-defined code. **Outlines** provides ways to control the generation of\nlanguage models to make their output more predictable.\n\nThe following methods of structured generation are supported:\n\n- [Multiple choices](#multiple-choices)\n- [Type constraints](#type-constraint)\n- [Efficient regex-structured generation](#efficient-regex-structured-generation)\n- [Efficient JSON generation following a Pydantic model](#efficient-json-generation-following-a-pydantic-model)\n- [Using context-free grammars to guide generation](#using-context-free-grammars-to-guide-generation)\n- [Open functions](#open-functions)\n\n### Chat template tokens\n\nOutlines does not manage chat templating tokens when using instruct models. You must apply the chat template tokens to the prompt yourself. Chat template tokens are not needed for base models.\n\nPlease see [the documentation](https://dottxt-ai.github.io/outlines/latest/reference/chat_templating) on chat templating for more.\n\n### Multiple choices\n\nYou can reduce the completion to a choice between multiple possibilities:\n\n``` python\nimport outlines\n\nmodel_name = \"HuggingFaceTB/SmolLM2-360M-Instruct\"\nmodel = outlines.models.transformers(model_name)\n\n# You must apply the chat template tokens to the prompt!\n# See below for an example.\nprompt = \"\"\"\n\u003c|im_start|\u003esystem\nYou extract information from text.\n\u003c|im_end|\u003e\n\n\u003c|im_start|\u003euser\nWhat food does the following text describe?\n\nText: I really really really want pizza.\n\u003c|im_end|\u003e\n\u003c|im_start|\u003eassistant\n\"\"\"\n\ngenerator = outlines.generate.choice(model, [\"Pizza\", \"Pasta\", \"Salad\", \"Dessert\"])\nanswer = generator(prompt)\n\n# Likely answer: Pizza\n```\n\nYou can also pass in choices with an `Enum`:\n\n````python\nfrom enum import Enum\n\nclass Food(str, Enum):\n    pizza = \"Pizza\"\n    pasta = \"Pasta\"\n    salad = \"Salad\"\n    dessert = \"Dessert\"\n\ngenerator = outlines.generate.choice(model, Food)\nanswer = generator(prompt)\n# Likely answer: Pizza\n````\n\n### Type constraints\n\nYou can instruct the model to only return integers or floats:\n\n\n``` python\nimport outlines\n\nmodel = outlines.models.transformers(\"WizardLM/WizardMath-7B-V1.1\")\n\nprompt = \"\u003cs\u003eresult of 9 + 9 = 18\u003c/s\u003e\u003cs\u003eresult of 1 + 2 = \"\nanswer = outlines.generate.format(model, int)(prompt)\nprint(answer)\n# 3\n\nprompt = \"sqrt(2)=\"\ngenerator = outlines.generate.format(model, float)\nanswer = generator(prompt, max_tokens=10)\nprint(answer)\n# 1.41421356\n```\n\n### Efficient regex-structured generation\n\nOutlines also comes with fast regex-structured generation. In fact, the `choice` and\n`format` functions above all use regex-structured generation under the\nhood:\n\n``` python\nimport outlines\n\nmodel = outlines.models.transformers(\"microsoft/Phi-3-mini-4k-instruct\")\n\nprompt = \"\"\"\n\u003c|im_start|\u003esystem You are a helpful assistant.\n\u003c|im_end|\u003e\n\n\u003c|im_start|\u003euser\nWhat is an IP address of the Google DNS servers?\n\u003c|im_end|\u003e\n\u003c|im_start|\u003eassistant\nThe IP address of a Google DNS server is\n\n\"\"\"\n\ngenerator = outlines.generate.text(model)\nunstructured = generator(prompt, max_tokens=30)\n\ngenerator = outlines.generate.regex(\n    model,\n    r\"((25[0-5]|2[0-4]\\d|[01]?\\d\\d?)\\.){3}(25[0-5]|2[0-4]\\d|[01]?\\d\\d?)\",\n    sampler=outlines.samplers.greedy(),\n)\nstructured = generator(prompt, max_tokens=30)\n\nprint(unstructured)\n# 8.8.8.8\n#\n# \u003c|im_end|\u003e\n\nprint(structured)\n# 8.8.8.8\n```\n\nUnlike other libraries, regex-structured generation in Outlines is almost as fast\nas non-structured generation.\n\n### Efficient JSON generation following a Pydantic model\n\nOutlines users can guide the generation process so the output is *guaranteed* to follow a [JSON schema](https://json-schema.org/) or [Pydantic model](https://docs.pydantic.dev/latest/):\n\n```python\nfrom enum import Enum\nfrom pydantic import BaseModel, constr\n\nimport outlines\n\nclass Weapon(str, Enum):\n    sword = \"sword\"\n    axe = \"axe\"\n    mace = \"mace\"\n    spear = \"spear\"\n    bow = \"bow\"\n    crossbow = \"crossbow\"\n\n\nclass Armor(str, Enum):\n    leather = \"leather\"\n    chainmail = \"chainmail\"\n    plate = \"plate\"\n\n\nclass Character(BaseModel):\n    name: constr(max_length=10)\n    age: int\n    armor: Armor\n    weapon: Weapon\n    strength: int\n\n\nmodel = outlines.models.transformers(\"microsoft/Phi-3-mini-4k-instruct\")\n\n# Construct structured sequence generator\ngenerator = outlines.generate.json(model, Character)\n\n# Draw a sample\nseed = 789001\n\ncharacter = generator(\"Give me a character description\", seed=seed)\n\nprint(repr(character))\n# Character(name='Anderson', age=28, armor=\u003cArmor.chainmail: 'chainmail'\u003e, weapon=\u003cWeapon.sword: 'sword'\u003e, strength=8)\n\ncharacter = generator(\"Give me an interesting character description\")\n\nprint(repr(character))\n# Character(name='Vivian Thr', age=44, armor=\u003cArmor.plate: 'plate'\u003e, weapon=\u003cWeapon.crossbow: 'crossbow'\u003e, strength=125)\n```\n\nThe method works with union types, optional types, arrays, nested schemas, etc. Some field constraints are [not supported yet](https://github.com/dottxt-ai/outlines/issues/215), but everything else should work.\n\n### Efficient JSON generation following a JSON Schema\n\nSometimes you just want to be able to pass a JSON Schema instead of a Pydantic model. We've got you covered:\n\n``` python\nimport outlines\n\nschema = '''{\n    \"title\": \"Character\",\n    \"type\": \"object\",\n    \"properties\": {\n        \"name\": {\n            \"title\": \"Name\",\n            \"maxLength\": 10,\n            \"type\": \"string\"\n        },\n        \"age\": {\n            \"title\": \"Age\",\n            \"type\": \"integer\"\n        },\n        \"armor\": {\"$ref\": \"#/definitions/Armor\"},\n        \"weapon\": {\"$ref\": \"#/definitions/Weapon\"},\n        \"strength\": {\n            \"title\": \"Strength\",\n            \"type\": \"integer\"\n        }\n    },\n    \"required\": [\"name\", \"age\", \"armor\", \"weapon\", \"strength\"],\n    \"definitions\": {\n        \"Armor\": {\n            \"title\": \"Armor\",\n            \"description\": \"An enumeration.\",\n            \"enum\": [\"leather\", \"chainmail\", \"plate\"],\n            \"type\": \"string\"\n        },\n        \"Weapon\": {\n            \"title\": \"Weapon\",\n            \"description\": \"An enumeration.\",\n            \"enum\": [\"sword\", \"axe\", \"mace\", \"spear\", \"bow\", \"crossbow\"],\n            \"type\": \"string\"\n        }\n    }\n}'''\n\nmodel = outlines.models.transformers(\"microsoft/Phi-3-mini-4k-instruct\")\ngenerator = outlines.generate.json(model, schema)\ncharacter = generator(\"Give me a character description\")\n```\n\n### Using context-free grammars to guide generation\n\nFormal grammars rule the world, and Outlines makes them rule LLMs too. You can pass any context-free grammar in the EBNF format and Outlines will generate an output that is valid to this grammar:\n\n``` python\nimport outlines\n\narithmetic_grammar = \"\"\"\n    ?start: expression\n\n    ?expression: term ((\"+\" | \"-\") term)*\n\n    ?term: factor ((\"*\" | \"/\") factor)*\n\n    ?factor: NUMBER\n           | \"-\" factor\n           | \"(\" expression \")\"\n\n    %import common.NUMBER\n\"\"\"\n\nmodel = outlines.models.transformers(\"WizardLM/WizardMath-7B-V1.1\")\ngenerator = outlines.generate.cfg(model, arithmetic_grammar)\nsequence = generator(\"Alice had 4 apples and Bob ate 2. Write an expression for Alice's apples:\")\n\nprint(sequence)\n# (8-2)\n```\n\nThis was a very simple grammar, and you can use `outlines.generate.cfg` to generate syntactically valid Python, SQL, and much more than this. Any kind of structured text, really. All you have to do is search for \"X EBNF grammar\" on the web, and take a look at the [Outlines `grammars` module](https://github.com/dottxt-ai/outlines/tree/main/outlines/grammars).\n\n### Open functions\n\nOutlines can infer the structure of the output from the signature of a function. The result is a dictionary, and can be passed directly to the function using the usual dictionary expansion syntax `**`:\n\n```python\nimport outlines\n\n\ndef add(a: int, b: int):\n    return a + b\n\nmodel = outlines.models.transformers(\"WizardLM/WizardMath-7B-V1.1\")\ngenerator = outlines.generate.json(model, add)\nresult = generator(\"Return json with two integers named a and b respectively. a is odd and b even.\")\n\nprint(add(**result))\n# 3\n```\n\nA great advantage of passing functions directly to specify the structure is that the structure of the LLM will change with the function's definition. No need to change the code at several places!\n\nYou can also embed various functions into an enum to generate params:\n\n```python\nfrom enum import Enum\nfrom functools import partial\n\nimport outlines\n\n\ndef add(a: int, b: int) -\u003e int:\n    return a + b\n\ndef mul(c: float, d: float) -\u003e float:\n    return c * d\n\nclass Operation(Enum):\n    add = partial(add)\n    mul = partial(mul)\n\nmodel = outlines.models.transformers(\"WizardLM/WizardMath-7B-V1.1\")\ngenerator = outlines.generate.json(model, Operation)\nresult = generator(\"Return json with two float named c and d respectively. c is negative and d greater than 1.0.\")\n\nprint(result)\n# {'c': -3.14, 'd': 1.5}\n```\n\n## Prompting\n\nBuilding prompts can get messy. **Outlines** makes it easier to write and manage\nprompts by encapsulating templates inside \"template functions\".\n\nThese functions make it possible to neatly separate the prompt logic from the\ngeneral program logic; they can be imported from other modules and libraries.\n\nTemplate functions require no superfluous abstraction, they use the Jinja2\ntemplating engine to help build complex prompts in a concise manner:\n\n``` python\nimport outlines\n\nexamples = [\n    (\"The food was disgusting\", \"Negative\"),\n    (\"We had a fantastic night\", \"Positive\"),\n    (\"Recommended\", \"Positive\"),\n    (\"The waiter was rude\", \"Negative\")\n]\n\n@outlines.prompt\ndef labelling(to_label, examples):\n    \"\"\"You are a sentiment-labelling assistant.\n\n    {% for example in examples %}\n    {{ example[0] }} // {{ example[1] }}\n    {% endfor %}\n    {{ to_label }} //\n    \"\"\"\n\nmodel = outlines.models.transformers(\"microsoft/Phi-3-mini-4k-instruct\")\nprompt = labelling(\"Just awesome\", examples)\nanswer = outlines.generate.text(model)(prompt, max_tokens=100)\n```\n\n## Join us\n\n- 💡 **Have an idea?** Come chat with us on [Discord][discord]\n- 🔨 **Want to contribute?** Consult our [contribution guide](https://dottxt-ai.github.io/outlines/latest/community/contribute/).\n- 🐞 **Found a bug?** Open an [issue](https://github.com/dottxt-ai/outlines/issues)\n\n\n## Cite Outlines\n\n```\n@article{willard2023efficient,\n  title={Efficient Guided Generation for LLMs},\n  author={Willard, Brandon T and Louf, R{\\'e}mi},\n  journal={arXiv preprint arXiv:2307.09702},\n  year={2023}\n}\n```\n\n[documentation]: https://dottxt-ai.github.io/outlines/latest/welcome/\n[documentation-badge]: https://img.shields.io/readthedocs/outlines\n[contributors]: https://github.com/dottxt-ai/outlines/graphs/contributors\n[contributors-badge]: https://img.shields.io/github/contributors/dottxt-ai/outlines?style=flat-square\u0026logo=github\u0026logoColor=white\u0026color=ECEFF4\n[dottxt-twitter]: https://twitter.com/dottxtai\n[discord]: https://discord.gg/R9DSu34mGd\n[discord-badge]: https://img.shields.io/discord/1182316225284554793?color=81A1C1\u0026logo=discord\u0026logoColor=white\u0026style=flat-square\n[downloads-badge]: https://img.shields.io/pypi/dm/outlines?color=89AC6B\u0026logo=python\u0026logoColor=white\u0026style=flat-square\n[pypistats]: https://pypistats.org/packages/outlines\n[dottxt-twitter-badge]: https://img.shields.io/twitter/follow/dottxtai?style=social\n[youtube-dottxt]: https://www.youtube.com/@dottxt-ai\n[blog-dottxt]: https://blog.dottxt.co/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnormal-computing%2Foutlines","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnormal-computing%2Foutlines","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnormal-computing%2Foutlines/lists"}