{"id":15012970,"url":"https://github.com/olivierduchenne/llm_json_schema","last_synced_at":"2025-08-26T12:07:53.786Z","repository":{"id":208089330,"uuid":"720788136","full_name":"olivierDuchenne/LLM_json_schema","owner":"olivierDuchenne","description":"Guaranty the output of an LLM to follow a json schema.","archived":false,"fork":false,"pushed_at":"2023-12-06T16:00:01.000Z","size":31,"stargazers_count":25,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-12T03:51:45.305Z","etag":null,"topics":["ai","generative-ai","jsonschema","large-language-models","llamacpp","llm"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/olivierDuchenne.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-19T15:53:25.000Z","updated_at":"2025-01-20T16:20:13.000Z","dependencies_parsed_at":"2023-12-05T19:25:54.370Z","dependency_job_id":"bd24f38b-06a9-4c7d-8542-1ac4d7b17ba6","html_url":"https://github.com/olivierDuchenne/LLM_json_schema","commit_stats":{"total_commits":16,"total_committers":2,"mean_commits":8.0,"dds":0.125,"last_synced_commit":"444a1ddf91764818e17a3a093d4482f78de09f57"},"previous_names":["olivierduchenne/llm_json_schema"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/olivierDuchenne/LLM_json_schema","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/olivierDuchenne%2FLLM_json_schema","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/olivierDuchenne%2FLLM_json_schema/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/olivierDuchenne%2FLLM_json_schema/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/olivierDuchenne%2FLLM_json_schema/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/olivierDuchenne","download_url":"https://codeload.github.com/olivierDuchenne/LLM_json_schema/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/olivierDuchenne%2FLLM_json_schema/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":272219198,"owners_count":24894374,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-26T02:00:07.904Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","generative-ai","jsonschema","large-language-models","llamacpp","llm"],"created_at":"2024-09-24T19:43:32.364Z","updated_at":"2025-08-26T12:07:53.761Z","avatar_url":"https://github.com/olivierDuchenne.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# What is LLM_json_schema?\n\nLLM_json_schema can enforce the output of an LLM model to follow a given json schema. The following types are available: string, number, boolean, array, object.\n\nThe output is guaranteed to have the correct format.\n\n# Examples\n\n```bash\npython3 LLM_json_schema.py \\\n  --model models/Mistral-7B-Instruct-v0.1.gguf \\\n  --json-schema '{\"type\":\"object\", \"properties\":{\"country\":{\"type\":\"string\"}, \"capital\":{\"type\":\"string\"}}}' \\\n  --prompt \"What is the capital of France?\\n\\n\"\n```\n\noutput:\n```json\n{\"country\":\"France\", \"capital\":\"Paris\"}\n```\n\n```bash\npython3 LLM_json_schema.py \\\n  --model models/Mistral-7B-Instruct-v0.1.gguf \\\n  --json-schema '{\"type\":\"array\", \"items\":{\"type\":\"number\"}}' \\\n  --prompt \"Count until 20.\\n\\n\"\n```\n\noutput:\n```json\n[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20]\n```\n\n# How does it work?\n\nIt adds biases to the logits outputted by the LLM to enforce that only valid tokens can be chosen.\n\n# Installation\n\n## Install LLM_json_schema\n\n```bash\ncd LLM_json_schema\npip3 install -r requirements.txt\n```\n\n## Download an convert an LLM model\n\nDownload an LLM model, and convert it to the gguf format.\n\nExample:\n```bash\nmkdir models\ncd models\ngit clone https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1\ngit clone https://github.com/ggerganov/llama.cpp.git\npip install -r llama.cpp/requirements.txt\npython3 llama.cpp/convert.py Mistral-7B-Instruct-v0.1 \\\n  --outfile Mistral-7B-Instruct-v0.1.gguf \\\n  --outtype q8_0\ncd ..\n```\n\n# Usage from CLI\n\n```\nusage: LLM_json_schema.py [-h] --model-path MODEL_PATH --prompt PROMPT [--json-schema JSON_SCHEMA]\n\noptions:\n  -h, --help            show this help message and exit\n  --model-path MODEL_PATH\n                        Path to the LLM model in gguf format\n  --prompt PROMPT       Input prompt\n  --json-schema JSON_SCHEMA\n                        JSON schema to enforce\n```\n\n```bash\npython3 LLM_json_schema.py --model models/Mistral-7B-Instruct-v0.1.gguf --json-schema '{\"type\":\"object\", \"properties\":{\"country\":{\"type\":\"string\"}, \"captial\":{\"type\":\"string\"}}}' --prompt \"What is the capital of France?\\n\\n\"\n```\n\n# Usage from Python\n\n```python\nfrom LLM_json_schema import run_inference_constrained_by_json_schema\nimport os\nscript_path = os.path.dirname(os.path.realpath(__file__))\nmodel_path=os.environ.get('MODEL_PATH', os.path.join(script_path, \"./models/Mistral-7B-Instruct-v0.1.gguf\"))\nprompt = \"\\n\\n### Instruction:\\nWhat is the capital of France?\\n\\n### Response:\\n\"\njson_schema = {\"type\":\"object\", \"properties\":{\"country\":{\"type\":\"string\"}, \"capital\":{\"type\":\"string\"}}}\nfor chunk in run_inference_constrained_by_json_schema(model_path=model_path, json_schema=json_schema, prompt=prompt):\n    print(chunk, end=\"\", flush=True)\nprint(\"\")\n```\n\n# Citation\n\nIf you use this work please cite the following:\n\n```\n@article{duchenne2023llm_json_schema,\n  title={LLM Json Schema},\n  author={Olivier Duchenne},\n  journal={Github},\n  url={https://github.com/olivierDuchenne/LLM_json_schema},\n  year={2023}\n}\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Folivierduchenne%2Fllm_json_schema","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Folivierduchenne%2Fllm_json_schema","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Folivierduchenne%2Fllm_json_schema/lists"}