{"id":14964959,"url":"https://github.com/nova-land/gbnf-compiler","last_synced_at":"2025-10-06T08:32:11.134Z","repository":{"id":202838529,"uuid":"708247187","full_name":"nova-land/gbnf-compiler","owner":"nova-land","description":"Plug n Play GBNF Compiler for llama.cpp","archived":false,"fork":false,"pushed_at":"2023-11-08T14:45:38.000Z","size":28,"stargazers_count":23,"open_issues_count":1,"forks_count":5,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-18T10:30:10.553Z","etag":null,"topics":["gbnf","grammar","llama","llamacpp","llm"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nova-land.png","metadata":{"files":{"readme":"ReadMe.MD","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-10-22T01:03:55.000Z","updated_at":"2025-01-17T11:02:17.000Z","dependencies_parsed_at":null,"dependency_job_id":"f4dbdf88-714b-4351-801d-ed9c40ab6997","html_url":"https://github.com/nova-land/gbnf-compiler","commit_stats":{"total_commits":14,"total_committers":2,"mean_commits":7.0,"dds":0.0714285714285714,"last_synced_commit":"afd2d84881d1b6df63b17832f4648d962c072b17"},"previous_names":["nova-land/gbnf-compiler"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nova-land%2Fgbnf-compiler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nova-land%2Fgbnf-compiler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nova-land%2Fgbnf-compiler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nova-land%2Fgbnf-compiler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nova-land","download_url":"https://codeload.github.com/nova-land/gbnf-compiler/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":235208946,"owners_count":18953003,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gbnf","grammar","llama","llamacpp","llm"],"created_at":"2024-09-24T13:34:01.781Z","updated_at":"2025-10-06T08:32:05.818Z","avatar_url":"https://github.com/nova-land.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# GBNF Compiler (Python)\n\n\u003e Dependency-free GBNF Compiler, plug and play for llama.cpp GBNF.\n\nSimple yet powerful GBNF Compiler, use it like handlebars.js with better response from LLMs.\n\n## Why ?\n\nGBNF is very useful to confine the response format from LLMs.\n\nIn most of the time, using sentence-based GBNF can produce better result than JSON-based GBNF in most LLMs without fine-tuning, `gbnf-compiler` provides the flexibility to construct or parse sentence / JSON / (anything you can think of) GBNF.\n\n## Getting Started\n\n`pip install gbnf-compiler`\n\n## How to Use\n\n1. Define the LLM Response Template\n2. Create the Rule\n3. Send it out, done!\n\n```python\nimport requests\nfrom gbnf_compiler import *\n\n# Define your Prompt\nprompt = \"What tool will you use to calculate 2^5 ?\"\n\n# Define the LLM Response Template\n# Each {{}} is a variable with a rule\ntemplate = \"I choose {{tool}} because {{reason}}\"\n\ntools = MultipleChoice('tool', ['calculator', 'web-search', 'web-browse'])\nc = GBNFCompiler(template, { 'tool': tools, 'reason': SingleSentence() })\nprint(c.grammar())\n\n# Try a dummy result\ntext = \"I choose calculator because it is the most efficient and accurate way to calculate 2^5.\"\nresult = c.parse(text)\nprint(result)\n\n\"\"\"\nResult: \n{'tool': 'calculator', 'reason': 'it is the most efficient and accurate way to calculate 2^5.'}\n\"\"\"\n\n# Example: Send it out to local llama.cpp\ndef template(role: str, prompt: str):\n    return \"\"\"[INST] \u003c\u003cSYS\u003e\u003e\n{role}\n\u003c\u003c/SYS\u003e\u003e\n{prompt}\n[/INST]\"\"\".format(role=role, prompt=prompt)\n\ndata_json = {\n    \"prompt\": template(\"\", prompt), \"temperature\": 0.0,\n    \"n_predict\": 512, \"top_p\": 0.2, \"top_k\": 10,\n    \"stream\": False, \"grammar\": c.grammar() }\n\nresp = requests.post(\n    url=\"http://127.0.0.1:9999/completion\",\n    headers={\"Content-Type\": \"application/json\"},\n    json=data_json,\n)\nresult = resp.json()[\"content\"]\nprint(c.parse(result))\n```\n\n### Useful Rule Examples\n\n1. [`ItemList`](./examples/item_list.py): The Result will automatically compiled into a list of string.\n    - Drawback: The LLM might keep generating more items until max tokens.\n\n2. [`Point Form`](./examples/point_form.py): Create a Point Form Result with specific numbers.\n    - This can restrict the LLM to provide limited items.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnova-land%2Fgbnf-compiler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnova-land%2Fgbnf-compiler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnova-land%2Fgbnf-compiler/lists"}