{"id":25894609,"url":"https://github.com/google/minja","last_synced_at":"2025-03-02T22:02:20.255Z","repository":{"id":266529774,"uuid":"865364319","full_name":"google/minja","owner":"google","description":"A minimalistic C++ Jinja templating engine for LLM chat templates","archived":false,"fork":false,"pushed_at":"2025-02-18T22:27:27.000Z","size":338,"stargazers_count":120,"open_issues_count":9,"forks_count":7,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-02-18T23:27:22.918Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/google.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-09-30T12:13:48.000Z","updated_at":"2025-02-18T22:27:31.000Z","dependencies_parsed_at":null,"dependency_job_id":"c626f60a-7b5e-45ab-b0f1-c8606a19a715","html_url":"https://github.com/google/minja","commit_stats":null,"previous_names":["google/minja"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google%2Fminja","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google%2Fminja/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google%2Fminja/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google%2Fminja/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/google","download_url":"https://codeload.github.com/google/minja/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241577073,"owners_count":19984941,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-03-02T22:01:50.020Z","updated_at":"2025-03-02T22:02:20.230Z","avatar_url":"https://github.com/google.png","language":"C++","readme":"# minja.hpp - A minimalistic C++ Jinja templating engine for LLM chat templates\n\n_**This is not an official Google product**_\n\nMinja is a minimalistic reimplementation of the [Jinja](https://github.com/pallets/jinja/) templating engine to integrate in/with C++ LLM projects (it's used in [llama.cpp](https://github.com/ggerganov/llama.cpp/pull/11016) and [GPT4All](https://github.com/nomic-ai/gpt4all/pull/3433)).\n\nIt is **not general purpose**: it includes just what’s needed for actual chat templates (very limited set of filters, tests and language features). Users with different needs should look at third-party alternatives such as [Jinja2Cpp](https://github.com/jinja2cpp/Jinja2Cpp), [Jinja2CppLight](https://github.com/hughperkins/Jinja2CppLight), or [inja](https://github.com/pantor/inja) (none of which we endorse).\n\n\u003e [!WARNING]  \n\u003e TL;DR: use of Minja is *at your own risk*, and the risks are plenty! See [Security \u0026 Privacy](#security--privacy) section below.\n\n[![CI](https://github.com/google/minja/actions/workflows/build.yml/badge.svg)](https://github.com/google/minja/actions/workflows/build.yml)\n\n## Design goals:\n\n- Support each and every major LLM found on HuggingFace\n  - See `MODEL_IDS` in [tests/CMakeLists.txt](./tests/CMakeLists.txt) for the list of models currently supported\n- Easy to integrate to/with projects such as [llama.cpp](https://github.com/ggerganov/llama.cpp) or [gemma.cpp](https://github.com/google/gemma.cpp):\n  - Header-only\n  - C++17\n  - Only depend on [nlohmann::json](https://github.com/nlohmann/json) (no Boost)\n  - Keep codebase small (currently 2.5k LoC) and easy to understand\n- *Decent* performance compared to Python.\n\n## Non-goals:\n\n- Address glaring Prompt injection risks in current Jinja chat templating practices. See [Security \u0026 Privacy](#security--privacy) below\n- Additional features from Jinja that aren't used by the template(s) of any major LLM (no feature creep!)\n  - Please don't submit PRs with such features, they will unfortunately be rejected.\n- Full Jinja compliance (neither syntax-wise, nor filters / tests / globals)\n\n## Usage:\n\nThis library is header-only: just copy the header(s) you need, make sure to use a compiler that handles C++17 and you're done. Oh, and get [nlohmann::json](https://github.com/nlohmann/json) in your include path.\n\nSee API in [minja/minja.hpp](./include/minja/minja.hpp) and [minja/chat-template.hpp](./include/minja/chat-template.hpp) (experimental).\n\nFor raw Jinja templating (see [examples/raw.cpp](./examples/raw.cpp)):\n\n```c++\n#include \u003cminja.hpp\u003e\n#include \u003ciostream\u003e\n\nusing json = nlohmann::ordered_json;\n\nint main() {\n    auto tmpl = minja::Parser::parse(\"Hello, {{ location }}!\", /* options= */ {});\n    auto context = minja::Context::make(minja::Value(json {\n        {\"location\", \"World\"},\n    }));\n    auto result = tmpl-\u003erender(context);\n    std::cout \u003c\u003c result \u003c\u003c std::endl;\n}\n```\n\nTo apply a template to a JSON array of `messages` and `tools` in the HuggingFace standard (see [examples/chat-template.cpp](./examples/chat-template.cpp)):\n\n```c++\n#include \u003cchat-template.hpp\u003e\n#include \u003ciostream\u003e\n\nusing json = nlohmann::ordered_json;\n\nint main() {\n    minja::chat_template tmpl(\n        \"{% for message in messages %}\"\n        \"{{ '\u003c|' + message['role'] + '|\u003e\\\\n' + message['content'] + '\u003c|end|\u003e' + '\\\\n' }}\"\n        \"{% endfor %}\",\n        /* bos_token= */ \"\u003c|start|\u003e\",\n        /* eos_token= */ \"\u003c|end|\u003e\"\n    );\n    std::cout \u003c\u003c tmpl.apply(\n        json::parse(R\"([\n            {\"role\": \"user\", \"content\": \"Hello\"},\n            {\"role\": \"assistant\", \"content\": \"Hi there\"}\n        ])\"),\n        json::parse(R\"([\n            {\"type\": \"function\", \"function\": {\"name\": \"google_search\", \"arguments\": {\"query\": \"2+2\"}}}\n        ])\"),\n        /* add_generation_prompt= */ true,\n        /* extra_context= */ {}) \u003c\u003c std::endl;\n}\n```\n\n(Note that some template quirks are worked around by [minja/chat-template.hpp](./include/minja/chat-template.hpp) so that all templates can be used the same way)\n\n## Supported features\n\nModels have increasingly complex templates (see [some examples](https://gist.github.com/ochafik/15881018fa0aeff5b7ddaa8ff14540b0)), so a fair bit of Jinja's language constructs is required to execute their templates properly.\n\nMinja supports the following subset of the [Jinja2/3 template syntax](https://jinja.palletsprojects.com/en/3.1.x/templates/):\n\n- Full expression syntax\n- Statements `{{% … %}}`, variable sections `{{ … }}`, and comments `{# … #}` with pre/post space elision `{%- … -%}` / `{{- … -}}` / `{#- … -#}`\n- `if` / `elif` / `else` / `endif`\n- `for` (`recursive`) (`if`) / `else` / `endfor` w/ `loop.*` (including `loop.cycle`) and destructuring\n- `break`, `continue` (aka [loop controls extensions](https://github.com/google/minja/pull/39))\n- `set` w/ namespaces \u0026 destructuring\n- `macro` / `endmacro`\n- `filter` / `endfilter`\n- Extensible filters collection: `count`, `dictsort`, `equalto`, `e` / `escape`, `items`, `join`, `joiner`, `namespace`, `raise_exception`, `range`, `reject` / `rejectattr` / `select` / `selectattr`, `tojson`, `trim`\n\nMain limitations (non-exhaustive list):\n\n- Not supporting [most filters](https://jinja.palletsprojects.com/en/3.0.x/templates/#builtin-filters). Only the ones actually used in templates of major (or trendy) models are/will be implemented.\n- No difference between `none` and `undefined`\n- Single namespace with all filters / tests / functions / macros / variables\n- No tuples (templates seem to rely on lists only)\n- No `if` expressions w/o `else` (but `if` statements are fine)\n- No `{% raw %}`, `{% block … %}`, `{% include … %}`, `{% extends … %},\n\n## Roadmap / TODOs\n\n- [ ] Fix known line difference issues on Windows\n- [ ] Document the various capabilities detectors + backfill strategies used\n- [ ] Propose integration w/ https://github.com/google/gemma.cpp\n- [x] Integrate to llama.cpp: https://github.com/ggerganov/llama.cpp/pull/11016 + https://github.com/ggerganov/llama.cpp/pull/9639\n- Improve fuzzing coverage:\n    - use thirdparty jinja grammar to guide exploration of inputs (or implement prettification of internal ASTs and use them to generate arbitrary values)\n    - fuzz each filter / test\n- Measure / track test coverage\n- Setup performance tests\n- Simplify two-pass parsing\n    - Pass tokens to IfNode and such\n- Macro nested set scope = global?\n- Get listed in https://jbmoelker.github.io/jinja-compat-tests/, https://en.cppreference.com/w/cpp/links/libs\n\n## Developer corner\n\n### Design overview\n\n- `minja::Parser` does two-phased parsing:\n  - its `tokenize()` method creates coarse template \"tokens\" (plain text section, or expression blocks or opening / closing blocks). Tokens may have nested expressions ASTs, parsed with `parseExpression()`\n  - its `parseTemplate()` method iterates on tokens to build the final `TemplateNode` AST.\n- `minja::Value` represents a Python-like value\n  - It relies on `nlohmann/json` for primitive values, but does its own JSON dump to be exactly compatible w/ the Jinja / Python implementation of `dict` string representation\n- `minja::chat_template` wraps a template and provides an interface similar to HuggingFace's chat template formatting. It also normalizes the message history to accommodate different expectations from some templates (e.g. `message.tool_calls.function.arguments` is typically expected to be a JSON string representation of the tool call arguments, but some templates expect the arguments object instead)\n- Testing involves a myriad of simple syntax tests and full e2e chat template rendering tests. For each model in `MODEL_IDS` (see [tests/CMakeLists.txt](./tests/CMakeLists.txt)), we fetch the `chat_template` field of the repo's `tokenizer_config.json`, use the official jinja2 Python library to render them on each of the (relevant) test contexts (in [tests/contexts](./tests/contexts)) into a golden file, and run a C++ test that renders w/ Minja and checks we get exactly the same output.\n\n### Adding new Templates / Building\n\n- Install Prerequisites:\n\n    - cmake\n    - GCC / clang\n    - python 3.8+ (for tests)\n    - flake8\n    - editorconfig-checker\n\n- Optional: test additional templates:\n\n    - Add their HuggingFace model identifier to `MODEL_IDS` in [tests/CMakeLists.txt](./tests/CMakeLists.txt) (e.g. `meta-llama/Llama-3.2-3B-Instruct`)\n    - For [gated models](https://huggingface.co/docs/transformers.js/en/guides/private) you have access to, first authenticate w/ HuggingFace:\n\n        ```bash\n        pip install huggingface_hub\n        huggingface-cli login\n        ```\n\n- Build \u0026 run tests (shorthand: `./scripts/tests.sh`):\n\n    ```bash\n    rm -fR build \u0026\u0026 \\\n        cmake -B build \u0026\u0026 \\\n        cmake --build build -j \u0026\u0026 \\\n        ctest --test-dir build -j --output-on-failure\n    ```\n\n- Fuzzing tests\n\n    - Note: `fuzztest` **[doesn't work](https://github.com/google/fuzztest/issues/179)** natively on Windows or MacOS.\n\n        \u003cdetails\u003e\n        \u003csummary\u003eShow instructions to run it inside a Docker container\u003c/summary\u003e\n\n        Beware of Docker Desktop's licensing: you might want to check out alternatives such as [colima](https://github.com/abiosoft/colima) (we'll still use the docker *client* in the example below).\n\n        ```bash\n        docker run --rm -it -v $PWD:/src:rw $( echo \"\n            FROM python:3.12-slim-bookworm\n            COPY requirements.txt /tmp\n            RUN apt update \u0026\u0026 \\\n                apt install -y cmake clang ccache git python3 python-is-python3 python3-pip \u0026\u0026 \\\n                apt-get clean \u0026\u0026 \\\n                rm -rf /var/lib/apt/lists/*\n            RUN pip install setuptools pip --upgrade --force-reinstall\n            RUN pip install -r /tmp/requirements.txt\n            CMD /usr/bin/bash\n            WORKDIR /src\n        \" | docker build . -f - -q )\n        ```\n\n        \u003c/details\u003e\n\n    - Build in [fuzzing mode](https://github.com/google/fuzztest/blob/main/doc/quickstart-cmake.md#fuzzing-mode) \u0026 run all fuzzing tests (optionally, set a higher `TIMEOUT` as env var):\n\n        ```bash\n        ./scripts/fuzzing_tests.sh\n        ```\n\n- If your model's template doesn't run fine, please consider the following before [opening a bug](https://github.com/googlestaging/minja/issues/new):\n\n    - Is the template using any unsupported filter / test / method / global function, and which one(s)?\n    - Is the template publicly available? Non-gated models are more likely to become supported.\n    - Which version of GCC / clang did you compile the tests with? On which OS version?\n    - If you intend to contribute a fix:\n        - Please read [CONTRIBUTING](./CONTRIBUTING.md) first. You'd have to sign a CLA, which your employer may need to accept.\n        - Please test as many gated models as possible (use `cmake -B build -DMINJA_TEST_GATED_MODELS=1 ...` and edit MODEL_LIST appropriately)\n\n- For bonus points, check the style of your edits with:\n\n    ```bash\n    flake8\n    editorconfig-checker\n    ```\n\n## Security \u0026 Privacy\n\n### Data protection\n\nThis library doesn't store any data by itself, it doesn't access files or the web, it only transforms a template (string) and context (JSON w/ fields `\"messages\"`, `\"tools\"`...) into a formatted string.\n\nYou should still be careful about untrusted third-party chat templates, as these could try and trigger bugs in Minja to exfiltrate user chat data (we only have limited fuzzing tests in place).\n\nRisks are even higher with any user-defined functions.\n\n### Do NOT produce HTML or JavaScript with this!\n\nHTML processing with this library is UNSAFE: no escaping of is performed (and the `safe` filter is a passthrough), leaving users vulnerable to XSS. Minja is not intended to produce HTML.\n\n### Beware of Prompt injection risks!\n\nPrompt injection is NOT protected against by this library.\n\nThere are many types of prompt injection, some quite exotic (cf. [data exfiltration exploits leveraging markdown image previews](https://promptarmor.substack.com/p/data-exfiltration-from-writercom)).\n\nFor the simpler cases, it is perfectly possible for a user to craft a message that will look like a system prompt, like an assistant response or like the results of tool calls. While some models might be fine-tuned to ignore system calls not at the very start of the prompt or out of order messages / tool call results, it is expected that most models will be very confused \u0026 successfully manipulated by such prompt injections.\n\nNote that injection of tool calls should typically not result in their execution as LLM inference engines should not try to parse the template output (just generated tokens), but this is something to watch out for when auditing such inference engines.\n\nAs there isn't any standard mechanism to escape special tokens to prevent those attacks, it is advised users of this library take their own message sanitization measures before applying chat templates. We do not recommend any specific such measure as each model reacts differently (some even understand l33tcode as instructions).\n","funding_links":[],"categories":["Miscellaneous","Recently Updated"],"sub_categories":["[Feb 28, 2025](/content/2025/02/28/README.md)"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogle%2Fminja","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgoogle%2Fminja","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogle%2Fminja/lists"}