{"id":47233754,"url":"https://github.com/globalaiplatform/langdiff","last_synced_at":"2026-03-13T21:24:18.772Z","repository":{"id":309310450,"uuid":"1033583182","full_name":"globalaiplatform/langdiff","owner":"globalaiplatform","description":"Progressive UI from LLM","archived":false,"fork":false,"pushed_at":"2025-09-10T08:58:24.000Z","size":402,"stargazers_count":275,"open_issues_count":4,"forks_count":11,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-02-18T07:19:34.953Z","etag":null,"topics":["ai","ai-engineering","generative-ui","json","llm","llm-streaming","llm-structured-output","partial-json","progressive-ui","python","streaming-json","streaming-ui"],"latest_commit_sha":null,"homepage":"https://langdiff.readthedocs.io/en/latest/","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/globalaiplatform.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-08-07T03:40:50.000Z","updated_at":"2026-02-15T15:03:00.000Z","dependencies_parsed_at":"2025-09-10T10:28:59.187Z","dependency_job_id":"b1bf4255-12f1-4e0c-a4dd-2107631bbadb","html_url":"https://github.com/globalaiplatform/langdiff","commit_stats":null,"previous_names":["globalaiplatform/langdiff"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/globalaiplatform/langdiff","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/globalaiplatform%2Flangdiff","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/globalaiplatform%2Flangdiff/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/globalaiplatform%2Flangdiff/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/globalaiplatform%2Flangdiff/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/globalaiplatform","download_url":"https://codeload.github.com/globalaiplatform/langdiff/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/globalaiplatform%2Flangdiff/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30476111,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-13T20:45:58.186Z","status":"ssl_error","status_checked_at":"2026-03-13T20:45:20.133Z","response_time":60,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","ai-engineering","generative-ui","json","llm","llm-streaming","llm-structured-output","partial-json","progressive-ui","python","streaming-json","streaming-ui"],"created_at":"2026-03-13T21:24:17.975Z","updated_at":"2026-03-13T21:24:18.752Z","avatar_url":"https://github.com/globalaiplatform.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ⚖️ LangDiff: Progressive UI from LLM\n\n[![build](https://img.shields.io/github/actions/workflow/status/globalaiplatform/langdiff/ci.yml?label=build)](https://github.com/globalaiplatform/langdiff/actions/workflows/ci.yml)\n[![docs](https://img.shields.io/badge/docs-latest-blue)](https://langdiff.readthedocs.io/en/latest/)\n[![pypi](https://img.shields.io/pypi/v/langdiff.svg)](https://pypi.python.org/pypi/langdiff)\n[![license](https://img.shields.io/github/license/globalaiplatform/langdiff.svg)](https://github.com/globalaiplatform/langdiff/blob/main/LICENSE)\n[![Global AI Platform](https://img.shields.io/badge/made%20by-Global%20AI%20Platform-646EFF)](https://globalaiplatform.com/)\n\nLangDiff is a Python library that solves the hard problems of streaming structured LLM outputs to frontends.\n\n![Diagram](/docs/diagram.png)\n\nLangDiff provides intelligent partial parsing with granular, type-safe events as JSON structures build token by token, plus automatic JSON Patch generation for efficient frontend synchronization. Build responsive AI applications where your backend structures and frontend experiences can evolve independently. Read more about it on the [Motivation](#motivation) section.\n\n## Demo\n\nClick the image below.\n\n[\u003cimg width=\"1175\" height=\"537\" alt=\"image\" src=\"https://github.com/user-attachments/assets/3ce97baf-8856-44b0-8f40-b9db75e05950\" /\u003e](https://globalaiplatform.github.io/langdiff/)\n\n\n## Core Features\n\n### Streaming Parsing\n- Define schemas for streaming structured outputs using Pydantic-style models.\n- Receive granular, type-safe callbacks (`on_append`, `on_update`, `on_complete`) as tokens stream in.\n- Derive Pydantic models from LangDiff models for seamless interop with existing libraries and SDKs like OpenAI SDK.\n\n\u003ctable\u003e\n\u003ctr\u003e\n\u003ctd\u003eWithout LangDiff\u003c/td\u003e \u003ctd\u003eWith LangDiff\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003e\n\n```python\nparse_partial('{\"it')\nparse_partial('{\"items\":')\nparse_partial('{\"items\": [\"Buy a b')\nparse_partial('{\"items\": [\"Buy a banana\", \"')\nparse_partial('{\"items\": [\"Buy a banana\", \"Pack b')\nparse_partial('{\"items\": [\"Buy a banana\", \"Pack bags\"]}')\n```\n\n\u003c/td\u003e\n\u003ctd\u003e\n\n```python\non_item_list_append(\"\", index=0)\non_item_append(\"Buy a b\")\non_item_append(\"anana\")\non_item_list_append(\"\", index=1)\non_item_append(\"Pack b\")\non_item_append(\"ags\")\n```\n\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/table\u003e\n\n### Change Tracking\n- Track mutations without changing your code patterns by instrumenting existing Pydantic models, or plain Python dict/list/objects.\n- Generate JSON Patch diffs automatically for efficient state synchronization between frontend and backend.\n\n\u003ctable\u003e\n\u003ctr\u003e\n\u003ctd\u003eWithout LangDiff\u003c/td\u003e \u003ctd\u003eWith LangDiff\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003e\n\n```http\ndata: {\"it\ndata: ems\":\ndata:  [\"Buy a b\ndata: anana\", \"\ndata: Pack b\ndata: ags\"]}\n```\n\n\u003c/td\u003e\n\u003ctd\u003e\n\n```http\ndata: {\"op\": \"add\", \"path\": \"/items/-\", \"value\": \"Buy a b\"}\ndata: {\"op\": \"append\", \"path\": \"/items/0\", \"value\": \"anana\"}\ndata: {\"op\": \"add\", \"path\": \"/items/-\", \"value\": \"Pack b\"}\ndata: {\"op\": \"append\", \"path\": \"/items/1\", \"value\": \"ags\"}\n```\n\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/table\u003e\n\n## Usage\n\n### Installation\n\n```\nuv add langdiff\n```\n\nFor pip,\n\n```\npip install langdiff\n```\n\n### Streaming Parsing\n\nSuppose you want to generate a multi-section article with an LLM. Rather than waiting for the entire response, \nyou can stream the article progressively by first generating section titles as they're determined, \nthen streaming each section's content as it's written.\n\n![Demo Video](/docs/demo.gif)\n\nStart by defining model classes that specify your streaming structure:\n\n```python\nimport langdiff as ld\n\nclass ArticleGenerationResponse(ld.Object):\n    section_titles: ld.List[ld.String]\n    section_contents: ld.List[ld.String]\n```\n\nThe `ld.Object` and `ld.List` classes handle internal streaming progression automatically. \nCreate an instance and attach event handlers to respond to streaming events:\n\n```python\nui = Article(sections=[])\nresponse = ArticleGenerationResponse()\n\n@response.section_titles.on_append\ndef on_section_title_append(title: ld.String, index: int):\n    ui.sections.append(Section(title=\"\", content=\"\", done=False))\n\n    @title.on_append\n    def on_title_append(chunk: str):\n        ui.sections[index].title += chunk\n\n@response.section_contents.on_append\ndef on_section_content_append(content: ld.String, index: int):\n    if index \u003e= len(ui.sections):\n        return\n\n    @content.on_append\n    def on_content_append(chunk: str):\n        ui.sections[index].content += chunk\n\n    @content.on_complete\n    def on_content_complete(_):\n        ui.sections[index].done = True\n```\n\nCreate a streaming parser with `ld.Parser` and feed token chunks from your LLM stream (`push()`):\n\n```python\nimport openai\nclient = openai.OpenAI()\n\nwith client.chat.completions.stream(\n    model=\"gpt-5-mini\",\n    messages=[{\"role\": \"user\", \"content\": \"Write me a guide to open source a Python library.\"}],\n    \n    # You can derive a Pydantic model\n    # from a LangDiff model and use it with OpenAI SDK.\n    response_format=ArticleGenerationResponse.to_pydantic(),\n\n) as stream:\n    with ld.Parser(response) as parser:\n        for event in stream:\n            if event.type == \"content.delta\":\n                parser.push(event.delta)\n                print(ui)\n    print(ui)\n```\n\n### Change Tracking\n\nTo automatically track changes to your `Article` object, wrap it with `ld.track_change()`:\n\n```diff\n- ui = Article(sections=[])\n+ ui, diff_buf = ld.track_change(Article(sections=[]))\n```\n\nNow all modifications to `ui` and its nested objects are automatically captured in `diff_buf`.\n\nAccess the accumulated changes using `diff_buf.flush()`:\n\n```python\nimport openai\nclient = openai.OpenAI()\n\nwith client.chat.completions.stream(\n    ...\n) as stream:\n    with ld.Parser(response) as parser:\n        for event in stream:\n            if event.type == \"content.delta\":\n                parser.push(event.delta)\n                print(diff_buf.flush())  # list of JSON Patch objects\n    print(diff_buf.flush())\n\n# Output:\n# [{\"op\": \"add\", \"path\": \"/sections/-\", \"value\": {\"title\": \"\", \"content\": \"\", \"done\": false}}]\n# [{\"op\": \"append\", \"path\": \"/sections/0/title\", \"value\": \"Abs\"}]\n# [{\"op\": \"append\", \"path\": \"/sections/0/title\", \"value\": \"tract\"}]\n# ...\n```\n\nNotes:\n\n- `flush()` returns and clears the accumulated changes, so each call gives you only new modifications\n- Send these lightweight diffs to your frontend instead of retransmitting entire objects\n- Diffs use JSON Patch format ([RFC 6902](https://datatracker.ietf.org/doc/html/rfc6902)) with an additional `append` operation for efficient string building\n- For standard JSON Patch compatibility, use `ld.track_change(..., tracker_cls=ld.JSONPatchChangeTracker)`\n\n## Motivation\n\nModern AI applications increasingly rely on LLMs to generate structured data rather than just conversational text. While LLM providers offer structured output capabilities (like OpenAI's JSON mode), streaming these outputs poses unique challenges that existing tools don't adequately address.\n\n### The Problem with Traditional Streaming Approaches\n\nWhen LLMs generate complex JSON structures, waiting for the complete response creates poor user experiences. Standard streaming JSON parsers can't handle incomplete tokens - for example, `{\"sentence\": \"Hello,` remains unparseable until the closing quote arrives. This means users see nothing until substantial chunks complete, defeating the purpose of streaming.\n\nEven partial JSON parsing libraries that \"repair\" incomplete JSON don't fully solve the issues:\n- **No type safety**: You lose static type checking when dealing with partial objects\n- **No granular control**: Can't distinguish between complete and incomplete fields\n\n### The Coupling Problem\n\nA more fundamental issue emerges in production applications: tightly coupling frontend UIs to LLM output schemas. When you stream raw JSON chunks from backend to frontend, several problems arise:\n\n**Schema Evolution**: Improving prompts often requires changing JSON schemas. If your frontend directly consumes LLM output, every schema change may cause a breaking change.\n\n**Backward Compatibility**: Consider a restaurant review summarizer that originally outputs:\n```json\n{\"summary\": [\"Food is great\", \"Nice interior\"]}\n```\n\nAdding emoji support requires a new schema:\n```json\n{\"summaryV2\": [{\"emoji\": \"🍽️\", \"text\": \"Food is great\"}]}\n```\n\nSupporting both versions in a single LLM output creates inefficiencies and synchronization issues between the redundant fields.\n\n**Implementation Detail Leakage**: Frontend code becomes dependent on LLM provider specifics, prompt engineering decisions, and token streaming patterns.\n\n### The LangDiff Approach\n\nLangDiff solves these problems through two key innovations:\n\n1. **Intelligent Streaming Parsing**: Define schemas that understand the streaming nature of LLM outputs. Get type-safe callbacks for partial updates, complete fields, and new array items as they arrive.\n2. **Change-Based Synchronization**: Instead of streaming raw JSON, track mutations on your application objects and send lightweight JSON Patch diffs to frontends. This decouples UI state from LLM output format.\n\nThis architecture allows:\n- **Independent Evolution**: Change LLM prompts and schemas without breaking frontends\n- **Efficient Updates**: Send only what changed, not entire objects\n- **Type Safety**: Maintain static type checking throughout the streaming process\n\nLangDiff enables you to build responsive, maintainable AI applications where the backend prompt engineering and frontend user experience can evolve independently.\n\n## License\n\nApache-2.0. See the [LICENSE](/LICENSE) file for details.\n\n## Demo\n\nSee [`py/example.py`](/py/example.py) for a runnable end-to-end demo using streaming parsing and diff tracking.\n\n## Star History\n\n[![Star History Chart](https://api.star-history.com/svg?repos=globalaiplatform/langdiff\u0026type=Date)](https://www.star-history.com/#globalaiplatform/langdiff\u0026Date)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fglobalaiplatform%2Flangdiff","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fglobalaiplatform%2Flangdiff","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fglobalaiplatform%2Flangdiff/lists"}