{"id":15829110,"url":"https://github.com/THUDM/LongCite","last_synced_at":"2025-10-16T21:31:33.463Z","repository":{"id":255565629,"uuid":"850338335","full_name":"THUDM/LongCite","owner":"THUDM","description":"LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QA","archived":false,"fork":false,"pushed_at":"2024-12-31T03:25:42.000Z","size":15889,"stargazers_count":453,"open_issues_count":9,"forks_count":33,"subscribers_count":11,"default_branch":"main","last_synced_at":"2025-01-26T16:01:44.441Z","etag":null,"topics":["benchmark","citation-generation","fine-tuning","llm","long-context"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/THUDM.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-31T13:53:10.000Z","updated_at":"2025-01-17T04:00:44.000Z","dependencies_parsed_at":"2024-12-29T12:02:03.637Z","dependency_job_id":"026df1ce-f5f8-4ca0-8b4b-b778b3162b2d","html_url":"https://github.com/THUDM/LongCite","commit_stats":{"total_commits":53,"total_committers":5,"mean_commits":10.6,"dds":"0.24528301886792447","last_synced_commit":"89b8f1ba59583803895ddf3110d808a2c1224050"},"previous_names":["thudm/longcite"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUDM%2FLongCite","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUDM%2FLongCite/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUDM%2FLongCite/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUDM%2FLongCite/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/THUDM","download_url":"https://codeload.github.com/THUDM/LongCite/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":236749064,"owners_count":19198617,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmark","citation-generation","fine-tuning","llm","long-context"],"created_at":"2024-10-05T11:00:37.292Z","updated_at":"2025-10-16T21:31:25.217Z","avatar_url":"https://github.com/THUDM.png","language":"Python","funding_links":[],"categories":["A01_文本生成_文本对话"],"sub_categories":["大语言对话模型及数据"],"readme":"\n\u003cp align=\"center\" width=\"100%\"\u003e\n\u003cimg src=\"https://github.com/user-attachments/assets/089ccebe-2ebc-4c8e-a9e0-9d82f5e253f9\" width=\"80%\" alt=\"LongCite\"\u003e\n\u003c/p\u003e\n\n# LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QA\n\n\u003cp align=\"center\"\u003e\n    🤗 \u003ca href=\"https://huggingface.co/datasets/THUDM/LongCite-45k\" target=\"_blank\"\u003eHF Repo\u003c/a\u003e • 📃 \u003ca href=\"https://arxiv.org/abs/2409.02897\" target=\"_blank\"\u003ePaper\u003c/a\u003e • 🚀 \u003ca href=\"https://huggingface.co/spaces/THUDM-HF-SPACE/LongCite\" target=\"_blank\"\u003eHF Space\u003c/a\u003e\n\u003c/p\u003e\n\n[English](./README.md) | [中文](./README_zh.md)\n\nhttps://github.com/user-attachments/assets/68f6677a-3ffd-41a8-889c-d56a65f9e3bb\n\n## 🔍 Table of Contents\n- [⚙️ LongCite Deployment](#deployment)\n- [🤖️ CoF pipeline](#pipeline)\n- [🖥️ Model Training](#training)\n- [📊 Evaluation](#evaluation)\n- [📝 Citation](#citation)\n\n\u003ca name=\"deployment\"\u003e\u003c/a\u003e\n## ⚙️ LongCite Deployment\n\n**Environmental Setup**:\nWe recommend using `transformers\u003e=4.43.0` to successfully deploy our models.\n\nWe open-source two models: [LongCite-glm4-9b](https://huggingface.co/THUDM/LongCite-glm4-9b) and [LongCite-llama3.1-8b](https://huggingface.co/THUDM/LongCite-llama3.1-8b), which are trained based on [GLM-4-9B](https://huggingface.co/THUDM/glm-4-9b) and [Meta-Llama-3.1-8B](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B), respectively, and support up to 128K context. These two models point to the \"LongCite-9B\" and \"LongCite-8B\" models in our paper. Given a long-context-based query, these models can generate accurate responses and precise sentence-level citations, making it easy for users to verify the output information. Try the model:\n```python\nimport json\nimport torch\nfrom transformers import AutoTokenizer, AutoModelForCausalLM\n\ntokenizer = AutoTokenizer.from_pretrained('THUDM/LongCite-glm4-9b', trust_remote_code=True)\nmodel = AutoModelForCausalLM.from_pretrained('THUDM/LongCite-glm4-9b', torch_dtype=torch.bfloat16, trust_remote_code=True, device_map='auto')\n\ncontext = '''\nW. Russell Todd, 94, United States Army general (b. 1928). February 13. Tim Aymar, 59, heavy metal singer (Pharaoh) (b. 1963). Marshall \\\"Eddie\\\" Conway, 76, Black Panther Party leader (b. 1946). Roger Bonk, 78, football player (North Dakota Fighting Sioux, Winnipeg Blue Bombers) (b. 1944). Conrad Dobler, 72, football player (St. Louis Cardinals, New Orleans Saints, Buffalo Bills) (b. 1950). Brian DuBois, 55, baseball player (Detroit Tigers) (b. 1967). Robert Geddes, 99, architect, dean of the Princeton University School of Architecture (1965–1982) (b. 1923). Tom Luddy, 79, film producer (Barfly, The Secret Garden), co-founder of the Telluride Film Festival (b. 1943). David Singmaster, 84, mathematician (b. 1938).\n'''\nquery = \"What was Robert Geddes' profession?\"\nresult = model.query_longcite(context, query, tokenizer=tokenizer, max_input_length=128000, max_new_tokens=1024)\n\nprint(\"Answer:\\n{}\\n\".format(result['answer']))\nprint(\"Statement with citations:\\n{}\\n\".format(\n  json.dumps(result['statements_with_citations'], indent=2, ensure_ascii=False)))\nprint(\"Context (divided into sentences):\\n{}\\n\".format(result['splited_context']))\n```\nYou may deploy your own LongCite chatbot (like the one we show in the above video) by running\n```\nCUDA_VISIBLE_DEVICES=0 streamlit run demo.py --server.fileWatcherType none\n```\nAlternatively, you can deploy the model with [vllm](https://github.com/vllm-project/vllm), which allows faster generation and multiconcurrent server. See the code example in [vllm_inference.py](https://github.com/THUDM/LongCite/blob/main/vllm_inference.py).\n\n\u003ca name=\"pipeline\"\u003e\u003c/a\u003e\n## 🤖️ CoF Pipeline\n![cof](https://github.com/user-attachments/assets/dae25838-3ce0-4a2c-80f7-307c8128e5c4)\n\nWe are also open-sourcing CoF (Coarse to Fine) under `CoF/`, our automated SFT data construction pipeline for generating high-quality long-context QA instances with fine-grained citations. Please configure your API key in the `utils/llm_api.py`, then run the following four scripts to obtain the final data: \n`1_qa_generation.py`, `2_chunk_level_citation.py`, `3_sentence_level_citaion.py`, and `4_postprocess_and_filter.py`.\n\n\n\u003ca name=\"training\"\u003e\u003c/a\u003e\n## 🖥️ Model Training\n\nYou can download and save the **LongCite-45k** dataset through the Hugging Face datasets ([🤗 HF Repo](https://huggingface.co/datasets/THUDM/LongCite-45k)):\n```python\nfrom datasets import load_dataset\ndataset = load_dataset('THUDM/LongCite-45k')\nfor split, split_dataset in dataset.items():\n    split_dataset.to_json(\"train/long.jsonl\")\n```\nYou can mix it with general SFT data such as [ShareGPT](https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered/tree/main/HTML_cleaned_raw_dataset). We adopt [Metragon-LM](https://github.com/NVIDIA/Megatron-LM) for model training. For a more lightweight implementation, you may adopt the code and environment from [LongAlign](https://github.com/THUDM/LongAlign), which can support a max training sequence length of 32k tokens for GLM-4-9B and Llama-3.1-8B. \n\n\u003ca name=\"evaluation\"\u003e\u003c/a\u003e\n## 📊 Evaluation\nWe introduce an automatic benchmark: **LongBench-Cite**, which adopt long-context QA pairs from [LongBench](https://github.com/THUDM/LongBench) and [LongBench-Chat](https://github.com/THUDM/LongAlign), to measure the citation quality as well as response correctness in long-context QA scenarios. \n\nWe provide our evaluation data and code under `LongBench-Cite/`. Run `pred_sft.py` and `pred_one_shot.py` to get responses from fine-tuned models (e.g., LongCite-glm4-9b) and normal models (e.g., GPT-4o). Then run `eval_cite.py` and `eval_correct.py` to evaluate the citation quality and response correctness. Remember to configure your OpenAI API key in `utils/llm_api.py` since we adopt GPT-4o as the judge.\n\nHere are the evaluation results on **LongBench-Cite**:\n![eval_results](https://github.com/user-attachments/assets/1ef68c5f-63f0-4041-8f19-b8694b0d68c2)\n\n\u003ca name=\"citation\"\u003e\u003c/a\u003e\n## 📝 Citation\n\nIf you find our work useful, please consider citing LongCite:\n\n```\n@article{zhang2024longcite,\n  title = {LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QA} \n  author={Jiajie Zhang and Yushi Bai and Xin Lv and Wanjun Gu and Danqing Liu and Minhao Zou and Shulin Cao and Lei Hou and Yuxiao Dong and Ling Feng and Juanzi Li},\n  journal={arXiv preprint arXiv:2409.02897},\n  year={2024}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FTHUDM%2FLongCite","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FTHUDM%2FLongCite","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FTHUDM%2FLongCite/lists"}