{"id":25319361,"url":"https://github.com/HITsz-TMG/FilmAgent","last_synced_at":"2025-10-28T21:31:08.506Z","repository":{"id":267408164,"uuid":"849386436","full_name":"HITsz-TMG/FilmAgent","owner":"HITsz-TMG","description":"Resources of our paper \"FilmAgent: A Multi-Agent Framework for End-to-End Film Automation in Virtual 3D Spaces\". New versions in the making!","archived":false,"fork":false,"pushed_at":"2025-02-11T08:25:21.000Z","size":20065,"stargazers_count":820,"open_issues_count":8,"forks_count":111,"subscribers_count":31,"default_branch":"main","last_synced_at":"2025-02-11T09:40:00.997Z","etag":null,"topics":["filmmaking","multi-agent-system","unity3d"],"latest_commit_sha":null,"homepage":"https://filmagent.github.io/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/HITsz-TMG.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-29T13:58:59.000Z","updated_at":"2025-02-11T08:25:24.000Z","dependencies_parsed_at":"2025-01-30T14:32:17.492Z","dependency_job_id":null,"html_url":"https://github.com/HITsz-TMG/FilmAgent","commit_stats":null,"previous_names":["hitsz-tmg/filmagent"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HITsz-TMG%2FFilmAgent","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HITsz-TMG%2FFilmAgent/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HITsz-TMG%2FFilmAgent/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HITsz-TMG%2FFilmAgent/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/HITsz-TMG","download_url":"https://codeload.github.com/HITsz-TMG/FilmAgent/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":238725857,"owners_count":19520115,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["filmmaking","multi-agent-system","unity3d"],"created_at":"2025-02-13T20:03:17.218Z","updated_at":"2025-10-28T21:31:08.500Z","avatar_url":"https://github.com/HITsz-TMG.png","language":"Python","funding_links":[],"categories":["Python","Repos"],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n    \u003cimg src=\"pics/cover_page.jpg\" width=\"300\" style=\"margin-bottom: 0.2;\"/\u003e\n\u003cp\u003e\n\n\u003ch2 align=\"center\"\u003e \u003ca href=\"https://github.com/HITsz-TMG/FilmAgent\"\u003eFilmAgent: A Multi-Agent Framework for End-to-End Film Automation in Virtual 3D Spaces\u003c/a\u003e\u003c/h2\u003e\n\u003c!-- \u003ch5 align=\"center\"\u003e If you like our project, please consider giving us a star ⭐ on GitHub to stay updated with the latest developments.  \u003c/h2\u003e --\u003e\n\u003ch4 align=\"center\"\u003e\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"https://img.shields.io/badge/Version-1.0.0-blue.svg\" alt=\"Version\"\u003e \n\u003cimg src=\"https://img.shields.io/badge/License-Apache%202.0-green.svg\" alt=\"License\"\u003e\n\u003cimg src=\"https://img.shields.io/github/stars/HITsz-TMG/FilmAgent?color=yellow\" alt=\"Stars\"\u003e\n\u003cimg src=\"https://img.shields.io/github/issues/HITsz-TMG/FilmAgent?color=red\" alt=\"Issues\"\u003e\n\u003cimg src=\"https://img.shields.io/badge/python-3.8-purple.svg\" alt=\"Python\"\u003e\n\n\u003c!-- \u003cimg src=\"https://img.shields.io/github/stars/AIDC-AI/Marco-o1?color=yellow\" alt=\"Stars\"\u003e --\u003e\n\n\u003c!-- [![Project Page](https://img.shields.io/badge/Project_Page-FilmAgent-blue)](https://filmagent.github.io/)\n[![Project Page](https://img.shields.io/badge/Paper-Arxiv-yellow)](https://arxiv.org/abs/2501.12909)\n[![Project Page](https://img.shields.io/badge/Video-Youtube-red)](https://www.youtube.com/watch?v=hTI-0777iHU)\n![Gitea Stars](https://img.shields.io/gitea/stars/HITsz-TMG/FilmAgent) --\u003e\n\n\u003c/h4\u003e\n\n\u003cdiv align=\"center\"\u003e\n\n\u003c!-- **Affiliations:** --\u003e\n\n_**Zhenran Xu, Longyue Wang, Jifang Wang, Zhouyi Li, Senbao Shi, Xue Yang, Yiyu Wang, Baotian Hu, Jun Yu, Min Zhang**_\n\n🎯  [**Project Page**](https://filmagent.github.io)  :octocat:  [**Code**](https://github.com/HITsz-TMG/FilmAgent)  📝  [**Paper**](https://arxiv.org/abs/2501.12909) 🧑‍💻  [**Slides**](https://filmagent.github.io/static/SA24_FilmAgent.pdf)  📽️  [**Video**](https://www.youtube.com/watch?v=hTI-0777iHU)\n\n\u003ca href=\"https://trendshift.io/repositories/12871\" target=\"_blank\"\u003e\u003cimg src=\"https://trendshift.io/api/badge/repositories/12871\" alt=\"HITsz-TMG%2FFilmAgent | Trendshift\" style=\"width: 250px; height: 55px;\" width=\"250\" height=\"55\"/\u003e\u003c/a\u003e\n\n\u003c/div\u003e\n\n#\n\n**FilmAgent** is a multi-agent collaborative system for end-to-end film automation in 3D virtual spaces. \nFilmAgent simulates key crew roles—directors, screenwriters, actors, and cinematographers, and integrates efficient human workflows within a sandbox environment.\n\n\u003cdiv align=center\u003e\u003cimg src=\"https://github.com/HITsz-TMG/FilmAgent/blob/main/pics/intro.png\" height=\"100%\" width=\"78%\"/\u003e\u003c/div\u003e\n\n## 💥 News\n\n- `2025/2/24`: 🚀 We have integrated **DeepSeek**-v3 and r1 for model selection, allowing for more sophisticated decision-making processes.\n- `2025/2/11`: 🎬 We’ve just released a fan-made video about ***NeZha2***, in celebration of its record-breaking success at the box office. Click the image below to watch the video! 👇\n  [\u003cdiv align=center\u003e\u003cimg src=\"https://github.com/HITsz-TMG/FilmAgent/blob/main/pics/nezha_youtube.png\" width=\"450\"\u003e\u003c/div\u003e](https://www.youtube.com/watch?v=jY3n-AzBtUQ)\n- `2025/1/23`:  🙌 We're excited that FilmAgent is recommended by [AK](https://x.com/_akhaliq/status/1882268452716728789), [el.cine](https://x.com/EHuanglu/status/1882294685919772928) and [Theoretically Media](https://www.youtube.com/watch?v=0ebXKegfxWk\u0026t=899s). Thanks!\n- `2025/1/22`: 📄 Our paper is now accessible at https://arxiv.org/abs/2501.12909.\n\n\n\n## 🚀 Framework\n\nFollowing the traditional film studio workflow, we divide the whole film automation process into three sequential stages: idea development, scriptwriting and cinematography, and apply the **Critique-Correct-Verify**, **Debate-Judge** collaboration strategies. After these stages, each line in the script is specified with the positions of the actors, their actions, their dialogue, and the chosen camera shots.\n\n\u003cdiv align=center\u003e\u003cimg src=\"https://github.com/HITsz-TMG/FilmAgent/blob/main/pics/framework.png\" height=\"100%\" width=\"85%\"/\u003e\u003c/div\u003e\n\n## 🌟 Build Your own Film with FilmAgent\n\n1. Install Package\n```Shell\nconda create -n filmagent python==3.9.18\nconda activate filmagent\npip install -r env.txt\n```\n\n2. Create `Script` and `Logs` folders in the Filmagent directory, then replace the absolute pathname '/path/to/' with your specific path and modify the `topic` in the `main.py`. Modify the api_key in `LLMCaller.py`. Run the following commands to get the movie script created by the agents collaboratively:\n```bash\ncd /path/to/FilmAgent\nconda activate filmagent\npython main.py --model \"gpt-4o\" # openai\n# python main.py --model \"deepseek-chat\" ## deepseek-v3\n# python main.py --model \"deepseek-reasoner\" ## deepseek-r1 Using this model for a multi-agent process will be very slow, you could try using a single-agent process instead. (Refer to step 6)\n```\n\n3. We use [ChatTTS](https://github.com/2noise/ChatTTS) to provide voice acting for the characters in the script. You need to download the [ChatTTS](https://github.com/2noise/ChatTTS) repository to the `TTS` directory. Then replace the absolute pathname '/path/to/' with your specific path in the `tts_main.py`. Run the following commands to deploy the text-to-speech service:\n```bash\ncd /path/to/TTS\nconda create -n tts python==3.9.18\nconda activate tts\npip install -r env_tts.txt\npython tts_main.py\n```\n\n4. Modify the `Script_path`, `actos_path`, `Audio_path` and `url` in the `GenerateAudio.py`. Run the following commands to get the audio files:\n```bash\ncd /path/to/FilmAgent\nconda activate filmagent\npython GenerateAudio.py\n```\n\n5. We now have the `script.json`, `actors_profile.json`, and a series of `.wav` audio files. Next, we need to execute the script in Unity. The recommended version of the Unity editor is **Unity 2022.3.14f1c1**. You need to download the Unity project file we provide [[Dropbox](https://www.dropbox.com/scl/fi/atxhxnqppeofmt471dxr5/TheBigBang.zip?rlkey=o7zuqficetabkk2h1w1npkv3v\u0026st=5s41rzdv\u0026dl=0)][[Baidu Disk](https://pan.baidu.com/s/10wgfvtcG-xnx1fA9Rd_zwg?pwd=5aif)]. After decompression, open `TheBigBang\\Assets\\TheBigBang\\Manyrooms.unity` with Unity. Then replace all the absolute pathnames '/path/to/' with your specific path in `TheBigBang\\Assets\\Scirpts\\StartVideo.cs` and `TheBigBang\\Assets\\Scirpts\\ScriptExecute.cs`. Press **'ctrl+R'** in the unity interface to recompile, click **'Play'** to enter Game mode, then press **'E'** to start executing the script (sometimes the audio files load slowly, so you may need to play it 2 or 3 times before it can run normally).\n\n\u003e [!IMPORTANT]\n\u003e Please add `\"com.unity.nuget.newtonsoft-json\": \"3.0.2\"` inside `Packages/manifest.json` if it throws an error, \"*the type or namespace name jobject could not be found.*\"\n\n\u003cdiv align=center\u003e\u003cimg src=\"https://github.com/HITsz-TMG/FilmAgent/blob/main/pics/unity_1.png\" height=\"100%\" width=\"50%\"/\u003e\u003cimg src=\"https://github.com/HITsz-TMG/FilmAgent/blob/main/pics/unity_2.png\" height=\"100%\" width=\"50%\"/\u003e\u003c/div\u003e  \n\n6. For the tests on 15 topics in our experimental section, we provide three .py files: `test_full.py` (The full FilmAgent framework, utilizing multi-agent collaboration.), `test_no_interation.py` (A single agent is responsible for planning, scriptwriting, and cinematography, representing our FilmAgent framework without multi-agent collaboration algorithms.) and `test_cot.py` (A single agent generates the chain-of-thought rationale and the complete script). Modify the `model` in these `.py` files, you can try different LLMs.\n\n## 🌈 Case Study\n\n### 🤝 What does Multi-Agent Collaboration do?\nThe following table records some comparisons of the scripts and camera settings **before (⬅️) and after (➡️)** multi-agent collaboration, with excerpts from their discussion process.\n\n\u003cdiv align=center\u003e\u003cimg src=\"https://github.com/HITsz-TMG/FilmAgent/blob/main/pics/cases.png\" height=\"100%\" width=\"70%\"/\u003e\u003c/div\u003e\n\n📌 **Case Highlights:**\n- **Case #1** shows that Director-Screenwriter discussion reduces hallucinations in non-existent actions (e.g., standing suggest), enhances plot coherence, and ensures consistency across scenes.\n- **Case #2** shows that Actor-Director-Screenwriter discussion improves the alignment of dialogue with character profiles.\n- **Case #3**, in the Debate-Judge method in cinematography, demonstrates the correction of an inappropriate dynamic shot, which is replaced with a medium shot to better convey body language.\n- **Case #4** replaces a series of identical static shots with a mix of dynamic and static shots, resulting in a more diverse camera setup.\n\n### ⚖️ Comparison with Sora\n\n\u003cdiv align=center\u003e\u003cimg src=\"https://github.com/HITsz-TMG/FilmAgent/blob/main/pics/sora.png\" height=\"100%\" width=\"70%\"/\u003e\u003c/div\u003e\n\nWhile Sora (🔗 [Video](https://github.com/user-attachments/assets/65bb4c12-cba0-4ee9-a673-63ea5103fd76)) shows great adaptability to diverse locations, characters and shots, it **struggles with consistency and narrative delivery**, along with **strange artifacts**. \n\nIn contrast, FilmAgent requires pre-built 3D spaces, but it produces **coherent, physics-compliant** videos with strong **storytelling capabilities** (▶️ Watch on [Youtube](https://www.youtube.com/watch?v=yOOycdfolFY)).\n\n### ✨ Integration with Text-to-Video Models (e.g. Sora, Vidu)\n\nWe are currently working towards **merging the power of text-to-video models with FilmAgent**, unlocking enhanced storytelling, greater adaptability, and improved consistency. 🚀\n\nHere is a preview of our work—a fan-made tribute to the box office phenomenon *NeZha2* [▶️[video link](https://www.youtube.com/watch?v=jY3n-AzBtUQ)]. This video celebrates *NeZha2* surpassing a CNY 8 billion at the box office, officially crowning it as the highest-grossing film in a single territory! 🎉 \n\n## 📚 Citation\n\nIf you find FilmAgent useful for your research and applications, please cite using this BibTeX:\n```bibtex\n@misc{xu2025filmagent,\n      title={FilmAgent: A Multi-Agent Framework for End-to-End Film Automation in Virtual 3D Spaces}, \n      author={Zhenran Xu and Longyue Wang and Jifang Wang and Zhouyi Li and Senbao Shi and Xue Yang and Yiyu Wang and Baotian Hu and Jun Yu and Min Zhang},\n      year={2025},\n      eprint={2501.12909},\n      archivePrefix={arXiv},\n      primaryClass={cs.CL},\n      url={https://arxiv.org/abs/2501.12909}, \n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FHITsz-TMG%2FFilmAgent","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FHITsz-TMG%2FFilmAgent","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FHITsz-TMG%2FFilmAgent/lists"}