{"id":26245213,"url":"https://github.com/evolvinglmms-lab/egolife","last_synced_at":"2025-04-05T15:02:29.601Z","repository":{"id":279872761,"uuid":"795465838","full_name":"EvolvingLMMs-Lab/EgoLife","owner":"EvolvingLMMs-Lab","description":"[CVPR 2025] EgoLife: Towards Egocentric Life Assistant","archived":false,"fork":false,"pushed_at":"2025-03-19T01:42:16.000Z","size":37439,"stargazers_count":248,"open_issues_count":1,"forks_count":17,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-04-05T01:15:34.823Z","etag":null,"topics":["egocentric-vision","omnimodal","rag"],"latest_commit_sha":null,"homepage":"https://egolife-ai.github.io/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/EvolvingLMMs-Lab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-05-03T10:44:56.000Z","updated_at":"2025-04-04T13:17:01.000Z","dependencies_parsed_at":"2025-03-07T09:24:25.929Z","dependency_job_id":null,"html_url":"https://github.com/EvolvingLMMs-Lab/EgoLife","commit_stats":null,"previous_names":["evolvinglmms-lab/egolife"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EvolvingLMMs-Lab%2FEgoLife","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EvolvingLMMs-Lab%2FEgoLife/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EvolvingLMMs-Lab%2FEgoLife/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EvolvingLMMs-Lab%2FEgoLife/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/EvolvingLMMs-Lab","download_url":"https://codeload.github.com/EvolvingLMMs-Lab/EgoLife/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247353729,"owners_count":20925329,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["egocentric-vision","omnimodal","rag"],"created_at":"2025-03-13T12:32:10.101Z","updated_at":"2025-04-05T15:02:29.584Z","avatar_url":"https://github.com/EvolvingLMMs-Lab.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# The EgoLife Project\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://arxiv.org/abs/2503.03803\" target='_blank'\u003e\n    \u003cimg src=\"https://img.shields.io/badge/Paper-CVPR2025-b31b1b?style=flat-square\"\u003e\n  \u003c/a\u003e\n  \u0026nbsp;\u0026nbsp;\u0026nbsp;\n  \u003ca href=\"https://egolife-ai.github.io/\" target='_blank'\u003e\n    \u003cimg src=\"https://img.shields.io/badge/Page-egolife--ai.github.io-228c22?style=flat-square\"\u003e\n  \u003c/a\u003e\n  \u0026nbsp;\u0026nbsp;\u0026nbsp;\n  \u003ca href=\"https://huggingface.co/collections/lmms-lab/egolife-67c04574c2a9b64ab312c342\" target='_blank'\u003e\n    \u003cimg src=\"https://img.shields.io/badge/Data-HuggingFace-FFD21E?style=flat-square\"\u003e\n  \u003c/a\u003e\n  \u0026nbsp;\u0026nbsp;\u0026nbsp;\n  \u003ca href=\"https://github.com/egolife-ai/EgoLife\" target='_blank'\u003e\n    \u003cimg src=\"https://hits.seeyoufarm.com/api/count/incr/badge.svg?url=https%3A%2F%2Fgithub.com%2Fegolife-ai%2FEgoLife\u0026count_bg=%23FFA500\u0026title_bg=%23555555\u0026icon=\u0026icon_color=%23E7E7E7\u0026title=visitors\u0026edge_flat=true\"\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n| ![teaser.png](assets/egolife_teaser.png) |\n|:---|\n| \u003cp align=\"justify\"\u003e\u003cb\u003eFigure 1. The Overview of EgoLife Project.\u003c/b\u003e EgoLife is an ambitious egocentric AI project capturing multimodal daily activities of six participants over a week. Using Meta Aria glasses, synchronized third-person cameras, and mmWave sensors, it provides a rich dataset for long-term video understanding. Leveraging this dataset, the project enables AI assistants—powered by EgoGPT and EgoRAG—to support memory, habit tracking, event recall, and task management, advancing real-world egocentric AI applications.\n\u003c/p\u003e\n\n\n## 🚀 News\n🤹 2025-02: We provide [HuggingFace gradio demo]() and [self-deployed demo]() for EgoGPT.\n\n🌟 2025-02: The EgoLife video is released at [HuggingFace](https://huggingface.co/datasets/lmms-lab/EgoLife) and uploaded to [Youtube](https://www.youtube.com/playlist?list=PLlweuFnfdo6F9Fu2Kyhc-kXu3qnaVsYOu) as video collection.\n\n🌟 2025-02: We release the EgoIT-99K dataset at [HuggingFace](https://huggingface.co/collections/lmms-lab/egolife-67c04574c2a9b64ab312c342). \n\n🌟 2025-02: We release the first version of [EgoGPT](./EgoGPT/) and [EgoRAG](./EgoRAG/) codebase.\n\n📖 2025-02: Our arXiv submission is currently on hold. For an overview, please visit our [academic page](https://egolife-ai.github.io/blog/).\n\n🎉 2025-02: The paper is accepted to CVPR 2025. Please be invited to our [online EgoHouse](https://egolife-ai.github.io/).\n\n\n## What is in this repo?\n### 🧠 EgoGPT: Clip-Level Multimodal Understanding\nEgoGPT is an **omni-modal vision-language model** fine-tuned on egocentric datasets. It performs **continuous video captioning**, extracting key events, actions, and context from first-person video and audio streams. \n\n**Key Features:**\n- **Dense captioning** for visual and auditory events.\n- **Fine-tuned for egocentric scenarios** (optimized for EgoLife data).\n\n### 📖 EgoRAG: Long-Context Question Answering\nEgoRAG is a **retrieval-augmented generation (RAG) module** that enables long-term reasoning and memory reconstruction. It retrieves **relevant past events** and synthesizes contextualized answers to user queries.\n\n**Key Features:**\n- **Hierarchical memory bank** (hourly, daily summaries).\n- **Time-stamped retrieval** for context-aware Q\u0026A.\n\n\n## 📂 Code Structure\n```bash\nEgoLife/\n│── assets/                # General assets used across the project\n│── EgoGPT/                # Core module for egocentric omni-modal model\n│── EgoRAG/                # Retrieval-augmented generation (RAG) module\n│── README.md              # Main documentation for the overall project\n```\nPlease dive in to the project of [EgoGPT](./EgoGPT/) and [EgoRAG](./EgoRAG/) for more details.\n\n## 📢 Citation\n\nIf you use EgoLife in your research, please cite our work:\n\n```bibtex\n@misc{yang2025egolifeegocentriclifeassistant,\n      title={EgoLife: Towards Egocentric Life Assistant}, \n      author={Jingkang Yang and Shuai Liu and Hongming Guo and Yuhao Dong and Xiamengwei Zhang and Sicheng Zhang and Pengyun Wang and Zitang Zhou and Binzhu Xie and Ziyue Wang and Bei Ouyang and Zhengyu Lin and Marco Cominelli and Zhongang Cai and Yuanhan Zhang and Peiyuan Zhang and Fangzhou Hong and Joerg Widmer and Francesco Gringoli and Lei Yang and Bo Li and Ziwei Liu},\n      year={2025},\n      eprint={2503.03803},\n      archivePrefix={arXiv},\n      primaryClass={cs.CV},\n      url={https://arxiv.org/abs/2503.03803}, \n}\n```\n\n## 📝 License\nThis project is licensed under the S-Lab license. See the [LICENSE](LICENSE) file for details.\n\n## Star History\n\n[![Star History Chart](https://api.star-history.com/svg?repos=EvolvingLMMs-Lab/EgoLife\u0026type=Date)](https://star-history.com/#EvolvingLMMs-Lab/EgoLife\u0026Date)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fevolvinglmms-lab%2Fegolife","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fevolvinglmms-lab%2Fegolife","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fevolvinglmms-lab%2Fegolife/lists"}