{"id":9524157,"url":"https://github.com/hukz18/DeFog","last_synced_at":"2025-08-23T04:30:59.164Z","repository":{"id":239497359,"uuid":"664615578","full_name":"hukz18/DeFog","owner":"hukz18","description":"Code release for the ICLR 2023 conference paper \"DeFog: Decision Transformer under Random Frame Dropping\"","archived":false,"fork":false,"pushed_at":"2023-07-10T11:19:45.000Z","size":3978,"stargazers_count":6,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-05-12T22:32:32.240Z","etag":null,"topics":["decision-transformers","frame-dropping","reinforcement-learning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hukz18.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-07-10T11:19:36.000Z","updated_at":"2024-05-12T22:32:44.432Z","dependencies_parsed_at":"2024-05-12T22:32:37.992Z","dependency_job_id":"a4aa3042-c092-48b3-961b-3a23c6a14d9d","html_url":"https://github.com/hukz18/DeFog","commit_stats":null,"previous_names":["hukz18/defog"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hukz18%2FDeFog","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hukz18%2FDeFog/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hukz18%2FDeFog/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hukz18%2FDeFog/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hukz18","download_url":"https://codeload.github.com/hukz18/DeFog/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":217307012,"owners_count":16157340,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["decision-transformers","frame-dropping","reinforcement-learning"],"created_at":"2024-05-12T22:09:13.693Z","updated_at":"2024-08-28T08:30:23.872Z","avatar_url":"https://github.com/hukz18.png","language":"Python","funding_links":[],"categories":["Papers"],"sub_categories":["ICLR 2023"],"readme":"# Official Repo for DeFog\n\n[![python](https://img.shields.io/badge/python-3.10-blue.svg)](https://www.python.org/downloads/release/python-3109/)\n[![arXiv](https://img.shields.io/badge/arXiv-2303.03391-b31b1b.svg)](https://arxiv.org/abs/2303.03391)\n[![License](https://img.shields.io/badge/License-MIT-green.svg)](https://github.com/hukz18/DeFog/blob/master/LICENSE)\n\n- [Overview](#overview)\n- [Setup](#setup)\n- [Datasets](#datasets)\n- [Acknowledgments](#acknowledgments)\n\n\n## Overview\n\nCode release for the ICLR 2023 conference paper:\n\n\u003ctable\u003e\u003ctr\u003e\u003ctd\u003e\n    \u003cstrong\u003e\n        \u003ca href=\"https://arxiv.org/abs/2303.03391\"\u003e\n            DeFog: Decision Transformer under Random Frame Dropping\n        \u003c/a\u003e\u003cbr/\u003e\n    \u003c/strong\u003e\n    Kaizhe Hu*\u003csup\u003e1,3\u003c/sup\u003e,  Ray Chen Zheng*\u003csup\u003e1,3\u003c/sup\u003e, Yang Gao\u003csup\u003e1,2,3\u003c/sup\u003e, Huazhe Xu\u003csup\u003e1,2,3\u003c/sup\u003e\u003cbr/\u003e\n    Conference on Learning Representations (ICLR) 2023\n\u003c/td\u003e\u003c/tr\u003e\u003c/table\u003e\n\n\u003csup\u003e1\u003c/sup\u003e\u003cem\u003eTsinghua Universtiy,\u003c/em\u003e\n\u003csup\u003e2\u003c/sup\u003e\u003cem\u003eShanghai AI Lab,\u003c/em\u003e\n\u003csup\u003e3\u003c/sup\u003e\u003cem\u003eShanghai Qi Zhi Institute\u003c/em\u003e\n\n---\n\nDeFog is an offline RL algorithm\nthat enables agents to act robustly in frame dropping scenarios (where observations are not arrived at each timestep, but have a possibility to be dropped). DeFog is able to remain much of the performance when the dropping rate is as high as 90% (as shown in the figure below), while the naive approach to repeat the last action has near zero perfermance once the dropping rate is above 50%.\n\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./images/defog_demo.gif\" width=\"90%\"/\u003e\n\u003c/p\u003e\n\nDeFog first randomly masks out data in the offline datasets and explicitly adds the time span of frame dropping as inputs. After that, a finetuning stage on  the same offline dataset with a higher mask rate would further boost the performance. Please refer to the conference paper for further information.\n\n## Setup\nTo create the environment of defog, firstly run the following command:\n```\nconda env create -f environment.yml\n```\nThis will create a conda environment named \"defog\". \n\nThe repo uses [Hydra](https://hydra.cc) to manage the configuration of the experiments. It enables users to overwrite hypermeters in the command line and store the training outputs in separate folders automatically. Please refer to the [documentation](https://hydra.cc/docs/intro/) for more details.\n\n### MuJoCo Experiments\n\nFor experiments in the MuJoCo environment, first cd to the `mujoco` folder, then run `pip install requirements.txt` to install the additional dependencies for mujoco.\n\nYou will need to download the dataset manually by running the `download_d4rl_dataset.py` first, this will download the [D4RL](https://github.com/Farama-Foundation/D4RL) datasets of mujoco to the `$HOME/.d4rl/datasets` folder. \n\nAfter downloading the datasets, run \n```python main.py```\nfor training DeFog on the hopper medium dataset. The result curves can be find under a autogenerated folder named after the current timestamp under the path `mujoco/runs`.\n\nFor other environments and datasets, overwrite the settings in default config file (`cfgs/config.yaml`) in the command line. For example, use  ```python main.py env=walker2d buffer.dataset=expert``` to run DeFog on the walker2d expert dataset. You can directly modify the config file as well.\n\nYou can also use the multiprocessing version to train multiple seeds in parallel:\n```python main_mp.py```\n\n### Atari Experiments\n\nFor experiments in the Atari environment, first cd to the `atari` folder, then run `pip install requirements.txt` to install the additional dependencies.\n\nAfter installing the dependencies, you can use ```python main.py``` for training DeFog on the breakout expert dataset. You can override the environment and dataset like ```python main.py env=qbert buffer.dataset_type=medium```. Please refer to the [documentation](https://github.com/takuseno/d4rl-atari) of d4rl-atari for a description of the datasets.\n\nYou may want to contrain the cpu usage in a multi-core platform by setting the environment variable `export OMP_NUM_THREADS=8`.\n\nYou can also use the multiprocessing version to train multiple seeds in parallel:\n```python main_mp.py```\n\n## Acknowledgements\n\nThe repo is a re-implementation of the code used in the paper, as we clean up and refactor the code a lot. The repo now utilizes a unified training pipeline, data buffer, and model architecture among the MuJoCo and Atari environment, which is not the case in the original Decision Transformer repo, we also optimized the data buffer for better performance. The code is based on the implementation of [min-decision-transformer](https://github.com/nikhilbarhate99/min-decision-transformer/tree/master). This repo is not necessarily an exact reproduction of the original paper (however the performance is basically the same), but a version that is easy to use, more understandable, and aims to better deliver the idea of DeFog.\n\n## Citation\n\nIf you find this code useful, consider citing our work:\n\n```bibtex\n@article{Hu2023DecisionTU,\n  title={Decision Transformer under Random Frame Dropping},\n  author={Kaizhe Hu and Ray Chen Zheng and Yang Gao and Huazhe Xu},\n  journal={ArXiv},\n  year={2023},\n  volume={abs/2303.03391}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhukz18%2FDeFog","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhukz18%2FDeFog","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhukz18%2FDeFog/lists"}