{"id":26738325,"url":"https://github.com/danelpeng/rdmae_nav","last_synced_at":"2025-06-15T10:04:56.939Z","repository":{"id":284120857,"uuid":"656446790","full_name":"danelpeng/RDMAE_Nav","owner":"danelpeng","description":"A robust embodied navigation agent to various visual corruptions.","archived":false,"fork":false,"pushed_at":"2025-03-24T08:40:37.000Z","size":11604,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-24T09:38:34.489Z","etag":null,"topics":["embodied-ai","masked-autoencoder","robustness","visual-navigation"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/danelpeng.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-06-21T01:32:49.000Z","updated_at":"2025-03-24T08:40:41.000Z","dependencies_parsed_at":"2025-03-24T09:48:36.590Z","dependency_job_id":null,"html_url":"https://github.com/danelpeng/RDMAE_Nav","commit_stats":null,"previous_names":["danelpeng/rdmae_nav"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danelpeng%2FRDMAE_Nav","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danelpeng%2FRDMAE_Nav/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danelpeng%2FRDMAE_Nav/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danelpeng%2FRDMAE_Nav/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/danelpeng","download_url":"https://codeload.github.com/danelpeng/RDMAE_Nav/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248897261,"owners_count":21179566,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["embodied-ai","masked-autoencoder","robustness","visual-navigation"],"created_at":"2025-03-28T03:29:06.013Z","updated_at":"2025-04-14T14:32:25.718Z","avatar_url":"https://github.com/danelpeng.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Regularized Denoising Masked Visual Pretraining for Robust Embodied PointGoal Navigation\n\n\n\n## Overall\n\n![](https://github.com/dniwsac/RDMAE_Nav/blob/master/assets/rdmae.png)\n\n\n\n![](https://github.com/dniwsac/RDMAE_Nav/blob/master/assets/overall.png)\n\n\n\n## Installation\n\nNVIDIA GeForce RTX 3060, python3.6, torch1.10.2+cu113\n\n1. First install [Anaconda](https://docs.anaconda.com/anaconda/install/linux/)\n\n2. Use conda environment\n\n   ```\n   conda env create --file robustnav.yml --name \u003cENV-NAME\u003e\n   \n   e.g. conda env create --file robustnav.yml --name rdmaenav\n   ```\n\n3. Activate the conda environment\n\n   ```\n   conda activate \u003cENV-NAME\u003e\n   ```\n\n\n\n## Train\n\n### Setting\n\nSpecify the path of the pretrained visual module in \"allenact/embodiedai/preprocessors/vit.py \"\n\n```\ne.g. \npretrain_path = \"/home/pengjie/projects/RDMAE_Nav/checkpoints/rdmae.pth\"\n```\n\n### Begin\n\nRun the following command to train a navigation agent\n\n```\nbash train.sh\n```\n\n### Description\n\n```\npython main.py \\\n    -o storage/robothor-pointnav-rgb-rdmae-ddppo \\\n    -exp pointnav_robothor_vanilla_rgb_rdmae_ddppo \\\n    -b projects/robustnav/experiments/train \\\n    -s 12345 \\\n    -et rnav_pointnav_vanilla_rgb_rdmae_ddppo_clean\n    \n#Parameters Description\n-o: \tModel and log storage location\n-exp:  \tExperimental configuration, together with -b to form the task file and path\n-b: \tTask folder path\n-s: \tRandom seed\n-et: \tThe name of the folder where the tensorboard is saved\n```\n\n\n\n## Test\n\n### Setting\n\nThe pretraining model consists of two parts：visual module and policy module\n\n1. Specify the path of the pretrained visual module in \"allenact/embodiedai/preprocessors/vit.py \"\n\n   ```\n   e.g. \n   pretrain_path = \"/home/pengjie/projects/RDMAE_Nav/checkpoints/rdmae.pth\"\n   ```\n\n2. Specify the path of the pretrained policy module in \"test.sh\"\n\n   ```\n   e.g.\n   agent=/home/pengjie/projects/RDMAE_Nav/checkpoints/rdmae_nav.pt\n   ```\n\n### Begin\n\nRun the following command to train a navigation agent\n\n```\nbash test.sh\n```\n\n### Description\n\nTests for clean and visual corruptions. Results are saved in \"storage/robothor-pointnav-rgb-rdmae-ddppo-eval\".\n\n\n\n## Pretrain Visualization\n![](https://github.com/dniwsac/RDMAE_Nav/blob/master/assets/viz.png)\n\n\n## Visualization\n\n### Setting\n\nSpecify the episodes to be visualized in projects/tutorials/running_inference_tutorial.py, running_inference_tutorial_fov.py, and running_inference_tutorial_cam_crack.py.\n\n```\nviz_ep_ids: \tSpecify the episodes to be visualized in the top-down trajectory\nviz_video_ids： Specify the episodes to be visualized for egocentric navigation\n```\n\n### Begin\n\n```\nbash viz.sh\n```\n\n## Description\n\nThe visualization results are saved in the tensorboard file in \"storage/robothor-pointnav-rgb-rdmae-ddppo-eval\"\n\n**egocentric navigation**\n\n![](https://github.com/dniwsac/RDMAE_Nav/blob/master/assets/video_example.gif)\n\n**top-down trajectory**\n\n![](https://github.com/dniwsac/RDMAE_Nav/blob/master/assets/scene_viz.png)\n\n![](https://github.com/dniwsac/RDMAE_Nav/blob/master/assets/coor_viz.png)\n\n\n\n## Caution\n\nThis project requires internet connection to render the UI interface when running, so **internet connection is required** during training/testing and can be disconnected after training/testing starts.\n\n\n\n## Acknowledgement\n\nThis project has heavily referenced code from [robustnav](https://github.com/allenai/robustnav), thanks!\n\n\n\n## Others\n\nThis project was completed by [Daniel Peng](https://github.com/danelpeng).\n\n\n\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdanelpeng%2Frdmae_nav","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdanelpeng%2Frdmae_nav","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdanelpeng%2Frdmae_nav/lists"}