{"id":19532370,"url":"https://github.com/ankur-deka/hiding-leader-identity","last_synced_at":"2026-02-24T02:32:05.564Z","repository":{"id":112199965,"uuid":"390172143","full_name":"Ankur-Deka/Hiding-Leader-Identity","owner":"Ankur-Deka","description":"IROS 2021 - Hiding  Leader’s  Identity  in  Leader-Follower  Navigation through  MARL","archived":false,"fork":false,"pushed_at":"2022-03-07T16:44:27.000Z","size":3026,"stargazers_count":9,"open_issues_count":1,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-19T03:03:45.194Z","etag":null,"topics":["adversarial-learning","iros","machine-learning","marl","multi-agent-reinforcement-learning","multi-agent-systems","reinforcement-learning","robotics"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Ankur-Deka.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-07-28T01:15:06.000Z","updated_at":"2024-04-17T17:17:44.000Z","dependencies_parsed_at":"2023-03-09T04:15:45.561Z","dependency_job_id":null,"html_url":"https://github.com/Ankur-Deka/Hiding-Leader-Identity","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Ankur-Deka/Hiding-Leader-Identity","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ankur-Deka%2FHiding-Leader-Identity","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ankur-Deka%2FHiding-Leader-Identity/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ankur-Deka%2FHiding-Leader-Identity/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ankur-Deka%2FHiding-Leader-Identity/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Ankur-Deka","download_url":"https://codeload.github.com/Ankur-Deka/Hiding-Leader-Identity/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ankur-Deka%2FHiding-Leader-Identity/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29769176,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-24T01:40:24.820Z","status":"online","status_checked_at":"2026-02-24T02:00:07.497Z","response_time":75,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["adversarial-learning","iros","machine-learning","marl","multi-agent-reinforcement-learning","multi-agent-systems","reinforcement-learning","robotics"],"created_at":"2024-11-11T01:50:37.402Z","updated_at":"2026-02-24T02:32:05.527Z","avatar_url":"https://github.com/Ankur-Deka.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Hiding Leader's Identity\n\nThis is the official repository for the following two papers:\n\n1. **Hiding Leader's Identity in Leader-Follower Navigation through Multi-Agent Reinforcement Learning**\u003cbr/\u003e\n   Ankur Deka, Wenhao Luo, Huao Li, Michael Lewis, Katia Sycara\u003cbr/\u003e\n   *Accepted to IEEE/RSJ International Conference on Intelligent Robots and Systems ([IROS](https://www.iros2021.org/)) 2021*\u003cbr/\u003e\n   Paper Link: [Arxiv](https://arxiv.org/abs/2103.06359), [IROS](https://ieeexplore.ieee.org/abstract/document/9636314)\n2. **Human vs. Deep Neural Network Performance at a Leader Identification Task**\u003cbr/\u003e\n   Ankur Deka, Michael Lewis, Huao Li, Phillip Walker, Katia Sycara*\u003cbr/\u003e\n   *Accepted to Human Factors and Ergonomics Society ([HFES](https://www.hfes.org/Events/International-Annual-Meeting)) Annual Meeting 2021*\u003cbr/\u003e\n   Paper Link: [PITT](https://sites.pitt.edu/~cmlewis/pubs/ankur-hfes.pdf), [HFES](https://journals.sagepub.com/doi/abs/10.1177/1071181321651127)\n\n## Installation\n\nI have tested this repository with  Python 3.6 on Ubuntu 18.04. First install [Anaconda](https://docs.anaconda.com/anaconda/install/linux/) and then run:\n\n```\ngit clone git@github.com:Ankur-Deka/Hiding-Leader-Identity.git\ncd Hiding-Leader-Identity\nconda create python=3.6 pip --name HidingIdentity\nconda activate HidingIdentity\npip install -r requirements.txt\n```\n\nNote: `gym_vecenv` MUST be installed from the link in `requirements.txt`. `pip install gym_vecenv` will NOT give the same results!\n\n## Quick start - Pretrained policies\n\u003cp float=\"center\"\u003e\n    \u003cimg src=\"Figures/naive_marl.gif\" width=\"150\" style='border:1px solid #ff0a0a'\u003e\n    \u003cimg src=\"Figures/leader_hiding.gif\" width=\"150\" style='border:3px solid #07d600'\u003e\n    \u003cimg src=\"Figures/co_training.gif\" width=\"150\" style='border:1px solid #ff0a0a'\u003e\n    \u003cimg src=\"Figures/scripted_pd.gif\" width=\"150\" style='border:1px solid #ff0a0a'\u003e\n    \u003cimg src=\"Figures/zheng.gif\" width=\"150\" style='border:1px solid #ff0a0a'\u003e\n    \u003cfigcaption\u003eFrom left to right: Naive MARL, Our proposed strategy (Paper 1), Co-training (Stage 4 in Paper 2), Scripted PD, Zheng et al.\u003c/figcaption\u003e\n\u003c/p\u003e\n\nDownload the folder `marlsave` from this [Drive link](https://drive.google.com/drive/folders/1JJpcSdAleKvkq9ZA8gubrIcXHU_ZpgyJ?usp=sharing) and store it in the root directory.\n\n1. Naive MARL\n```\npython joint_main.py --mode test --load-mode individual --swarm-load-run 0 --swarm-load-ckpt latest --adversary-load-run 0 --adversary-load-ckpt latest --out-dir naive_marl --plot-trajectories --record --goal-at-top --seed 0\n```\n1. Our proposed leader identity hiding policy in Paper 1\n```\npython joint_main.py --mode test --load-mode joint --load-run 1 --load-ckpt latest --out-dir leader_hiding --plot-trajectories --record --goal-at-top --seed 0\n```\n1. Co-training - stage 4 in Paper 2\nI am providing 3 training runs (2,3 and 4) due to stochasticity of results (refer to Paper 2).\n```\npython joint_main.py --mode test --load-mode joint --load-run 2 --load-ckpt latest --out-dir co_training --plot-trajectories --record --goal-at-top --seed 0\n```\n1. Scripted PD\n```\npython joint_main.py --mode test --algo scripted --load-mode individual --adversary-load-run 4 --adversary-load-ckpt latest --out-dir scripted_pd --plot-trajectories --record --goal-at-top --seed 0\n```\n1. Zheng et al.\n```\npython -W ignore joint_main_genetic.py --algo genetic --adversary-hidden-dim 512 --num-processes 1 --mode test --load-mode joint --load-run 5 --load-ckpt latest --adversary-version V2 --out-dir zheng --plot-trajectories --record --goal-at-top --seed 0\n```\n\n## Full details - Train and test yourself\n\n![Stages](Figures/stages.svg)\u003cbr/\u003e\n\nThere are multiple stages of training as described in the above mentioned papers: Stage 1 to 3 in Paper 1, Stage 1 to 4 in Paper 2. For each stage, for both train and test mode, we need to `joint_main.py` or `adversary_training/main.py` with the right arguments as explained below.\n\n### Stage 1\n#### Train multi-robot team on goal reaching reward\n```shell\npython joint_main.py --mode train --use-adversary 0\n```\nSaves training files in a folder `marlsave/run_n1`. `n1` is generated automatically starting from 0 and increasing everytime we run train. Trained model checkpoints and tensorboard logs are saved here. It is important to note down the run number `n1` for use below.\n#### Test\nReplace `--load-run 0` with the same `n1` as above. `load-ckpt` can be `latest` or a valid number. Results are saved in `output/stage_1`\n```shell\npython joint_main.py --mode test --use-adversary 0 --load-mode joint --load-run 0 --load-ckpt latest --out-dir stage_1 --record --goal-at-top\n```\n\n### Stage 2\n#### Train adversary to identify leader\n\n1. Generate trajectory data \u003cbr/\u003e\nReplace `--load-run 0` with the right value of `n1` corresponding to training in stage 1.\n```shell\npython joint_main.py --mode test --use-adversary 0 --load-mode joint --load-run 0 --load-ckpt latest --out-dir stage_1_train --num-eval-episodes 1000\npython joint_main.py --mode test --use-adversary 0 --load-mode joint --load-run 0 --load-ckpt latest --out-dir stage_1_test --num-eval-episodes 100 --goal-at-top\n```\nSaves trajectories in `output/stage_1_train/trajs` and `output/stage_1_test/trajs`.\n1. Create a dataset folder \u003cbr/\u003e\n```shell\nmkdir -p trajectory_datasets/dataset_1\nmv output/stage_1_train/trajs trajectory_datasets/dataset_1/train_dataset\nmv output/stage_1_test/trajs trajectory_datasets/dataset_1/test_dataset\n```\n1. Train adversary \u003cbr/\u003e\n```shell\ncd adversary_training\npython main.py --mode train --dataDir ../trajectory_datasets/dataset_1\n```\nThis will save training files in `runs/run_n2` where `n2` is generated automatically. It is important to note down `n2` for use below.\n1. Test adversary \u003cbr/\u003e\nReplace `0` in `--swarm-load-run 0` with `n1`. Replace `0` in `--adversary-load-run n2` with `n2`.\n```shell\ncd ..\npython joint_main.py --mode test --load-mode individual --swarm-load-run 0 --swarm-load-ckpt latest --adversary-load-run 0 --adversary-load-ckpt latest --out-dir stage_2_results --plot-trajectories --record --goal-at-top\n```\n\n### Stage 3\n#### Train swarm with goal reaching + identity hiding reward\nReplace `0` in `--adversary-load-run 0` with `n2`. \n```shell\npython joint_main.py --mode train --load-mode individual --adversary-load-run 0 --adversary-load-ckpt latest --train-adversary 0\n```\nSaves training files in a folder `marlsave/run_n2`.\n#### Test\nReplace `1` in `--load-run 0` with `n2`.\n```shell\npython joint_main.py --mode test --load-mode joint --load-run 1 --load-ckpt latest --out-dir stage_3_results --plot-trajectories --record --goal-at-top\n```\nThis is our proposed policy in Paper 1.\n### Stage 4\n#### Joint training of swarm and adversary\n```shell\npython joint_main.py --mode train\n```\n#### Test\nReplace `--load-run 1`\n```shell\npython joint_main.py --mode test --load-mode joint --load-run 1 --load-ckpt latest --out-dir stage_4_results --plot-trajectories --record --goal-at-top\n```\n\n## Baselines\n### Scripted PD\nset `algo=scripted`\n\n```shell\npython joint_main.py --mode test --use-adversary 0 --algo scripted --out-dir scripted_pd_train --num-eval-episodes 1000\npython joint_main.py --mode test --use-adversary 0 --algo scripted --out-dir scripted_pd_test --num-eval-episodes 100 --goal-at-top\nmkdir -p trajectory_datasets/dataset_scripted_pd\nmv output/scripted_pd_train/trajs trajectory_datasets/dataset_scripted_pd/train_dataset\nmv output/scripted_pd_test/trajs trajectory_datasets/dataset_scripted_pd/test_dataset\n\ncd adversary_training\npython main.py --mode train --dataDir ../trajectory_datasets/dataset_scripted_pd\n\ncd ..\npython joint_main.py --mode test --algo scripted --load-mode individual --adversary-load-run 4 --adversary-load-ckpt latest --out-dir scripted_pd_results --plot-trajectories --record --goal-at-top --seed 0\n```\n\n### Zheng et al. - Genetic\n\n#### Pretraining Adversary\n```\ncd adversary_training\npython main.py --mode train --dataDir ../trajectory_datasets/dataset_genetic_pretraining --lr 0.025 --version V2 --hiddenDim 512 --optimizer SGD\n```\n#### Co-optimization\n```\ncd ..\ncmd python -W ignore joint_main_genetic.py --adversary-num-trajs 100 --algo genetic --adversary-load-ckpt latest --num-frames 1000000 --adversary-hidden-dim 512 --num-processes 1 --mode train --env-name simple_flocking --adversary-load-run 8 --load-mode individual --adversary-num-epochs 1 --adversary-version V2\n```\nTest\n```\npython -W ignore joint_main_genetic.py --algo genetic --adversary-hidden-dim 512 --num-processes 1 --mode test --env-name simple_flocking --load-mode joint --load-run 228 --load-ckpt latest --adversary-version V2 \n```\n\n## Utilities\n### Generating plotting data\n```\npython gen_plot_data.py --load-run 32\n```\n\n### Recording and playing videos\nUse `run_grid_search.py` file to generate multiple videos together. Open `swarm_training/output/video_previewing_tool/video_preview.html` on browser (tested on Firefox 75.0 beta and Chrome Version 83.0.4103.61 (Official Build) (64-bit)). Browse and select the videos you wish to play.\n\n`--store-video-together` to store videos in common folder\n\n\n### Analysing\n1. For checking results of different ckpts of same run (1) pass an array of 'load-ckpt': [10,20,30,40,50,60,70,80,90,100], (2) 'store-video-together': [''], (3) DON'T pass 'out_dir' - auto generates out-dir names\n\n#### Useful arguments\nArguments are defined in `arguments.py`\n1. `num_frames`: No. of environment frames to train on \n1. `num_iters`: `num_frames // num_processes`\n1. `update_every`: Updated after this many frames\n1. `num_updates`: No. of updates for each `update_every`\n1. `batch_size`  \n1. `buffer_size`: Should be larger than max possible episode length     \n\n#### Evaluation\n1. `load-mode = {individual, joint}`\n    1. `individual`: loads from `swarm-load-path` and `adversary-load-path`\n    1. `joint`: loads from `load-path`\n\n#### Automate\n1. `joint_run.py` provides a convenient way to run multiple experiments. Earlier, I didn't have any problem but lately I'm having issues training in parallel. run_ID of one experiment is clashing with another\n\n### User Data\n1. `web_form` contains complete web based UI. In our experiments we hosted on a Apache web server running on a google cloud instance.\n1. Data is saved in `user_data.txt`. `cd web_form \u0026\u0026 python conv2csv.py user_data.txt` to convert to csv format. If there is any space/new line at the end of `user_data.txt`, remove it before converting.\n\n## Folders and files\n1. `out_files`: trajectories of swarm robots, one episode per file\n1. `adversary_training`: code for training adversary\n    1. `Prototyping_notebook.ipynb`\n    1. `runs`: checkpoints and tensorboard\n    1. `main.py`: training/validating\n    1. `dataset.py`: dataset class\n    1. `models.py`: model classes\n1. `swarm_training`: code for training swarm\n    1. `main.py`: main file for running \n    1. `arguments.py`:  arguments\n    1. `learner.py`: Learner - master object for the swarm\n    1. `runs`: checkpoints and tensorboard \n1. `mape`: environment\n\n## Environment Description\n1. `simple_flocking` - goal reaching, leader observes goal location\n1. `simple_trajectory` - trajectory following, leader observes next loc on trajectory (crude implementation). 'is_success' is always False\n1. Successful if all agents are within a thershold distance to the goal\n1. Reward: difference in distance + additional goal on task completion (disable right now)\n1. Goal at the top half of the window during training and at y = 0.9 during testing (to get roughly uniform goal reaching time for human trials)\n1. Adversary cannot see the goal\n1. Implementation details:\n    1. `done` is a list with `True`/`False` repeated num_agents times. All should have save value. Env is reset (by `gym_vecenv` or `PseudoVecEnv`) even if one of them is `True`\n    1. `info['env_done']` contains done for overall team\n    1. `info['is_success']` contains is_success for overall team \n    1. `env` reset when it's done by gym_vecenv or PseudoVecEnv, last obs stored in info['terminal_observation']\n    1. `mape/multiagent/environment`: generates multiagent environment\n    1. `mape/environment/scenarios/simple_flocking`: goal reaching environment\n\n\n## Output files\n1. stored in `output/args.out_file`\n1. `adversary_preds` has csv files for different episodes. First column is true leader ID in corresponding video, second column is adversary's prediction.\n\n## Common Issues\n1. Pyglet:\n```\nFile \"./mape/multiagent/rendering.py\", line 120, in render\n    arr = np.fromstring(image_data.data, dtype=np.uint8, sep='')\nAttributeError: 'ImageData' object has no attribute 'data'\n```\nSolution: `pip install pyglet==1.3.2`\n\n## Acknowledgements\n1. `swarm_training` directory is adapted from: [marl_transfer](https://github.com/sumitsk/marl_transfer).\n2. Parts of the code for [Zheng et al.](https://arxiv.org/pdf/1909.10387.pdf) are adapted from the corresponding [private_flocking](https://github.com/proroklab/private_flocking) repository.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fankur-deka%2Fhiding-leader-identity","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fankur-deka%2Fhiding-leader-identity","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fankur-deka%2Fhiding-leader-identity/lists"}