{"id":20853575,"url":"https://github.com/quantaji/amd-ssd","last_synced_at":"2025-04-10T20:06:46.529Z","repository":{"id":176271502,"uuid":"627360599","full_name":"quantaji/AMD-SSD","owner":"quantaji","description":"Adaptive Mechanism Design (AMD) in Sequential Social Dilemma (SSD). Foundations of Reinforcement Learning 2023 Spring, team project.","archived":false,"fork":false,"pushed_at":"2023-06-18T20:14:57.000Z","size":16378,"stargazers_count":3,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-24T17:52:38.351Z","etag":null,"topics":["mechanism-design","multi-agent-reinforcement-learning","reinforcement-learning","reinforcement-learning-algorithms","reinforcement-learning-environments"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/quantaji.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-04-13T09:58:24.000Z","updated_at":"2024-08-08T12:02:58.000Z","dependencies_parsed_at":"2023-09-28T07:17:41.223Z","dependency_job_id":null,"html_url":"https://github.com/quantaji/AMD-SSD","commit_stats":{"total_commits":76,"total_committers":11,"mean_commits":6.909090909090909,"dds":"0.35526315789473684","last_synced_commit":"38d86618094e860d802f52dca3997aa17e939175"},"previous_names":["quantaji/amd-ssd"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quantaji%2FAMD-SSD","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quantaji%2FAMD-SSD/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quantaji%2FAMD-SSD/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quantaji%2FAMD-SSD/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/quantaji","download_url":"https://codeload.github.com/quantaji/AMD-SSD/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248288340,"owners_count":21078900,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["mechanism-design","multi-agent-reinforcement-learning","reinforcement-learning","reinforcement-learning-algorithms","reinforcement-learning-environments"],"created_at":"2024-11-18T03:22:17.954Z","updated_at":"2025-04-10T20:06:46.512Z","avatar_url":"https://github.com/quantaji.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Adaptive Mechanism Design (AMD) in Sequential Social Dilemma (SSD)\n\nThis is the code for project of cource Foundations of Reinforcement Learning (FoRL) Spring Semester 2023. The major contributions of this project is\n- A full implementation of AMD algorithm [[original paper]](https://arxiv.org/abs/1806.04067) for arbitrary environments, using `ray==2.3.1`. \n  - migrating to higher version of `ray` needs additional effort\n- Two RL environments, Wolfpack and Gathering. [[original paper]](https://arxiv.org/abs/1702.03037)\n\nFor further information, please refer to our [report](https://www.overleaf.com/read/nqnprygyhqkf).\n\n# Instruction for local and Euler environment setup\nI decide to use Python 3.10.4 and CUDA 11.8 as standard version. This is the default version on Euler.\n\nThese versions can be modifyed. Depending on the repo we are migrating.\n\n## Euler setup\nOn Euler, for each time you need to load the module.\n```sh\nmodule load gcc/8.2.0 python_gpu/3.10.4 cuda/11.8.0 git-lfs/2.3.0 git/2.31.1 eth_proxy\n```\nPackage `gcc/8.2.0` is necessary. Only this module is loaded, then can you search out result about python and cuda. You can search for the version of python and cuda you want by command \n```shell\nmodule avail ${package name: cuda, python, etc}\n```\n\ncreate a virtual env with venv\n```sh\npy_venv_dir=\"${SCRATCH}/.python_venv\"\npython -m venv ${py_venv_dir}/forl-proj --upgrade-deps\n# To install python packages, run\n${SCRATCH}/.python_venv/forl-proj/bin/pip install -r requirements.txt --cache-dir ${SCRATCH}/pip_cache\n# actiavte\nsource \"${SCRATCH}/.python_venv/forl-proj/bin/activate\"\n# deactivate\ndeactivate\n```\n\n\n## Local setup\nOn local machine, to install this exact python version I use conda (you can also use venv).\n```sh\nconda create --name=forl-proj python=3.10\n# activate\nconda activate forl-proj\n# deactivate\nconda deactivate\n```\n\n## Submit jobs on Euler/ Slurm\n\nYou can first edit the resources needed in `start-ray.sbatch` file, and submit jobs by\n```shell\n# command\necho \"$(cat start-ray-nodes.sbatch; echo command_to_submit_jobs )\" \u003e temp \u0026\u0026 sbatch \u003c temp \u0026\u0026 rm temp\n# command from file\necho \"$(cat start-ray-nodes.sbatch file_of_job )\" \u003e temp \u0026\u0026 sbatch \u003c temp \u0026\u0026 rm temp\n```\n\nFor example\n\n```shell\necho \"$(cat start-ray-nodes.sbatch exp_scripts/wolfpack/amd-qadj-delay_model-conv_assump-neural.bash)\" \u003e temp \u0026\u0026 sbatch \u003c temp \u0026\u0026 rm temp\n```\n\n### Tips for using LSF or Slurm\nThere are some links here \n- [Migration from LSF to SLURM](https://scicomp.ethz.ch/wiki/LSF_to_Slurm_quick_reference)\n- [How to use GPU](https://scicomp.ethz.ch/wiki/Getting_started_with_GPUs)\n\nMost importantly, [this](https://scicomp.ethz.ch/public/lsla/index2.html) interactive website can generate sbatch scripts.\n\n\n\n### Ray on SLURM\nSee the following link:\n- https://docs.ray.io/en/latest/cluster/vms/user-guides/community/slurm.html#slurm-network-ray\n- https://docs.ray.io/en/latest/cluster/vms/user-guides/community/slurm-basic.html\n- https://github.com/NERSC/slurm-ray-cluster/blob/master/submit-ray-cluster.sbatch\n- https://github.com/pengzhenghao/use-ray-with-slurm\n- https://github.com/klieret/ray-tune-slurm-demo\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fquantaji%2Famd-ssd","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fquantaji%2Famd-ssd","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fquantaji%2Famd-ssd/lists"}