{"id":26184465,"url":"https://github.com/dashdeckers/graph_rl","last_synced_at":"2025-12-16T10:50:29.886Z","repository":{"id":279202447,"uuid":"711598159","full_name":"dashdeckers/graph_rl","owner":"dashdeckers","description":"Combining Graph-Based Planning and Deep Reinforcement Learning. Masters Thesis. Written in Rust.","archived":false,"fork":false,"pushed_at":"2025-02-24T10:06:06.000Z","size":4524,"stargazers_count":11,"open_issues_count":2,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-11T22:56:05.644Z","etag":null,"topics":["graph-algorithms","reinforcement-learning","rust"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dashdeckers.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-10-29T18:44:46.000Z","updated_at":"2025-03-01T04:23:14.000Z","dependencies_parsed_at":"2025-02-24T11:35:24.869Z","dependency_job_id":null,"html_url":"https://github.com/dashdeckers/graph_rl","commit_stats":null,"previous_names":["dashdeckers/graph_rl"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/dashdeckers/graph_rl","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dashdeckers%2Fgraph_rl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dashdeckers%2Fgraph_rl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dashdeckers%2Fgraph_rl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dashdeckers%2Fgraph_rl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dashdeckers","download_url":"https://codeload.github.com/dashdeckers/graph_rl/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dashdeckers%2Fgraph_rl/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":27763120,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-12-16T02:00:10.477Z","response_time":57,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["graph-algorithms","reinforcement-learning","rust"],"created_at":"2025-03-11T22:56:08.178Z","updated_at":"2025-12-16T10:50:29.858Z","avatar_url":"https://github.com/dashdeckers.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# On Combining the Fields of Graph-Based Planning and Deep Reinforcement Learning\n\n[Paper](https://github.com/dashdeckers/graph_rl_thesis/blob/thesis/Thesis.pdf) / [Slides](https://github.com/dashdeckers/graph_rl_thesis/blob/thesis/Slides.pdf) / [Demo](https://drive.google.com/file/d/13QPwt5N2HikDYtOqxfZGFswLPVnx1U5U/view?usp=sharing)\n\n# Run Experiments\n\nWindows (powershell) and Linux (bash) commands are provided,\nthey each enable the `RUST_BACKTRACE` environment variable for easier\ndebugging in case something goes wrong, and the Linux experiment commands are\nadditionally run with the `nohup` command to run in the background\n(because I usually run them on a remote server and they would\notherwise terminate when the connection is dropped).\n\n\n## H-DDPG / HGB-DDPG on PointEnv-(Empty, OneLine, Hook)-(Close, Mid, Far)\n\n```powershell\n$REPS=50; `\n$ALG=\"ddpg_hgb\"; `\n$ENV=\"pointenv\"; `\n$ENV_V1=\"hooks\"; `\n$ENV_V2=\"mid\"; `\n$EXAMPLE=\"${ENV}_${ALG}\"; `\n$NAME=\"${ALG}_${ENV}_${ENV_V1}_${ENV_V2}\"; `\n$ALG_CONFIG=\"${ALG}.ron\"; `\n$TRAIN_CONFIG=\"${ENV}_training.ron\"; `\n$ENV_CONFIG=\"${ENV}_${ENV_V1}_${ENV_V2}.ron\"; `\n$PRETRAIN_TRAIN_CONFIG=\"${ENV}_pretraining.ron\"; `\n$PRETRAIN_ENV_CONFIG=\"${ENV}_pretrain.ron\"; `\n$env:RUST_BACKTRACE=1; `\ncargo run `\n    --release `\n    --example $EXAMPLE `\n    -- `\n    --log warn `\n    --name $NAME `\n    --pretrain-train-config \".\\examples\\configs\\$PRETRAIN_TRAIN_CONFIG\" `\n    --pretrain-env-config \".\\examples\\configs\\env_configs\\$PRETRAIN_ENV_CONFIG\" `\n    --train-config \".\\examples\\configs\\$TRAIN_CONFIG\" `\n    --env-config \".\\examples\\configs\\env_configs\\$ENV_CONFIG\" `\n    --alg-config \".\\examples\\configs\\$ALG_CONFIG\" `\n    --n-repetitions $REPS `\n```\n```powershell\n# GUI commands\n    --load-model \".\\data\" \"decent-ddpg-$ENV\" `\n    --gui `\n```\n```bash\nREPS=50; \\\nALG=\"ddpg_hgb\"; \\\nENV=\"pointenv\"; \\\nENV_V1=\"hooks\"; \\\nENV_V2=\"far\"; \\\nEXAMPLE=\"${ENV}_${ALG}\"; \\\nNAME=\"${ALG}_${ENV}_${ENV_V1}_${ENV_V2}\"; \\\nALG_CONFIG=\"${ALG}.ron\"; \\\nTRAIN_CONFIG=\"${ENV}_training.ron\"; \\\nENV_CONFIG=\"${ENV}_${ENV_V1}_${ENV_V2}.ron\"; \\\nPRETRAIN_TRAIN_CONFIG=\"${ENV}_pretraining.ron\"; \\\nPRETRAIN_ENV_CONFIG=\"${ENV}_pretrain.ron\"; \\\nRUST_BACKTRACE=1; \\\nnohup \\\ncargo run \\\n    --release \\\n    --example ${EXAMPLE} \\\n    -- \\\n    --log warn \\\n    --name ${NAME} \\\n    --pretrain-train-config \"./examples/configs/${PRETRAIN_TRAIN_CONFIG}\" \\\n    --pretrain-env-config \"./examples/configs/env_configs/${PRETRAIN_ENV_CONFIG}\" \\\n    --train-config \"./examples/configs/${TRAIN_CONFIG}\" \\\n    --env-config \"./examples/configs/env_configs/${ENV_CONFIG}\" \\\n    --alg-config \"./examples/configs/${ALG_CONFIG}\" \\\n    --device cuda \\\n    --n-repetitions ${REPS} \\\n    \u0026\n```\n```bash\n# GUI commands\n    --gui\n    --load-model \"./data\" \"decent-ddpg-${ENV}\" \\\n```\n\n\n\n\n\n# Plot\n\n```bash\nNAME=\"plot_1\"; \\\nTITLE=\"H-DDPG vs HGB-DDPG on PointEnv-Hooks-far\"; \\\ndeclare -a files=(); \\\ndeclare -a algs=(\"ddpg_hgb ddpg\"); \\\ndeclare -a envs=(\"pointenv\"); \\\ndeclare -a env_v1s=(\"hooks\"); \\\ndeclare -a env_v2s=(\"far\"); \\\nfor alg in \"${algs[@]}\"; do \\\n    for env in \"${envs[@]}\"; do \\\n        for env_v1 in \"${env_v1s[@]}\"; do \\\n            for env_v2 in \"${env_v2s[@]}\"; do \\\n                files+=(\"./data/${alg}_${env}_${env_v1}_${env_v2}\"); \\\n            done; \\\n        done; \\\n    done; \\\ndone; \\\npython \\\n    \"./scripts/viz_data.py\" \\\n    -d \"${files[@]}\" \\\n    -o \"${NAME}.png\" \\\n    -t \"${TITLE}\" \\\n    -s\n```\n\n\n\n\n# Misc\n\n## View the Docs locally\n\nWindows\n```powershell\n$env:RUSTDOCFLAGS=\"--html-in-header katex-header.html\"; `\n    cargo doc `\n    --document-private-items `\n    --no-deps `\n    --open\n```\n\nLinux\n```bash\nRUSTDOCFLAGS=\"--html-in-header katex-header.html\" \\\n    cargo doc \\\n    --document-private-items \\\n    --no-deps \\\n    --open\n```\n\n\n## Install \u0026 Setup Gymnasium\n\n```bash\n# Setup Gymnasium:\n## Install Python as a dynamic/shared library\nenv PYTHON_CONFIGURE_OPTS=\"--enable-shared\" pyenv install 3.11\n## Install missing libraries\nsudo apt-get install libpython3.11-dev\n## Setup virtualenv and install gymnasium\npyenv virtualenv 3.11 thesis\npyenv local thesis\npip install gymnasium\n```\n\n## Might need to fix a Pyenv bug with this on Linux:\n\n```bash\nexport LD_LIBRARY_PATH=/home/travis/.pyenv/versions/3.10.13/lib:$LD_LIBRARY_PATH\n```\n\n## Watch the GPU\n\n```bash\nwatch --differences=permanent -n 0.3 nvidia-smi\n```\n\n## Check your GPU's compute cap\n```bash\nnvidia-smi --query-gpu=compute_cap --format=csv,noheader\n```\n\n## CuDNN Setup\n```bash\n# CuDNN\n## (Assuming CUDA 12.2 and Ubuntu 20.04)\n## Go to https://developer.nvidia.com/rdp/cudnn-download\n## Download: Local Installer for Ubuntu20.04 x86_64 (Deb)\n## Filename: cudnn-local-repo-ubuntu2004-8.9.5.29_1.0-1_amd64.deb\nsudo apt-get install zlib1g\nsudo dpkg -i cudnn-local-repo-ubuntu2004-8.9.5.29_1.0-1_amd64.deb\nsudo cp /var/cudnn-local-repo-*/cudnn-local-*-keyring.gpg /usr/share/keyrings/\nsudo apt-get update\nsudo apt-get install libcudnn8=8.9.5.29-1+cuda12.2\nsudo apt-get install libcudnn8-dev=8.9.5.29-1+cuda12.2\n\n# Verify CuDNN installation:\nsudo apt-get install libfreeimage3 libfreeimage-dev\nsudo apt-get install libcudnn8-samples=8.9.5.29-1+cuda12.2\ncp -r /usr/src/cudnn_samples_v8/ $HOME # copy the samples to a writable path\ncd  $HOME/cudnn_samples_v8/mnistCUDNN\nmake clean \u0026\u0026 make\n./mnistCUDNN\n```\n\n\n## Libtorch Setup\n```bash\n\n# Libtorch\ncd /home/travis\n# Clicking to the right version from (https://pytorch.org/get-started/locally/): linux, libtorch, C++, cuda 12.1\nwget https://download.pytorch.org/libtorch/cu121/libtorch-cxx11-abi-shared-with-deps-2.1.0%2Bcu121.zip\nunzip libtorch-cxx11-abi-shared-with-deps-2.1.0+cu121.zip\nrm libtorch-cxx11-abi-shared-with-deps-2.1.0+cu121.zip\nexport LIBTORCH=/home/travis/libtorch\n```\n\n## Install \u0026 Setup Gym (the old library, needed by tch-rs)\n```bash\n# Tch: Gym / AutoROM\npip install gym[atari]==0.21.0  # gym version of last working tch(ddpg) commit Feb 3. 2022 (tch: #453)\nexport PATH=\"/home/travis/.local/bin:$PATH\"\npip install --upgrade AutoROM\nAutoROM --accept-license\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdashdeckers%2Fgraph_rl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdashdeckers%2Fgraph_rl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdashdeckers%2Fgraph_rl/lists"}