{"id":27195631,"url":"https://github.com/barisyazici/deep-rl-grasping","last_synced_at":"2025-04-09T19:47:24.742Z","repository":{"id":37006788,"uuid":"302409901","full_name":"BarisYazici/deep-rl-grasping","owner":"BarisYazici","description":"Train deep reinforcement learning model for robotics grasping. Choose from different perception layers raw Depth, RGBD and autoencoder. Test the learned models in different scenes and object datasets","archived":false,"fork":false,"pushed_at":"2022-07-10T19:40:36.000Z","size":105187,"stargazers_count":161,"open_issues_count":5,"forks_count":40,"subscribers_count":2,"default_branch":"master","last_synced_at":"2023-11-07T17:04:23.484Z","etag":null,"topics":["deep-learning","kuka","machine-learning","open-ai","pybullet","reinfrocement-learning","robotics"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/BarisYazici.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-10-08T17:09:01.000Z","updated_at":"2023-11-07T02:35:46.000Z","dependencies_parsed_at":"2022-07-13T15:59:46.101Z","dependency_job_id":null,"html_url":"https://github.com/BarisYazici/deep-rl-grasping","commit_stats":null,"previous_names":[],"tags_count":0,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BarisYazici%2Fdeep-rl-grasping","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BarisYazici%2Fdeep-rl-grasping/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BarisYazici%2Fdeep-rl-grasping/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BarisYazici%2Fdeep-rl-grasping/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/BarisYazici","download_url":"https://codeload.github.com/BarisYazici/deep-rl-grasping/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248102726,"owners_count":21048231,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","kuka","machine-learning","open-ai","pybullet","reinfrocement-learning","robotics"],"created_at":"2025-04-09T19:47:24.079Z","updated_at":"2025-04-09T19:47:24.735Z","avatar_url":"https://github.com/BarisYazici.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Deep Reinforcement Learning on Robotics Grasping\nTrain robotics model with integrated curriculum learning-based gripper environment. Choose from different perception layers depth, RGB-D. Run pretrained models with SAC, BDQ and DQN algorithms. Test trained algorithms in different scenes and domains. \n\nMaster's thesis [PDF](https://github.com/BarisYazici/masters_thesis/blob/master/final_report.pdf)\n\n### Prerequisites (CPU)\n\nInstall anaconda. Start a clean conda environment.\n\n```\nconda create -n grasp_env python=3.6\nconda activate grasp_env\n```\npython manipulation_main/training/train_stable_baselines.py train --config config/gripper_grasp.yaml --algo SAC --model_dir trained_models/SAC_full --timestep 100000 -v\n### Prerequisites (GPU)\n```\nconda create -n grasp_env python=3.6\nconda activate grasp_env\nconda install -c conda-forge cudatoolkit=10.0 cudnn=7.6.5\nexport LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$CONDA_PREFIX/lib/\n```\n\nIn [setup.py](setup.py) change:\n```\n'tensorflow==1.14.0',\nto\n'tensorflow_gpu==1.14.0',\n```\n\n## Installation \n\nUse pip to install the dependencies.\n\n```\npip install -e .\n```\n\nIf using GPU you can check if it was successful with:\n´´´\npython -c \"import tensorflow as tf; print(tf.config.experimental.list_physical_devices('GPU'))\"\n´´´\n\n## Run Models\ntrain_stable_baselines script provides the functionality of **running** and **training** models.\n\nFor running models *'manipulation_main/training/train_stable_baselines.py'* takes the following arguments\n\n* --model - trained model file e.g trained_models/SAC_full_depth_1mbuffer/best_model/best_model.zip\n* -t - use test dataset if not given runs on training dataset\n* -v - visualize the model (faster without the -v option)\n* -s - run stochastic model if not deterministic\n\nFor running functionality *run* sub-parser needs to be passed to the script.\n\n```\npython manipulation_main/training/train_stable_baselines.py run --model trained_models/SAC_full_depth_1mbuffer/best_model/best_model.zip -v -t\n```\n\n\n## Train models\n\nFor training models *'manipulation_main/training/train_stable_baselines.py'* takes the following arguments\n\n* --config - config file (e.g *'config/simplified_object_picking.yaml'* or *'config/gripper_grasp.yaml'*)\n* --algo - algorithm to use(e.g BDQ, DQN, SAC, TRPO)\n* --model_dir - name of the folder to host the trained model logs and best performing model on validation set.\n* -sh - use shaped reward function (Only makes sense for Full Environment version)\n* -v - visualize the model\n\nFor training functionality *train* sub-parser needs to be passed to the script.\n\n```\npython manipulation_main/training/train_stable_baselines.py train --config config/gripper_grasp.yaml --algo SAC --model_dir trained_models/SAC_full --timestep 100000 -v\n```\n\n## Running the tests\n\nTo run the gripperEnv related test use\n\n```\npytest tests_gripper\n```\n\n* **Domain and Scene Transfer**\n\n \u003cimg src=\"https://github.com/BarisYazici/masters_thesis/blob/master/figures/testtraining.jpg\" width=\"75%\"\u003e\n\n* **Different Perception Layers**\n\n \u003cimg src=\"https://github.com/BarisYazici/masters_thesis/blob/master/figures/NewPerception.png\" width=\"75%\"\u003e\n\n* **Ablation Studies**\n\n  \u003cimg src=\"https://github.com/BarisYazici/masters_thesis/blob/master/figures/ablation/SAC_performance_shaped_reward_vs_sparse_reward.png\" width=\"45%\"\u003e\n  \u003cimg src=\"https://github.com/BarisYazici/masters_thesis/blob/master/figures/ablation/SAC_performance_wo_actuator_width.png\" width=\"45%\"\u003e\n  \u003cimg src=\"https://github.com/BarisYazici/masters_thesis/blob/master/figures/ablation/SAC_performance_wo_curriculum_strategy.png\" width=\"45%\"\u003e\n  \u003cimg src=\"https://github.com/BarisYazici/masters_thesis/blob/master/figures/ablation/SAC_performance_wo_normalization.png\" width=\"45%\"\u003e\n\n* **Training Environment**\n\n  \u003cimg src=\"https://github.com/BarisYazici/masters_thesis/blob/master/figures/trainingEnv.gif\" width=\"50%\"\u003e\n  \n* **Domain transfer performance**\n\n  \u003cimg src=\"https://github.com/BarisYazici/masters_thesis/blob/master/figures/SACGripperEnvRes.png\"\u003e\n  \u003cimg src=\"https://github.com/BarisYazici/masters_thesis/blob/master/figures/GripperEnv.gif\" width=\"45%\"\u003e\n\n  \u003cimg src=\"https://github.com/BarisYazici/masters_thesis/blob/master/figures/SACKukaEnv.png\"\u003e\n  \u003cimg src=\"https://github.com/BarisYazici/masters_thesis/blob/master/figures/kukaGif.gif\" width=\"45%\"\u003e  \n\n## Authors\n\n* **Baris Yazici** - *Initial work* - [BarisYazici](https://github.com/BarisYazici)\n\n## Citing the Project\n\nTo cite the master's thesis:\n```\n@MastersThesis{Yazici2020,\n    author     =     {Yazici Baris},\n    title     =     {{Branch Dueling Deep Q-Networks for Robotics Applications}},\n    school     =     {Technical University of Munich},\n    year     =     {2020},\n    howpublished = {\\url{https://github.com/BarisYazici/tum_masters_thesis}}\n}\n```\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE.md](LICENSE.md) file for details\n\n## Acknowledgments\n\n* Mahmoud Akl (supervisor)\n* Breyer Michel (author of https://arxiv.org/abs/1803.04996)\n* https://github.com/atavakol/action-branching-agents\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbarisyazici%2Fdeep-rl-grasping","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbarisyazici%2Fdeep-rl-grasping","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbarisyazici%2Fdeep-rl-grasping/lists"}