{"id":13728255,"url":"https://github.com/dougsm/ggcnn","last_synced_at":"2025-05-08T00:31:31.422Z","repository":{"id":43676044,"uuid":"132820235","full_name":"dougsm/ggcnn","owner":"dougsm","description":"Generative Grasping CNN from \"Closing the Loop for Robotic Grasping: A Real-time, Generative Grasp Synthesis Approach\" (RSS 2018)","archived":false,"fork":false,"pushed_at":"2020-07-21T00:34:15.000Z","size":48,"stargazers_count":486,"open_issues_count":15,"forks_count":139,"subscribers_count":13,"default_branch":"master","last_synced_at":"2024-08-04T02:08:01.153Z","etag":null,"topics":["deep-learning","grasping","robotics"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dougsm.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-05-09T22:43:22.000Z","updated_at":"2024-07-22T12:18:06.000Z","dependencies_parsed_at":"2022-08-12T10:42:25.302Z","dependency_job_id":null,"html_url":"https://github.com/dougsm/ggcnn","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dougsm%2Fggcnn","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dougsm%2Fggcnn/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dougsm%2Fggcnn/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dougsm%2Fggcnn/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dougsm","download_url":"https://codeload.github.com/dougsm/ggcnn/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224679879,"owners_count":17351883,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","grasping","robotics"],"created_at":"2024-08-03T02:00:39.334Z","updated_at":"2024-11-14T19:30:55.565Z","avatar_url":"https://github.com/dougsm.png","language":"Python","funding_links":[],"categories":["[Grasp Implementation](#implementations)"],"sub_categories":[],"readme":"**Note:** This is a cleaned-up, PyTorch port of the GG-CNN code.  For the original Keras implementation, see the `RSS2018` branch.  \nMain changes are major code clean-ups and documentation, an improved GG-CNN2 model, ability to use the Jacquard dataset and simpler evaluation.    \n\n\n# Generative Grasping CNN (GG-CNN)\n\nThe GG-CNN is a lightweight, fully-convolutional network which predicts the quality and pose of antipodal grasps at every pixel in an input depth image.  The lightweight and single-pass generative nature of GG-CNN allows for fast execution and closed-loop control, enabling accurate grasping in dynamic environments where objects are moved during the grasp attempt.\n\nThis repository contains the implementation of the Generative Grasping Convolutional Neural Network (GG-CNN) from the paper:\n\n**Closing the Loop for Robotic Grasping: A Real-time, Generative Grasp Synthesis Approach**\n\n*[Douglas Morrison](http://dougsm.com), [Peter Corke](http://petercorke.com), [Jürgen Leitner](http://juxi.net)*\n\nRobotics: Science and Systems (RSS) 2018\n\n[arXiv](https://arxiv.org/abs/1804.05172) | [Video](https://www.youtube.com/watch?v=7nOoxuGEcxA)\n\nIf you use this work, please cite:\n\n```text\n@inproceedings{morrison2018closing,\n\ttitle={{Closing the Loop for Robotic Grasping: A Real-time, Generative Grasp Synthesis Approach}},\n\tauthor={Morrison, Douglas and Corke, Peter and Leitner, J\\\"urgen},\n\tbooktitle={Proc.\\ of Robotics: Science and Systems (RSS)},\n\tyear={2018}\n}\n```\n\n**Contact**\n\nAny questions or comments contact [Doug Morrison](mailto:doug.morrison@roboticvision.org).\n\n## Installation\n\nThis code was developed with Python 3.6 on Ubuntu 16.04.  Python requirements can installed by:\n\n```bash\npip install -r requirements.txt\n```\n\n## Datasets\n\nCurrently, both the [Cornell Grasping Dataset](http://pr.cs.cornell.edu/grasping/rect_data/data.php) and\n[Jacquard Dataset](https://jacquard.liris.cnrs.fr/) are supported.\n\n### Cornell Grasping Dataset\n\n1. Download the and extract [Cornell Grasping Dataset](http://pr.cs.cornell.edu/grasping/rect_data/data.php). \n2. Convert the PCD files to depth images by running `python -m utils.dataset_processing.generate_cornell_depth \u003cPath To Dataset\u003e`\n\n### Jacquard Dataset\n\n1. Download and extract the [Jacquard Dataset](https://jacquard.liris.cnrs.fr/).\n\n## Pre-trained Models\n\nSome example pre-trained models for GG-CNN and GG-CNN2 can be downloaded from [here](https://github.com/dougsm/ggcnn/releases/tag/v0.1).  The models are trained on the Cornell grasping\ndataset using the depth images.  Each zip file contains 1) the full saved model from `torch.save(model)` and 2) the weights state dict from `torch.save(model.state_dict())`. \n\nFor example loading GG-CNN (replace ggcnn with ggcnn2 as required):\n\n```bash\n# Enter the directory where you cloned this repo\ncd /path/to/ggcnn\n\n# Download the weights\nwget https://github.com/dougsm/ggcnn/releases/download/v0.1/ggcnn_weights_cornell.zip\n\n# Unzip the weights.\nunzip ggcnn_weights_cornell.zip\n\n# Load the weights in python, e.g.\npython\n\u003e\u003e\u003e import torch\n\n# Option 1) Load the model directly.\n# (this may print warning based on the installed version of python)\n\u003e\u003e\u003e model = torch.load('ggcnn_weights_cornell/ggcnn_epoch_23_cornell')\n\u003e\u003e\u003e model\n\nGGCNN(\n  (conv1): Conv2d(1, 32, kernel_size=(9, 9), stride=(3, 3), padding=(3, 3))\n  (conv2): Conv2d(32, 16, kernel_size=(5, 5), stride=(2, 2), padding=(2, 2))\n  (conv3): Conv2d(16, 8, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))\n  (convt1): ConvTranspose2d(8, 8, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), output_padding=(1, 1))\n  (convt2): ConvTranspose2d(8, 16, kernel_size=(5, 5), stride=(2, 2), padding=(2, 2), output_padding=(1, 1))\n  (convt3): ConvTranspose2d(16, 32, kernel_size=(9, 9), stride=(3, 3), padding=(3, 3), output_padding=(1, 1))\n  (pos_output): Conv2d(32, 1, kernel_size=(2, 2), stride=(1, 1))\n  (cos_output): Conv2d(32, 1, kernel_size=(2, 2), stride=(1, 1))\n  (sin_output): Conv2d(32, 1, kernel_size=(2, 2), stride=(1, 1))\n  (width_output): Conv2d(32, 1, kernel_size=(2, 2), stride=(1, 1))\n)\n\n\n# Option 2) Instantiate a model and load the weights.\n\u003e\u003e\u003e from models.ggcnn import GGCNN\n\u003e\u003e\u003e model = GGCNN()\n\u003e\u003e\u003e model.load_state_dict(torch.load('ggcnn_weights_cornell/ggcnn_epoch_23_cornell_statedict.pt'))\n\n\u003cAll keys matched successfully\u003e\n\n```\n\n## Training\n\nTraining is done by the `train_ggcnn.py` script.  Run `train_ggcnn.py --help` to see a full list of options, such as dataset augmentation and validation options.\n\nSome basic examples:\n\n```bash\n# Train GG-CNN on Cornell Dataset\npython train_ggcnn.py --description training_example --network ggcnn --dataset cornell --dataset-path \u003cPath To Dataset\u003e\n\n# Train GG-CNN2 on Jacquard Datset\npython train_ggcnn.py --description training_example2 --network ggcnn2 --dataset jacquard --dataset-path \u003cPath To Dataset\u003e\n```\n\nTrained models are saved in `output/models` by default, with the validation score appended.\n\n## Evaluation/Visualisation\n\nEvaluation or visualisation of the trained networks are done using the `eval_ggcnn.py` script.  Run `eval_ggcnn.py --help` for a full set of options.\n\nImportant flags are:\n* `--iou-eval` to evaluate using the IoU between grasping rectangles metric.\n* `--jacquard-output` to generate output files in the format required for simulated testing against the Jacquard dataset.\n* `--vis` to plot the network output and predicted grasping rectangles.\n\nFor example:\n\n```bash\npython eval_ggcnn.py --network \u003cPath to Trained Network\u003e --dataset jacquard --dataset-path \u003cPath to Dataset\u003e --jacquard-output --iou-eval\n```\n\n\n## Running on a Robot\n\nOur ROS implementation for running the grasping system see [https://github.com/dougsm/mvp_grasp](https://github.com/dougsm/mvp_grasp).\n\nThe original implementation for running experiments on a Kinva Mico arm can be found in the repository [https://github.com/dougsm/ggcnn_kinova_grasping](https://github.com/dougsm/ggcnn_kinova_grasping).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdougsm%2Fggcnn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdougsm%2Fggcnn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdougsm%2Fggcnn/lists"}