{"id":19134175,"url":"https://github.com/um-arm-lab/val_pbvs_pybullet","last_synced_at":"2025-07-07T14:33:00.036Z","repository":{"id":79651070,"uuid":"353750758","full_name":"UM-ARM-Lab/val_pbvs_pybullet","owner":"UM-ARM-Lab","description":"Pybullet simulation for visual servoing of a dual arm robot val","archived":false,"fork":false,"pushed_at":"2021-05-17T00:04:49.000Z","size":4968,"stargazers_count":2,"open_issues_count":0,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-02-22T17:47:28.406Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/UM-ARM-Lab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-04-01T15:49:56.000Z","updated_at":"2024-05-08T23:27:06.000Z","dependencies_parsed_at":"2023-05-23T14:30:53.967Z","dependency_job_id":null,"html_url":"https://github.com/UM-ARM-Lab/val_pbvs_pybullet","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/UM-ARM-Lab/val_pbvs_pybullet","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/UM-ARM-Lab%2Fval_pbvs_pybullet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/UM-ARM-Lab%2Fval_pbvs_pybullet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/UM-ARM-Lab%2Fval_pbvs_pybullet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/UM-ARM-Lab%2Fval_pbvs_pybullet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/UM-ARM-Lab","download_url":"https://codeload.github.com/UM-ARM-Lab/val_pbvs_pybullet/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/UM-ARM-Lab%2Fval_pbvs_pybullet/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264092040,"owners_count":23556223,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T06:25:43.951Z","updated_at":"2025-07-07T14:32:59.996Z","avatar_url":"https://github.com/UM-ARM-Lab.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Pybullet Simulation for markerless Position-Based Visual Servoing (PBVS) of a robot arm\n\n**Author:** [Haoran Mike Cheng](https://www.linkedin.com/in/hrcheng/)\n\n**Email:** hrcheng@umich.edu\n\n**References:** [Project Report](https://drive.google.com/file/d/1RTxMGtyoRdckZbu1weEqGqMV1YU7K-99/view?usp=sharing)\n\n**video:** [link](https://drive.google.com/file/d/1CJzujuzSmLbED6PKnHZAKvR8Pr7eE_Hm/view?usp=sharing)\n\n![Sample Image](https://drive.google.com/uc?export=view\u0026id=1SN8ObK_XjMm3gslmQTcdCA_5doAEI9a0)\n\n## Abstract\nLight-weight service robot arms and humanoids usually have inaccurate kinematics model as opposed to industrial robots. Visual servoing tackles this problem by continuously measuring the robot and target state and creating a feedback signal from camera images. In this project, a **particle filter-based Position-Based Visual Servoing (PBVS)** method is implemented in **PyBullet** simulation. As opposed to methods that attach markers to the end-effector for pose estimate, our method uses particle filter, which **renders image at the pose of each particle**, and uses **HOG likelihood** with the actual image as the measurement update, without the need of attaching a marker. The result in simulation shows that the end-effector can reach the target pose within the error threshold. \n\n## Prerequisite\n### **Linux**\nThe code is developed and tested on Ubuntu 20.04, other versions might also work with the following libraries installed. Numpy, Scipy, Matplotlib are required.\n\n### **PyBullet**\nCan use pip to install (this repo uses PyBullet 3.1.0), no issue encountered.[Official Documentation](https://pybullet.org/wordpress/)\n \n### **sophuspy**\nsophuspy library is used to perform Lie theory related calculations. \nDirectly installation using pip has know issues \nPlease use following steps to install **pybind11** locally, and then install **sophuspy**\n[Installation step references](https://github.com/craigstar/SophusPy/issues/3)\n```\ngit clone https://github.com/pybind/pybind11.git\ncd pybind11\nmkdir build \u0026\u0026 cd build\ncmake .. -DCMAKE_INSTALL_PREFIX=../install/\nmake -j8\nmake install\nexport pybind11_DIR= **local directory to pybind11**\neg: /home/mike/Pybullet/lib/pybind11/install/share/cmake/pybind11/\npip3 install --user sophuspy\n```\n### **scikit-image**\n```\npip install scikit-image\n```\n## Examples \nPlease run `demo_script.py` for sample codes to run the velocity controller, PBVS with added noise, and Particle-filter based PBVS\n\n### Acknowledgement\nI would like to thank all members in Autonomous Robotic Manipulation Lab, University of Michigan for providing support and valuable feedback on this project. Last but not least, thank God for His mercy and faithfulness during this difficult semester. 5/16/2021\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fum-arm-lab%2Fval_pbvs_pybullet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fum-arm-lab%2Fval_pbvs_pybullet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fum-arm-lab%2Fval_pbvs_pybullet/lists"}