{"id":17320172,"url":"https://github.com/youtalk/iknet","last_synced_at":"2025-06-12T22:06:27.654Z","repository":{"id":50151339,"uuid":"326091293","full_name":"youtalk/iknet","owner":"youtalk","description":"Inverse kinematics estimation of ROBOTIS Open Manipulator X with neural networks","archived":false,"fork":false,"pushed_at":"2021-07-09T14:02:15.000Z","size":7076,"stargazers_count":42,"open_issues_count":2,"forks_count":6,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-14T14:15:50.813Z","etag":null,"topics":["dynamixel","jetson-nano","neural-network","pytorch","robotis","ros2"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/youtalk.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-01-02T02:07:14.000Z","updated_at":"2025-02-17T14:46:27.000Z","dependencies_parsed_at":"2022-08-22T06:21:01.171Z","dependency_job_id":null,"html_url":"https://github.com/youtalk/iknet","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/youtalk/iknet","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/youtalk%2Fiknet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/youtalk%2Fiknet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/youtalk%2Fiknet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/youtalk%2Fiknet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/youtalk","download_url":"https://codeload.github.com/youtalk/iknet/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/youtalk%2Fiknet/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259539134,"owners_count":22873341,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dynamixel","jetson-nano","neural-network","pytorch","robotis","ros2"],"created_at":"2024-10-15T13:29:18.967Z","updated_at":"2025-06-12T22:06:27.633Z","avatar_url":"https://github.com/youtalk.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# IKNet: Inverse kinematics neural networks\n\nIKNet is an inverse kinematics estimation with simple neural networks.\nThis repository also contains the training and test dataset by manually moving the 4 DoF manipulator [ROBOTIS Open Manipulator X](https://emanual.robotis.com/docs/en/platform/openmanipulator_x/overview/).\n\nIKNet can be trained on tested on [NVIDIA Jetson Nano 2GB](https://nvda.ws/2HQcb1Y), [Jetson family](https://developer.nvidia.com/EMBEDDED/Jetson-modules) or PC with/without NVIDIA GPU.\nThe training needs 900MB of GPU memory under default options.\n\n## Data collection\n\n### Set up\n\nInstall ROS 2 on Ubuntu 18.04 by following the ROBOTIS e-Manual.\n\n[https://emanual.robotis.com/docs/en/platform/openmanipulator_x/ros2_setup/#ros-setup](https://emanual.robotis.com/docs/en/platform/openmanipulator_x/ros2_setup/#ros-setup)\n\nThen build some additional packages to modify a message `open_manipulator_msgs/msg/KinematicsPose`  to add timestamp.\n\n```shell\n$ mkdir -p ~/ros2/src \u0026\u0026 cd ~/ros2/src\n$ git clone https://github.com/youtalk/open_manipulator.git -b kinematics-pose-header\n$ git clone https://github.com/youtalk/open_manipulator_msgs.git -b kinematics-pose-header\n$ cd ~/ros2\n$ colcon build\n$ . install/setup.bash\n```\n\n### Demo\n\nFirst launch Open Manipulator X controller and turn the servo off to manually move it around.\n\n```shell\n$ ros2 launch open_manipulator_x_controller open_manipulator_x_controller.launch.py\n```\n\n```shell\n$ ros2 service call /set_actuator_state open_manipulator_msgs/srv/SetActuatorState\n```\n\nThen collect the pair of the kinematics pose and the joint angles by recording `/kinematics_pose` and `/joint_states` topics under csv format.\n\n```shell\n$ ros2 topic echo --csv /kinematics_pose \u003e kinematics_pose.csv \u0026 \\\n  ros2 topic echo --csv /joint_states \u003e joint_states.csv\n```\n\nFinally append the headers into them to load by Pandas `DataFrame`.\n\n```shell\n$ sed -i \"1s/^/sec,nanosec,frame_id,position_x,position_y,position_z,orientation_x,orientation_y,orientation_z,orientation_w,max_accelerations_scaling_factor,max_velocity_scaling_factor,tolerance\\n/\" kinematics_pose.csv\n$ sed -i \"1s/^/sec,nanosec,frame_id,name0,name1,name2,name3,name4,position0,position1,position2,position3,position4,velocity0,velocity1,velocity2,velocity3,velocity4,effort0,effort1,effort2,effort3,effort4\\n/\" joint_states.csv\n```\n\n[![IKNet data collection with Open Manipulator X](https://img.youtube.com/vi/dsHGYwkQ5Ag/0.jpg)](https://www.youtube.com/watch?v=dsHGYwkQ5Ag)\n\n## Training\n\n### Set up\n\nInstall PyTorch and the related packages.\n\n```shell\n$ conda install pytorch cudatoolkit=11.0 -c pytorch\n$ pip3 install pytorch-pfn-extras matplotlib\n```\n\n### Demo\n\nTrain IKNet with training dataset which is inside dataset/train directory or prepared by yourself. The dataset/train dataset contains a 5-minutes movement at 100 [Hz] sampling.\n\nThe training may be stopped before maximum epochs by the early stopping trigger.\n\n```shell\n$ python3 iknet_training.py --help\nusage: iknet_training.py [-h] [--kinematics-pose-csv KINEMATICS_POSE_CSV]\n                         [--joint-states-csv JOINT_STATES_CSV] [--train-val-ratio TRAIN_VAL_RATIO]\n                         [--batch-size BATCH_SIZE] [--epochs EPOCHS] [--lr LR] [--save-model]\n\noptional arguments:\n  -h, --help            show this help message and exit\n  --kinematics-pose-csv KINEMATICS_POSE_CSV\n  --joint-states-csv JOINT_STATES_CSV\n  --train-val-ratio TRAIN_VAL_RATIO\n  --batch-size BATCH_SIZE\n  --epochs EPOCHS\n  --lr LR\n  --save-model\n\n$ python3 iknet_training.py\nepoch       iteration   train/loss  lr          val/loss\n1           3           0.0188889   0.01        0.0130676\n2           6           0.0165503   0.01        0.0132546\n3           9           0.0167138   0.01        0.0134633\n...\n61          183         0.00267084  0.01        0.00428417\n62          186         0.00266047  0.01        0.00461381\n63          189         0.00260262  0.01        0.00461737\n```\n\nThe training can be run on NVIDIA Jetson Nano 2GB.\n\n[![IKNet training on NVIDIA Jetson Nano 2GB](https://img.youtube.com/vi/R_RtWAhCt8o/0.jpg)](https://www.youtube.com/watch?v=R_RtWAhCt8o)\n\nThe loss indicates the L1 norm of the joint angles. So the final networks solved 0.00461737 [rad] accuracy on average.\n\n![train/loss and val/loss](https://user-images.githubusercontent.com/579333/103491840-44a38880-4e6a-11eb-946c-222c46b97878.png)\n\n## Test\n\n### Demo\n\nEvaluate accuracy of IKNet with test dataset which is inside dataset/test directory or prepared by yourself.\nThe dataset/test dataset contains a 1-minute movement at 100 [Hz] sampling.\n\n```shell\n$ python3 iknet_test.py --help\nusage: iknet_test.py [-h] [--kinematics-pose-csv KINEMATICS_POSE_CSV]\n                     [--joint-states-csv JOINT_STATES_CSV] [--batch-size BATCH_SIZE]\n\noptional arguments:\n  -h, --help            show this help message and exit\n  --kinematics-pose-csv KINEMATICS_POSE_CSV\n  --joint-states-csv JOINT_STATES_CSV\n  --batch-size BATCH_SIZE\n\n$ python3 iknet_test.py\nTotal loss = 0.006885118103027344\n```\n## Inference\n\n### Demo\n\nEstimate the inverse kinematics of IKNet using Open Manipulator X.\nFirst launch Open Manipulator X controller.\n\n```shell\n$ ros2 launch open_manipulator_x_controller open_manipulator_x_controller.launch.py\n```\n\nThen run `iknet_inference.py` to input the pose (position and orientation) and move the robot.\nNote that the orientation is described by quaternion (`qx`, `qy`, `qz`, `qw`).\n\n```shell\n$ . ~/ros2/install/setup.bash\n$ python3 iknet_inference.py --help\nusage: iknet_inference.py [-h] [--model MODEL] [--trt] [--x X] [--y Y] [--z Z]\n                          [--qx QX] [--qy QY] [--qz QZ] [--qw QW]\n\noptional arguments:\n  -h, --help     show this help message and exit\n  --model MODEL\n  --trt\n  --x X\n  --y Y\n  --z Z\n  --qx QX\n  --qy QY\n  --qz QZ\n  --qw QW\n\n$ python3 iknet_inference.py --x 0.1 --z 0.1\ninput dimentsions: [400, 300, 200, 100, 50]\ndropout: 0.1\ninput: tensor([0.1000, 0.0000, 0.1000, 0.0000, 0.0000, 0.0000, 1.0000],\n       device='cuda:0')\noutput: tensor([-0.0769, -0.9976,  1.3582, -0.2827], device='cuda:0',\n       grad_fn=\u003cAddBackward0\u003e)\n```\n\n[![Inverse kinematics estimation by IKNet](https://img.youtube.com/vi/62_zIF_PvxU/0.jpg)](https://www.youtube.com/watch?v=62_zIF_PvxU)\n\n### Demo with TensorRT\n\nIf you would like to use the inference with [TensorRT](https://developer.nvidia.com/tensorrt), first convert the PyTorch model to TensorRT enabled model using [torch2trt](https://github.com/NVIDIA-AI-IOT/torch2trt).\n\n```shell\n$ python3 iknet_trt_export.py --help\nusage: iknet_trt_export.py [-h] [--input-model INPUT_MODEL] [--output-model OUTPUT_MODEL]\n\noptional arguments:\n  -h, --help            show this help message and exit\n  --input-model INPUT_MODEL\n  --output-model OUTPUT_MODEL\n```\n\nThen run the `iknet_inference.py` mentioned above with the options.\n\n```shell\n$ python3 iknet_inference.py --trt --model iknet-trt.pth --x 0.1 --z 0.1\ninput dimentsions: [400, 300, 200, 100, 50]\ndropout: 0.1\ninput: tensor([0.1000, 0.0000, 0.1000, 0.0000, 0.0000, 0.0000, 1.0000],\n       device='cuda:0')\noutput: tensor([-0.0769, -0.9976,  1.3582, -0.2827], device='cuda:0',\n       grad_fn=\u003cAddBackward0\u003e)\n```\n\n## Reference\n\n- Theofanidis, Michail \u0026 Sayed, Saif \u0026 Cloud, Joe \u0026 Brady, James \u0026 Makedon, Fillia. (2018). Kinematic Estimation with Neural Networks for Robotic Manipulators: 27th International Conference on Artificial Neural Networks, Rhodes, Greece, October 4–7, 2018, Proceedings, Part III. 10.1007/978-3-030-01424-7_77. \n- Duka, Adrian-Vasile. (2014). Neural Network based Inverse Kinematics Solution for Trajectory Tracking of a Robotic Arm. Procedia Technology. 12. 20–27. 10.1016/j.protcy.2013.12.451.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyoutalk%2Fiknet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyoutalk%2Fiknet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyoutalk%2Fiknet/lists"}