Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/ivy-llc/robot
Functions and classes for gradient-based robot motion planning, written in Ivy.
https://github.com/ivy-llc/robot
deep-learning ivy machine-learning motion-planning mxnet numpy path-planning python pytorch robot tensorflow trajectory-optimization
Last synced: about 5 hours ago
JSON representation
Functions and classes for gradient-based robot motion planning, written in Ivy.
- Host: GitHub
- URL: https://github.com/ivy-llc/robot
- Owner: ivy-llc
- License: apache-2.0
- Created: 2021-01-19T15:10:08.000Z (almost 4 years ago)
- Default Branch: main
- Last Pushed: 2023-08-28T06:54:19.000Z (over 1 year ago)
- Last Synced: 2024-11-15T03:04:47.672Z (about 1 month ago)
- Topics: deep-learning, ivy, machine-learning, motion-planning, mxnet, numpy, path-planning, python, pytorch, robot, tensorflow, trajectory-optimization
- Language: Python
- Homepage: https://unify.ai
- Size: 20 MB
- Stars: 51
- Watchers: 19
- Forks: 10
- Open Issues: 0
-
Metadata Files:
- Readme: README.rst
Awesome Lists containing this project
README
.. image:: https://github.com/unifyai/unifyai.github.io/blob/main/img/externally_linked/logo.png?raw=true#gh-light-mode-only
:width: 100%
:class: only-light
.. image:: https://github.com/unifyai/unifyai.github.io/blob/main/img/externally_linked/logo_dark.png?raw=true#gh-dark-mode-only
:width: 100%
:class: only-dark
.. raw:: html
**Functions and classes for gradient-based robot motion planning, written in Ivy.**
.. raw:: html
Contents
--------
* `Overview`_
* `Run Through`_
* `Interactive Demos`_
* `Get Involved`_
Overview
--------
.. _docs: https://unify.ai/docs/robot/
**What is Ivy Robot?**
Ivy robot provides functions and classes for gradient-based motion planning and trajectory optimization.
Classes are provided both for mobile robots and robot manipulators. Check out the docs_ for more info!
The library is built on top of the Ivy machine learning framework.
This means all functions and classes simultaneously support:
Jax, Tensorflow, PyTorch, MXNet, and Numpy.
**Ivy Libraries**
There are a host of derived libraries written in Ivy, in the areas of mechanics, 3D vision, robotics, gym environments,
neural memory, pre-trained models + implementations, and builder tools with trainers, data loaders and more. Click on the icons below to learn more!
.. raw:: html
**Quick Start**
Ivy robot can be installed like so: ``pip install ivy-robot==0.0.1.post0``
.. _demos: https://github.com/unifyai/robot/tree/main/ivy_robot_demos
.. _interactive: https://github.com/unifyai/robot/tree/main/ivy_robot_demos/interactive
To quickly see the different aspects of the library, we suggest you check out the demos_!
We suggest you start by running the script ``run_through.py``,
and read the "Run Through" section below which explains this script.
For more interactive demos, we suggest you run either
``drone_spline_planning.py`` or ``manipulator_spline_planning.py`` in the interactive_ demos folder.
Run Through
-----------
We run through some of the different parts of the library via a simple ongoing example script.
The full script is available in the demos_ folder, as file ``run_through.py``.
First, we select a random backend framework to use for the examples, from the options
``ivy.jax``, ``ivy.tensorflow``, ``ivy.torch``, ``ivy.mxnet`` or ``ivy.numpy``,
and use this to set the ivy backend framework.
.. code-block:: python
import ivy
ivy.set_backend(ivy.choose_random_backend())
**Spline Planning**
We now show how a spline path can be generated from a set of spline anchor points,
using the method ``ivy_robot.sample_spline_path``.
In this example, we generate a spline path representing full 6DOF motion from a starting pose to a target pose.
However, for simplicitly we fix the z translation and 3DOF rotation to zeros in this case.
.. code-block:: python
# config
num_free_anchors = 3
num_samples = 100
constant_rot_vec = ivy.array([[0., 0., 0.]])
constant_z = ivy.array([[0.]])
# xy positions
# 1 x 2
start_xy = ivy.array([[0., 0.]])
target_xy = ivy.array([[1., 1.]])
# 1 x 2
anchor1_xy = ivy.array([[0.6, 0.2]])
anchor2_xy = ivy.array([[0.5, 0.5]])
anchor3_xy = ivy.array([[0.4, 0.8]])
# as 6DOF poses
# 1 x 6
start_pose = ivy.concat((start_xy, constant_z, constant_rot_vec), axis=-1)
anchor1_pose = ivy.concat((anchor1_xy, constant_z, constant_rot_vec), axis=-1)
anchor2_pose = ivy.concat((anchor2_xy, constant_z, constant_rot_vec), axis=-1)
anchor3_pose = ivy.concat((anchor3_xy, constant_z, constant_rot_vec), axis=-1)
target_pose = ivy.concat((target_xy, constant_z, constant_rot_vec), axis=-1)
num_anchors = num_free_anchors + 2
# num_anchors x 6
anchor_poses = ivy.concat((start_pose, anchor1_pose, anchor2_pose, anchor3_pose, target_pose), axis=0)
# uniform sampling for spline
# num_anchors x 1
anchor_points = ivy.expand_dims(ivy.linspace(0., 1., num_anchors), axis=-1)
# num_samples x 1
query_points = ivy.expand_dims(ivy.linspace(0., 1., num_samples), axis=-1)
# interpolated spline poses
# num_samples x 6
interpolated_poses = ivy_robot.sample_spline_path(anchor_points, anchor_poses, query_points)
# xy motion
# num_samples x 2
anchor_xy_positions = anchor_poses[..., 0:2]
# num_samples x 2
interpolated_xy_positions = interpolated_poses[..., 0:2]
The interpolated xy positions and anchor positions from the path are shown below in the x-y plane.
.. image:: https://github.com/unifyai/robot/blob/main/docs/images/interpolated_drone_poses.png?raw=true
:width: 100%
**Rigid Mobile Class**
We now introduce the ``RigidMobile`` robot class,
which can be used to represent rigid jointless robots which are able to move freely.
In this case, we consider the case of a drone executing 6DOF motion in a scene.
The body of the drone is specified by a number of relative body points.
In this case, we represent the drone with 5 points: one in the centre, and one in each of the four corners.
We assume the same target position in the x-y plane as before,
but this time with a self-rotation of 180 degrees about the z-axis.
.. code-block:: python
# drone relative body points
rel_body_points = ivy.array([[0., 0., 0.],
[-0.1, -0.1, 0.],
[-0.1, 0.1, 0.],
[0.1, -0.1, 0.],
[0.1, 0.1, 0.]])
# create drone as ivy rigid mobile robot
drone = RigidMobile(rel_body_points)
# rotatin vectors
# 1 x 3
start_rot_vec = ivy.array([[0., 0., 0.]])
target_rot_vec = ivy.array([[0., 0., np.pi]])
# 1 x 3
anchor1_rot_vec = ivy.array([[0., 0., np.pi/4]])
anchor2_rot_vec = ivy.array([[0., 0., 2*np.pi/4]])
anchor3_rot_vec = ivy.array([[0., 0., 3*np.pi/4]])
# as 6DOF poses
# 1 x 6
start_pose = ivy.concat((start_xy, constant_z, start_rot_vec), axis=-1)
anchor1_pose = ivy.concat((anchor1_xy, constant_z, anchor1_rot_vec), axis=-1)
anchor2_pose = ivy.concat((anchor2_xy, constant_z, anchor2_rot_vec), axis=-1)
anchor3_pose = ivy.concat((anchor3_xy, constant_z, anchor3_rot_vec), axis=-1)
target_pose = ivy.concat((target_xy, constant_z, target_rot_vec), axis=-1)
# num_anchors x 6
anchor_poses = ivy.concat((start_pose, anchor1_pose, anchor2_pose, anchor3_pose, target_pose), axis=0)
# interpolated spline poses
# num_samples x 6
interpolated_poses = ivy_robot.sample_spline_path(anchor_points, anchor_poses, query_points)
# as matrices
# num_anchors x 3 x 4
anchor_matrices = ivy_mech.rot_vec_pose_to_mat_pose(anchor_poses)
# num_samples x 3 x 4
interpolated_matrices = ivy_mech.rot_vec_pose_to_mat_pose(interpolated_poses)
# sample drone body
# num_anchors x num_body_points x 3
anchor_body_points = drone.sample_body(anchor_matrices)
# num_samples x num_body_points x 3
interpolated_body_points = drone.sample_body(interpolated_matrices)
The sampled drone body xy positions during motion are shown below in the x-y plane.
By tracing the body points for each of the four corners of the drone,
we can see how the drone performs the 180 degree self-rotation about the z-axis during the motion.
.. image:: https://github.com/unifyai/robot/blob/main/docs/images/sampled_drone_body_positions.png?raw=true
:width: 100%
**Manipulator Class**
We now introduce the robot Manipulator class,
which can be used to represent arbitrary robot manipulators.
In this case, we consider the case of very simple 2-link manipulator,
which is constrained to move in the x-y plane.
The manipulator is specified by the Denavit–Hartenberg parameters,
as outlined in the newly derived class below.
We assume a manipulator with two 0.5m links,
where a configuration with both joints angles at 0 degrees represents a upright link configuration.
We specify a new set of target joint angles which corresponds with
a forward reaching motion in the positive x direction.
.. code-block:: python
class SimpleManipulator(Manipulator):
def __init__(self, base_inv_ext_mat=None):
a_s = ivy.array([0.5, 0.5])
d_s = ivy.array([0., 0.])
alpha_s = ivy.array([0., 0.])
dh_joint_scales = ivy.ones((2,))
dh_joint_offsets = ivy.array([-np.pi/2, 0.])
super().__init__(a_s, d_s, alpha_s, dh_joint_scales, dh_joint_offsets, base_inv_ext_mat)
# create manipulator as ivy manipulator
manipulator = SimpleManipulator()
# joint angles
# 1 x 2
start_joint_angles = ivy.array([[0., 0.]])
target_joint_angles = ivy.array([[-np.pi/4, -np.pi/4]])
# 1 x 2
anchor1_joint_angles = -ivy.array([[0.2, 0.6]])*np.pi/4
anchor2_joint_angles = -ivy.array([[0.5, 0.5]])*np.pi/4
anchor3_joint_angles = -ivy.array([[0.8, 0.4]])*np.pi/4
# num_anchors x 2
anchor_joint_angles = ivy.concat(
(start_joint_angles, anchor1_joint_angles, anchor2_joint_angles, anchor3_joint_angles,
target_joint_angles), axis=0)
# interpolated joint angles
# num_anchors x 2
interpolated_joint_angles = ivy_robot.sample_spline_path(anchor_points, anchor_joint_angles, query_points)
The interpolated joint angles are presented below.
.. image:: https://github.com/unifyai/robot/blob/main/docs/images/interpolated_manipulator_joint_angles.png?raw=true
:width: 100%
In a similar fashion to how the drone body was sampled in the previous example,
we next use these interpolated joint angles to sample the link positions for the manipulator.
.. code-block:: python
# sample links
# num_anchors x num_link_points x 3
anchor_link_points = manipulator.sample_links(anchor_joint_angles, samples_per_metre=5)
# num_anchors x num_link_points x 3
interpolated_link_points = manipulator.sample_links(interpolated_joint_angles, samples_per_metre=5)
we show the sampled link positions during the course of the forward reaching motion in the x-y plane below.
.. image:: https://github.com/unifyai/robot/blob/main/docs/images/sampled_manipulator_links.png?raw=true
:width: 100%
Interactive Demos
-----------------
The main benefit of the library is not simply the ability to sample paths, but to optimize these paths using gradients.
For exmaple, the body or link sample positions can be used to query the signed distance function (SDF) of a 3D scene in batch.
Then, assuming the spline anchor points to be free variables,
the gradients of the mean sampled SDF and a path length metric can be computed with respect to the anchor points.
The anhcor points can then be incrementally updated using gradient descent on this loss function.
We provide two further demo scripts which outline this gradient-based path optimization in a 3D scene.
Rather than presenting the code here, we show visualizations of the demos.
The scripts for these demos can be found in the interactive_ demos folder.
**RigidMobile Planning**
The first demo uses the ``RigidMobile`` class to optimzie the motion of a drone to a target pose,
making use of functions ``ivy_robot.sample_spline_path`` and ``ivy_robot.RigidMobile.sample_body``.
.. raw:: html
**Manipulator Planning**
The second demo uses the ``MicoManipulator`` class, derived from ``Manipulator``,
to optimzie the motion of a mico robot manipulator to a set of target joint angles,
making use of functions ``ivy_robot.sample_spline_path`` and ``ivy_robot.Manipulator.sample_links``.
.. raw:: html
Get Involved
------------
We hope the functions in this library are useful to a wide range of machine learning developers.
However, there are many more areas of gradient-based motion planning and broader robotics
which could be covered by this library.
If there are any particular robotics functions you feel are missing,
and your needs are not met by the functions currently on offer,
then we are very happy to accept pull requests!
We look forward to working with the community on expanding and improving the Ivy robot library.
Citation
--------
::
@article{lenton2021ivy,
title={Ivy: Templated deep learning for inter-framework portability},
author={Lenton, Daniel and Pardo, Fabio and Falck, Fabian and James, Stephen and Clark, Ronald},
journal={arXiv preprint arXiv:2102.02886},
year={2021}
}