{"id":13818883,"url":"https://github.com/natolambert/dynamicslearn","last_synced_at":"2025-04-30T07:09:03.944Z","repository":{"id":67861214,"uuid":"127788906","full_name":"natolambert/dynamicslearn","owner":"natolambert","description":"Working directory for dynamics learning for experimental robots.","archived":false,"fork":false,"pushed_at":"2021-03-26T16:53:01.000Z","size":119439,"stargazers_count":56,"open_issues_count":7,"forks_count":19,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-04-30T07:08:57.916Z","etag":null,"topics":["dynamics","machine-learning","quadrotor","research"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/natolambert.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2018-04-02T17:26:07.000Z","updated_at":"2024-11-11T14:32:02.000Z","dependencies_parsed_at":"2023-02-21T19:15:12.165Z","dependency_job_id":null,"html_url":"https://github.com/natolambert/dynamicslearn","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/natolambert%2Fdynamicslearn","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/natolambert%2Fdynamicslearn/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/natolambert%2Fdynamicslearn/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/natolambert%2Fdynamicslearn/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/natolambert","download_url":"https://codeload.github.com/natolambert/dynamicslearn/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251658212,"owners_count":21622822,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dynamics","machine-learning","quadrotor","research"],"created_at":"2024-08-04T08:00:34.240Z","updated_at":"2025-04-30T07:09:03.926Z","avatar_url":"https://github.com/natolambert.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"\n\n# dynamics-learn\nWorking directory for my work on model-based reinforcement learning for novel robots. Best for robots with high test cost and difficult to model dynamics. Contact: [nol@berkeley.edu](mailto:nol@berkeley.edu)\nFirst paper website: [https://sites.google.com/berkeley.edu/mbrl-quadrotor/](https://sites.google.com/berkeley.edu/mbrl-quadrotor/)\nThere is current future work using this library, such as attempting to control the Ionocraft with model-based RL.  [https://sites.google.com/berkeley.edu/mbrl-ionocraft/](https://sites.google.com/berkeley.edu/mbrl-ionocraft/)\n\n*Note that I have been very actively developing in this repo, please reach out if you have any questions of accuracy in the readme*.\n\nThis directory is working towards an implementation of many simulated model-based approaches on real robots. For current state of the art in simulation, see this work from Prof Sergey Levine's group: [Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models](https://arxiv.org/abs/1805.12114).\n\nFuture implementations work towards controlled flight of the ionocraft,\n[with a recent publication in Robotics and Automation Letters](https://ieeexplore.ieee.org/document/8373697/)\nand in the future for transfer learning of dynamics on the Crazyflie 2.0 Platform.\n\nSome potentially noteable implementations include:\n- probablistic nueral network in pytorch\n- gaussian loss function for said pytorch probablistic neural network\n- random shooting MPC implementation with customizable cost / reward function (See cousin repo: https://github.com/natolambert/ros-crazyflie-mbrl)\n\nUsage is generally of the form, with hydra enabling more options:\n```\n$ python learn/trainer.py robot=iono\n```\n\nMain Scripts:\n---------------\n- `learn/trainer.py`: is for training dynamics models (P,PE,D,DE) on experimental data. The training process uses [Hydra](https://github.com/facebookresearch/hydra) to allow easy configuration of which states are used and how the predictions are formatted. \n- `learn/simulate_mpc.py`: a script that runs MBRL with a MPC on a simulated environment.\n- `learn/bo.py`: For generating PID parameters using a dynamics model as a simulation environment. This will eventually extend beyond PID control. See the controllers directory `learn/control`. I am working to integrate [opto](https://github.com/robertocalandra/opto).\n- `learn/plot.py`: a script for viewing different types of predictions, under improvement\n\nIn Development:\n-------------\n\n- `learn/pipps_experiment.py`: A reimplementation of the paper [\"PIPPS: Flexible Model-Based Policy Search Robust to the Curse of Chaos\"](https://arxiv.org/abs/1902.01240). I wrote a blog post summarizing the main derivation behind this work [here](https://medium.com/@natolambert/deep-rl-case-study-policy-based-vs-model-conditioned-gradients-in-rl-4546434c84b0).\n\nRelated Code for Experiments:\n-----------------------------\nCF Firmware: https://github.com/natolambert/crazyflie-firmware-pwm-control\n  - forked from: https://github.com/bitcraze/crazyflie-firmware\n\nRos code: https://github.com/natolambert/ros-crazyflie-mbrl\n  - uh from: https://github.com/whoenig/crazyflie_ros\n  - want to use: https://github.com/USC-ACTLab/crazyswarm\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnatolambert%2Fdynamicslearn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnatolambert%2Fdynamicslearn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnatolambert%2Fdynamicslearn/lists"}