{"id":14238495,"url":"https://github.com/rail-berkeley/soar","last_synced_at":"2025-12-30T01:27:12.540Z","repository":{"id":250040390,"uuid":"830222923","full_name":"rail-berkeley/soar","owner":"rail-berkeley","description":"Code release for paper \"Autonomous Improvement of Instruction Following Skills via Foundation Models\" | CoRL 2024","archived":false,"fork":false,"pushed_at":"2024-09-09T18:35:18.000Z","size":8681,"stargazers_count":40,"open_issues_count":0,"forks_count":2,"subscribers_count":10,"default_branch":"main","last_synced_at":"2024-09-09T22:59:37.138Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://auto-improvement.github.io","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rail-berkeley.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-17T21:04:10.000Z","updated_at":"2024-09-09T18:35:22.000Z","dependencies_parsed_at":"2024-10-21T19:28:15.764Z","dependency_job_id":null,"html_url":"https://github.com/rail-berkeley/soar","commit_stats":null,"previous_names":["rail-berkeley/soar"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rail-berkeley%2Fsoar","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rail-berkeley%2Fsoar/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rail-berkeley%2Fsoar/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rail-berkeley%2Fsoar/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rail-berkeley","download_url":"https://codeload.github.com/rail-berkeley/soar/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":229516967,"owners_count":18085475,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-21T03:00:52.211Z","updated_at":"2025-12-30T01:27:12.532Z","avatar_url":"https://github.com/rail-berkeley.png","language":"Python","readme":"# SOAR: Autonomous Improvement of Instruction Following Skills via Foundation Models\n[](media/soar_logo.jpeg)\n[![arXiv](https://img.shields.io/badge/arXiv-2407.20635-df2a2a.svg)](https://arxiv.org/pdf/2407.20635)\n[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://githubtocolab.com/rail-berkeley/soar/blob/main/soar_data/load_soar_data.ipynb)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![Static Badge](https://img.shields.io/badge/Project-Page-a)](https://auto-improvement.github.io/)\n\n[Zhiyuan Zhou](https://zhouzypaul.github.io/), [Pranav Atreya](https://pranavatreya.github.io/), [Abraham Lee](https://www.linkedin.com/in/abraham-lee-4a0497242?original_referer=https%3A%2F%2Fwww.google.com%2F), [Homer Walke](https://homerwalke.com/), [Oier Mees](https://www.oiermees.com/), [Sergey Levine](https://people.eecs.berkeley.edu/~svlevine/)\n\u003chr style=\"border: 2px solid gray;\"\u003e\u003c/hr\u003e\n\nWe present SOAR,  an approach to autonomously improve instruction following policies leveraging\nfoundation models. SOAR breaks down the autonomous improvement problem into components that import\nInternet-scale knowledge from VLMs and a component that learns from autonomous data with a purely self-supervised objective.\n\n![](media/soar_teaser.png)\n\nThis repository contains three components: (1) the VLM powered semantics-aware autonomous data collection pipeline, (2) converting the collected raw data into the RLDS format, and (3) Jax/Flax code for training the policies used in the paper.\n\n## Using SOAR-Data\n\nWe have released SOAR-Data for public access [here](https://rail.eecs.berkeley.edu/datasets/soar_release/). It is available in both [RLDS](https://rail.eecs.berkeley.edu/datasets/soar_release/1.0.0/) format and the [raw numpy](https://rail.eecs.berkeley.edu/datasets/soar_release/numpy_source/) source format.\n\nWe also provide download scripts for both formats, check the `soar_data` directory for more details. In essence, you can download the dataset with:\n```bash\nbash soar_data/rlds/download_dataset.sh  # for RLDS format\nbash soar_data/numpy_source/download_dataset.sh  # for numpy source format\n```\nThe RLDS format requires 136G of disk space and will take around 20 minutes to download if you use the parallel download option, so we recommend downloading inside a tmux session.\n\nTo load the dataset for training and other downstream use cases, we have provided a minimal example [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://githubtocolab.com/rail-berkeley/soar/blob/main/soar_data/load_soar_data.ipynb) that loads the RLDS dataset and visualizes it.\n\n## Installation\n```bash\nconda create -n soar python=3.10\nconda activate soar\n\n# model training requirements\npip install -e model_training\npip install -r model_training/requirements.txt\n\n# data collection requirements (you also need the jaxrl_m library above)\npip install -e data_collection\npip install -r data_collection/requirements.txt\n\n# rlds conversion requirements\npip install -e rlds_converter\npip install -r rlds_converter/requirements.txt\n```\n\nIf you would like to train models with Jax,\nFor GPU:\n```bash\npip install --upgrade \"jax[cuda11_pip]==0.4.20\" -f https://storage.googleapis.com/jax-releases/jax_cuda_releases.html\n```\n\nFor TPU:\n```bash\npip install --upgrade \"jax[tpu]==0.4.20\" -f https://storage.googleapis.com/jax-releases/libtpu_releases.html\n```\n\n\n## Autonomous Data Collection\n\nWe provide a ready-to-use implementation of autonomous data collection on a fleet of WidowX robot arms. This data collection system is designed around deploying instruction following policies at scale to collect autonomous datasets that are semantically relevant, diverse, and large. Special care is taken to minimize human supervision during data collection, with features like automatic reset detection (and subsequent Slack notification).\n\n![](media/autonomous_data_collection.png)\n\nRun autonomous data collection on the robot with:\n```\npython data_collection/orchestrator/robot/main.py --config_dir config/\u003crobot_config_dir\u003e\n```\n\nSee [data_collection/README.md](data_collection/README.md) for more information on the setup required before running data collection.\n\n## Model Training\nThis directory contains a self-contained python project for training goal-conditioned and language conditioned policies on Bridge and on Soar-Data.\n\nTo launch a training run, run:\n```bash\ncd model_training\nbash experiments/scripts/launch.sh\n```\nThis will launch [train.py](model_training/experiments/train.py) with the default arguments specified in [train_config.py](model_training/experiments/configs/train_config.py) and [data_config.py](model_training/experiments/configs/data_config.py).\n\n## RLDS Data Conversion\nWe convert the raw data logged in the `data_collection/*` directories into the commonly used RLDS format. The conversion code is\nlocated in the `rlds_converter` directory. See [rlds_converter/README.md](rlds_converter/README.md) for more information.\n\nTo build the SOAR dataset\n```bash\ncd rlds_converter/soar_dataset\nCUDA_VISIBLE_DEVICES=\"\" tfds build --manual_dir \u003cpath_to_raw_data\u003e\n```\n\n## Citation\n```\n@article{zhou2024autonomous,\n    title={Autonomous Improvement of Instruction Following Skills via Foundation Models},\n    author={Zhiyuan Zhou and Pranav Atreya and Abraham Lee and Homer Walke and Oier Mees and Sergey Levine},\n    journal = {arXiv preprint arXiv:407.20635},\n    year={2024},\n}\n```\n\n## Contributing\nWe welcome pull requests and bug reports to this repo.\n\nTo enable code checks and auto-formatting, please install pre-commit hooks (run this in the root directory):\n```bash\npre-commit install\n```\nThe hooks should now run before every commit. If files are modified during the checks, you'll need to re-stage them and commit again.\n","funding_links":[],"categories":["Robot Arm"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frail-berkeley%2Fsoar","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frail-berkeley%2Fsoar","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frail-berkeley%2Fsoar/lists"}