{"id":31841573,"url":"https://github.com/dirvine/saorsa-robotics","last_synced_at":"2025-10-12T05:21:10.696Z","repository":{"id":309210862,"uuid":"1035458443","full_name":"dirvine/saorsa-robotics","owner":"dirvine","description":"Scaffold for training Hugging Face SO-101 robotic arms using Vision-Language-Action policies without demonstrations","archived":false,"fork":false,"pushed_at":"2025-09-02T13:38:14.000Z","size":807,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-09-02T14:21:56.084Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dirvine.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2025-08-10T12:53:56.000Z","updated_at":"2025-09-02T13:37:56.000Z","dependencies_parsed_at":"2025-08-10T16:24:17.199Z","dependency_job_id":"cabb178a-8f87-4d0c-9e76-f32eda0f7aa6","html_url":"https://github.com/dirvine/saorsa-robotics","commit_stats":null,"previous_names":["dirvine/saorsa-robotics"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/dirvine/saorsa-robotics","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dirvine%2Fsaorsa-robotics","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dirvine%2Fsaorsa-robotics/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dirvine%2Fsaorsa-robotics/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dirvine%2Fsaorsa-robotics/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dirvine","download_url":"https://codeload.github.com/dirvine/saorsa-robotics/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dirvine%2Fsaorsa-robotics/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279010341,"owners_count":26084738,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-12T02:00:06.719Z","response_time":53,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-10-12T05:21:06.843Z","updated_at":"2025-10-12T05:21:10.689Z","avatar_url":"https://github.com/dirvine.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🤖 Saorsa Robotics\n\n\u003e **Production-ready Rust framework for autonomous robotic control with local AI models**\n\n[![Rust](https://img.shields.io/badge/rust-1.75%2B-orange.svg)](https://www.rust-lang.org)\n[![Safety](https://img.shields.io/badge/safety-critical-red.svg)](./crates/safety-guard)\n[![Tests](https://img.shields.io/badge/tests-100%25-brightgreen.svg)](./crates)\n[![License](https://img.shields.io/badge/license-MIT-blue.svg)](./LICENSE)\n\nSaorsa Robotics provides a comprehensive, safety-first framework for robotic control systems with Vision-Language-Action (VLA) models running entirely on local hardware. Built in Rust for memory safety, performance, and reliability.\n\n## 🌟 Key Features\n\n### 🧠 Local AI Models\n- **MolmoAct Integration**: Action Reasoning Model with 3D spatial understanding and Chain-of-Thought planning\n- **Candle ML Framework**: Lightweight Rust-native inference without Python dependencies\n- **OpenVLA Support**: Compatible with cloud and local VLA models\n- **On-Device Learning**: Continual improvement through OFT adapters and intervention learning\n\n### 🛡️ Safety-Critical Design\n- **Formal Constraint System**: Expression-based DSL for defining safety boundaries\n- **Real-Time Monitoring**: Watchdog systems with automatic intervention\n- **Zero Panic Guarantee**: No `unwrap()`, `expect()`, or `panic!()` in production code\n- **Comprehensive Testing**: 100% test coverage on safety-critical paths\n\n### 🎯 Advanced Capabilities\n- **Multi-Modal Control**: Voice commands, vision processing, and haptic feedback\n- **Stereo Vision**: Depth perception with dual camera calibration\n- **CAN Bus Integration**: Direct hardware control for motors and actuators  \n- **Intent Parsing**: Natural language to robot action conversion\n\n### ⚡ Performance\n- **Zero-Copy Data Paths**: Efficient memory usage\n- **Async/Await**: Non-blocking I/O throughout\n- **Action Chunking**: Smooth control despite network latency\n- **Real-Time Capable**: Deterministic timing for critical operations\n\n## 🚀 Quick Start\n\n### Prerequisites\n\n```bash\n# Install Rust\ncurl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh\n\n# Clone repository\ngit clone https://github.com/dirvine/saorsa-robotics\ncd saorsa-robotics\n```\n\n### Build and Test\n\n```bash\n# Build all crates\ncargo build --release\n\n# Run all tests\ncargo test --all\n\n# Run with safety checks\ncargo run --bin sr-cli -- --safety-enabled\n```\n\n### Run Examples\n\n```bash\n# VLA Policy Demo\ncargo run --example vla_policy_demo\n\n# Wake Word Detection\ncargo run --example wake_word_demo\n\n# Safety Constraints Demo\ncargo run --bin safety-demo\n```\n\n## 📦 Architecture\n\n```\nsaorsa-robotics/\n├── apps/                       # Application binaries\n│   ├── sr-cli/                # Main CLI interface\n│   ├── brain-daemon/          # Central coordination daemon\n│   ├── safety-demo/           # Safety system demonstration\n│   └── kyutai-stt-app/       # Speech-to-text application\n├── crates/                    # Core library crates\n│   ├── vla-policy/           # Vision-Language-Action models\n│   ├── safety-guard/         # Safety constraint engine\n│   ├── voice-local/          # On-device voice processing\n│   ├── vision-stereo/        # Stereo vision and depth\n│   ├── intent-parser/        # NLU and command parsing\n│   ├── can-transport/        # CAN bus communication\n│   ├── device-registry/      # Hardware device management\n│   └── continual-learning/   # Online learning framework\n├── examples/                  # Example applications\n├── configs/                   # Device and system configs\n└── docs/                     # Technical documentation\n```\n\n## 🔧 Core Components\n\n### VLA Policy System (`vla-policy`)\n\nImplements multiple Vision-Language-Action models for robot control:\n\n```rust\nuse vla_policy::{create_policy, PolicyConfig, Observation};\n\n// Create MolmoAct policy with 3D reasoning\nlet config = PolicyConfig {\n    model_type: \"molmoact\".to_string(),\n    model_path: \"models/molmoact-7b\".to_string(),\n    // ... configuration\n};\n\nlet policy = create_policy(config)?;\nlet action = policy.predict(\u0026observation).await?;\n```\n\n**Features:**\n- MolmoAct with Chain-of-Thought reasoning\n- Waypoint generation for complex tasks\n- Skills framework (Pick, Place, Reach)\n- Mock policy for testing\n\n### Safety Guard (`safety-guard`)\n\nExpression-based constraint system ensuring safe operation:\n\n```rust\nuse safety_guard::{SafetyGuard, Constraint};\n\nlet mut guard = SafetyGuard::new();\n\n// Define workspace boundaries\nguard.add_constraint(Constraint::expression(\n    \"workspace_x\",\n    \"x \u003e= -0.5 \u0026\u0026 x \u003c= 0.5\"\n)?);\n\n// Check if action is safe\nif guard.check_action(\u0026action)? {\n    robot.execute(action)?;\n}\n```\n\n**Features:**\n- Mathematical expression constraints\n- Real-time evaluation with evalexpr\n- Watchdog monitoring\n- Automatic intervention on violations\n\n### Voice Control (`voice-local`)\n\nOn-device speech recognition and wake word detection:\n\n```rust\nuse voice_local::{KyutaiProvider, WakeWordDetector};\n\nlet provider = KyutaiProvider::new(config)?;\nlet detector = WakeWordDetector::new(\"hey robot\")?;\n\n// Process audio stream\nif detector.detect(\u0026audio_frame)? {\n    let command = provider.transcribe(\u0026audio_buffer)?;\n    execute_command(command)?;\n}\n```\n\n**Features:**\n- Kyutai/Mimi model integration\n- Real-time transcription\n- Wake word detection\n- Plugin architecture for custom models\n\n### Stereo Vision (`vision-stereo`)\n\nDepth perception and 3D scene understanding:\n\n```rust\nuse vision_stereo::{StereoCamera, DepthEstimator};\n\nlet camera = StereoCamera::new(config)?;\ncamera.calibrate()?;\n\nlet (left, right) = camera.capture()?;\nlet depth_map = DepthEstimator::compute(\u0026left, \u0026right)?;\nlet tags = detect_april_tags(\u0026left)?;\n```\n\n**Features:**\n- Dual camera calibration\n- Real-time depth estimation\n- AprilTag detection\n- Point cloud generation\n\n### CAN Transport (`can-transport`)\n\nHardware control via CAN bus:\n\n```rust\nuse can_transport::{SlcanTransport, Message};\n\nlet transport = SlcanTransport::new(\"/dev/ttyUSB0\")?;\n\n// Send motor command\nlet msg = Message::new(0x123, \u0026[0x01, 0x02, 0x03])?;\ntransport.send(\u0026msg)?;\n```\n\n**Features:**\n- SLCAN protocol support\n- ODrive motor control\n- T-Motor actuator support\n- Mock transport for testing\n\n## 🧪 Testing\n\nAll crates maintain 100% test coverage on critical paths:\n\n```bash\n# Run all tests\ncargo test --all\n\n# Run with coverage\ncargo tarpaulin --out Html\n\n# Run safety-critical tests\ncargo test -p safety-guard\n\n# Run benchmarks\ncargo bench\n```\n\nCurrent test status:\n- ✅ `safety-guard`: 13/13 passing\n- ✅ `vla-policy`: 21/21 passing\n- ✅ `voice-local`: All doctests passing\n- ✅ `intent-parser`: 1/1 passing\n- ✅ Zero compilation warnings\n\n## 🔐 Safety \u0026 Security\n\n### Production Standards\n- **No Panics**: Zero `unwrap()`, `expect()`, or `panic!()` in production\n- **Error Handling**: All errors properly propagated with `Result\u003cT, E\u003e`\n- **Memory Safety**: Guaranteed by Rust's ownership system\n- **Concurrency Safety**: Safe parallelism with Send/Sync traits\n\n### Safety Features\n- Formal constraint verification\n- Watchdog timers on all operations\n- Automatic failsafe modes\n- Comprehensive audit logging\n\n## 📚 Documentation\n\n- [Architecture Overview](./docs/README.md)\n- [VLA Policy Design](./docs/SPEC.md)\n- [Safety System](./crates/safety-guard/README.md)\n- [CAN Protocol](./docs/CAN.md)\n- [Vision System](./docs/VISION.md)\n- [Voice Control](./docs/VOICE.md)\n- [Research Notes](./docs/RESEARCH.md)\n\n## 🗺️ Roadmap\n\n### Near Term\n- [ ] ONNX runtime integration for broader model support\n- [ ] ROS2 bridge for ecosystem compatibility\n- [ ] Web dashboard for monitoring and control\n- [ ] Simulation environment with Bevy\n\n### Long Term\n- [ ] Distributed multi-robot coordination\n- [ ] Federated learning across robot fleets\n- [ ] Custom silicon accelerator support\n- [ ] Formal verification of safety properties\n\n## 🤝 Contributing\n\nWe welcome contributions! Please see [CONTRIBUTING.md](./CONTRIBUTING.md) for guidelines.\n\nKey areas for contribution:\n- Additional VLA model implementations\n- Hardware device drivers\n- Safety constraint patterns\n- Documentation and examples\n\n## 📄 License\n\nMIT License - see [LICENSE](./LICENSE) for details.\n\n## 🙏 Acknowledgments\n\n- Built with [Candle](https://github.com/huggingface/candle) for ML inference\n- Inspired by [LeRobot](https://github.com/huggingface/lerobot) for robot learning\n- Safety patterns from aerospace and automotive industries\n\n## 📬 Contact\n\n- GitHub: [@dirvine](https://github.com/dirvine)\n- Project: [Saorsa Labs](https://saorsa.org)\n\n---\n\n*For the original Python implementation for SO-101 arms, see [archive/python-so101](./archive/python-so101/README.md)*","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdirvine%2Fsaorsa-robotics","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdirvine%2Fsaorsa-robotics","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdirvine%2Fsaorsa-robotics/lists"}