{"id":50816708,"url":"https://github.com/ikatsov/phy-harness","last_synced_at":"2026-06-13T10:04:34.887Z","repository":{"id":360951604,"uuid":"1248721117","full_name":"ikatsov/phy-harness","owner":"ikatsov","description":null,"archived":false,"fork":false,"pushed_at":"2026-06-06T22:08:05.000Z","size":22121,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-06T23:08:12.013Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://ikatsov.github.io/phy-harness/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ikatsov.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-25T01:44:35.000Z","updated_at":"2026-06-06T22:08:09.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/ikatsov/phy-harness","commit_stats":null,"previous_names":["ikatsov/robotics-devagent","ikatsov/phy-harness"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ikatsov/phy-harness","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ikatsov%2Fphy-harness","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ikatsov%2Fphy-harness/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ikatsov%2Fphy-harness/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ikatsov%2Fphy-harness/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ikatsov","download_url":"https://codeload.github.com/ikatsov/phy-harness/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ikatsov%2Fphy-harness/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34279899,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-13T02:00:06.617Z","response_time":62,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-06-13T10:04:33.733Z","updated_at":"2026-06-13T10:04:34.871Z","avatar_url":"https://github.com/ikatsov.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# PhyHarness - A Dev Harness for Physical AI\n\nPython harness for agentic development of embodied control policies using perception-less motion control, computer vision, and motion generation models.\n\nProject website: [ikatsov.github.io/phy-harness](https://ikatsov.github.io/phy-harness/)\n\nThe current development workflow is:\n\n```\n  ┌──────────────────────────────────────────────────────────────────┐\n  │ INPUTS                                                           │\n  │  Start with policies/impl/\u003ctask\u003e/\u003ctask\u003e.yaml (task spec)         │\n  └───────────────────────────────┬──────────────────────────────────┘\n                                  ▼\n  ┌──────────────────────────────────────────────────────────────────┐\n  │ DESIGN \u0026 IMPLEMENT                                               │\n  │  Coding model implements/updates policy, analyzers, and tests    │\n  └───────────────────────────────┬──────────────────────────────────┘\n                                  ▼\n  ┌──────────────────────────────────────────────────────────────────┐\n  │ SIMULATE                                                         │\n  │  simulate_policy.py generates artifacts/\u003ctask\u003e/ with:            │\n  │  - augmented rollout video (overlays)                            │\n  │  - joints.csv                                                    │\n  │  - VLM transcript JSON (if enabled)                              │\n  │  - custom analyzer outputs                                       │\n  └───────────────────────────────┬──────────────────────────────────┘\n                                  ▼\n  ┌──────────────────────────────────────────────────────────────────┐\n  │ CODING FEEDBACK LOOP                                             │\n  │  Coding model analyzes simulation outputs (video frames, logs,   │\n  │  transcripts, analyzer results), adds focused analyzers/tests,   │\n  │  and loops back to DESIGN \u0026 IMPLEMENT.                           │\n  └──────────────────────────────────────────────────────────────────┘\n```\n\n## Setup\n\n```bash\npython3 -m venv .venv\nsource .venv/bin/activate\npip install -e \".[dev]\"\n```\n\nFor rollout validation with Gemini / VLM analyzers (and `.env` loading):\n\n```bash\npip install -e \".[vlm]\"\ncp .env.example .env   # set GEMINI_API_KEY; do not commit .env\n```\n\nMuJoCo uses an OpenGL backend for offscreen RGB (`mujoco.Renderer`). On a normal desktop, the default platform GL is fine. In headless CI, use `UR5GripperEnv(enable_rgb=False)` or configure OSMesa/EGL per the [MuJoCo rendering docs](https://mujoco.readthedocs.io/en/stable/python.html#passive-viewer). Run full RGB rollouts (`simulate_policy.py`) in a terminal where rendering already works.\n\n### Optional: inverse kinematics (``mink``)\n\nDifferential IK on the **same** simulator MJCF uses [mink](https://github.com/kevinzakka/mink) (MuJoCo-based) plus a QP backend:\n\n```bash\npip install -e \".[ik]\"\n```\n\n## Task workflow\n\n### Inputs\n\nCreate a **task bundle** at `policies/impl/\u003ctask\u003e/`:\n\n| File | Purpose |\n|------|---------|\n| `\u003ctask\u003e.yaml` | **`task_spec.inline`** (intent), optional **`policy_module`** |\n| `\u003ctask\u003e.py` | `policy(obs, step, env) -\u003e ctrl` (and optional `reset`) |\n| `\u003canalyzer_type\u003e.py` | Optional policy-specific analyzer (`build(params) -\u003e analyzer`) |\n| `tests/test_\u003ctask\u003e.py` | Headless unit tests (paired with the task) |\n\n### Canonical loop (per task)\n\nFrom the repo root with the venv active:\n\n```bash\n# 1 — Headless unit tests\npytest -q tests/test_\u003ctask\u003e.py\n\n# 2 — Simulation rollout (writes artifacts/\u003ctask\u003e/)\npython scripts/simulate_policy.py --config policies/simulate_policy.example.yaml \\\n  policies/impl/\u003ctask\u003e/\u003ctask\u003e.py --run-dir artifacts/\u003ctask\u003e\n\n# 3 — Coding agent analyzes the simulation outputs and makes changes\n```\n\n**Design \u0026 implement:** done in policy/analyzer/test files for the task.\n\n**Simulate:** produces augmented rollout video, joints log, and analyzer artifacts.\n\n**Coding feedback loop:** analyze artifacts (including extracted video frames and analyzer JSON), implement fixes, add focused tests/analyzers, and repeat.\n\n**Outputs** under `artifacts/\u003ctask\u003e/`: `rollout.mp4`, `metrics.txt`, `joints.csv`, `rollout.vlm_transcript.json` (when VLM transcriber is enabled), and optional custom analyzer JSON files.\n\n`joints.csv` logs joint `qpos` (and free-joint pose columns), then for each actuator a **`target_*`** column (the vector returned by the policy for that step) and a **`ctrl_*`** column (`data.ctrl` after physics, i.e. applied command). Floats are rounded to five decimal places to keep files smaller.\n\n## Third-party assets\n\nThe robot uses **vendored** [MuJoCo Menagerie](https://github.com/google-deepmind/mujoco_menagerie) `universal_robots_ur5e` OBJ meshes under `src/robot_manipulation_sim/mjcf/menagerie_ur5e/` (see `NOTICE.txt` and `MENAGERIE_LICENSE` there). Default scene: `ur5e_two_finger_scene.xml`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fikatsov%2Fphy-harness","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fikatsov%2Fphy-harness","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fikatsov%2Fphy-harness/lists"}