{"id":28416865,"url":"https://github.com/mlshukai/pamiq-vrchat","last_synced_at":"2025-06-25T05:31:18.728Z","repository":{"id":289601479,"uuid":"971789956","full_name":"MLShukai/pamiq-vrchat","owner":"MLShukai","description":"VRChat interfaces for PAMIQ","archived":false,"fork":false,"pushed_at":"2025-06-12T02:47:26.000Z","size":1668,"stargazers_count":3,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-12T03:44:25.304Z","etag":null,"topics":["agent","ai","ai-agents-framework","ami","autonoumous-machine-intelligence","game","interface","linux","machine-learning","machine-learning-library","ml","pamiq","vrchat","vrchat-tool","windows"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MLShukai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-04-24T04:13:12.000Z","updated_at":"2025-06-12T02:47:29.000Z","dependencies_parsed_at":"2025-05-09T00:18:42.451Z","dependency_job_id":"7d387c0b-3f2c-4081-be3c-24443184e2d8","html_url":"https://github.com/MLShukai/pamiq-vrchat","commit_stats":null,"previous_names":["mlshukai/pamiq-vrchat"],"tags_count":1,"template":false,"template_full_name":"Geson-anko/python-uv-template","purl":"pkg:github/MLShukai/pamiq-vrchat","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MLShukai%2Fpamiq-vrchat","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MLShukai%2Fpamiq-vrchat/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MLShukai%2Fpamiq-vrchat/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MLShukai%2Fpamiq-vrchat/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MLShukai","download_url":"https://codeload.github.com/MLShukai/pamiq-vrchat/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MLShukai%2Fpamiq-vrchat/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261812887,"owners_count":23213568,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agent","ai","ai-agents-framework","ami","autonoumous-machine-intelligence","game","interface","linux","machine-learning","machine-learning-library","ml","pamiq","vrchat","vrchat-tool","windows"],"created_at":"2025-06-04T01:14:30.549Z","updated_at":"2025-06-25T05:31:18.720Z","avatar_url":"https://github.com/MLShukai.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"![logo](./docs/images/logo.svg)\n\n# PAMIQ VRChat\n\nInterface for PAMIQ to interact with VRChat.\n\n## ✨ Features\n\n- **Visual Input**: Capture VRChat gameplay through OBS virtual camera with the `ImageSensor`\n- **Mouse Control**: Simulate mouse movements and clicks with natural-feeling transitions using `MouseActuator` and `SmoothMouseActuator`\n- **Avatar Control**: Send OSC commands to VRChat for avatar movement, jumping, and actions via `OscActuator` and `SmoothOscActuator`\n- **Smooth Transitions**: Natural-feeling controls with gradual acceleration and realistic button timings\n- **PAMIQ Integration**: Seamlessly works with [PAMIQ-Core](https://mlshukai.github.io/pamiq-core/) for AI agent development\n\n## 📦 Installation\n\n\u003e \\[!NOTE\\]\n\u003e If **Linux 🐧** user, install [**inputtino**](https://github.com/games-on-whales/inputtino/tree/stable/bindings/python#installation) first which is a required dependency.\n\n```sh\n# Install via pip\npip install pamiq-vrchat\n\n# Install from source\ngit clone https://github.com/MLShukai/pamiq-vrchat.git\ncd pamiq-vrchat\npip install .\n```\n\n## 🛠️ Setup VRChat Environment\n\n### Prerequisites\n\n- Linux or Windows machine with Desktop environment.\n- Machine capable of running VRChat\n\n### Install Steam\n\nDownload and install Steam from the [official website](https://store.steampowered.com/about/).\n\n### **(🐧 Linux User Only)** Enable Proton\n\nOpen Steam → Settings → Compatibility and enable `Enable Steam Play for all other titles`.\n\n![steam_compatibility](./docs/images/steam_compatibility.png)\n\n#### (Optional) Install Proton GE\n\nIf you want to use video players in VRChat on Linux, install [Proton GE](https://github.com/GloriousEggroll/proton-ge-custom?tab=readme-ov-file#installation).\n\nAfter installation, select `GE-Proton` as the compatibility tool in Steam → Settings → Compatibility under `Run other titles with:`.\n\n### Install VRChat\n\nAdd **VRChat** to your library from the [Steam store](https://store.steampowered.com/app/438100/VRChat/) and install it.\n\nAfter install, launch VRChat and login.\n\n### Setup OBS\n\nFor OBS installation and virtual camera setup, refer to [pamiq-io documentation](https://github.com/MLShukai/pamiq-io?tab=readme-ov-file#obs-virtual-camera).\n\n\u003e \\[!NOTE\\]\n\u003e The `Output (Scaled) Resolution` and `FPS Value` in OBS Video settings will affect the output of the `ImageSensor` class.\n\u003e ![obs-video-setting](./docs/images/obs_video_setting.png)\n\nCapture the VRChat window in OBS and enable the virtual camera.\n\nYou can also use our pre-configured [Scene Collection](./obs_settings/). Import it from the OBS `Scene Collection` tab → `Import`, and ensure the checkbox is checked.\n\n### Enable OSC\n\n1. Open the \"Launch Pad\" (press the `Esc` key)\n2. Go to the main menu\n3. Open settings (click the ⚙️ icon)\n4. Click `Search all settings` and type \"OSC\", then press Enter\n5. Enable the `OSC` button\n\n![enable_osc](./docs/images/osc_enable.png)\n\n## 🚀 Quick Examples\n\n### Image Sensor\n\n```python\nfrom pamiq_vrchat.sensors import ImageSensor\n\n# Connect to OBS virtual camera automatically\nsensor = ImageSensor()\n# Or specify a camera index\n# sensor = ImageSensor(camera_index=0)\n# (Windows only) you can specify width and height\n# sensor = ImageSensor(width=1920, height=1080)\n\n# Capture a frame\nframe = sensor.read()\n# frame is a numpy array with shape (height, width, channels)\n```\n\n### Mouse Control\n\n\u003e \\[!NOTE\\]\n\u003e When using mouse control, remember to keep the VRChat game window focused (selected) on your desktop.\n\n```python\nfrom pamiq_vrchat.actuators import MouseActuator, MouseButton, SmoothMouseActuator\n\n# Basic mouse control\nmouse = MouseActuator()\n# Move mouse at 100 pixels/sec horizontally, 50 pixels/sec vertically\nmouse.operate({\"move_velocity\": (100.0, 50.0)})\n# Click the left button\nmouse.operate({\"button_press\": {MouseButton.LEFT: True}})\n# Release the left button\nmouse.operate({\"button_press\": {MouseButton.LEFT: False}})\n\n# Smooth mouse control with gradual acceleration and natural-feeling clicks\nsmooth_mouse = SmoothMouseActuator(\n    delta_time=0.05,     # Update interval\n    time_constant=0.2,   # Movement smoothing\n    press_delay=0.05,    # Button press delay\n    release_delay=0.1    # Button release delay\n)\nsmooth_mouse.operate({\"move_velocity\": (100.0, 50.0)})\n```\n\n### OSC Avatar Control\n\n```python\nfrom pamiq_vrchat.actuators import OscActuator, OscAxes, OscButtons, SmoothOscActuator\n\n# Basic OSC control\nosc = OscActuator()\n# Move forward\nosc.operate({\"axes\": {OscAxes.Vertical: 1.0}})\n# Jump\nosc.operate({\"buttons\": {OscButtons.Jump: True}})\n# Move forward while running\nosc.operate({\n    \"axes\": {OscAxes.Vertical: 1.0},\n    \"buttons\": {OscButtons.Run: True}\n})\n\n# Smooth OSC control for more natural movement\nsmooth_osc = SmoothOscActuator(\n    delta_time=0.05,     # Update interval\n    time_constant=0.2,   # Axis value smoothing\n    press_delay=0.05,    # Button press delay\n    release_delay=0.1    # Button release delay\n)\nsmooth_osc.operate({\"axes\": {OscAxes.Vertical: 0.5}})\n```\n\n## 🎮 Running the Sample Project\n\nAfter setting up VRChat environment and cloning this repository, you can run the sample project. It trains an agent that interacts with VRChat.\n\n- **For Linux Users**\n\n  ```bash\n  # Run the sample\n  ./run-sample.linux.sh\n  ```\n\n- **For Windows Users**\n\n  ```powershell\n  # Run the sample (in PowerShell)\n  .\\Run-Sample.Windows.ps1\n  # Or Bypass execution policy.\n  powershell -noexit -ExecutionPolicy Bypass -File .\\Run-Sample.Windows.ps1\n  ```\n\nThese scripts will:\n\n- Check and install dependencies automatically\n- Verify CUDA availability\n- Check if VRChat and OBS are running\n- Start the keyboard control interface ([**`pamiq-kbctl`**](https://mlshukai.github.io/pamiq-core/user-guide/console/#keyboard-shortcut-controller))\n- Launch the autonomous learning agent\n\n\u003e \\[!IMPORTANT\\]\n\u003e **Mouse Control Notice:** When the agent starts, it will take control of your mouse for VRChat interaction. To pause the system, press **`Alt+Shift+P`**. This is essential for regaining mouse control when needed.\n\nFor detailed implementation, see [`src/run_sample.py`](src/run_sample.py) which contains the architecture, hyperparameters and training procedure.\n\n## 🤝 Contributing\n\nSee our [Contributing Guide](CONTRIBUTING.md) for details on how to set up the development environment and contribute to the project.\n\n## 📄 License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmlshukai%2Fpamiq-vrchat","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmlshukai%2Fpamiq-vrchat","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmlshukai%2Fpamiq-vrchat/lists"}