{"id":27119394,"url":"https://github.com/nengelmann/meetingcam","last_synced_at":"2025-04-07T08:52:41.212Z","repository":{"id":195018722,"uuid":"689600536","full_name":"nengelmann/MeetingCam","owner":"nengelmann","description":"Run your AI and CV algorithms in meetings such as Zoom, Meets or Teams! 🚀","archived":false,"fork":false,"pushed_at":"2024-02-18T20:31:01.000Z","size":18244,"stargazers_count":13,"open_issues_count":0,"forks_count":2,"subscribers_count":5,"default_branch":"main","last_synced_at":"2024-11-08T07:39:32.279Z","etag":null,"topics":["aritificial-intelligence","computer-vision","meeting-tools","meetings","meets","online-meetings","teams","webcam","webcam-hacking","zoom"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nengelmann.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-09-10T10:36:16.000Z","updated_at":"2024-11-04T08:56:22.000Z","dependencies_parsed_at":null,"dependency_job_id":"d4bd49fc-2422-4a05-9246-370e36d6d175","html_url":"https://github.com/nengelmann/MeetingCam","commit_stats":null,"previous_names":["nengelmann/meetingcam"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nengelmann%2FMeetingCam","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nengelmann%2FMeetingCam/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nengelmann%2FMeetingCam/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nengelmann%2FMeetingCam/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nengelmann","download_url":"https://codeload.github.com/nengelmann/MeetingCam/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247623011,"owners_count":20968574,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aritificial-intelligence","computer-vision","meeting-tools","meetings","meets","online-meetings","teams","webcam","webcam-hacking","zoom"],"created_at":"2025-04-07T08:52:40.708Z","updated_at":"2025-04-07T08:52:41.195Z","avatar_url":"https://github.com/nengelmann.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n[![image](https://github.com/nengelmann/MeetingCam/assets/120744129/f49b189c-1e3f-4396-baff-8fd4a8633b89)](https://www.hackster.io/poseestimators/meetingcam-run-your-ai-and-cv-algorithms-in-meetings-de7bc6)\n\n\u003cp align=\"center\"\u003e\n  \u003ch1 align=\"center\"\u003eMeetingCam\u003c/h1\u003e\n  \u003cp align=\"center\"\u003e\u0026#x2728 Special effects for not so special meetings. \u0026#x1F440\u003c/p\u003e\n\u003c/p\u003e\n\u003chr /\u003e\n\u003cp align=\"center\"\u003e\n    Run your AI and CV algorithms in online meetings such as Zoom, Meets or Teams! 🚀\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e  \n    This Project is one of the \u003ca href=\"https://www.hackster.io/poseestimators/meetingcam-run-your-ai-and-cv-algorithms-in-meetings-de7bc6\"\u003ePopular Vote Finalists\u003c/a\u003e of the OpenCV AI Competition 2023. 🎉\n\u003c/p\u003e\n\u003chr /\u003e\n\nhttps://github.com/nengelmann/MeetingCam/assets/120744129/ec2e608d-e785-4179-ba33-c692da05a95b\n\n[First-person face detection](src/meetingcam/plugins/openvino_face_detection/README.md) prints your face detection and name in your webcam stream. \\\nMore plugins are listed further down the readme. \n\n## Intro video\n\n[![Intro Video](https://img.youtube.com/vi/EkfZDEayFqM/0.jpg)](https://www.youtube.com/watch?v=EkfZDEayFqM)\n\n## Installation\n\n1. Clone this repo\n\n   ```bash\n   git clone https://github.com/nengelmann/MeetingCam.git \u0026\u0026 cd MeetingCam\n   ```\n\n2. Install [virtualenv](https://virtualenv.pypa.io/en/stable/installation.html)\n\n3. Create a virtual python3.10 environment\n   ```bash\n   virtualenv -p /usr/bin/python3.10 .venv \u0026\u0026 source .venv/bin/activate\n   ```\n4. Install the dependencies\n   ```bash\n   python -m pip install -r requirements.txt\n   ```\n5. Setup [pyvirtualcam](https://github.com/letmaik/pyvirtualcam)\n   ```bash\n   sudo apt update\n   sudo apt install v4l2loopback-dkms\n   sudo apt install v4l-utils\n   ```\n## Usage\n\n### First add a camera device to MeetingCam\n\nTo show a modified camera stream in online meetings it is necessary to create a virtual camera for each real camera you want to use.\n\n1. Make sure your webcam is connected.\n\n2. Activate the virtual environment\n   ```bash\n   source .venv/bin/activate\n   ```\n3. Run the main.py to see general commands and plugin options.\n   ```bash\n   python ./src/meetingcam/main.py\n   ```\n4. List available cameras\n   ```bash\n   python src/meetingcam/main.py list-devices\n   ```\n5. Get command to add a camera device\n   ```bash\n   python src/meetingcam/main.py add-devices\n   ```\n   Then **copy and execute** the **command of the camera you want to use**.\n6. You can now see that the camera of your choice has a virtual counterpart.\n   ```bash\n   python src/meetingcam/main.py list-devices\n   ```\n\n### Run a plugin\n\nThe general command to run a plugin is:\n```bash\npython src/meetingcam/main.py PLUGIN_NAME [OPTIONS] DEVICE_PATH\n```\n\ne.g. the face detection plugin:\n```bash\npython src/meetingcam/main.py face-detector --name yourname /dev/video0\n```\n\nOptions and usage for each plugin are documented in the plugins help function \\\nand in it's readme (link below 👇)\n\n## Available plugins\n### [**first-person face detector**](src/meetingcam/plugins/openvino_face_detection/)\n[\u003cimg src=\"./assets/example_face_detection_n_+_f_trigger.png\" height=150\u003e](src/meetingcam/plugins/openvino_face_detection/)  \\\nDetection of first persons face and imprint of bounding box with name.\n\n### [**roboflow general**](src/meetingcam/plugins/roboflow_general/)\n[\u003cimg src=\"https://2486075003-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M6S9nPJhEX9FYH6clfW%2Fuploads%2FVW4FMckhfS5GGlUcpuY2%2F642746dba53a59a614a64b35_roboflow-open-graph.png?alt=media\u0026token=d120c000-46a4-411b-aba3-db055d48a904\" height=\"150\"\u003e](src/meetingcam/plugins/roboflow_general/) \\\n Runs roboflow object-detection and instance-segmentation models.\n\n### [**depthai yolov5**](src/meetingcam/plugins/depthai_yolov5_coco/)\n[\u003cimg src=\"https://user-images.githubusercontent.com/56075061/144863247-fa819d1d-28d6-498a-89a8-c3f94d9e9357.gif\" height=\"150\"\u003e](src/meetingcam/plugins/depthai_yolov5_coco/) \\\nRuns a Yolov5 model trained on COCO. Computation on a depthai device.\n\n## Custom plugins\n\nYou can create custom plugins from a predefined template to kickstart your project. 🚀\n\n1. Create plugin from template:\n   ```bash\n   python src/meetingcam/main.py create-plugin\n   ```\n   Alternative:\n   ```bash\n   python src/meetingcam/main.py create-plugin --name your_plugin --short-description \"My awesome plugin\"  --description \"Plugin which runs some custom AI and CV algorithms\"\n   ```\n   Make sure to add quotes `\"` to sentences with spaces otherwise it will be interpreted as an extra argument.\n2. Run the template to ensure the creation worked out, see [Run a plugin](#run-a-plugin)\n   ```bash\n   python src/meetingcam/main.py your_plugin /dev/your_device\n   ```\n   \u003cimg src=\"assets/example_custom_plugin_template.png\" height=\"350\"\u003e \\\n   You'll see an imprint if you access the camera in a meeting tool.\n3. Go to [src/meetingcam/plugins/](src/meetingcam/plugins/) and you see your plugin as a directory which contains a `plugin.py` file. \\\nThis file can be customized to run your CV and AI on the webcam stream! \\\nIt boils down to a `CustomPlugin` class where you can initialize and process the images as well as a typer main function where you need to adapt custom arguments. \n\n   - `CustomPlugin` class: \\\n   This class needs to have an `init` and `process` function, for initialization and processing of a camera images respectively.\n   \n      ```python\n      class YourPluginClass(PluginBase):\n      \n         def __init__(self, your_arg) -\u003e None:\n            super().__init__()\n            # some more custom init, e.g. your AI model\n            self.your_arg = your_arg\n            self.model = YourModel()\n\n         def process(\n            self,\n            image: NDArray[Any],\n            detection: Any,\n            keyhandler: Type[KeyHandler],\n         ) -\u003e NDArray[Any]:\n            # custom image processing\n            image = self.model(image, your_arg)\n            cv2.putText(image,\"That's printed in every frame!\", (10, image.shape[0]//2), cv2.FONT_HERSHEY_SIMPLEX, 1, 255)\n            return image\n      ```\n   - Last step is to adapt your plugins entry point (typer app), if you have some arguments you want to pass via CLI.\n      ```python\n      @plugin_app.callback(rich_help_panel=\"Plugin-Commands\")\n      def main(\n         device_path: DevicePath = DevicePathWebcam,\n         your_arg: Optional[str] = typer.Option(\n            default=None, help=\"Your custom argument.\"\n         ),\n      ):\n         # instantiate your plugin\n         plugin = YourPluginClass(your_arg)\n         # instantiate runner with plugin and device path\n         runner = Runner(plugin, device_path)\n         # run\n         runner.run()\n      ```\n\n\n## Hotkey triggers\n\nThere are two default **triggers** build in. They allow you to switch the image color channel from BGR to RGB and to mirror the video stream.\nThe main purpose is, that if your camera is outputting an BGR stream by default you would look bluish, you can switch without modifying the code. Also the mirroring is helpful in some online meeting tools.\n\n**`\u003cCtrl\u003e+\u003cAlt\u003e+r`** -\u003e Switch color channels (RGB\u003c\u003eBGR). \\\n**`\u003cCtrl\u003e+\u003cAlt\u003e+m`** -\u003e Mirror the video stream.\n\nPlugins can use additional triggers within their processing function. This is useful to hide some imprints or run the model inference when needed.\n\nCustom hotkey triggers can be defined in the plugins main class initialization with by specifying a hotkeys list with `Hotkey(key_combination, variable_name, is_enabled, description)`.\n\n```python\nself.hotkeys = [\n            Hotkey(\"\u003cCtrl\u003e+\u003cAlt\u003e+z\", \"z_trigger\", True, \"toggle something\"),\n        ]\n```\n\nThis trigger can be accessed during runtime in the plugins processing function as follows.\n\n```python\n# If z_trigger \u003cCtrl\u003e+\u003cAlt\u003e+z is True, print in additional text\nif keyhandler.z_trigger:\n   image = cv2.putText(\n                image,\n                \"Toggle this text with \u003cCtrl\u003e+\u003cAlt\u003e+z\",\n                (100, 100)\n   )\n```\n\nHave a look at the available [example plugins](#available-plugins) for more information or [create your own](#custom-plugins).\n\n## Further information\n\n### Restrictions\n\n**Resolution** \\\nIn general most meeting tools have the restriction on camera/video resolution, mostly 720p (1270x720 pixels). Make sure to not send a virtual video stream with higher resolution. If it breaks due to resolution you can select the virtual camera in the meeting tool but it will either show \"Camera failed\" or a black video stream.\n\n**Chrome/Chromium Browser** \\\nChrome/Chromium might need to be ran with `exclusive_caps=1` in the `sudo modprobe v4l2loopback` command which is not (yet) supported by MeetingCam.\n\n### Tool compatibility\n\n#### Meets ✅\n\nWorks on Firefox.\n\n#### Zoom ✅\n\nWorks on Firefox.\n\n#### Teams 🚧\n\nThere are several Problems running MeetingCam on Linux with Teams. \\\nTeams, at least on Linux is just running on Chromium, Edge and it's Progressive Web App (PWA). Therefore we need to run the `sudo modprobe v4l2loopback` command with `exclusive_caps=1` argument, which makes troubles down the road. For some cameras this might work but is likely to fail. Even with the `exclusive_caps=1` argument it is not reliably working. \\\nA **workaround** is described [here](https://medium.com/@dan_ringwald/make-microsoft-teams-work-on-linux-with-firefox-browser-867fa0485ac). On Firefox Version 117.0.1 (snap install) it was working with the overwrite `Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.74 Safari/537.36 Edg/79.0.309.43`. This overwrite might cause unwanted behaviour (including Google Meets calls are failing). So it is advisable to not add the overwrite in your main firefox profile. However you can run `firefox -p` and create a new dedicated profile and apply the above changes. Then you can open it with `firefox -p` or `firefox -P PROFILE_NAME` and run it isolated.\n\n### Camera setup info\n\nMore information on how to setup virtual cameras can be found [here](https://wiki.archlinux.org/title/V4l2loopback)\n\n### Development and modifications\n\nYou can customize this repo for your needs, you can also write your own AI-Plugin for running your models on Zoom, Teams or Meets. \\\nMore information about that in the section [Custom plugins](#custom-plugins) section and in [DEVELOP.md](DEVELOP.md)\nTemplates and documentation on how to run your custom AI will follow.\n\n### Usage with depthai camera\n\n#### Run OAK devices as webcam (uvc)\n\nRun OAK devices as uvc before normal usage of MeetingCam.\n\n```bash\n source .venv/bin/activate\n python ./tools/oak_as_uvc.py\n```\n\n#### Run OAK devices with on device compute\n\nFollow the steps in the [Usage](#Usage) section normally, just append a `--type depthai` flag to the `list-devices` and `add-devices` command and use a plugin which is of type depthai.\n\n```bash\n python src/meetingcam/main.py list-devices --type depthai\n python src/meetingcam/main.py add-devices --type depthai\n```\n\n### Usage with Roboflow\n\nFor roboflow it is currently needed to run a roboflow inference server in a separate terminal.\n\nYou can run it e.g. on CPU with the following command:\n```bash\ndocker run --net=host roboflow/roboflow-inference-server-cpu:latest\n```\nSee [here](https://github.com/roboflow/inference) for more options.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnengelmann%2Fmeetingcam","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnengelmann%2Fmeetingcam","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnengelmann%2Fmeetingcam/lists"}