{"id":13712316,"url":"https://github.com/achalddave/segment-any-moving","last_synced_at":"2025-07-17T19:42:13.720Z","repository":{"id":142334046,"uuid":"199077972","full_name":"achalddave/segment-any-moving","owner":"achalddave","description":"Code for \"Towards Segmenting Anything That Moves\"","archived":false,"fork":false,"pushed_at":"2019-12-22T20:24:03.000Z","size":363,"stargazers_count":109,"open_issues_count":4,"forks_count":12,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-06-23T10:53:47.866Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/achalddave.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"licenses/pycococreator-LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-07-26T20:45:50.000Z","updated_at":"2025-01-14T03:28:01.000Z","dependencies_parsed_at":null,"dependency_job_id":"e2900c1f-5a46-43ba-8a1f-52c519458888","html_url":"https://github.com/achalddave/segment-any-moving","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/achalddave/segment-any-moving","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/achalddave%2Fsegment-any-moving","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/achalddave%2Fsegment-any-moving/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/achalddave%2Fsegment-any-moving/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/achalddave%2Fsegment-any-moving/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/achalddave","download_url":"https://codeload.github.com/achalddave/segment-any-moving/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/achalddave%2Fsegment-any-moving/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265653509,"owners_count":23805837,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T23:01:17.086Z","updated_at":"2025-07-17T19:42:13.658Z","avatar_url":"https://github.com/achalddave.png","language":"Python","funding_links":[],"categories":["Relevant Repos","Open Source Projects"],"sub_categories":["Follow-up Papers"],"readme":"# Towards Segmenting Anything That Moves\n\n[\u003cimg src=\"http://www.achaldave.com/projects/anything-that-moves/videos/ZXN6A-tracked-with-objectness-trimmed.gif\" width=\"32%\" /\u003e](http://www.achaldave.com/projects/anything-that-moves/videos/ZXN6A-tracked-with-objectness-trimmed.mp4)[\u003cimg src=\"http://www.achaldave.com/projects/anything-that-moves/videos/c95cd17749.gif\" width=\"32%\" /\u003e](http://www.achaldave.com/projects/anything-that-moves/videos/c95cd17749.mp4)[\u003cimg src=\"http://www.achaldave.com/projects/anything-that-moves/videos/e0bdb5dfae.gif\" width=\"32%\" /\u003e](http://www.achaldave.com/projects/anything-that-moves/videos/e0bdb5dfae.mp4)\n\n[[Pre-print](https://arxiv.org/abs/1902.03715)] [[Website](http://www.achaldave.com/projects/anything-that-moves/)]\n\n[Achal Dave](http://www.achaldave.com/), [Pavel Tokmakov](http://thoth.inrialpes.fr/people/tokmakov/), [Deva Ramanan](http://www.cs.cmu.edu/~deva/)\n\n## Setup\n\n1. Download\n   [models](https://drive.google.com/file/d/1qckICZRzX_GBTJSRhn2NDMoJuVppgWUS/view?usp=sharing)\n   and extract them to release/models\n1. Install pytorch 0.4.0.\n1. Run `git submodule update --init`.\n1. Setup [detectron-pytorch](https://github.com/achalddave/segment-any-moving_detectron/).\n1. Setup [flownet2](https://github.com/lmb-freiburg/flownet2). If you just\nwant to use the appearance stream, you can skip this step.\n1. Install requirements with `pip install -r requirements.txt`\u003csup\u003e[1](#footnote1)\u003c/sup\u003e.\n1. Copy [`./release/example_config.yaml`](./release/example_config.yaml) to\n   `./release/config.yaml`, and edit fields marked with `***EDIT THIS***`.\n1. Add root directory to `PYTHONPATH`: `source ./env.sh activate`.\n\n## Running models\n\nAll scripts needed for running our models on standard datasets, as well as on\nnew videos, are provided in the [`./release`](./release) directory. Outside\nof the `release` directory, this repository contains a number of scripts\nwhich are not used for the final results. They can be safely ignored, but are\nprovided in case anyone finds them useful.\n\n## Run on your own video\n\n1. **Extract frames**: To run the model on your own video, first dump the frames from your video.\nFor a single video, you can just use\n\n    ```ffmpeg -i video.mp4 %04d.jpg```\n\n    Alternatively, you can use [this\nscript](https://github.com/achalddave/video-tools/blob/master/dump_frames.py)\nto extract frames in parallel on multiple videos.\n\n1. **Run joint model**: To run the joint model, run the following commands:\n    ```bash\n    # Inputs\n    FRAMES_DIR=/path/to/frames/dir\n    # Outputs\n    OUTPUT_DIR=/path/to/output/dir\n\n    python release/custom/run.py \\\n    --model joint \\\n    --frames-dir ${FRAMES_DIR} \\\n    --output-dir ${OUTPUT_DIR}\n    ```\n\n1. **Run appearance only model**: To run only the appearance model, you don't\n    need to compute optical flow, or set up flownet2:\n    ```bash\n    python release/custom/run.py \\\n    --model appearance \\\n    --frames-dir ${FRAMES_DIR} \\\n    --output-dir ${OUTPUT_DIR}\n    ```\n\n\n## FBMS, DAVIS 2016/2017, YTVOS\n\nThe instructions for FBMS, DAVIS 2016/2017 and YTVOS datasets are roughly the\nsame. Once you have downloaded the dataset and edited the paths in\n`./release/config.yaml`, run the following scripts:\n\n```bash\n# or davis16, davis17, ytvos\ndataset=fbms\npython release/${dataset}/compute_flow.py\npython release/${dataset}/infer.py\npython release/${dataset}/track.py\n# For evaluation:\npython release/${dataset}/evaluate.py\n```\n\nNote that by default, we use our final model trained on COCO, FlyingThings3D,\nDAVIS, and YTVOS. For YTVOS, we provide the option to run using a model that\nwas trained without YTVOS, to evaluate generalization. To activate this, pass\n`--without-ytvos-train` to `release/ytvos/infer.py` and\n`release/ytvos/track.py`.\n\n---\n\u003ca name=\"footnote1\"\u003e1\u003c/a\u003e: This should contain all the requirements, but this was created manually so I may be missing some pip modules. If you run into an import error, try pip installing the module, and/or [file an issue](https://github.com/achalddave/segment-any-moving/issues).\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fachalddave%2Fsegment-any-moving","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fachalddave%2Fsegment-any-moving","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fachalddave%2Fsegment-any-moving/lists"}