{"id":13936385,"url":"https://github.com/cvalenzuela/scenescoop","last_synced_at":"2025-09-19T11:32:12.151Z","repository":{"id":54301862,"uuid":"113128808","full_name":"cvalenzuela/scenescoop","owner":"cvalenzuela","description":"A tool to describe the content of videos and suggest similar scenes in other videos/films.","archived":false,"fork":false,"pushed_at":"2021-02-25T15:03:40.000Z","size":21446,"stargazers_count":137,"open_issues_count":5,"forks_count":21,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-04-02T13:11:16.761Z","etag":null,"topics":["learning","machine","movie","python","scene","tensorflow","video"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cvalenzuela.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-12-05T03:43:15.000Z","updated_at":"2025-01-01T01:18:55.000Z","dependencies_parsed_at":"2022-08-13T11:30:24.221Z","dependency_job_id":null,"html_url":"https://github.com/cvalenzuela/scenescoop","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/cvalenzuela/scenescoop","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cvalenzuela%2Fscenescoop","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cvalenzuela%2Fscenescoop/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cvalenzuela%2Fscenescoop/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cvalenzuela%2Fscenescoop/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cvalenzuela","download_url":"https://codeload.github.com/cvalenzuela/scenescoop/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cvalenzuela%2Fscenescoop/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":275927996,"owners_count":25554320,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-19T02:00:09.700Z","response_time":108,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["learning","machine","movie","python","scene","tensorflow","video"],"created_at":"2024-08-07T23:02:37.196Z","updated_at":"2025-09-19T11:32:10.763Z","avatar_url":"https://github.com/cvalenzuela.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Scenescoop\n\nScenescoop is a tool to get similar semantic scenes from a pair of videos. Basically, you input a video and get a scene that has a similar meaning in another video. You can run it as a python script or as a web app.\n\n![description](static/imgs/description2.png)\n\n## How it works\n\nScenescoop uses the [im2text](https://github.com/tensorflow/models/tree/master/research/im2txt) tensorflow model to analyze videos on a frame to frames basis and get a description of the content of those images. Frames with the same description are grouped together to create a sequence or scene. \n\nScenes are then analyzed with [spaCy](https://spacy.io/), for sentence parsing and built-in word vectors, using the average of the word vectors in the sentence. \n\n[Annoy](https://github.com/spotify/annoy) is finally used to create an index for fast nearest-neighbor lookup (based on [@aparrish](https://github.com/aparrish) [Plot to poem](https://github.com/aparrish/plot-to-poem/blob/master/plot-to-poem.ipynb))\n\nThis project is inspired by [Thingscoop](https://github.com/agermanidis/thingscoop).\n\n## Video Demos\n\n### [A man sitting at a table with a plate of food](https://youtu.be/ZF5W_tcnF4s)\n[![A man sitting at a table with a plate of food](static/imgs/food.png)](https://youtu.be/ZF5W_tcnF4s)\n\n### [A group of people walking down the street](https://youtu.be/aaYVMsMMEjc)\n[![A group of people walking down the street](static/imgs/street.png)](https://youtu.be/aaYVMsMMEjc)\n\n## Usage\n\nTo run this you'll need to install a few dependencies. You can follow the [original repository](https://github.com/tensorflow/models/tree/master/research/im2txt) or the instructions [Edouard Fouché](https://edouardfouche.com/Fun-with-Tensorflow-im2txt/) wrote.\n(I plan to write a step-by-step guide on how to install everything)\n\nYou can also get the pretrained model I'm using [here](https://drive.google.com/open?id=1tSTzD21qXXOiXlfgJllgXNZ9lREy6yij).\n\nOnce everything is installed, clone the repo and install the project dependencies:\n\n```\ngit clone https://github.com/cvalenzuela/scenescoop.git\ncd scenescoop\npip install -r requirements.txt\n```\n\nYou can then run Scenescoop in two modes:\n\n### 1) Frame Analysis Mode\n\nGiven a video file `--video` (.mp4, .avi, .mkv or .mov), this will analyse the file frame by frame and output a `.json` file containing the descriptions of the those frames. The `--name` argument should be the output name of the transcript.\n\nExample:\n```\npython scenescoop.py --video videos/moonrisekingdom.mp4 --name moonrisekingdom\n```\n\nThe `.json` file should look something like this:\n\n```\n{ \n...\n\"a person is taking a picture of themselves in a mirror \": [4834], \n\"a man sitting in the back of a pickup truck \": [2265, 2266], \n\"a man sitting on a bench in front of a building \": [1935, 1937, \n1938, 3950, 3951, 3952, 3953, 3960, 4072, 4073, 4074, 4075, \n4077, 4079, 4080, 4082, 4115, 4467], \n\"a man standing next to a tree holding a surfboard \": [2470]\n...\n}\n```\n\n### 2) Transfer Mode\n\nTwo videos are required for this mode and both should have their corresponding `transcript.json` file created in the Frame Analysis Mode.\n\nThe `--input_data` argument should be the `.json` file containing the data for the input video and `--transform_data` is the `.json` file for the transfer video. `--input_seconds` is the input time frame to transfer and `--transform_src` is the video source of the transfer video. \n\nExample:\n```\npython scenescoop.py --input_data transcripts/street.json --input_seconds 0,5 --transform_src videos/her.avi --transform_data transcripts/her.json\n```\n\nYou can print all options with `python scenescoop.py -h`:\n\n```\nusage: scenescoop.py [-h] [--video VIDEO] [--name NAME]\n                     [--input_data INPUT_DATA] [--input_seconds INPUT_SECONDS]\n                     [--transform_src TRANSFORM_SRC]\n                     [--transform_data TRANSFORM_DATA] [--api API]\n\nStoriescoop\n\noptional arguments:\n  -h, --help            show this help message and exit\n  --video VIDEO         Video Source to transform\n  --name NAME           Name of the video\n  --input_data INPUT_DATA\n                        Input Video. Must be a json file.\n  --input_seconds INPUT_SECONDS\n                        Input Video Seconds to create transformation. Example:\n                        1,30\n  --transform_src TRANSFORM_SRC\n                        Transform Video Source.\n  --transform_data TRANSFORM_DATA\n                        Transform Video Data. Must be a json file.\n  --api API             API Request\n```\n\n## Web App\n\nYou can also launch an interactive web app, using a flask server, to run the Frame Analysis Mode and Transfer Mode in a webpage. You'll still need all the dependencies installed.\n\n![description](static/imgs/demo.gif)\n\n\nTo run the app in a local server:\n\n```\npython server.py\n```\n\nThe visit `localhost:8080`.\n\nTo modify the source code:\n```\ncd static\nyarn watch\n```\n\n## MMS\n\nLocal development of the MMS application:\n\nStart ngrok\n```\n./ngrok http 7676\n```\n\nConfigure the url in Twilio and in the server in `NGROK_URL`\n\nStart the Redis server\n```\nredis-server\n```\n\nStart the Celery worker:\n```\ncelery -A server.celery worker\n```\n\nFinally start the server\n```\npython server.py\n```\n\n## License\n\nMIT","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcvalenzuela%2Fscenescoop","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcvalenzuela%2Fscenescoop","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcvalenzuela%2Fscenescoop/lists"}