{"id":15126196,"url":"https://github.com/modelscope/funclip","last_synced_at":"2025-05-13T19:18:08.874Z","repository":{"id":170560245,"uuid":"641795100","full_name":"modelscope/FunClip","owner":"modelscope","description":"Open-source, accurate and easy-to-use video speech recognition \u0026 clipping tool, LLM based AI clipping intergrated.","archived":false,"fork":false,"pushed_at":"2025-03-11T06:18:06.000Z","size":45292,"stargazers_count":4541,"open_issues_count":33,"forks_count":519,"subscribers_count":42,"default_branch":"main","last_synced_at":"2025-05-13T07:03:26.352Z","etag":null,"topics":["gradio","gradio-python-llm","llm","speech-recognition","speech-to-text","subtitles-generator","video-clip","video-subtitles"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/modelscope.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-05-17T07:23:44.000Z","updated_at":"2025-05-13T03:01:25.000Z","dependencies_parsed_at":"2025-03-27T06:01:13.448Z","dependency_job_id":"a3da2a4e-019b-4828-84de-a37fc003d29b","html_url":"https://github.com/modelscope/FunClip","commit_stats":null,"previous_names":["alibaba-damo-academy/funasr-app","alibaba-damo-academy/paraclipper","alibaba-damo-academy/funclip","modelscope/funclip"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/modelscope%2FFunClip","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/modelscope%2FFunClip/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/modelscope%2FFunClip/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/modelscope%2FFunClip/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/modelscope","download_url":"https://codeload.github.com/modelscope/FunClip/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254010830,"owners_count":21999004,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gradio","gradio-python-llm","llm","speech-recognition","speech-to-text","subtitles-generator","video-clip","video-subtitles"],"created_at":"2024-09-26T02:02:29.731Z","updated_at":"2025-05-13T19:18:03.855Z","avatar_url":"https://github.com/modelscope.png","language":"Python","readme":"[![SVG Banners](https://svg-banners.vercel.app/api?type=rainbow\u0026text1=FunClip%20%20🥒\u0026width=800\u0026height=210)](https://github.com/Akshay090/svg-banners)\n\n### \u003cp align=\"center\"\u003e「[简体中文](./README_zh.md) | English」\u003c/p\u003e\n\n**\u003cp align=\"center\"\u003e ⚡ Open-source, accurate and easy-to-use video clipping tool \u003c/p\u003e**\n**\u003cp align=\"center\"\u003e 🧠 Explore LLM based video clipping with FunClip \u003c/p\u003e**\n\n\u003cp align=\"center\"\u003e \u003cimg src=\"docs/images/interface.jpg\" width=444/\u003e\u003c/p\u003e\n\n\u003cp align=\"center\" class=\"trendshift\"\u003e\n\u003ca href=\"https://trendshift.io/repositories/10126\" target=\"_blank\"\u003e\u003cimg src=\"https://trendshift.io/api/badge/repositories/10126\" alt=\"alibaba-damo-academy%2FFunClip | Trendshift\" style=\"width: 250px; height: 55px;\" width=\"300\" height=\"55\"/\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n\u003cdiv align=\"center\"\u003e  \n\u003ch4\u003e\n\u003ca href=\"#What's New\"\u003e What's New \u003c/a\u003e\n｜\u003ca href=\"#On Going\"\u003e On Going \u003c/a\u003e\n｜\u003ca href=\"#Install\"\u003e Install \u003c/a\u003e\n｜\u003ca href=\"#Usage\"\u003e Usage \u003c/a\u003e\n｜\u003ca href=\"#Community\"\u003e Community \u003c/a\u003e\n\u003c/h4\u003e\n\u003c/div\u003e\n\n**FunClip** is a fully open-source, locally deployed automated video clipping tool. It leverages Alibaba TONGYI speech lab's open-source [FunASR](https://github.com/alibaba-damo-academy/FunASR) Paraformer series models to perform speech recognition on videos. Then, users can freely choose text segments or speakers from the recognition results and click the clip button to obtain the video clip corresponding to the selected segments (Quick Experience [Modelscope⭐](https://modelscope.cn/studios/iic/funasr_app_clipvideo/summary) [HuggingFace🤗](https://huggingface.co/spaces/R1ckShi/FunClip)).\n\n## Highlights🎨\n\n- 🔥Try AI clipping using LLM in FunClip now.\n- FunClip integrates Alibaba's open-source industrial-grade model [Paraformer-Large](https://modelscope.cn/models/iic/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch/summary), which is one of the best-performing open-source Chinese ASR models available, with over 13 million downloads on Modelscope. It can also accurately predict timestamps in an integrated manner.\n- FunClip incorporates the hotword customization feature of [SeACo-Paraformer](https://modelscope.cn/models/iic/speech_seaco_paraformer_large_asr_nat-zh-cn-16k-common-vocab8404-pytorch/summary), allowing users to specify certain entity words, names, etc., as hotwords during the ASR process to enhance recognition results.\n- FunClip integrates the [CAM++](https://modelscope.cn/models/iic/speech_campplus_sv_zh-cn_16k-common/summary) speaker recognition model, enabling users to use the auto-recognized speaker ID as the target for trimming, to clip segments from a specific speaker.\n- The functionalities are realized through Gradio interaction, offering simple installation and ease of use. It can also be deployed on a server and accessed via a browser.\n- FunClip supports multi-segment free clipping and automatically returns full video SRT subtitles and target segment SRT subtitles, offering a simple and convenient user experience.\n\n\u003ca name=\"What's New\"\u003e\u003c/a\u003e\n## What's New🚀\n- 2024/06/12 FunClip supports recognize and clip English audio files now. Run `python funclip/launch.py -l en` to try.\n- 🔥2024/05/13 FunClip v2.0.0 now supports smart clipping with large language models, integrating models from the qwen series, GPT series, etc., providing default prompts. You can also explore and share tips for setting prompts, the usage is as follows:\n  1. After the recognition, select the name of the large model and configure your own apikey;\n  2. Click on the 'LLM Inference' button, and FunClip will automatically combine two prompts with the video's srt subtitles;\n  3. Click on the 'AI Clip' button, and based on the output results of the large language model from the previous step, FunClip will extract the timestamps for clipping;\n  4. You can try changing the prompt to leverage the capabilities of the large language models to get the results you want;\n- 2024/05/09 FunClip updated to v1.1.0, including the following updates and fixes:\n  - Support configuration of output file directory, saving ASR intermediate results and video clipping intermediate files;\n  - UI upgrade (see guide picture below), video and audio cropping function are on the same page now, button position adjustment;\n  - Fixed a bug introduced due to FunASR interface upgrade, which has caused some serious clipping errors;\n  - Support configuring different start and end time offsets for each paragraph;\n  - Code update, etc;\n- 2024/03/06 Fix bugs in using FunClip with command line.\n- 2024/02/28 [FunASR](https://github.com/alibaba-damo-academy/FunASR) is updated to 1.0 version, use FunASR1.0 and SeACo-Paraformer to conduct ASR with hotword customization.\n- 2023/10/17 Fix bugs in multiple periods chosen, used to return video with wrong length.\n- 2023/10/10 FunClipper now supports recognizing with speaker diarization ability, choose 'yes' button in 'Recognize Speakers' and you will get recognition results with speaker id for each sentence. And then you can clip out the periods of one or some speakers (e.g. 'spk0' or 'spk0#spk3') using FunClipper.\n\n\u003ca name=\"On Going\"\u003e\u003c/a\u003e\n## On Going🌵\n\n- [x] FunClip will support Whisper model for English users, coming soon (ASR using Whisper with timestamp requires massive GPU memory, we support timestamp prediction for vanilla Paraformer in FunASR to achieving this).\n- [x] FunClip will further explore the abilities of large langage model based AI clipping, welcome to discuss about prompt setting and clipping, etc.\n- [ ] Reverse periods choosing while clipping.\n- [ ] Removing silence periods.\n\n\u003ca name=\"Install\"\u003e\u003c/a\u003e\n## Install🔨\n\n### Python env install\n\nFunClip basic functions rely on a python environment only.\n```shell\n# clone funclip repo\ngit clone https://github.com/alibaba-damo-academy/FunClip.git\ncd FunClip\n# install Python requirments\npip install -r ./requirements.txt\n```\n\n### imagemagick install (Optional)\n\nIf you want to clip video file with embedded subtitles\n\n1. ffmpeg and imagemagick is required\n\n- On Ubuntu\n```shell\napt-get -y update \u0026\u0026 apt-get -y install ffmpeg imagemagick\nsed -i 's/none/read,write/g' /etc/ImageMagick-6/policy.xml\n```\n- On MacOS\n```shell\nbrew install imagemagick\nsed -i 's/none/read,write/g' /usr/local/Cellar/imagemagick/7.1.1-8_1/etc/ImageMagick-7/policy.xml \n```\n- On Windows\n\nDownload and install imagemagick https://imagemagick.org/script/download.php#windows\n\nFind your python install path and change the `IMAGEMAGICK_BINARY` to your imagemagick install path in file `site-packages\\moviepy\\config_defaults.py`\n\n2. Download font file to funclip/font\n\n```shell\nwget https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ClipVideo/STHeitiMedium.ttc -O font/STHeitiMedium.ttc\n```\n\u003ca name=\"Usage\"\u003e\u003c/a\u003e\n## Use FunClip\n\n### A. Use FunClip as local Gradio Service\nYou can establish your own FunClip service which is same as [Modelscope Space](https://modelscope.cn/studios/iic/funasr_app_clipvideo/summary) as follow:\n```shell\npython funclip/launch.py\n# '-l en' for English audio recognize\n# '-p xxx' for setting port number\n# '-s True' for establishing service for public accessing\n```\nthen visit ```localhost:7860``` you will get a Gradio service like below and you can use FunClip following the steps:\n\n- Step1: Upload your video file (or try the example videos below)\n- Step2: Copy the text segments you need to 'Text to Clip'\n- Step3: Adjust subtitle settings (if needed)\n- Step4: Click 'Clip' or 'Clip and Generate Subtitles'\n\n\u003cimg src=\"docs/images/guide.jpg\"/\u003e\n\nFollow the guide below to explore LLM based clipping:\n\n\u003cimg src=\"docs/images/LLM_guide.png\" width=360/\u003e\n\n### B. Experience FunClip in Modelscope\n\n[FunClip@Modelscope Space⭐](https://modelscope.cn/studios/iic/funasr_app_clipvideo/summary)\n\n[FunClip@HuggingFace Space🤗](https://huggingface.co/spaces/R1ckShi/FunClip)\n\n### C. Use FunClip in command line\n\nFunClip supports you to recognize and clip with commands:\n```shell\n# step1: Recognize\npython funclip/videoclipper.py --stage 1 \\\n                       --file examples/2022云栖大会_片段.mp4 \\\n                       --output_dir ./output\n# now you can find recognition results and entire SRT file in ./output/\n# step2: Clip\npython funclip/videoclipper.py --stage 2 \\\n                       --file examples/2022云栖大会_片段.mp4 \\\n                       --output_dir ./output \\\n                       --dest_text '我们把它跟乡村振兴去结合起来，利用我们的设计的能力' \\\n                       --start_ost 0 \\\n                       --end_ost 100 \\\n                       --output_file './output/res.mp4'\n```\n\n\u003ca name=\"Community\"\u003e\u003c/a\u003e\n## Community Communication🍟\n\nFunClip is firstly open-sourced bu FunASR team, any useful PR is welcomed.\n\nYou can also scan the following DingTalk group or WeChat group QR code to join the community group for communication.\n\n|                           DingTalk group                            |                     WeChat group                      |\n|:-------------------------------------------------------------------:|:-----------------------------------------------------:|\n| \u003cdiv align=\"left\"\u003e\u003cimg src=\"docs/images/dingding.png\" width=\"250\"/\u003e | \u003cimg src=\"docs/images/wechat.png\" width=\"215\"/\u003e\u003c/div\u003e |\n\n## Find Speech Models in FunASR\n\n[FunASR](https://github.com/alibaba-damo-academy/FunASR) hopes to build a bridge between academic research and industrial applications on speech recognition. By supporting the training \u0026 finetuning of the industrial-grade speech recognition model released on ModelScope, researchers and developers can conduct research and production of speech recognition models more conveniently, and promote the development of speech recognition ecology. ASR for Fun！\n\n📚FunASR Paper: \u003ca href=\"https://arxiv.org/abs/2305.11013\"\u003e\u003cimg src=\"https://img.shields.io/badge/Arxiv-2305.11013-orange\"\u003e\u003c/a\u003e \n\n📚SeACo-Paraformer Paper: \u003ca href=\"https://arxiv.org/abs/2308.03266\"\u003e\u003cimg src=\"https://img.shields.io/badge/Arxiv-2308.03266-orange\"\u003e\u003c/a\u003e\n\n🌟Support FunASR: \u003ca href='https://github.com/alibaba-damo-academy/FunASR/stargazers'\u003e\u003cimg src='https://img.shields.io/github/stars/alibaba-damo-academy/FunASR.svg?style=social'\u003e\u003c/a\u003e\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmodelscope%2Ffunclip","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmodelscope%2Ffunclip","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmodelscope%2Ffunclip/lists"}