{"id":13551695,"url":"https://github.com/l3uddz/plex_dupefinder","last_synced_at":"2025-04-06T01:11:25.468Z","repository":{"id":31070860,"uuid":"126598830","full_name":"l3uddz/plex_dupefinder","owner":"l3uddz","description":"Find and delete duplicate files in Plex","archived":false,"fork":false,"pushed_at":"2024-02-21T16:01:18.000Z","size":4417,"stargazers_count":317,"open_issues_count":46,"forks_count":54,"subscribers_count":12,"default_branch":"master","last_synced_at":"2025-03-30T00:09:30.034Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/l3uddz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":".github/SUPPORT.md","governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-03-24T12:51:24.000Z","updated_at":"2025-03-25T07:01:45.000Z","dependencies_parsed_at":"2024-01-13T16:26:15.961Z","dependency_job_id":"0c47208f-95ac-4918-8e2f-36afa507ecc7","html_url":"https://github.com/l3uddz/plex_dupefinder","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/l3uddz%2Fplex_dupefinder","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/l3uddz%2Fplex_dupefinder/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/l3uddz%2Fplex_dupefinder/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/l3uddz%2Fplex_dupefinder/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/l3uddz","download_url":"https://codeload.github.com/l3uddz/plex_dupefinder/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247419862,"owners_count":20936012,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T12:01:52.342Z","updated_at":"2025-04-06T01:11:25.445Z","avatar_url":"https://github.com/l3uddz.png","language":"Python","funding_links":["https://www.paypal.me/l3uddz"],"categories":["Install from Source","Python"],"sub_categories":["Media Server"],"readme":"\u003cimg src=\"assets/logo.svg\" width=\"600\" alt=\"Plex DupeFinder\"\u003e\n\n[![made-with-python](https://img.shields.io/badge/Made%20with-Python-blue.svg?style=flat-square)](https://www.python.org/)\n[![License: GPL v3](https://img.shields.io/badge/License-GPL%203-blue.svg?style=flat-square)](https://github.com/l3uddz/plex_dupefinder/blob/master/LICENSE.md)\n[![last commit (develop)](https://img.shields.io/github/last-commit/l3uddz/plex_dupefinder/develop.svg?colorB=177DC1\u0026label=Last%20Commit\u0026style=flat-square)](https://github.com/l3uddz/plex_dupefinder/commits/develop)\n[![Discord](https://img.shields.io/discord/381077432285003776.svg?colorB=177DC1\u0026label=Discord\u0026style=flat-square)](https://discord.io/cloudbox)\n[![Contributing](https://img.shields.io/badge/Contributing-gray.svg?style=flat-square)](CONTRIBUTING.md)\n[![Donate](https://img.shields.io/badge/Donate-gray.svg?style=flat-square)](#donate)\n\n---\n\n\u003c!-- TOC depthFrom:1 depthTo:2 withLinks:1 updateOnSave:1 orderedList:0 --\u003e\n\n- [Introduction](#introduction)\n- [Demo](#demo)\n- [Requirements](#requirements)\n- [Installation](#installation)\n- [Configuration](#configuration)\n  - [Sample](#sample)\n  - [Foreword](#foreword)\n  - [Details](#details)\n- [Plex](#plex)\n- [Usage](#usage)\n- [Donate](#donate)\n\n\u003c!-- /TOC --\u003e\n\n---\n\n# Introduction\n\nPlex DupeFinder is a python script that finds duplicate versions of media (TV episodes and movies) in your Plex Library and tells Plex to remove the lowest rated files/versions (based on user-specified scoring) to leave behind a single file/version.\n\nDuplicates can be either in bulk (automatic) or on-by-one (interactively).\n\n\n# Demo\n\nClick to enlarge.\n\n[![asciicast](assets/demo.gif)](https://asciinema.org/a/180157?cols=180\u0026rows=50)\n\n\n# Requirements\n\n1. Python 3.6+.\n\n1. Required Python modules (see below).\n\n\n# Installation\n\n_Note: Steps below are for Debian-based distros (other operating systems will require tweaking to the steps)._\n\n1. Install Python 3 and PIP\n\n   ```\n   sudo apt install python3 python3-pip\n   ```\n\n1. Clone the Plex DupeFinder repo.\n\n   ```\n   sudo git clone https://github.com/l3uddz/plex_dupefinder /opt/plex_dupefinder\n   ```\n\n1. Find your user \u0026 group.\n\n   ```\n   id\n   ```\n\n1. Fix permissions of the Plex DupeFinder folder (replace `user`/`group` with yours).\n\n   ```\n   sudo chown -R user:group /opt/plex_dupefinder\n   ```\n\n1. Go into the Plex DupeFinder folder.\n\n   ```\n   cd /opt/plex_dupefinder\n   ```\n\n1. Install the required python modules.\n\n   ```\n   sudo python3 -m pip install -r requirements.txt\n   ```\n\n1. Create a shortcut for Plex DupeFinder.\n\n   ```\n   sudo ln -s /opt/plex_dupefinder/plex_dupefinder.py /usr/local/bin/plex_dupefinder\n   ```\n\n1. Generate a `config.json` file.\n\n   ```\n   plex_dupefinder\n   ```\n\n1. Fill in Plex URL and credentials at the prompt to generated a Plex Access Token (optional).\n\n   ```\n   Dumping default config to: /opt/plex_dupefinder/config.json\n   Plex Server URL: http://localhost:32400\n   Plex Username: your_plex_username\n   Plex Password: your_plex_password\n   Auto Delete duplicates? [y/n]: n\n   Please edit the default configuration before running again!\n   ```\n\n1. Configure the `config.json` file.\n\n   ```\n   nano config.json\n   ```\n\n\n# Configuration\n\n## Sample\n\n```json\n{\n  \"AUDIO_CODEC_SCORES\": {\n    \"Unknown\": 0,\n    \"aac\": 1000,\n    \"ac3\": 1000,\n    \"dca\": 2000,\n    \"dca-ma\": 4000,\n    \"eac3\": 1250,\n    \"flac\": 2500,\n    \"mp2\": 500,\n    \"mp3\": 1000,\n    \"pcm\": 2500,\n    \"truehd\": 4500,\n    \"wmapro\": 200\n  },\n  \"AUTO_DELETE\": false,\n  \"FIND_DUPLICATE_FILEPATHS_ONLY\": false,\n  \"FILENAME_SCORES\": {\n    \"*.avi\": -1000,\n    \"*.ts\": -1000,\n    \"*.vob\": -5000,\n    \"*1080p*BluRay*\": 15000,\n    \"*720p*BluRay*\": 10000,\n    \"*HDTV*\": -1000,\n    \"*PROPER*\": 1500,\n    \"*REPACK*\": 1500,\n    \"*Remux*\": 20000,\n    \"*WEB*CasStudio*\": 5000,\n    \"*WEB*KINGS*\": 5000,\n    \"*WEB*NTB*\": 5000,\n    \"*WEB*QOQ*\": 5000,\n    \"*WEB*SiGMA*\": 5000,\n    \"*WEB*TBS*\": -1000,\n    \"*WEB*TROLLHD*\": 2500,\n    \"*WEB*VISUM*\": 5000,\n    \"*dvd*\": -1000\n  },\n  \"PLEX_LIBRARIES\": [\n    \"Movies\",\n    \"TV\"\n  ],\n  \"PLEX_SERVER\": \"https://plex.your-server.com\",\n  \"PLEX_TOKEN\": \"\",\n  \"SCORE_FILESIZE\": true,\n  \"SKIP_LIST\": [],\n  \"VIDEO_CODEC_SCORES\": {\n    \"Unknown\": 0,\n    \"h264\": 10000,\n    \"h265\": 5000,\n    \"hevc\": 5000,\n    \"mpeg1video\": 250,\n    \"mpeg2video\": 250,\n    \"mpeg4\": 500,\n    \"msmpeg4\": 100,\n    \"msmpeg4v2\": 100,\n    \"msmpeg4v3\": 100,\n    \"vc1\": 3000,\n    \"vp9\": 1000,\n    \"wmv2\": 250,\n    \"wmv3\": 250\n  },\n  \"VIDEO_RESOLUTION_SCORES\": {\n    \"1080\": 10000,\n    \"480\": 3000,\n    \"4k\": 20000,\n    \"720\": 5000,\n    \"Unknown\": 0,\n    \"sd\": 1000\n  }\n}\n```\n## Foreword\n\nThe scoring is based on: non-configurable and configurable parameters.\n\n  - Non-configurable parameters are: _bitrate_, _duration_, _height_, _width_, and _audio channel_.\n\n  - Configurable parameters are: _audio codec scores_, _video codec scores_, _video resolution scores_, _filename scores_, and _file sizes_ (can only be toggled on or off).\n\n  - Note: bitrate, duration, height, width, audio channel, audio and video codecs, video resolutions (e.g. SD, 480p, 720p, 1080p, 4K, etc), and file sizes are all taken from the metadata Plex retrieves during media analysis.\n\n## Details\n\n### Audio Codec Scores\n\n- You can set `AUDIO_CODEC_SCORES` to your preference.\n\n- The default settings should be sufficient for most.\n\n\n### Auto Delete\n\n- Under `AUTO_DELETE`, set your desired option.\n\n  - `\"AUTO_DELETE\": true,`  - Plex DupeFinder will run in automatic mode.\n\n  - `\"AUTO_DELETE\": false,` -  Plex DupeFinder will run in interactive mode. (Default)\n\n    - Options:\n\n      - Skip (i.e. keep both): `0`\n\n      - Choose the best one (and delete the rest): `b`\n\n      - Select the item to keep (and delete the rest): `#` (i.e. `1`, `2`, `3`, etc).\n\n### Find Duplicate File Paths Only\n\n- Finds duplicates that only share the same file path.\n\n  ```json\n  \"FIND_DUPLICATE_FILEPATHS_ONLY\": false,\n  ```\n\n- This option has a very limited use case, i.e. in instances where Plex may have glitched and created multiple duplicates of the same media item.\n\n- If using this setting, we recommend using UnionFS-Fuse that can generate whiteout files (`*_HIDDEN~`) to prevent the deletion of the actual file on the system. The `_HIDDEN~` files can then be removed afterwards or even during the dupe cleanup (e.g. `watch -n 5 rm -rf /mnt/local/.unionfs-fuse/*`).\n\n- The default settings should be sufficient for most.\n\n### Filename Scores\n\n- You can set `FILENAME_SCORES` to your preference.\n\n- The default settings should be sufficient for most.\n\n### Plex Libraries\n\n1. Go to Plex and get all the names of your Plex Libraries you want to find duplicates in.\n\n   - Example Library:\n\n     ![](https://i.imgur.com/JFRTD1m.png)\n\n1. Under `PLEX_LIBRARIES`, list your Plex Libraries exactly as they are named in your Plex.\n\n   - Format:\n\n     ```json\n     \"PLEX_LIBRARIES\": [\n       \"LIBRARY_NAME_1\",\n       \"LIBRARY_NAME_2\"\n     ],\n     ```\n     or\n\n     ```json\n     \"PLEX_LIBRARIES\": [\"LIBRARY_NAME_1\", \"LIBRARY_NAME_2\"],\n     ```\n     \n   - Example:\n\n     ```json\n     \"PLEX_LIBRARIES\": [\n       \"Movies\",\n       \"TV\"\n     ],\n     ```\n\n### Plex Server URL\n\n- Your Plex server's URL.\n\n- This can be any format (e.g. \u003chttp://localhost:32400\u003e, \u003chttps://plex.domain.ltd\u003e).\n\n### Plex Token\n\n1. Obtain a Plex Access Token:\n\n   - Fill in the Plex URL and Plex login credentials, at the prompt, on first run. This only occurs when there is no `config.json` present.\n\n     or\n\n   - Visit https://support.plex.tv/hc/en-us/articles/204059436-Finding-an-authentication-token-X-Plex-Token\n\n1. Add the Plex Access Token to `\"PLEX_TOKEN\"` so that it now appears as `\"PLEX_TOKEN\": \"abcd1234\",`.\n\n   - Note: Make sure it is within the quotes (`\"`) and there is a comma (`,`) after it.\n\n### Filesize Scores\n\n- `\"SCORE_FILESIZE\": true` will add more points to the overall score based on the actual file size.\n\n- The default settings should be sufficient for most.\n\n- Note: In some situations (e.g. a bad encode resulting in a large size), this may be something you want to turn it off (i.e. `false`).\n\n### Skip List\n\n- In Auto Delete mode, any file paths matching the patterns (i.e folders), listed in `SKIP_LIST`, will be ignored.\n\n- Example:\n\n  ```json\n  \"SKIP_LIST\": [\"/Movies4K/\"]\n  ```\n\n- The default settings should be sufficient for most.\n\n### Video Codec Scores\n\n- You can set `VIDEO_CODEC_SCORES` to your preference.\n\n- The default settings should be sufficient for most.\n\n### Video Resolution Scoring\n\n- You can set `VIDEO_RESOLUTION_SCORES` to your preference.\n\n- The default settings should be sufficient for most.\n\n\n# Plex\n\nYou will need to make sure that **Allow media deletion** is enabled in Plex.\n\n1. In Plex, click the **Settings** icon -\u003e **Server** -\u003e **Library**.\n\n1. Set the following:\n\n   - **Allow media deletion**: `enabled`\n\n1. Click **SAVE CHANGES**.\n\n\n![](http://i.imgur.com/D82n8vh.png)\n\n\n# Usage\n\nSimply run the script/command:\n\n```\nplex_dupefinder\n```\n\n***\n\n# Donate\n\nIf you find this project helpful, feel free to make a small donation to the developer:\n\n  - [Monzo](https://monzo.me/today): Credit Cards, Apple Pay, Google Pay\n\n  - [Beerpay](https://beerpay.io/l3uddz/plex_dupefinder): Credit Cards\n\n  - [Paypal: l3uddz@gmail.com](https://www.paypal.me/l3uddz)\n\n  - BTC: 3CiHME1HZQsNNcDL6BArG7PbZLa8zUUgjL\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fl3uddz%2Fplex_dupefinder","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fl3uddz%2Fplex_dupefinder","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fl3uddz%2Fplex_dupefinder/lists"}