{"id":13581743,"url":"https://github.com/hekmon/rcgdip","last_synced_at":"2025-05-01T21:31:19.776Z","repository":{"id":64305715,"uuid":"452484871","full_name":"hekmon/rcgdip","owner":"hekmon","description":"RClone GDrive Inotify for Plex","archived":false,"fork":false,"pushed_at":"2022-03-28T18:52:44.000Z","size":974,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-11-05T21:45:07.519Z","etag":null,"topics":["google-drive","google-drive-api","inotify","plex","plex-api","plex-media-server","plex-server","rclone","rclone-crypt","rclone-mount","synchro","synchronization"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hekmon.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-01-27T00:17:44.000Z","updated_at":"2022-04-13T19:05:18.000Z","dependencies_parsed_at":"2023-01-15T10:30:44.468Z","dependency_job_id":null,"html_url":"https://github.com/hekmon/rcgdip","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hekmon%2Frcgdip","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hekmon%2Frcgdip/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hekmon%2Frcgdip/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hekmon%2Frcgdip/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hekmon","download_url":"https://codeload.github.com/hekmon/rcgdip/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224278438,"owners_count":17285080,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["google-drive","google-drive-api","inotify","plex","plex-api","plex-media-server","plex-server","rclone","rclone-crypt","rclone-mount","synchro","synchronization"],"created_at":"2024-08-01T15:02:13.133Z","updated_at":"2024-11-12T13:11:41.235Z","avatar_url":"https://github.com/hekmon.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"# rcgdip\n\nRClone GDrive Inotify for Plex is a rclone companion for your plex server using a rclone mount on Google Drive. It will monitor changes on GDrive and launch targeted scans on your matching Plex libraries to automatically refresh your plex libraries as soon as your rclone drive mount can see the modifications.\n\nIt supports (directly from your rclone config file):\n\n* [x] GDrive backend (scope `drive` only for now, `drive.file` support planned, see [GDrive scope](#gdrive-scope) for more details)\n* [x] Custom root folder ID\n* [x] Team Drives\n* [x] Crypt backend on top of your GDrive backend for path decryption\n* [x] OAuth2 token\n* [ ] Service Account (planned)\n\n- [rcgdip](#rcgdip)\n  - [Installation](#installation)\n    - [Get the software](#get-the-software)\n      - [From official releases](#from-official-releases)\n      - [From source](#from-source)\n    - [Configure your system](#configure-your-system)\n      - [Execution environment](#execution-environment)\n      - [systemd service](#systemd-service)\n        - [Mono instance](#mono-instance)\n        - [Multi instances](#multi-instances)\n    - [Configure rcgdip](#configure-rcgdip)\n      - [Mono instance](#mono-instance-1)\n      - [Multi instances](#multi-instances-1)\n    - [Start rcgdip](#start-rcgdip)\n      - [Mono instance](#mono-instance-2)\n      - [Multi instances](#multi-instances-2)\n  - [Things to consider](#things-to-consider)\n    - [rclone mount config values](#rclone-mount-config-values)\n    - [deletion events](#deletion-events)\n    - [rclone version](#rclone-version)\n    - [scan list optimizations](#scan-list-optimizations)\n      - [same path optimization](#same-path-optimization)\n      - [same ancester optimization](#same-ancester-optimization)\n    - [GDrive scope](#gdrive-scope)\n      - [change the original rclone config](#change-the-original-rclone-config)\n      - [keep the original rclone config](#keep-the-original-rclone-config)\n    - [db backup](#db-backup)\n      - [Mono instance](#mono-instance-3)\n      - [Multi instances](#multi-instances-3)\n  - [Sponsoring](#sponsoring)\n\n## Installation\n\n### Get the software\n\n#### From official releases\n\nDownload a binary from the [releases page](https://github.com/hekmon/rcgdip/releases).\n\n#### From source\n\n```bash\nenv GOOS=linux GOARCH=amd64 go build -ldflags=\"-s -w -X 'main.appVersion=$(git describe)'\" -trimpath -o rcgdip github.com/hekmon/rcgdip/cmd\n```\n\n### Configure your system\n\nWhile technically rcgdip can run anywhere, it is recommended to run it alongside your plex server and your rclone mount.\n\n#### Execution environment\n\n```bash\nsudo useradd --home-dir '/var/lib/rcgdip' --create-home --system --shell '/usr/sbin/nologin' 'rcgdip'\nsudo chown 'rcgdip:' '/var/lib/rcgdip'\nsudo chmod 750 '/var/lib/rcgdip'\nsudo wget 'https://github.com/hekmon/rcgdip/releases/download/v0.4.0/rcgdip_linux_amd64' -O '/usr/local/bin/rcgdip'\nsudo chmod +x '/usr/local/bin/rcgdip'\n# adapt to the group of your rclone config file, here the rclone config file is owned (and readable) by the rclone group\nsudo usermod -a -G 'rclone' 'rcgdip'\n```\n\n#### systemd service\n\nIf your rclone mount is started with systemd too (for example using [rclonemount](https://github.com/hekmon/rclonemount)), add it to `After=` and `BindsTo=` as well. Eg:\n\n```ini\n[Unit]\nAfter=rclonemount@\u003cyourinstance\u003e.service\nBindsTo=rclonemount@\u003cyourinstance\u003e.service\n```\n\nAdd it add while `systemd edit ...` on the next parts.\n\n##### Mono instance\n\n```bash\nsudo wget 'https://raw.githubusercontent.com/hekmon/rcgdip/v0.4.0/systemd/rcgdip.service' -O '/etc/systemd/system/rcgdip.service'\nsudo systemctl edit rcgdip.service # add your rclonemount unit as stated previously\nsudo systemctl daemon-reload # not needed if you did systemctl edit\n```\n\n##### Multi instances\n\nOptional. If you intend to run multiples instances. Each instance must have a custom edit.\n\n```bash\nsudo wget 'https://raw.githubusercontent.com/hekmon/rcgdip/v0.4.0/systemd/rcgdip%40.service' -O '/etc/systemd/system/rcgdip@.service'\nsudo systemctl edit rcgdip@instanceNameA.service # add the related rclonemount unit as stated previously\nsudo systemctl edit rcgdip@instanceNameB.service # add the (other) related rclonemount unit as stated previously\n# ... repeat for each instance (1 rclone mount == 1 rcgdip instance)\nsudo systemctl daemon-reload # not needed if you did systemctl edit\n```\n\n### Configure rcgdip\n\nCheck the [plex documentation](https://support.plex.tv/articles/204059436-finding-an-authentication-token-x-plex-token/) to recover your Plex token.\n\n#### Mono instance\n\nAdapt the values to yours.\n\n```bash\nconfFile=\"/etc/default/rcgdip\"\ncat \u003c\u003cEOF | sudo tee \"$confFile\"\n# Mandatory\nRCGDIP_RCLONE_CONFIG_PATH=\"/etc/rclone/instance.conf\"\nRCGDIP_RCLONE_BACKEND_DRIVE_NAME=\"DriveBackendName\"\nRCGDIP_RCLONE_MOUNT_PATH=\"/some/path/used/as/destination/for/rclone/mount\"\nRCGDIP_PLEX_URL=\"http://127.0.0.1:32400\"\nRCGDIP_PLEX_TOKEN=\"yourshere\"\n\n# Optional\nRCGDIP_RCLONE_BACKEND_CRYPT_NAME=\"\"\nRCGDIP_RCLONE_BACKEND_DRIVE_POLLINTERVAL=\"\"\nRCGDIP_RCLONE_BACKEND_DRIVE_DIRCACHETIME=\"\"\nRCGDIP_LOGLEVEL=\"DEBUG\"\nEOF\nsudo chown root:rcgdip \"$confFile\"\nsudo chmod 640 \"$confFile\"\n```\n\n#### Multi instances\n\nSame as mono instance but for the `instanceName` instance, change `confFile=\"/etc/default/rcgdip\"` to `confFile=\"/etc/default/rcgdip_instanceName\"`.\n\n### Start rcgdip\n\n#### Mono instance\n\n```bash\nsudo systemctl enable --now rcgdip.service\nsudo journalctl -f -u rcgdip.service\n```\n\n#### Multi instances\n\nFor an instance named `instanceName`.\n\n```bash\nsudo systemctl enable --now rcgdip@instanceName.service\nsudo journalctl -f -u rcgdip@instanceName.service\n```\n\n## Things to consider\n\n### rclone mount config values\n\n3 rclone mount values are importants to have rcgdip work as intended:\n\n* `--attr-timeout` it is the time the FUSE mount is allowed to cache the informations before asking them again to rclone, keep the default unless you know what you are doing but it should always be lower than...\n* `--dir-cache-time` which is the time rclone itself keeps the metadata of each folder without reasking the backend to answers the FUSE requests when `--attr-timeout` is elapsed. Default is fine but I like to increase it to `15m`.\n* `--poll-interval` the frequency used by rclone to ask the backend for changes, it allows for targetted updates of the dir cache even if it is still within the `--dir-cache-time` window. It should be lower than `--dir-cache-time`. Default value is fine here too but I like to increase it to `5m`.\n\nrcgdip bases its scan delay prediction to detect new and changes files on the `--poll-interval` value. So if you customize the rclone `--poll-interval` for your rclone mount remember to set the exact same value in `RCGDIP_RCLONE_BACKEND_DRIVE_POLLINTERVAL` as well as rcgdip will wait this interval between the change event timestamp and the plex scan in order to be sure the local rclone mount had the time to discover the new file(s). Note that this also applies to `--dir-cache-time` and `RCGDIP_RCLONE_BACKEND_DRIVE_DIRCACHETIME`, see next section for more details.\n\nIf you leave `RCGDIP_RCLONE_BACKEND_DRIVE_POLLINTERVAL` and `RCGDIP_RCLONE_BACKEND_DRIVE_DIRCACHETIME` empty, rcgdip will use the same defaults as rclone.\n\n### deletion events\n\nIt seems that while `--poll-interval` works very well in rclone mounting  gdrive for new files and file changes, it does not work for deleted files (it is actually tricky to support as you have to build and maintain your own index locally, which rcgdip does). It means that a new file will be seen by your rclone mount fairly quickly (respecting the `--poll-interval`) but deleted files will only disappears locally when rclone dir cache is expired (the `--dir-cache-time` flag).\n\nThis is why in rcgdip you can specify `RCGDIP_RCLONE_BACKEND_DRIVE_DIRCACHETIME` in addition to `RCGDIP_RCLONE_BACKEND_DRIVE_POLLINTERVAL`: deletion events will wait the `--dir-cache-time` duration before starting a scan while new or changed files will only wait the `--poll-interval` allowing fast detection when this is possible while still correctly handling deletion events.\n\nIf not specified, both `RCGDIP_RCLONE_BACKEND_DRIVE_POLLINTERVAL` and `RCGDIP_RCLONE_BACKEND_DRIVE_DIRCACHETIME` take exactly the same default as rclone, be sure to use the same rclone version as the version of rcgdip you are using has been built against ! (see next section).\n\n### rclone version\n\nrcgdip is built with original rclone parts directly (as libs or constant values source) so to avoid any unexpected behaviors you should always ensure your rclone mount is using the same version of rclone as rcgdip was built against. RClone version used by rcgdip is always mentioned on each [release](https://github.com/hekmon/rcgdip/releases) notes.\n\n### scan list optimizations\n\nTo avoid too many scan requests to be fired, severals optimizations are made inside a batch between the raw changes from drive and the scan requests to plex.\n\n#### same path optimization\n\nIf several files have changed within the same directoy, only one scan job will be created for this directory and the wait time will be determined by the most recent event to ensure all modifications are seen by the time of the scan. But it also means that if one of this changes is a deletion event, the wait time can be longer (see [deletion events](#deletion-events) for more details) even for new files.\n\n#### same ancester optimization\n\nIf 2 paths are scheduled for scan but one of them is actually a parent of the other, only the parent will be kept as it will also scan the child. But what about wait time ? If the parent was to be scanned at T+2 but the child was to be scanned at T+3, this optimization will remove the scan job for the child but adapt the scan time of the parent to T+3 in order for all changes to be detected within the scan.\n\n### GDrive scope\n\nTo work, rcgdip starts by indexing every file in the targeted drive in order to correctly process the changes event from the API (the deletion events can not be handled without an index). But the current method for doing this initial index will fail if the drive scope is `drive.file`. While this is the best scope for a rclone mount it actually prevent rcgdip from working as expected. I am currently thinking of ways to actually perform a working indexing with this restricted scope but in the meantime, to have rcgdip working properly, the scope needs to be `drive` and the oauth token must have been issued with that scope.\n\nIf you are actually using the `drive.file` scope and wants to have rcgdip working there is 2 ways of doing it.\n\n#### change the original rclone config\n\nSimply edit the rclone config and edit the scope. Once edited, issue a [rclone reconnect command](https://rclone.org/commands/rclone_config_reconnect/) to refresh the oauth token with the new scope.\n\n#### keep the original rclone config\n\nIf you want to keep the original file in place, copy it elsewhere, edit the copy, issue the [rclone reconnect command](https://rclone.org/commands/rclone_config_reconnect/) to refresh the oauth token with the new scope using the new rclone config file and use this new file as target for rcgdip.\n\n### db backup\n\nDo not directly backup db files while rcgdip is running ! By default a backup is performed at each start. If you need to make a backup of the db while rcgdip is running, send the `USR1` signal to the process: it will perform a backup in the backup directory which you can safely access after backup is done.\n\nIf you are using the systemd integration a simple reload of the unit will send the signal for you (see sub sections).\n\n#### Mono instance\n\ndb directory is `rcgdip_storage` in the current working directory (`/var/lib/rcgdip` if you followed the installation steps) and the backup directory is `rcgdip_storage_backup`. To start a backup while rcgdip is running just launch `systemctl reload rcgdip.service` and check the logs.\n\n#### Multi instances\n\nFor an instance named `instanceName`, the db directory is `rcgdip_storage_instanceName` in the current working directory (`/var/lib/rcgdip` if you followed the installation steps) and the backup directory is `rcgdip_storage_instanceName_backup`. To start a backup while rcgdip is running just launch `systemctl reload rcgdip@instanceName.service` and check the logs.\n\n## Sponsoring\n\nIf you like rcgdip, please consider sponsoring [rclone](https://github.com/rclone/rclone) directly.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhekmon%2Frcgdip","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhekmon%2Frcgdip","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhekmon%2Frcgdip/lists"}