{"id":15631978,"url":"https://github.com/jaymzh/recover-cyrus","last_synced_at":"2025-10-29T12:01:46.851Z","repository":{"id":177966450,"uuid":"657412606","full_name":"jaymzh/recover-cyrus","owner":"jaymzh","description":null,"archived":false,"fork":false,"pushed_at":"2023-08-07T16:21:38.000Z","size":10,"stargazers_count":1,"open_issues_count":1,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-24T01:43:55.696Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jaymzh.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-06-23T02:26:01.000Z","updated_at":"2023-07-04T13:32:40.000Z","dependencies_parsed_at":"2024-10-23T01:59:52.323Z","dependency_job_id":null,"html_url":"https://github.com/jaymzh/recover-cyrus","commit_stats":null,"previous_names":["jaymzh/recover-cyrus"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jaymzh%2Frecover-cyrus","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jaymzh%2Frecover-cyrus/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jaymzh%2Frecover-cyrus/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jaymzh%2Frecover-cyrus/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jaymzh","download_url":"https://codeload.github.com/jaymzh/recover-cyrus/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246230541,"owners_count":20744349,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-03T10:42:16.390Z","updated_at":"2025-10-29T12:01:46.840Z","avatar_url":"https://github.com/jaymzh.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Cyrus mailbox loss recovery\n\nThe upgrade of Cyrus to 3.6.1 in Debian [can cause complete mailbox loss](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1037346).\n\nThe script here (and this doc) will help you recover your mail.\n\n# Overview\n\nThere's a migration of the mailbox data from a previous per-user hierarchy, held in `/var/spool/cyrus/mail/\u003chashletter\u003e/user/\u003cuser\u003e` to a new user-agnostic per-folder hierarchy in `/var/spool/cyrus/mail/uuid/\u003chash_digit_1\u003e/\u003chash_digit_2\u003e/\u003cuuid\u003e`.\n\nFolder paths/names are mapped to these new UUID directories in a database, and you find out where a specific folder is with `/usr/lib/cyrus/bin/mbpath user.\u003cuser\u003e.\u003cfolder\u003e` for example: `/usr/lib/cyrus/bin/mbpath user.phil.work` for user `phil`'s \"work\" IMAP folder.\n\nDuring the upgrade to 3.6.1 in Debian, many users found all of their email appeared gone. For many users the mail had not been migrated to the new locations. For some especially unlucky users the old mail had also been deleted.\n\n# Recovery\n\n# Ensuring the data still exists (in the old format)\n\nThe first thing to do is check that you still have mail in the old format. Look in `/var/spool/cyrus/mail`. Pick a user, for example `phil`, and then look in the \u003cfirst-letter-of-that-username\u003e -\u003e user -\u003e \u003cusername\u003e, for example `p/user/phil`. If this directory is not empty, you (likely) still have all of your data. If not, the first thing you'll need to do is restore `/var/spool/cyrus/mail` from your most recent backups (you **do** have backups, right?). For me, I had to restore from the previous night's backups. There had been a temporary space issue on the device, and so there were a few blocks in the tar file corrupted, so I restored from 2 nights prior, then restored the previous nights on top of that to get as close as possible to a complete restoration.\n\n# Migrating/Recovering the data\n\nThe script in this repo will attempt to help you recover your data. Initially written by Kai Lindenberg \u003ckai@ldbg.de\u003e, I've extended and modified it.\n\nThe script is incredibly conservative - it will generate two scripts for you, and then have you inspect them and run them yourself. This is to ensure that your data isn't trampled on.\n\nPlace the script somewhere and make it executable (`chmod +x recover_cyrus_user.sh`).\n\nFor each user on your system, you'll run `recover_cyrus_user.sh \u003cfull_user_hash\u003e`. So for example, `recover_cyrus_user.sh p/user/phil`.\n\nThe script will first walk all folders it can find for that user, and then generate a script you will pass to `cyradm` to generate those folders (which will create the UUID directories). It will then spawn a shell so you may inspect the script and pass it to `cyradm`. It'll look like this:\n\n```shell\n# recover_cyrus_user.sh p/user/phil\nstarting shell to examine the situation\ncreatembx.cyradm created to feed cyradm\nplease review the file and then:\n   cat creatembx.cyradm | cyradm --user cyrus localhost\nonce complete, continue with \"exit\"'\n#\n```\n\nYou cna then look at `creatembox.cyradm` with your favorite editor, or `less` or whatever you choose. When you're happy run it as the script instructs with `cat creatembx.cyradm | cyradm --user cyrus localhost`, and enter the `cyrus` password. If you've setup cyrus to have a different user be the admin, adjust accordingly.\n\nThen type `exit` and the script will create a shell script which *hardlinks* files from the new path to the old path.\n\nWhy hardlinks? Well, for one, we're gauranteed to be on the same filesystem here, so they'll definitely work. Second, not copying avoids space issues. Third, by hardlinking (instead of symlinking), if we ever want to remove the original paths, we can.\n\nThe script will then tell you to inspect the script and run it, as well as tell you how to recover the inbox (it does not do this for you for safety reasons). It will look like this:\n\n```shell\n...\n# exit\nlinkmbx.bash created, please review before executing and then run it\nmanual work:\n1. might be too many argument, review output\n2. main inbox not linked, create inbox_recovered and link the contents\n```\n\nInspect `linkmbx.bash` and then run it (as root):\n\n```shell\n# bash ./linkmbx.bash\n```\n\nIf there's any errors, inspect, and fix.\n\nFinally you can recover the user's inbox an as `inbox_recovered` (so as not to mess with any email received since the migration) by doing the following. First create a new, empty `inbox_recovered` in the new strufture using `cyradm`:\n\n```shell\n# cyradm --user cyrus localhost\n\u003e cm user.\u003cwhatever\u003e.inbox_recovered\n\u003e ^D\n```\n\nFor example, that might be `cm user.phil.inbox_recovered`.\n\nThen cd into the path to this new directory and link the old inbox to it:\n\n```shell\ncd $(/usr/lib/cyrus/bin/mbpath user.\u003cuser\u003e.inbox_recovered)\nOLD_INBOX='/var/spool/cyrus/mail/\u003chashletter\u003e/user/\u003cuser\u003e'\nls -f ${OLD_INBOX?}/ | xargs -I{} ln -f \"${OLD_INBOX?}/{}\" .\n```\n\nFor example that would look like:\n\n```shell\ncd $(/usr/lib/cyrus/bin/mbpath user.phil.inbox_recovered)\nOLD_INBOX='/var/spool/cyrus/mail/p/user/phil'\nls -f \"${OLD_INBOX?}/\" | xargs -I{} ln -f \"${OLD_INBOX?}/{}\" .\n```\n\n# Final words\n\nThe script could do a lot more on its own with extra error checking. Both Kai and I only had a few users to recover, and so this was enough of the process automated to suffice. I've added a variety of safety checks to the script, but there's still many things that can go wrong. If you have many users, this may not be enough and it may be worth having the script actually run the commands instead of generating scripts and recovering the inboxes as well. PRs welcome.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjaymzh%2Frecover-cyrus","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjaymzh%2Frecover-cyrus","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjaymzh%2Frecover-cyrus/lists"}