Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/pimalaya/neverest
CLI to synchronize, backup and restore emails
https://github.com/pimalaya/neverest
backup cli email imap jmap maildir notmuch restore rust sync synchronizer terminal
Last synced: 22 days ago
JSON representation
CLI to synchronize, backup and restore emails
- Host: GitHub
- URL: https://github.com/pimalaya/neverest
- Owner: pimalaya
- License: mit
- Created: 2024-03-25T13:05:39.000Z (10 months ago)
- Default Branch: master
- Last Pushed: 2024-10-29T22:21:47.000Z (3 months ago)
- Last Synced: 2024-10-30T00:41:21.536Z (3 months ago)
- Topics: backup, cli, email, imap, jmap, maildir, notmuch, restore, rust, sync, synchronizer, terminal
- Language: Rust
- Homepage: https://pimalaya.org
- Size: 662 KB
- Stars: 183
- Watchers: 4
- Forks: 5
- Open Issues: 7
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- Funding: .github/FUNDING.yml
- License: LICENSE
Awesome Lists containing this project
README
*The project is under active development, do not use in production before the final `v1.0.0` (or at least do some manual backups).*
## Features
- Multi-accounting
- Interactive configuration via **wizard** (requires `wizard` feature)
- Synchronize pair of backends together (namely `left` and `right`)
- **Partial** synchronization based on **filters**
- **Restrictive** synchronization based on **permissions**
- **IMAP** backend (requires `imap` feature)
- **Maildir** backend (requires `maildir` feature)
- **Notmuch** backend (requires `notmuch` feature)
- **Backup** and **restore** emails using the Maildir backend
*Neverest CLI is written in [Rust](https://www.rust-lang.org/), and relies on [cargo features](https://doc.rust-lang.org/cargo/reference/features.html) to enable or disable functionalities. Default features can be found in the `features` section of the [`Cargo.toml`](https://github.com/pimalaya/neverest/blob/master/Cargo.toml#L18).*
## Installation
*The `v1.0.0` is currently being tested on the `master` branch, and is the prefered version to use. Previous versions (including GitHub beta releases and repositories published versions) are not recommended.*
### Pre-built binary
Neverest CLI `v1.0.0` can be installed with a pre-built binary. Find the latest [`pre-releases`](https://github.com/pimalaya/neverest/actions/workflows/pre-releases.yml) GitHub workflow and look for the *Artifacts* section. You should find a pre-built binary matching your OS.
### Cargo (git)
Neverest CLI `v1.0.0` can also be installed with [cargo](https://doc.rust-lang.org/cargo/):
```bash
$ cargo install --frozen --force --git https://github.com/pimalaya/neverest.git
```
### Other outdated methods
These installation methods should not be used until the `v1.0.0` is finally released, as they are all (temporarily) outdated:
Pre-built binary
Neverest CLI can be installed with a prebuilt binary:
```bash
# As root:
$ curl -sSL https://raw.githubusercontent.com/pimalaya/neverest/master/install.sh | sudo sh
# As a regular user:
$ curl -sSL https://raw.githubusercontent.com/pimalaya/neverest/master/install.sh | PREFIX=~/.local sh
```
These commands install the latest binary from the GitHub [releases](https://github.com/pimalaya/neverest/releases) section.
*Binaries are built with [default](https://github.com/pimalaya/neverest/blob/master/Cargo.toml#L18) cargo features. If you want to enable or disable a feature, please use another installation method.*
Cargo
Neverest CLI can be installed with [cargo](https://doc.rust-lang.org/cargo/):
```bash
$ cargo install neverest
# With only IMAP support:
$ cargo install neverest --no-default-features --features imap
```
You can also use the git repository for a more up-to-date (but less stable) version:
```bash
$ cargo install --git https://github.com/pimalaya/neverest.git neverest
```
Nix
Neverest CLI can be installed with [Nix](https://serokell.io/blog/what-is-nix):
```bash
$ nix-env -i neverest
```
You can also use the git repository for a more up-to-date (but less stable) version:
```bash
$ nix-env -if https://github.com/pimalaya/neverest/archive/master.tar.gz
# or, from within the source tree checkout
$ nix-env -if .
```
If you have the [Flakes](https://nixos.wiki/wiki/Flakes) feature enabled:
```bash
$ nix profile install neverest
# or, from within the source tree checkout
$ nix profile install
# you can also run Neverest directly without installing it:
$ nix run neverest
```
Sources
Neverest CLI can be installed from sources.
First you need to install the Rust development environment (see the [rust installation documentation](https://doc.rust-lang.org/cargo/getting-started/installation.html)):
```bash
$ curl https://sh.rustup.rs -sSf | sh
```
Then, you need to clone the repository and install dependencies:
```bash
$ git clone https://github.com/pimalaya/neverest.git
$ cd neverest
$ cargo check
```
Now, you can build Neverest:
```bash
$ cargo build --release
```
*Binaries are available under the `target/release` folder.*
## Configuration
Just run `neverest`, the wizard will help you to configure your default account.
You can also manually edit your own configuration, from scratch:
- Copy the content of the documented [`./config.sample.toml`](./config.sample.toml)
- Paste it in a new file `~/.config/neverest/config.toml`
- Edit, then comment or uncomment the options you want
Proton Mail (Bridge)
When using Proton Bridge, emails are synchronized locally and exposed via a local IMAP/SMTP server. This implies 2 things:
- Id order may be reversed or shuffled, but envelopes will still be sorted by date.
- SSL/TLS needs to be deactivated manually.
- The password to use is the one generated by Proton Bridge, not the one from your Proton Mail account.
```toml
[accounts.proton]
left.backend.type = "maildir"
left.backend.root-dir = "~/.Mail/proton"
right.backend.type = "imap"
right.backend.host = "127.0.0.1"
right.backend.port = 1143
right.backend.encryption = false
right.backend.login = "[email protected]"
right.backend.auth.type = "password"
right.backend.auth.raw = "*****"
```
Keeping your password inside the configuration file is good for testing purpose, but it is not safe. You have 2 better alternatives:
- Save your password in any password manager that can be queried via the CLI:
```toml
right.backend.auth.cmd = "pass show proton"
```
- Use the global keyring of your system (requires the `keyring` cargo feature):
```toml
right.backend.auth.keyring = "proton-example"
```
Running `neverest configure proton` will ask for your IMAP password, just paste the one generated previously.
Gmail
Google passwords cannot be used directly. There is two ways to authenticate yourself:
### Using [App Passwords](https://support.google.com/mail/answer/185833)
This option is the simplest and the fastest. First, be sure that:
- IMAP is enabled
- Two-step authentication is enabled
- Less secure app access is enabled
First create a [dedicated password](https://myaccount.google.com/apppasswords) for Neverest.
```toml
[accounts.gmail]
# this is important in order not to sync twice your account
folder.filters.exclude = ["[Gmail]/All Mail"]
left.backend.type = "maildir"
left.backend.root-dir = "~/.Mail/gmail"
right.backend.type = "imap"
right.backend.host = "imap.gmail.com"
right.backend.port = 993
right.backend.login = "[email protected]"
right.backend.auth.type = "password"
right.backend.auth.raw = "*****"
right.folder.aliases.inbox = "INBOX"
right.folder.aliases.sent = "[Gmail]/Sent Mail"
right.folder.aliases.drafts = "[Gmail]/Drafts"
right.folder.aliases.trash = "[Gmail]/Trash"
```
Keeping your password inside the configuration file is good for testing purpose, but it is not safe. You have 2 better alternatives:
- Save your password in any password manager that can be queried via the CLI:
```toml
right.backend.auth.cmd = "pass show gmail"
```
- Use the global keyring of your system (requires the `keyring` cargo feature):
```toml
right.backend.auth.keyring = "gmail-example"
```
Running `neverest configure gmail` will ask for your IMAP password, just paste the one generated previously.
### Using OAuth 2.0
This option is the most secure but the hardest to configure. It requires the `oauth2` and `keyring` cargo features.
First, you need to get your OAuth 2.0 credentials by following [this guide](https://developers.google.com/identity/protocols/oauth2#1.-obtain-oauth-2.0-credentials-from-the-dynamic_data.setvar.console_name-.). Once you get your client id and your client secret, you can configure your Neverest account this way:
```toml
[accounts.gmail]
# this is important in order not to sync twice your account
folder.filters.exclude = ["[Gmail]/All Mail"]
left.backend.type = "maildir"
left.backend.root-dir = "~/.Mail/gmail"
right.backend.type = "imap"
right.backend.host = "imap.gmail.com"
right.backend.port = 993
right.backend.login = "[email protected]"
right.backend.auth.type = "oauth2"
right.backend.auth.client-id = "*****"
right.backend.auth.auth-url = "https://accounts.google.com/o/oauth2/v2/auth"
right.backend.auth.token-url = "https://www.googleapis.com/oauth2/v3/token"
right.backend.auth.pkce = true
right.backend.auth.scope = "https://mail.google.com/"
right.folder.aliases.inbox = "INBOX"
right.folder.aliases.sent = "[Gmail]/Sent Mail"
right.folder.aliases.drafts = "[Gmail]/Drafts"
right.folder.aliases.trash = "[Gmail]/Trash"
```
Running `neverest configure gmail` will complete your OAuth 2.0 setup and ask for your client secret.
Outlook
```toml
[accounts.outlook]
left.backend.type = "maildir"
left.backend.root-dir = "~/.Mail/outlook"
right.backend.type = "imap"
right.backend.host = "outlook.office365.com"
right.backend.port = 993
right.backend.login = "[email protected]"
right.backend.auth.type = "password"
right.backend.auth.raw = "*****"
```
Keeping your password inside the configuration file is good for testing purpose, but it is not safe. You have 2 better alternatives:
- Save your password in any password manager that can be queried via the CLI:
```toml
right.backend.auth.cmd = "pass show outlook"
```
- Use the global keyring of your system (requires the `keyring` cargo feature):
```toml
right.backend.auth.keyring = "outlook-example"
```
Running `neverest configure outlook` will ask for your IMAP password, just paste the one generated previously.
### Using OAuth 2.0
This option is the most secure but the hardest to configure. First, you need to get your OAuth 2.0 credentials by following [this guide](https://learn.microsoft.com/en-us/exchange/client-developer/legacy-protocols/how-to-authenticate-an-imap-pop-smtp-application-by-using-oauth). Once you get your client id and your client secret, you can configure your Neverest account this way:
```toml
[accounts.outlook]
left.backend.type = "maildir"
left.backend.root-dir = "~/.Mail/outlook"
right.backend.type = "imap"
right.backend.host = "outlook.office365.com"
right.backend.port = 993
right.backend.login = "[email protected]"
right.backend.auth.type = "oauth2"
right.backend.auth.client-id = "*****"
right.backend.auth.auth-url = "https://login.microsoftonline.com/common/oauth2/v2.0/authorize"
right.backend.auth.token-url = "https://login.microsoftonline.com/common/oauth2/v2.0/token"
right.backend.auth.pkce = true
right.backend.auth.scope = "https://outlook.office.com/IMAP.AccessAsUser.All"
```
Running `neverest configure outlook` will complete your OAuth 2.0 setup and ask for your client secret.
iCloud Mail
From the [iCloud Mail](https://support.apple.com/en-us/HT202304) support page:
- IMAP port = `993`.
- IMAP login = name of your iCloud Mail email address (for example, `johnappleseed`, not `[email protected]`)
```toml
[accounts.icloud]
left.backend.type = "maildir"
left.backend.root-dir = "~/.Mail/icloud"
right.backend.type = "imap"
right.backend.host = "imap.mail.me.com"
right.backend.port = 993
right.backend.login = "johnappleseed"
right.backend.auth.type = "password"
right.backend.auth.raw = "*****"
```
Keeping your password inside the configuration file is good for testing purpose, but it is not safe. You have 2 better alternatives:
- Save your password in any password manager that can be queried via the CLI:
```toml
right.backend.auth.cmd = "pass show icloud"
```
- Use the global keyring of your system (requires the `keyring` cargo feature):
```toml
right.backend.auth.keyring = "icloud-example"
```
Running `neverest configure icloud` will ask for your IMAP password, just paste the one generated previously.
## FAQ
How to debug Neverest CLI?
The simplest way is to use `--debug` and `--trace` arguments.
The advanced way is based on environment variables:
- `RUST_LOG=`: determines the log level filter, can be one of `off`, `error`, `warn`, `info`, `debug` and `trace`.
- `RUST_SPANTRACE=1`: enables the spantrace (a span represent periods of time in which a program was executing in a particular context).
- `RUST_BACKTRACE=1`: enables the error backtrace.
- `RUST_BACKTRACE=full`: enables the full error backtrace, which include source lines where the error originated from.
Logs are written to the `stderr`, which means that you can redirect them easily to a file:
```
RUST_LOG=debug neverest 2>/tmp/neverest.log
```
How the wizard discovers IMAP configs?
All the lookup mechanisms use the email address domain as base for the lookup. It is heavily inspired from the Thunderbird [Autoconfiguration](https://udn.realityripple.com/docs/Mozilla/Thunderbird/Autoconfiguration) protocol. For example, for the email address `[email protected]`, the lookup is performed as (in this order):
1. check for `autoconfig.example.com`
2. look up of `example.com` in the ISPDB (the Thunderbird central database)
3. look up `MX example.com` in DNS, and for `mx1.mail.hoster.com`, look up `hoster.com` in the ISPDB
4. look up `SRV example.com` in DNS
5. try to guess (`imap.example.com`…)
## Sponsoring
[](https://nlnet.nl/)
Special thanks to the [NLnet foundation](https://nlnet.nl/) and the [European Commission](https://www.ngi.eu/) that helped the project to receive financial support from various programs:
- [NGI Assure](https://nlnet.nl/project/Himalaya/) in 2022
- [NGI Zero Entrust](https://nlnet.nl/project/Pimalaya/) in 2023
- [NGI Zero Core](https://nlnet.nl/project/Pimalaya-PIM/) in 2024 *(still ongoing)*
If you appreciate the project, feel free to donate using one of the following providers:
[](https://github.com/sponsors/soywod)
[](https://ko-fi.com/soywod)
[](https://www.buymeacoffee.com/soywod)
[](https://liberapay.com/soywod)
[](https://thanks.dev/soywod)
[](https://www.paypal.com/paypalme/soywod)
