Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/kdeldycke/mail-deduplicate
📧 CLI to deduplicate mails from mail boxes.
https://github.com/kdeldycke/mail-deduplicate
babyl cleanup cli dedupe deduplication email mail mailbox maildir mbox mh mmdf python
Last synced: 9 days ago
JSON representation
📧 CLI to deduplicate mails from mail boxes.
- Host: GitHub
- URL: https://github.com/kdeldycke/mail-deduplicate
- Owner: kdeldycke
- License: gpl-2.0
- Created: 2013-03-25T21:32:56.000Z (over 11 years ago)
- Default Branch: main
- Last Pushed: 2024-05-01T17:54:50.000Z (6 months ago)
- Last Synced: 2024-05-02T01:08:51.295Z (6 months ago)
- Topics: babyl, cleanup, cli, dedupe, deduplication, email, mail, mailbox, maildir, mbox, mh, mmdf, python
- Language: Python
- Homepage: https://kdeldycke.github.io/mail-deduplicate
- Size: 6.36 MB
- Stars: 159
- Watchers: 10
- Forks: 38
- Open Issues: 18
-
Metadata Files:
- Readme: readme.md
- Changelog: changelog.md
- Funding: .github/funding.yml
- License: license
- Code of conduct: .github/code-of-conduct.md
- Citation: citation.cff
Awesome Lists containing this project
- awesome-starred - kdeldycke/mail-deduplicate - 📧 CLI to deduplicate mails from mail boxes. (python)
README
[](https://pypi.python.org/pypi/mail-deduplicate)
[](https://pypi.python.org/pypi/mail-deduplicate)
[](https://github.com/kdeldycke/mail-deduplicate/actions/workflows/tests.yaml?query=branch%3Amain)
[](https://github.com/kdeldycke/mail-deduplicate/actions/workflows/docs.yaml?query=branch%3Amain)
[](https://app.codecov.io/gh/kdeldycke/mail-deduplicate)
[](https://doi.org/10.5281/zenodo.7364256)
**What is Mail Deduplicate?**
Provides the `mdedup` CLI, an utility to deduplicate mails from a set of boxes.
## Features
- Duplicate detection based on cherry-picked and normalized mail
headers.
- Fetch mails from multiple sources.
- Reads and writes to `mbox`, `maildir`, `babyl`, `mh` and `mmdf`
formats.
- Deduplication strategies based on size, content, timestamp, file path
or random choice.
- Copy, move or delete the resulting set of duplicates.
- Dry-run mode.
- Protection against false-positives with safety checks on size and content differences.
- Supports macOS, Linux and Windows.
- [Standalone executables](#executables) for Linux, macOS and Windows.
- Shell auto-completion for Bash, Zsh and Fish.
> ⚠️ **Warning**: Performances
>
> `mdedup` implementation is quite naive at the moment and everything resides in memory.
>
> If this is good enough for a volume of a couple of gigabytes, the more emails `mdedup` try to parse, the closer you'll reach the memory limits of your machine. In which case [`mdedup` will exit abrubtly](https://github.com/kdeldycke/mail-deduplicate/issues/362#issuecomment-1266743045), zapped by the [OOM killer](https://en.wikipedia.org/wiki/Out_of_memory) of your OS. Of course your mileage may vary depending on your hardware.
>
> You can influence implementation of this feature with pull requests, or [purchase of business support 🤝 and sponsorship 🫶](https://github.com/sponsors/kdeldycke).
## Example
## Installation
### From sources
Easiest way is to install `mdedup` from sources with [`pipx`](https://pipx.pypa.io):
```shell-session
$ pipx install mail-deduplicate
```
Other
[alternatives installation methods](https://kdeldycke.github.io/mail-deduplicate/install.html)
are available in the documentation.
### Executables
Standalone executables of `mdedup`'s latest version are available for several platforms and architectures:
| Platform | `x86_64` |`arm64` |
| ----------- | ---------------------------------------------------------------------------------------------------------------------------------- |-------------------------------------------------------------------------------------------------------------------------------- |
| **Linux** | [Download `mdedup-linux-x64.bin`](https://github.com/kdeldycke/mail-deduplicate/releases/latest/download/mdedup-linux-x64.bin) | |
| **macOS** | [Download `mdedup-macos-x64.bin`](https://github.com/kdeldycke/mail-deduplicate/releases/latest/download/mdedup-macos-x64.bin) | [Download `mdedup-macos-arm64.bin`](https://github.com/kdeldycke/mail-deduplicate/releases/latest/download/mdedup-macos-arm64.bin) |
| **Windows** | [Download `mdedup-windows-x64.exe`](https://github.com/kdeldycke/mail-deduplicate/releases/latest/download/mdedup-windows-x64.exe) | |