{"id":35893321,"url":"https://github.com/Mailuminati/Guardian","last_synced_at":"2026-01-15T19:00:47.573Z","repository":{"id":330838221,"uuid":"1123883503","full_name":"Mailuminati/Guardian","owner":"Mailuminati","description":null,"archived":false,"fork":false,"pushed_at":"2025-12-30T21:31:47.000Z","size":3284,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-12-30T22:29:26.197Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Mailuminati.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-12-27T20:49:57.000Z","updated_at":"2025-12-30T21:31:51.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/Mailuminati/Guardian","commit_stats":null,"previous_names":["mailuminati/guardian"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/Mailuminati/Guardian","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mailuminati%2FGuardian","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mailuminati%2FGuardian/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mailuminati%2FGuardian/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mailuminati%2FGuardian/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Mailuminati","download_url":"https://codeload.github.com/Mailuminati/Guardian/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mailuminati%2FGuardian/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28465785,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-15T18:56:24.589Z","status":"ssl_error","status_checked_at":"2026-01-15T18:56:23.992Z","response_time":62,"last_error":"SSL_read: (null) (tls_retry_write_records failure)","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-09T11:00:41.645Z","updated_at":"2026-01-15T19:00:47.562Z","avatar_url":"https://github.com/Mailuminati.png","language":null,"funding_links":[],"categories":["Sending"],"sub_categories":["SPAM Filtering"],"readme":"\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"./assets/images/Guardian_M.png\" alt=\"Logo Guardian\" width=\"400\"\u003e\n\u003c/div\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://img.shields.io/badge/Go-1.25-blue\" alt=\"Go\"\u003e\n  \u003cimg src=\"https://img.shields.io/badge/License-GPLv3-blue\" alt=\"License\"\u003e\n  \u003ca href=\"https://github.com/Mailuminati/Guardian/actions/workflows/go-tests.yml\"\u003e\u003cimg src=\"https://github.com/Mailuminati/Guardian/actions/workflows/go-tests.yml/badge.svg\" alt=\"Go Tests\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n\n# Mailuminati Guardian\n\n**Guardian** is a **high-performance, scalable spam/phishing detection and enforcement service** designed to run next to your MTA and filtering engine.\n\nIt analyzes incoming emails **ultra-fast** (structure fingerprinting + proximity detection), applies **immediate local learning** from operator/user reports, and only reaches out to the **Mailuminati Oracle** when needed for shared, collaborative intelligence.\n\nGuardian is built for anyone operating email infrastructure, from large providers to small and community-run servers, who wants fast decisions with minimal overhead.\n\n---\n\n## Table of Contents\n\n- [Role in the Mailuminati Ecosystem](#role-in-the-mailuminati-ecosystem)\n- [Local Intelligence vs Shared Intelligence](#local-intelligence-vs-shared-intelligence)\n  - [Local Intelligence](#local-intelligence)\n  - [Shared Intelligence via the Oracle](#shared-intelligence-via-the-oracle)\n- [How It Works](#how-it-works)\n  - [1. Local Analysis](#1-local-analysis)\n  - [2. Local Proximity Detection](#2-local-proximity-detection)\n  - [3. Oracle Confirmation (When Needed)](#3-oracle-confirmation-when-needed)\n  - [4. Learning and Feedback](#4-learning-and-feedback)\n- [Design Goals](#design-goals)\n- [Prerequisites](#prerequisites)\n  - [Mandatory](#mandatory)\n  - [Optional but Recommended](#optional-but-recommended)\n- [Installation Options](#installation-options)\n- [Installation](#installation)\n  - [Method 1: Install from GitHub Archive (Recommended)](#method-1-install-from-github-archive-recommended)\n  - [Method 2: Install using Git, developer friendly](#method-2-install-using-git-developer-friendly)\n- [Deployment Model](#deployment-model)\n- [API Endpoints](#api-endpoints)\n- [Relationship to Other Components](#relationship-to-other-components)\n- [License](#license)\n\n---\n\n## Role in the Mailuminati Ecosystem\n\nGuardian is responsible for:\n\n- Local spam/phishing analysis of incoming emails\n- Structural fingerprinting using TLSH\n- Fast proximity detection via locality sensitive hashing (LSH)\n- Immediate application of local learning\n- Remote confirmation through the Mailuminati Oracle\n- Enforcing final decisions (allow, spam, proximity match)\n\nIt acts as the **first line of defense**, minimizing latency and resource usage,\nwhile remaining connected to a broader community driven detection network.\n\n---\n\n## Local Intelligence vs Shared Intelligence\n\nGuardian is built around a clear separation of concerns.\n\n### Local Intelligence\n\nLocal analysis and learning allow Guardian to:\n\n- Apply new detections immediately after a report\n- Adapt instantly to operator specific threats and campaigns\n- Remain effective even when disconnected from the Oracle\n- Keep latency close to zero for the majority of messages\n\nThis ensures that confirmed spam or phishing reports have an **instant impact**\non subsequent mail flows for the same operator.\n\n### Shared Intelligence via the Oracle\n\nThe Mailuminati Oracle provides the indispensable collaborative dimension:\n\n- Cross operator correlation of campaigns\n- Shared clusters built from independent reports\n- Protection against large scale or fast moving threats\n- Early detection of campaigns unseen locally\n\nBy querying the Oracle only when meaningful proximity is detected,\nGuardian benefits from collective intelligence without sacrificing performance\nor privacy.\n\n---\n\n## How It Works\n\n### 1. Local Analysis\n\nFor each incoming email, Guardian:\n\n- Normalizes textual and HTML content\n- Extracts meaningful attachments\n- Computes one or more TLSH structural fingerprints\n\nThis process is fast, deterministic, and does not rely on external calls.\n\n### 2. Local Proximity Detection\n\nEach fingerprint is split into overlapping bands using LSH techniques.\n\nGuardian checks:\n- Its local learning database\n- A locally cached subset of Oracle band data\n\nIf sufficient proximity is detected, Guardian may:\n- Classify the message locally\n- Flag it as a partial or suspicious match\n- Escalate to the Oracle for confirmation\n\n### 3. Oracle Confirmation (When Needed)\n\nOnly when proximity thresholds are met, Guardian contacts the Oracle to:\n\n- Compute exact distances against known threat clusters\n- Compare fingerprints against cluster medoids built from confirmed reports\n- Receive a final verdict\n\nThis design ensures that **only a small fraction of messages** require remote\nconfirmation.\n\n\u003cpre\u003e\nIncoming Email\n      |\n      v\n+---------------------+\n|  Mailuminati        |\n|  Guardian (Local)   |\n+---------------------+\n   |           |\n   |           +--------------------+\n   |                                |\n   v                                v\nLocal Analysis                  Local Learning\n(TLSH + LSH)                (Immediate Effect)\n   |\n   |  No proximity\n   |-----------------------------\u003e  ALLOW / LOCAL DECISION\n   |\n   |  Proximity detected\n   v\n+---------------------+\n|   Mailuminati       |\n|   Oracle (Remote)   |\n+---------------------+\n        |\n        v\nShared Intelligence\n(Clusters, Medoids,\nCommunity Reports)\n        |\n        v\n   Verdict Returned\n        |\n        v\nLocal Enforcement\n(Spam / Allow / Flag)\n\u003c/pre\u003e\n\n\n### 4. Learning and Feedback\n\nGuardian supports learning through reports such as:\n\n- User complaints\n- Operator validation\n- Abuse desk signals\n\nConfirmed reports immediately reinforce local detection.\nThey can also be shared with the Oracle, contributing to the global\nMailuminati intelligence and benefiting other Guardian users.\n\n---\n\n## Design Goals\n\n- Very low latency\n- Immediate impact of local learning\n- Minimal CPU and memory usage\n- Privacy preserving by design\n- No raw email content sharing\n- Resilience to Oracle unavailability\n- Suitable for high volume and low volume operators alike\n\n---\n\n## Prerequisites\n\n### Mandatory\n\n- Linux server  \n- POSIX compatible shell (`/bin/sh` or `/bin/bash`)  \n- `curl`  \n- `tar`  \n- `sudo` (unless installing as root)  \n\n### Optional but Recommended\n\n- `systemd` for service management  \n- `redis` for local cache and learning  \n- An anti spam engine capable of calling HTTP APIs  \n  Examples: Rspamd, SpamAssassin, custom filters  \n- An IMAP server supporting Sieve  \n  Examples: Dovecot, Cyrus, or equivalent  \n\nGuardian does **not** require:\n\n- Git (unless using the developer installation method)  \n- IMAP credentials  \n- Access to raw mailbox content  \n- Heavy runtime dependencies  \n\n### Installation Options\n\nYou can customize the installation by passing arguments to the installer.\n\nTo see all available options:\n\n```sh\n./install.sh --help\n```\n\nCommon options:\n\n- **Redis Configuration**:\n  If your Redis instance is not on localhost (or `mi-redis` for Docker), specify it:\n  ```sh\n  ./install.sh --redis-host 192.168.1.50 --redis-port 6380\n  ```\n\n- **Filter Integration**:\n  Skip all filter integration prompts:\n  ```sh\n  ./install.sh --no-filter-integration\n  ```\n  Disable a specific integration even if installed:\n  ```sh\n  ./install.sh --no-rspamd\n  ./install.sh --no-spamassassin\n  ```\n\n---\n\n## Installation\n\nTwo installation methods are officially supported.\n\n### Method 1: Install from GitHub Archive (Recommended)\n\nThis method does not require Git and is suitable for production environments.\n\n```sh\ncurl -fsSL https://github.com/Mailuminati/Guardian/archive/refs/heads/main.tar.gz \\\n| tar xz\ncd Guardian-main\n./install.sh\n```\n\n### Method 2: Install using Git, developer friendly\n\nThis method is recommended if you plan to contribute or track changes easily.\n\n```sh\ngit clone https://github.com/Mailuminati/Guardian.git\ncd Guardian\n./install.sh\n```\n\n## Deployment Model\n\nGuardian typically runs as:\n\n- A local HTTP service\n- A bridge between the MTA and the Mailuminati ecosystem\n- A containerized service alongside Redis\n\nIt exposes endpoints such as:\n- `/analyze`\n- `/report`\n- `/status`\n\n---\n\n## API Endpoints\n\nBase URL: `http://\u003cguardian-host\u003e:12421`\n\n\u003e **Warning (Security)**\n\u003e\n\u003e Guardian listens on port **12421** and the API provides **no authentication**.\n\u003e It is therefore strongly recommended to **not expose** `:12421` to the Internet and to **block external access** with a firewall (allow only `localhost` or your internal network) to prevent fraudulent use.\n\n### GET /status\n\nHealth/info endpoint used by the installer post-start check.\n\n```bash\ncurl -sS http://localhost:12421/status | jq\n```\n\nExample response:\n\n```json\n{\n  \"node_id\": \"6c0a5e16-2b32-4f86-9b3d-2b2e3df5c7d8\",\n  \"current_seq\": 0,\n  \"version\": \"0.3.2\"\n}\n```\n\n### POST /analyze\n\nAnalyzes an email provided as raw RFC822/MIME bytes (the full message). Maximum request size is 15 MB.\n\nNotes:\n- If the email has no `Message-ID` header, Guardian will still analyze it, but `/report` will not be able to find its scan data later.\n- The response includes the computed TLSH signatures under `hashes`.\n\n```bash\ncurl -sS -X POST \\\n  -H 'Content-Type: message/rfc822' \\\n  --data-binary @message.eml \\\n  http://localhost:12421/analyze | jq\n```\n\nExample response:\n\n```json\n{\n  \"action\": \"allow\",\n  \"proximity_match\": false,\n  \"hashes\": [\n    \"T1A9B0E0F2D3C4B5A6...\"\n  ]\n}\n```\n\nPossible fields:\n- `action`: `allow` | `spam`\n- `label` (optional): e.g. `local_spam`\n- `proximity_match`: boolean\n- `distance` (optional): integer (TLSH distance when applicable)\n- `hashes` (optional): array of TLSH signatures computed for body/attachments\n\n### POST /report\n\nReports a previously scanned email by `Message-ID` (as seen in the original email headers). Guardian will:\n- Apply **local learning** immediately when `report_type` is `spam`\n- Forward the report to the Oracle\n\nRequest body:\n\n```json\n{\n  \"message-id\": \"\u003cyour-message-id@example\u003e\",\n  \"report_type\": \"spam\"\n}\n```\n\n```bash\ncurl -sS -X POST \\\n  -H 'Content-Type: application/json' \\\n  -d '{\"message-id\":\"\u003cyour-message-id@example\u003e\",\"report_type\":\"spam\"}' \\\n  http://localhost:12421/report\n```\n\nNotes:\n- If Guardian has no stored scan data for this `Message-ID`, it returns `404 No scan data found`.\n- The response body/status code are proxied from the Oracle when reachable.\n\n### Configuration (env vars)\n\nGuardian’s API behavior depends on these environment variables:\n\n- `REDIS_HOST` (default: `localhost`)\n- `REDIS_PORT` (default: `6379`)\n\n- **`SPAM_WEIGHT`**: Defines the weight (or score increment) applied to a hash when it is reported as spam. Default value: `1`.\n- **`HAM_WEIGHT`**: Defines the weight (or score decrement) applied to a hash when it is reported as ham (false positive). Default value: `2`.\n\nThese variables allow operators to fine-tune the impact of spam and ham reports on the local learning database. Adjust these values based on your specific requirements and the desired sensitivity of the system.\n\n---\n\n## Relationship to Other Components\n\n- **Guardian** performs local detection, learning, and enforcement\n- **Oracle** provides shared intelligence and collaborative confirmation\n\nGuardian can operate independently.\nIts effectiveness increases when connected to the Oracle,\nwhere local signals become part of a collective defense.\n\n---\n\n## License\n\nThis client is open-source software licensed under the GNU GPLv3.\n\nCopyright © 2025 Simon Bressier.\n\nPlease note: This license applies strictly to the client-side code contained in this repository.\n\nSee the [LICENSE](LICENSE) file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FMailuminati%2FGuardian","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FMailuminati%2FGuardian","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FMailuminati%2FGuardian/lists"}