{"id":27976525,"url":"https://github.com/misp/mail_to_misp","last_synced_at":"2025-05-08T01:27:39.342Z","repository":{"id":20356052,"uuid":"89592667","full_name":"MISP/mail_to_misp","owner":"MISP","description":"Connect your mail client/infrastructure to MISP in order to create events based on the information contained within mails.","archived":false,"fork":false,"pushed_at":"2023-11-11T06:18:12.000Z","size":892,"stargazers_count":67,"open_issues_count":14,"forks_count":25,"subscribers_count":12,"default_branch":"main","last_synced_at":"2024-03-26T04:54:07.885Z","etag":null,"topics":["misp","misp-api","threat-hunting","threat-intelligence","threatintel"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MISP.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2017-04-27T12:00:25.000Z","updated_at":"2024-03-12T20:35:31.000Z","dependencies_parsed_at":"2023-11-11T07:24:11.703Z","dependency_job_id":"5c96b233-843e-4489-a5f1-adfe169f6595","html_url":"https://github.com/MISP/mail_to_misp","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MISP%2Fmail_to_misp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MISP%2Fmail_to_misp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MISP%2Fmail_to_misp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MISP%2Fmail_to_misp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MISP","download_url":"https://codeload.github.com/MISP/mail_to_misp/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252980699,"owners_count":21835290,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["misp","misp-api","threat-hunting","threat-intelligence","threatintel"],"created_at":"2025-05-08T01:27:38.705Z","updated_at":"2025-05-08T01:27:39.311Z","avatar_url":"https://github.com/MISP.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Build Status](https://app.travis-ci.com/SteveClement/mail_to_misp.svg?branch=main)](https://app.travis-ci.com/SteveClement/mail_to_misp)\n[![codecov](https://codecov.io/gh/MISP/mail_to_misp/branch/main/graph/badge.svg)](https://codecov.io/gh/MISP/mail_to_misp)\n\n# mail_to_misp\n\nConnect your mail infrastructure to [MISP](https://github.com/MISP/MISP) in order to create events based on the information contained within mails.\n\n## Features\n\n- Extraction of URLs and IP addresses (and port numbers) from free text emails\n- Extraction of hostnames from URLs\n- Extraction of hashes (MD5, SHA1, SHA256)\n- DNS expansion\n- Custom filter list for lines containing specific words\n- Subject filters\n- Respecting TLP classification mentioned in free text (including optional spelling robustness)\n- Refanging of URLs ('hxxp://...')\n- Add tags automatically based on key words (configurable)\n- Add tags automatically depending on the presence of other tags (configurable)\n- Add tags automatically depending on presence of hashes (e.g. for automatic expansion)\n- Ignore 'whitelisted' domains (configurable)\n- Specify a stop word term to no further process input\n- Configurable list of attributes not to enable the IDS flag\n- Automatically create 'external analysis' links based on filter list (e.g. VirusTotal, malwr.com)\n- Automatically create 'internal reference' links based on filter list\n- Detection of forwarded messages\n- Process attachments as malware samples or specify that they are processed as benign files (`m2m_attachment_keyword`)\n\nYou can send mails with attachments to mail_to_misp and tell it, to treat the attachment as a benign document (in contrast to the default behaviour: treating it as a malware sample). You need to set a keyword in the configuration:\n`m2m_attachment_keyword = 'attachment:benign'`\n- Logging to syslog\n- Remove \"[tags]\", \"Re:\" and \"Fwd:\" from subjects\n- Optionally attach entire mail to event\n- Contains now a fake-smtpd spamtrap which delivers IoCs/mails to MISP\n- Automatically filter out attributes that are on a server side warning list (`enforcewarninglist=True`)\n- Support for value sighting (`sighting=True`, `sighting_source=\"YOUR_MAIL_TO_MISP_IDENTIFIER\"`)\n- Auto-publish when `key:yourkey` is specified in mail (configurable, `m2m_key`, `m2m_auto_distribution`)\nThe `m2m_key configuration` is used to specify a secret only you and your users know. If you know the key, you can send a mail to your mail_to_misp instance, and when this key is present in the body of the message, it will automatically publish the event. So let's assume your config says: `m2m_key = 'ABCDEFGHIJKLMN0PQRSTUVWXYZ'`\nIf you send a mail to mail_to_misp containing: `key:ABCDEFGHIJKLMN0PQRSTUVWXYZ` the event is automatically published.\nIf you don't want to use this feature, just don't put it in the message body.\nThe distribution is defined in the configuration as well: `m2m_auto_distribution = '3' # 3 = All communities`\n\nFor OSINT collection purposes (like collecting URLs to OSINT reports), you can tell `mail_to_misp` to only extract URLs (`--urlsonly`) and append them to a predefined MISP event (`--event N`). The subject of such a mail goes into the comment field of the value.\n\nExample:\n```\nosinturlcollection: \"|/path/to/mail_to_misp.py --urlsonly --event 12345 -\"\n```\n\n\n# Pass parameters in the email body\n\n```\nm2m:\u003cparameter\u003e:\u003cValue\u003e\n\n# Examples\nm2m:attachment:benign  # Email attachment considered benign (attachment in MISP, malware-sample by default)\nm2m:attach_original_mail:1  # Attach the full original email to the MISP Event (may contain private information)\n\nm2m:m2mkey:YOUSETYOURKEYHERE  # Key required for some actions\n# The following key are ignored if M2M:m2mkey is invalid\nm2m:distribution:\u003c0-3,5\u003e # Note: impossible to pass a sharing group yet.\nm2m:threat_level:\u003c0-2\u003e\nm2m:analysis:\u003c0-3\u003e\nm2m:publish:1  # Autopublish\n```\n\n## Implementation\n\nThe implemented workflow is mainly for mail servers like Postfix. Client side implementations exist but are no longer supported:\n\n1. Postfix and others\n\n`Email -\u003e mail_to_misp`\n\n2. Office 365\n\n`Email -\u003e Outlook -\u003e O365MISPClient -\u003e mail_to_misp`\n\n3. Apple Mail [unmaintained]\n\n`Email -\u003e Apple Mail -\u003e Mail rule -\u003e AppleScript -\u003e mail_to_misp -\u003e PyMISP -\u003e MISP`\n\n4. Mozilla Thunderbird [unmaintained]\n\n`Email -\u003e Thunderbird -\u003e Mail rule -\u003e filterscript -\u003e thunderbird_wrapper -\u003e mail_to_misp -\u003e PyMISP -\u003e MISP`\n\n\n## Installation\n\n### Postfix (or other MTA) - preferred method\n\n1. Setup a new email address in the aliases file (e.g. /etc/aliases) and configure the correct path:\n\n`misp_handler: \"|/path/to/mail_to_misp.py -\"`\n\n2. Rebuild the DB:\n\n`$ sudo newaliases`\n\n3. Configure mail_to_misp_config.py\n\nYou should now be able to send your IoC-containing mails to misp_handler@YOURDOMAIN.\n\n#### Bonus: Fake-SMTPD spamtrap\n\nIf you want to process all incoming junk mails automatically and collect the contained information in a separate throw-away MISP instance, you could use the fake_smtp.py script. It listens on port 25, accepts all mails and pushes them through mail_to_misp to a MISP instance.\nIt can also be configured to listen on an SSL port. (465)\n\n1. Configure mail_to_misp_config.py\n\n2. cp fake_smtp_config.py-example fake_smtp_config.py\n\n3. Make port 25 accessible to normal users\n\n```\n$ sudo apt install authbind\n$ sudo touch /etc/authbind/byport/25\n$ sudo chown misp:misp /etc/authbind/byport/25\n$ sudo chmod 770 /etc/authbind/byport/25\n```\n\n4. Run fake_smtp.py\n\n`$ python3 fake_smtp.py`\n\n### Office 365\n- Full [documentation](MUA/Microsoft/Office365/README.md) for getting started in MUA/Microsoft/Office365\n- Built-in O365MISPClient for Mail2MISP\n- Uses new Mail2MISP methods: `.load_o365_email` and `.process_o365_email_body`\n\nRun mail_to_misp_o365.py to get the last 1 day of messages\n\n`$ python3 mail_to_misp_0365.py -nd 1`\n\n### Apple Mail [unmaintained]\n\n1. Mail rule script\n- git clone this repository\n- open the AppleScript file MUA/Apple/Mail/MISP Mail Rule Action.txt in Apple's 'Script Editor'\n- adjust the path to the python installation and location of the mail_to_misp.py script\n- save it in ~/Library/Application Scripts/com.apple.mail/\n2. Create a mail rule based on your needs, executing the AppleScript defined before\n3. Configure mail_to_misp_config.py\n\n### Thunderbird [unmaintained]\n\n1. Git clone https://github.com/rommelfs/filterscript and install plugin (instructions within the project description)\n2. Mail rule script\n- git clone this repository\n- open the bash script MUA/Mozilla/Thunderbird/thunderbird_wrapper.sh and adujst the paths\n- adjust the path to the python installation and location of the mail_to_misp.py script\n3. Create a mail rule based on your needs, executing the thunderbird_wrapper.sh script\n4. Configure mail_to_misp_config.py\n\nYou should be able to create MISP events now.\n\n### Outlook [unmaintained]\n\nOutlook is not implemented due to lack of test environment. However, it should be feasible to do it this way:\n\n```\nimport win32com.client\nimport pythoncom\n\nclass Handler_Class(object):\n    def OnNewMailEx(self, receivedItemsIDs):\n        for ID in receivedItemsIDs.split(\",\"):\n            # Microsoft.Office.Interop.Outlook _MailItem properties:\n            # https://msdn.microsoft.com/en-us/library/microsoft.office.interop.outlook._mailitem_properties.aspx\n            mailItem = outlook.Session.GetItemFromID(ID)\n            print \"Subj: \" + mailItem.Subject\n            print \"Body: \" + mailItem.Body.encode( 'ascii', 'ignore' )\n            print \"========\"\n\noutlook = win32com.client.DispatchWithEvents(\"Outlook.Application\", Handler_Class)\npythoncom.PumpMessages()\n```\n(from: https://blog.matthewurch.ca/?p=236)\n\nObviously, you would like to filter mails based on subject or from address and pass subject and body to mail_to_misp.py in order to do something useful. Pull-requests welcome for actual implementations :)\n\n\n## Requirements\n\n### The easy way\n\n```bash\npip install --user poetry\n\n# Install other python requirements\npoetry install -E fileobjects -E openioc -E virustotal -E email -E url\n\n# Test if the script is working\n./mail_to_misp.py -h\n```\n\n### General\n\n- mail_to_misp requires access to a MISP instance (via API).\n- Python \u003e=3.6\n- dnspython\n- PyMISP\n- faup from https://github.com/stricaud/faup\n- urlmarker from https://github.com/rcompton/ryancompton.net/blob/master/assets/praw_drugs/urlmarker.py (contained in this project)\n- ftfy from https://github.com/LuminosoInsight/python-ftfy (to fix unicode text)\n- defang from https://github.com/Rafiot/defang.git (fork of: https://bitbucket.org/johannestaas/defang)\n\n### Office 365\n\n- O365 from https://github.com/O365/python-o365\n\n### Thunderbird [unmaintained]\n\n- https://github.com/rommelfs/filterscript (modified fork from https://github.com/adamnew123456/filterscript)\n\n## License\n\nThis software is licensed under [GNU Affero General Public License version 3](http://www.gnu.org/licenses/agpl-3.0.html)\n\n* Copyright (C) 2017 - 2019 Sascha Rommelfangen, Raphaël Vinot\n* Copyright (C) 2017 - 2021 CIRCL - Computer Incident Response Center Luxembourg\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmisp%2Fmail_to_misp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmisp%2Fmail_to_misp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmisp%2Fmail_to_misp/lists"}