{"id":51680729,"url":"https://github.com/pedramamini/ThreatIngestor","last_synced_at":"2026-08-03T18:01:00.547Z","repository":{"id":38899370,"uuid":"102033255","full_name":"pedramamini/ThreatIngestor","owner":"pedramamini","description":"Extract and aggregate threat intelligence.","archived":false,"fork":false,"pushed_at":"2026-05-26T15:01:24.000Z","size":1738,"stargazers_count":923,"open_issues_count":15,"forks_count":136,"subscribers_count":40,"default_branch":"master","last_synced_at":"2026-07-27T18:07:42.026Z","etag":null,"topics":["dfir","fraud-detection","indicators-of-compromise","intelligence-gathering","ioc","malware-research","misp","osint","security-tools","soar","threat-analysis","threat-feeds","threat-hunting","threat-intelligence","threat-intelligence-platform","threat-sharing","threatintel","yara"],"latest_commit_sha":null,"homepage":"https://inquest.readthedocs.io/projects/threatingestor/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pedramamini.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-08-31T18:41:55.000Z","updated_at":"2026-07-17T12:09:37.000Z","dependencies_parsed_at":"2023-10-04T05:45:45.255Z","dependency_job_id":"b41e602f-0e60-452c-8622-6941e848902c","html_url":"https://github.com/pedramamini/ThreatIngestor","commit_stats":{"total_commits":533,"total_committers":19,"mean_commits":28.05263157894737,"dds":"0.32457786116322707","last_synced_commit":"2ccc1a53e704fd5d213dd8307ff9b907088f8b34"},"previous_names":[],"tags_count":18,"template":false,"template_full_name":null,"purl":"pkg:github/pedramamini/ThreatIngestor","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pedramamini%2FThreatIngestor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pedramamini%2FThreatIngestor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pedramamini%2FThreatIngestor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pedramamini%2FThreatIngestor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pedramamini","download_url":"https://codeload.github.com/pedramamini/ThreatIngestor/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pedramamini%2FThreatIngestor/sbom","scorecard":{"id":66088,"data":{"date":"2025-08-11","repo":{"name":"github.com/InQuest/ThreatIngestor","commit":"51bd783314abfbc0da9c9b9ca43038b97c364221"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":2.5,"checks":[{"name":"Code-Review","score":0,"reason":"Found 0/20 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: no topLevel permission defined: .github/workflows/workflow.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: GNU General Public License v2.0: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/workflow.yml:13: update your workflow using https://app.stepsecurity.io/secureworkflow/InQuest/ThreatIngestor/workflow.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/workflow.yml:15: update your workflow using https://app.stepsecurity.io/secureworkflow/InQuest/ThreatIngestor/workflow.yml/master?enable=pin","Warn: containerImage not pinned by hash: Dockerfile:2: pin your Docker image by updating ubuntu:18.04 to ubuntu:18.04@sha256:152dc042452c496007f07ca9127571cb9c29697f42acbfad72324b2bb2e43c98","Warn: containerImage not pinned by hash: Dockerfile.dev:2: pin your Docker image by updating ubuntu:18.04 to ubuntu:18.04@sha256:152dc042452c496007f07ca9127571cb9c29697f42acbfad72324b2bb2e43c98","Warn: pipCommand not pinned by hash: Dockerfile:12","Warn: pipCommand not pinned by hash: Dockerfile:13","Warn: pipCommand not pinned by hash: Dockerfile:14","Warn: pipCommand not pinned by hash: Dockerfile:15","Warn: pipCommand not pinned by hash: Dockerfile.dev:17","Warn: pipCommand not pinned by hash: Dockerfile.dev:18","Warn: pipCommand not pinned by hash: Dockerfile.dev:19","Warn: pipCommand not pinned by hash: Dockerfile.dev:20","Warn: pipCommand not pinned by hash: Dockerfile.dev:21","Warn: pipCommand not pinned by hash: .github/workflows/workflow.yml:20","Warn: pipCommand not pinned by hash: .github/workflows/workflow.yml:21","Info:   0 out of   2 GitHub-owned GitHubAction dependencies pinned","Info:   0 out of   2 containerImage dependencies pinned","Info:   1 out of  12 pipCommand dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Vulnerabilities","score":0,"reason":"10 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: PYSEC-2023-117 / GHSA-mrwq-x4v8-fh7p","Warn: Project is vulnerable to: GHSA-v9hf-5j83-6xpp","Warn: Project is vulnerable to: PYSEC-2022-14 / GHSA-39ph-wr67-j4xq","Warn: Project is vulnerable to: PYSEC-2021-142 / GHSA-8q59-q68h-6hv4","Warn: Project is vulnerable to: PYSEC-2018-49 / GHSA-rprw-h62v-c2w7","Warn: Project is vulnerable to: PYSEC-2014-14 / GHSA-652x-xj99-gmcc","Warn: Project is vulnerable to: GHSA-9hjg-9r4m-mvj7","Warn: Project is vulnerable to: GHSA-9wx4-h78v-vm56","Warn: Project is vulnerable to: PYSEC-2014-13 / GHSA-cfj3-7x9c-4p3h","Warn: Project is vulnerable to: PYSEC-2018-28 / GHSA-x84v-xcm2-53pg"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 15 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-15T02:38:07.386Z","repository_id":38899370,"created_at":"2025-08-15T02:38:07.387Z","updated_at":"2025-08-15T02:38:07.387Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":36245385,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-07-20T02:08:10.276Z","status":"online","status_checked_at":"2026-08-03T02:00:06.975Z","response_time":56,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dfir","fraud-detection","indicators-of-compromise","intelligence-gathering","ioc","malware-research","misp","osint","security-tools","soar","threat-analysis","threat-feeds","threat-hunting","threat-intelligence","threat-intelligence-platform","threat-sharing","threatintel","yara"],"created_at":"2026-07-15T13:00:35.770Z","updated_at":"2026-08-03T18:01:00.517Z","avatar_url":"https://github.com/pedramamini.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n  \u003ch1\u003eThreatIngestor\u003c/h1\u003e\n\u003c/div\u003e\n\n[![Build Status](https://github.com/InQuest/ThreatIngestor/actions/workflows/workflow.yml/badge.svg)](https://github.com/InQuest/ThreatIngestor/actions/workflows/workflow.yml)\n![Documentation Status](https://readthedocs.org/projects/threatingestor/badge/?version=latest)\n![PyPI Version](http://img.shields.io/pypi/v/ThreatIngestor.svg)\n\nAn extendable tool to extract and aggregate [IOCs](https://en.wikipedia.org/wiki/Indicator_of_compromise) from threat feeds.\n\nIntegrates out-of-the-box with [ThreatKB](https://github.com/InQuest/ThreatKB) and [MISP](https://www.misp-project.org/), and can fit seamlessly into any existing workflow with [SQS](https://aws.amazon.com/sqs/), [Beanstalk](https://beanstalkd.github.io/), and [custom plugins](https://inquest.readthedocs.io/projects/threatingestor/en/latest/developing.html).\n\nCurrently used by InQuest Labs IOC-DB: https://labs.inquest.net/iocdb\n\n## Overview\n\nThreatIngestor can be configured to watch Twitter, RSS feeds, sitemap (XML) feeds, or other sources, and extract meaningful information such as malicious IPs/domains and YARA signatures, and send that information to another system for analysis.\n\n![ThreatIngestor flowchart with several sources feeding into multiple operators](https://inquest.readthedocs.io/projects/threatingestor/en/latest/_images/mermaid-multiple-operators.png)\n\nTry it out now with this [quick walkthrough](https://inquest.readthedocs.io/projects/threatingestor/en/latest/welcome.html#try-it-out), read more [ThreatIngestor walkthroughs](https://inquest.net/taxonomy/term/42) on the InQuest blog, and check out [labs.inquest.net/iocdb](https://labs.inquest.net/iocdb), an IOC aggregation and querying tool powered by ThreatIngestor.\n\n## Installation\n\nThreatIngestor requires Python 3.6+, with development headers.\n\nInstall ThreatIngestor from PyPI:\n\n```bash\npip install threatingestor\n```\n\nInstall optional dependencies for using some plugins, as needed:\n\n```bash\npip install threatingestor[all]\n```\n\nView the [full installation instructions](https://inquest.readthedocs.io/projects/threatingestor/en/latest/installation.html) for more information.\n\n## Usage\n\nCreate a new ``config.yml`` file, and configure each source and operator module you want to use. (See ``config.example.yml`` for layout.) Then run the script:\n\n```bash\nthreatingestor config.yml\n```\n\nBy default, it will run forever, polling each configured source every 15 minutes.\n\nIf you'd like to run the image extraction source, or include the image extraction functionality for other sources, you will need to be running Python 3.7 \u003e= due to the dependencies:\n\n```bash\npip install opencv-python pytesseract numpy\n```\n\nView the [full ThreatIngestor documentation](https://inquest.readthedocs.io/projects/threatingestor/) for more information.\n\n## Plugins\n\nThreatIngestor uses a plugin architecture with \"source\" (input) and \"operator\" (output) plugins. The currently supported integrations are:\n\n### Sources\n\n- [Git repositories](https://inquest.readthedocs.io/projects/threatingestor/en/latest/sources/git.html)\n- [GitHub repository search](https://inquest.readthedocs.io/projects/threatingestor/en/latest/sources/github.html)\n- [Gists by username](https://inquest.readthedocs.io/projects/threatingestor/en/latest/sources/github_gist.html)\n- [RSS feeds](https://inquest.readthedocs.io/projects/threatingestor/en/latest/sources/rss.html)\n- [Sitemap feeds](https://inquest.readthedocs.io/projects/threatingestor/en/latest/sources/sitemap.html)\n- [Image extraction](https://inquest.readthedocs.io/projects/threatingestor/en/latest/sources/image.html)\n- [Amazon SQS queues](https://inquest.readthedocs.io/projects/threatingestor/en/latest/sources/sqs.html)\n- [Twitter](https://inquest.readthedocs.io/projects/threatingestor/en/latest/sources/twitter.html)\n- [Generic web pages](https://inquest.readthedocs.io/projects/threatingestor/en/latest/sources/web.html)\n\n### Operators\n\n- [CSV files](https://inquest.readthedocs.io/projects/threatingestor/en/latest/operators/csv.html)\n- [MISP](https://inquest.readthedocs.io/projects/threatingestor/en/latest/operators/misp.html)\n- [MySQL table](https://inquest.readthedocs.io/projects/threatingestor/en/latest/operators/mysql.html)\n- [SQLite database](https://inquest.readthedocs.io/projects/threatingestor/en/latest/operators/sqlite.html)\n- [Amazon SQS queues](https://inquest.readthedocs.io/projects/threatingestor/en/latest/operators/sqs.html)\n- [ThreatKB](https://inquest.readthedocs.io/projects/threatingestor/en/latest/operators/threatkb.html)\n- [Twitter](https://inquest.readthedocs.io/projects/threatingestor/en/latest/operators/twitter.html)\n\nView the [full ThreatIngestor documentation](https://inquest.readthedocs.io/projects/threatingestor/) for more information on included plugins, and how to create your own.\n\n## Threat Intel Sources\n\nLooking for some threat intel sources to get started? InQuest has a Twitter List with several accounts that post C2 domains and IPs: https://twitter.com/InQuest/lists/ioc-feed. Note that you will need to apply for a Twitter developer account to use the ThreatIngestor Twitter Source. Take a look at ``config.example.yml`` to see how to set this list up as a source.\n\nFor quicker setup, RSS feeds can be a great source of intelligence. Check out this example [RSS config file](https://github.com/InQuest/ThreatIngestor/blob/master/rss.example.yml) for a few pre-configured security blogs.\n\n## Support\n\nIf you need help getting set up, or run into any issues, feel free to open an [issue](https://github.com/InQuest/ThreatIngestor/issues). You can also reach out to [@InQuest](https://twitter.com/InQuest) on Twitter or read more about us on the web at https://www.inquest.net.\n\nWe'd love to hear any feedback you have on ThreatIngestor, its documentation, or how you're putting it to work for you!\n\n## Contributing\n\nIssues and pull requests are welcomed. Please keep Python code PEP8 compliant. By submitting a pull request you agree to release your submissions under the terms of the [LICENSE](https://github.com/InQuest/ThreatIngestor/blob/master/LICENSE).\n\n## Docker\n\n\n### Production\n\nA `Dockerfile` is available for running ThreatIngestor within a Docker container.\n\nFirst, you'll need to build the container:\n\n```bash\ndocker build . -t threatingestor\n```\n\nAfter that, you can mount the container by using this command:\n\n```bash\ndocker run -it --mount type=bind,source=/,target=/dock threatingestor /bin/bash\n```\n\nAfter you've mounted the container and you're inside the `/bin/bash` shell, you can run ThreatIngestor like normal:\n\n```bash\nthreatingestor config.yml\n```\n\n### Development\n\nThere is also a Dockerfile.dev for building a development version of ThreatIngestor. All you need is an available .whl file, which can be generated with the following command:\n\n```bash\npython3 -m build \n```\n\nAfter you've built the project, you can build the container:\n\n```bash\ndocker build . -t threatingestor -f Dockerfile.dev\n```\n\nNOTE: If you run into any issues while building the development environment or running ThreatIngestor within the container, you may need to comment out the following lines in `Dockerfile.dev` to work properly:\n\n```\nFROM ubuntu:18.04\n...\n# RUN apt-get install tesseract-ocr -y\n...\n# RUN pip3 install opencv-python pytesseract numpy\n...\n```\n\n### Extra Scripts\n\nSome scripts are now provided to help with your local configuration of ThreatIngestor.\n\nA README.md with additional information is available [here](https://github.com/InQuest/ThreatIngestor/tree/master/scripts/README.md).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpedramamini%2FThreatIngestor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpedramamini%2FThreatIngestor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpedramamini%2FThreatIngestor/lists"}