{"id":13601161,"url":"https://github.com/assafmo/xioc","last_synced_at":"2025-06-22T06:35:47.869Z","repository":{"id":82611726,"uuid":"166091373","full_name":"assafmo/xioc","owner":"assafmo","description":"Extract indicators of compromise from text, including \"escaped\" ones.","archived":false,"fork":false,"pushed_at":"2020-04-19T17:42:40.000Z","size":66,"stargazers_count":159,"open_issues_count":4,"forks_count":13,"subscribers_count":9,"default_branch":"master","last_synced_at":"2025-04-08T03:02:01.416Z","etag":null,"topics":["command-line","command-line-tool","data-mining","defang","escaping","extract","extraction","indicators-of-compromise","ioc","iocs","regex","regexp","text-mining","text-processing"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/assafmo.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2019-01-16T18:38:35.000Z","updated_at":"2025-03-22T10:59:33.000Z","dependencies_parsed_at":null,"dependency_job_id":"b6e16685-c18f-4163-911a-9c5e9e0abfad","html_url":"https://github.com/assafmo/xioc","commit_stats":null,"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"purl":"pkg:github/assafmo/xioc","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/assafmo%2Fxioc","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/assafmo%2Fxioc/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/assafmo%2Fxioc/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/assafmo%2Fxioc/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/assafmo","download_url":"https://codeload.github.com/assafmo/xioc/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/assafmo%2Fxioc/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261249129,"owners_count":23130492,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["command-line","command-line-tool","data-mining","defang","escaping","extract","extraction","indicators-of-compromise","ioc","iocs","regex","regexp","text-mining","text-processing"],"created_at":"2024-08-01T18:00:57.067Z","updated_at":"2025-06-22T06:35:42.856Z","avatar_url":"https://github.com/assafmo.png","language":"Go","readme":"# xioc\n\nExtract indicators of compromise from text, including \"escaped\" ones like `hxxp://banana.com`, `1.1.1[.]1` and `phish at malicious dot com`.\n\n[![CircleCI](https://circleci.com/gh/assafmo/xioc.svg?style=shield\u0026circle-token=53b168115c42a883184dd01267d549aed80c2f49)](https://circleci.com/gh/assafmo/xioc)\n[![Coverage Status](https://coveralls.io/repos/github/assafmo/xioc/badge.svg?branch=master)](https://coveralls.io/github/assafmo/xioc?branch=master)\n[![Go Report Card](https://goreportcard.com/badge/github.com/assafmo/xioc)](https://goreportcard.com/report/github.com/assafmo/xioc)\n[![GoDoc](https://godoc.org/github.com/assafmo/xioc/xioc?status.svg)](https://godoc.org/github.com/assafmo/xioc/xioc)\n\n## Installation\n\n- Download a precompiled binary from https://github.com/assafmo/xioc/releases\n- Or... Use `go get`:\n\n  ```bash\n  go get -u github.com/assafmo/xioc\n  ```\n\n- Or... Use snap install (Ubuntu):\n\n  ```bash\n  snap install xioc\n  ```\n\n- Or use Ubuntu PPA:\n\n  ```bash\n  curl -SsL https://assafmo.github.io/ppa/ubuntu/KEY.gpg | sudo apt-key add -\n  sudo curl -SsL -o /etc/apt/sources.list.d/assafmo.list https://assafmo.github.io/ppa/ubuntu/assafmo.list\n  sudo apt update\n  sudo apt install xioc\n  ```\n\n## Features\n\n- Extract IOCs (indicators of compromise) from an input text:\n  - IPv4\n  - IPv6\n  - Domain\n  - URL\n  - Email\n  - MD5\n  - SHA1\n  - SHA256\n- Translate some kinds of \"escaping\"/\"defanging\" techniques:\n  - `(dot)`, `[dot]`, `(.)`, `[.]`, `{.}` to `.`.\n  - `(at)`, `[at]`, `(@)`, `[@]`, `{@}` to `@`.\n  - `hxxp`, `hzzzp`, `hxxxp`, `hXXp`, `h__p`, `h**p` to `http`.\n- Command line interface\n- Go library\n\n## Command line usage\n\n```bash\n$ xioc -h\nUsage of xioc:\n  -o string\n        Extract only specified types.\n        Types must be comma seperated. E.g: xioc -o \"ip4,domain,url,md5\"\n        Available types:\n                - ip4\n                - ip6\n                - domain\n                - url\n                - email\n                - md5\n                - sha1\n                - sha256\n  -v    Print version and exit\n```\n\n```bash\n$ REPORT=\"https://unit42.paloaltonetworks.com/digital-quartermaster-scenario-demonstrated-in-attacks-against-the-mongolian-government/\"\n$ lynx -dump \"$REPORT\" | xioc\nsha256  5beb50d95c1e720143ca0004f5172cb8881d75f6c9f434ceaff59f34fa1fe378\ndomain  energy.gov.mn\nemail   altangadas@energy.gov.mn\nsha256  10090692ff40758a08bd66f806e0f2c831b4b9742bbf3d19c250e778de638f57\n# ...\n```\n\n```bash\n$ REPORT=\"https://unit42.paloaltonetworks.com/digital-quartermaster-scenario-demonstrated-in-attacks-against-the-mongolian-government/\"\n$ lynx -dump \"$REPORT\" | xioc -o email,sha256\nsha256  5beb50d95c1e720143ca0004f5172cb8881d75f6c9f434ceaff59f34fa1fe378\nemail   altangadas@energy.gov.mn\nsha256  10090692ff40758a08bd66f806e0f2c831b4b9742bbf3d19c250e778de638f57\nemail   ganbat_g@bpo.gov.mn\n# ...\n```\n\n## Library usage\n\nFull API:  \n[![GoDoc](https://godoc.org/github.com/assafmo/xioc/xioc?status.svg)](https://godoc.org/github.com/assafmo/xioc/xioc)\n\n```golang\npackage main\n\nimport (\n\t\"fmt\"\n\n\t\"github.com/assafmo/xioc/xioc\"\n)\n\nfunc main() {\n\tinput := `e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\n\tbanana.com\n\thxxp://i.robot.com/robots.txt\n\t1.2.3.4\n\t1.1.1[.]1\n\tinfo at gmail dot com\n\thxxps://m.twitter[dot]com/`\n\n\tfmt.Println(xioc.ExtractDomains(input)) // =\u003e [i.robot.com m.twitter.com gmail.com banana.com]\n\tfmt.Println(xioc.ExtractSHA256s(input)) // =\u003e [e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855]\n\tfmt.Println(xioc.ExtractMD5s(input))    // =\u003e []\n\tfmt.Println(xioc.ExtractIPv4s(input))   // =\u003e [1.2.3.4 1.1.1.1]\n\tfmt.Println(xioc.ExtractURLs(input))    // =\u003e [http://i.robot.com/robots.txt https://m.twitter.com/]\n\tfmt.Println(xioc.ExtractEmails(input))  // =\u003e [info@gmail.com]\n}\n```\n\n## Sources\n\n- Test email address: http://codefool.tumblr.com/post/15288874550/list-of-valid-and-invalid-email-addresses\n- Domains can start with a number: https://serverfault.com/a/638270\n- IPv6 Examples: http://www.gestioip.net/docu/ipv6_address_examples.html\n- Fang and defang IOCs: https://github.com/ioc-fang/ioc_fanger\n- Indicator of Compromise (De)Fanging Project: https://ioc-fang.hightower.space/\n- InQuest/python-iocextract test data: https://github.com/InQuest/python-iocextract/tree/master/test_data\n- Email address can be treated as case-insensitive: https://stackoverflow.com/a/9808332\n","funding_links":[],"categories":["Go"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fassafmo%2Fxioc","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fassafmo%2Fxioc","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fassafmo%2Fxioc/lists"}