{"id":15663401,"url":"https://github.com/ybubnov/deep-lookup","last_synced_at":"2025-07-25T19:33:37.238Z","repository":{"id":45001253,"uuid":"335033301","full_name":"ybubnov/deep-lookup","owner":"ybubnov","description":"Deep Learning for Domain Name System","archived":false,"fork":false,"pushed_at":"2022-01-14T13:44:32.000Z","size":231500,"stargazers_count":19,"open_issues_count":0,"forks_count":1,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-07-19T05:41:24.883Z","etag":null,"topics":["deep-learning","deep-reinforcement-learning","dga","dga-detection","dns","dns-tunneling","dns-tunneling-detection"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ybubnov.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-02-01T17:43:58.000Z","updated_at":"2025-05-23T06:58:11.000Z","dependencies_parsed_at":"2022-09-02T13:10:21.746Z","dependency_job_id":null,"html_url":"https://github.com/ybubnov/deep-lookup","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/ybubnov/deep-lookup","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ybubnov%2Fdeep-lookup","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ybubnov%2Fdeep-lookup/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ybubnov%2Fdeep-lookup/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ybubnov%2Fdeep-lookup/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ybubnov","download_url":"https://codeload.github.com/ybubnov/deep-lookup/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ybubnov%2Fdeep-lookup/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267053007,"owners_count":24028030,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-25T02:00:09.625Z","response_time":70,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","deep-reinforcement-learning","dga","dga-detection","dns","dns-tunneling","dns-tunneling-detection"],"created_at":"2024-10-03T13:37:11.016Z","updated_at":"2025-07-25T19:33:37.179Z","avatar_url":"https://github.com/ybubnov.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Deep Lookup - Deep Learning for Domain Name System\n\n## Installation\n\n### Installation Using PyPi\n\n```sh\npip install deeplookup\n```\n\n## Using DeepLookup\n\nDeepLookup provides a `Resolver` instance that inherits [`dns.resolver.Resolver`](dns-resolver)\n```py\nfrom deeplookup import Resolver\n\n\nresolver = Resolver()\n\nfor ip in resolver.resolve(\"google.com\", \"A\"):\n    print(f\"ip: {ip.to_text()}\")\n```\n\nThe code above performs a verification of a queried name using a neural network trained\nto detect malicious queries ([DGAs][dga-wiki] and tunnels). For the example above the\noutput will look like following:\n```sh\nip: 142.250.184.206\n```\n\nWhen the queried name is generated using domain generation algorithm, the resolver throws\n[`dns.resolver.NXDOMAIN`](dns-nxdomain) without even accessing a remote name server.\n```py\nfor ip in resolver.resolve(\"mjewnjixnjaa.com\", \"A\"):\n    print(f\"ip: {ip.to_text()}\")\n```\n\nThe example above throws [`dns.resolver.NXDOMAIN`](dns-nxdomain) error with the following\nmessage:\n```sh\ndns.resolver.NXDOMAIN: The DNS query name does not exist: mjewnjixnjaa.com.\n```\n\n## Training\n\nThe model is trained using [tfx](txf) pipeline, where the training dataset is uploaded,\nsplit into the training and evaluation subsets and then used to fit the neural network.\n\nIn order to trigger the training pipeline use the following command:\n```sh\npython -m deeplookup.pipeline.gta1\n```\n\nThis command creates a folder called \"tfx\", where all artifacts are persisted. See the\n`tfx/pipelines/gta1/serving_model/gta1/*` folder to access the model in HDF5 format.\n\n## Publications\n1. Bubnov Y., Ivanov N. (2020) Text analysis of DNS queries for data exfiltration protection of computer networks, [_Informatics_][Informatics, 2020], 3, 78-86.\n2. Bubnov Y., Ivanov N. (2020) Hidden Markov model for malicious hosts detection in a computer network, [_Journal of BSU. Mathematics and Informatics_][BSU, 2020], 3, 73-79.\n3. Bubnov Y., Ivanov N. (2021) DGA domain detection and botnet prevention using Q-learning for POMDP, [_Doklady BGUIR_][BGUIR, 2021], 2, 91-99.\n\n## Datasets\n\nThe most robust dataset [DGTA-BENCH][DGTA, 2021] is available through\n[tensorflow datasets](https://www.tensorflow.org/datasets) API and used for training\nother neural network architectures:\n```py\nimport deeplookup.datasets as dlds\nimport tensorflow_datasets as tfds\n\nds = tfds.load(\"gta1\", shuffle_files=True)\n\nfor example in ds.take(1):\n  domain, label = example[\"domain\"], example[\"class\"]\n```\n\n1. Bubnov Y. (2019) DNS Tunneling Queries for Binary Classification, [_Mendeley Data_][DTQBC, 2019], v1.\n2. Zago M., Perez. M.G., Perez G.M. (2020) UMUDGA - University of Murcia Domain Generation Algorithm Dataset, [_Mendeley Data_][UMUDGA, 2020], v1.\n3. Bybnov Y. (2021) DGTA-BENCH - Domain Generation and Tunneling Algorithms for Benchmark, [_Mendeley Data_][DGTA, 2021], v1.\n\n\n[Informatics, 2020]: https://doi.org/10.37661/1816-0301-2020-17-3-78-86\n[BSU, 2020]: https://doi.org/10.33581/2520-6508-2020-3-73-79\n[BGUIR, 2021]: https://doi.org/10.35596/1729-7648-2021-19-2-91-99\n[UMUDGA, 2020]: http://dx.doi.org/10.17632/y8ph45msv8.1\n[DTQBC, 2019]: http://dx.doi.org/10.17632/mzn9hvdcxg.1\n[DGTA, 2021]: http://dx.doi.org/10.17632/2wzf9bz7xr.1\n\n[dga-wiki]: https://en.wikipedia.org/wiki/Domain_generation_algorithm\n[dns-resolver]: https://dnspython.readthedocs.io/en/latest/resolver-class.html\n[dns-nxdomain]: https://dnspython.readthedocs.io/en/latest/exceptions.html#dns.resolver.NXDOMAIN\n[tfx]: https://www.tensorflow.org/tfx\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fybubnov%2Fdeep-lookup","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fybubnov%2Fdeep-lookup","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fybubnov%2Fdeep-lookup/lists"}