{"id":46128480,"url":"https://github.com/melonsmasher/turkeybite","last_synced_at":"2026-03-02T03:12:09.551Z","repository":{"id":70802359,"uuid":"288859994","full_name":"MelonSmasher/TurkeyBite","owner":"MelonSmasher","description":"A domain and host context analysis pipeline.","archived":false,"fork":false,"pushed_at":"2025-12-05T18:13:28.000Z","size":5029,"stargazers_count":8,"open_issues_count":0,"forks_count":2,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-12-09T06:36:45.096Z","etag":null,"topics":["audit","browser-history","browserbeat","dns","docker","elastic-beats","elasticsearch","kibana","opensearch","packetbeat","python","python3","redis","security","security-tools","traffic-analysis","traffic-context","valkey","webbrowser"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MelonSmasher.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2020-08-19T23:35:47.000Z","updated_at":"2025-12-05T18:13:32.000Z","dependencies_parsed_at":null,"dependency_job_id":"ce429966-f692-4ef9-92c7-5294668535cb","html_url":"https://github.com/MelonSmasher/TurkeyBite","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/MelonSmasher/TurkeyBite","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MelonSmasher%2FTurkeyBite","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MelonSmasher%2FTurkeyBite/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MelonSmasher%2FTurkeyBite/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MelonSmasher%2FTurkeyBite/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MelonSmasher","download_url":"https://codeload.github.com/MelonSmasher/TurkeyBite/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MelonSmasher%2FTurkeyBite/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29991319,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-02T01:47:34.672Z","status":"online","status_checked_at":"2026-03-02T02:00:07.342Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["audit","browser-history","browserbeat","dns","docker","elastic-beats","elasticsearch","kibana","opensearch","packetbeat","python","python3","redis","security","security-tools","traffic-analysis","traffic-context","valkey","webbrowser"],"created_at":"2026-03-02T03:12:07.130Z","updated_at":"2026-03-02T03:12:09.542Z","avatar_url":"https://github.com/MelonSmasher.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# TurkeyBite\n\n[![GitHub license](https://img.shields.io/github/license/MelonSmasher/TurkeyBite)](https://github.com/MelonSmasher/TurkeyBite/blob/master/LICENSE)\n![Codacy grade](https://img.shields.io/codacy/grade/25d2ad332ca1453cb24aef58f3c10728)\n\n![TurkeyBite Logo](docs/img/turkey_bite_spy.png)\n\n## What is TurkeyBite\n\nA domain and host context analysis pipeline.\n\nTurkeyBite analyzes client network traffic to glean some context into each request. TB allows you to identify clients who are requesting domains associated with anything from porn to gambling to shopping and everything in between.\n\n### Whats under the hood\n\nTurkeyBite relies on the following technologies\n\n*   Docker\n*   Python3\n*   Valkey\n*   Bind9\n*   [Packetbeat](https://www.elastic.co/products/beats/packetbeat) and/or [Browserbeat](https://github.com/MelonSmasher/browserbeat)\n*   OpenSearch\n*   OpenSearch Dashboards\n*   Domain and host lists from many sources\n\nIn practice the analysis pipeline looks like this:\n\n![flow-chart](docs/img/flow.png)\n\nWhen conceptualizing the diagram above replace redis, elasticsearch, and kibana with valkey, opensearch, and opensearch dashboards respectively.\n\n### What DNS servers does this work with\n\nAs of right now I have tested this with a Microsoft DNS server and I am running this in production with multiple Bind9 servers. Since Packetbeat is used to grab and send packets to Redis this should work with any DNS server that can also run Packetbeat.\n\n### What browsers does this work with\n\nAny browsers that [Browserbeat](https://github.com/MelonSmasher/browserbeat) supports should work with TurkeyBite.\n\n### Will this block clients\n\nShort answer: no.\n\nLong answer: TB is an analysis tool not a blocking tool. For something like that check out [pi-hole](https://pi-hole.net/). In theory there is no reason why you couldn't run both pi-hole and TB in tandem. TB is designed to be as unobtrusive as possible so that it's implementation impact is never felt by clients.\n\n## Setup\n\n### Prerequisites\n\n* Docker and Docker Compose installed on your host system\n* Git to clone the repository\n\n### Installation\n\n1. **Clone the repository**\n\n   ```bash\n   git clone https://github.com/MelonSmasher/TurkeyBite.git\n   cd TurkeyBite\n   ```\n\n2. **Initialize the project**\n\n   Run the setup script to create required directories and configuration files:\n\n   ```bash\n   bash setup.sh\n   ```\n\n   The setup script will guide you through configuration options including:\n   \n   - Deployment type (Development, Small Scale, or Full Scale)\n   - DNS lookup configuration for client IPs\n   - Output options (OpenSearch and/or Syslog)\n   - Service passwords and connection settings\n   \n   For distributed deployments, you'll run this script on each node with the appropriate configuration.\n\n3. **Review configuration (optional)**\n\n   The setup script automatically generates the following configuration files:\n\n   - `.env` - Environment variables for Docker containers\n   - `config.yaml` - TurkeyBite application configuration\n   - `docker-compose.yml` - Container orchestration configuration\n\n   While the setup script configures these files based on your selections, you can review and adjust them if needed:\n\n   **Environment Variables** in `.env`:\n\n   ```bash\n   # Key environment variables (automatically configured by setup)\n   OPENSEARCH_INITIAL_ADMIN_PASSWORD=******      # Password for OpenSearch admin\n   OPENSEARCH_HOSTS='[\"https://opensearch:9200\"]'  # OpenSearch connection URL array\n   bootstrap.memory_lock=true                     # Enable memory locking for OpenSearch\n   node.name=${OPENSEARCH_HOST}                  # Set node name to match host\n   discovery.type=single-node                    # Run in single node mode\n   OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m        # Configure Java memory limits\n   VALKEY_HOST=valkey                            # Valkey/Redis hostname or IP\n   VALKEY_PORT=6379                             # Valkey/Redis port\n   OPENSEARCH_PORT=9200                         # OpenSearch API port\n   OPENSEARCH_DASHBOARD_PORT=5601               # OpenSearch Dashboards port\n   BIND9_IP=172.172.0.100                       # Static IP for Bind9 in Docker network\n   TURKEYBITE_WORKER_PROCS=2                    # Number of worker processes\n   TURKEYBITE_HOSTS_INTERVAL_MIN=720            # Host list refresh interval (minutes)\n   TURKEYBITE_IGNORELIST_INTERVAL_MIN=5         # Ignorelist refresh interval (minutes)\n   ```\n   \n   **Application Configuration** in `config.yaml`:\n\n   ```yaml\n   redis:\n     host: valkey\n     port: 6379\n     password: your_password_from_secrets\n     db: 0\n     host_list_db: 1\n     channel: turkeybite\n   # ... other configuration sections\n   ```\n\n4. **Secrets Setup**\n\n   The setup script automatically creates the required password files in the `vols/secrets/` directory. These include:\n\n   - `valkey_password.txt` - Password for Valkey/Redis authentication\n\n   You can review and modify these secrets if needed.\n\n   **Important for Distributed Setups:** In distributed deployments where Valkey runs on its own dedicated node, the `valkey_password.txt` file must be copied from the Valkey server to all Core and Worker nodes. The setup script will prompt you to enter this password when configuring nodes that don't run Valkey directly.\n\n5. **Configure Bind9 (if using as DNS server)**\n\n   The setup script copies example Bind9 configuration files to the `vols/bind/` directory. Review and modify these files:\n   \n   * `named.conf.local` - Local DNS configuration\n   * `named.conf.options` - DNS server options\n   * `slave.conf` - Zone configurations for slave DNS setup\n\n   For more information on Bind9 configuration see [docs/bind9.md](docs/bind9.md).\n\n### Running TurkeyBite\n\n1. **Start the containers**\n\n   ```bash\n   docker compose up -d\n   ```\n\n2. **Verify containers are running**\n\n   ```bash\n   docker compose ps\n   ```\n\n3. **Access OpenSearch Dashboards**\n\n   Navigate to `http://localhost:5601` in your web browser\n   \n   * Username: `admin`\n   * Password: The password you set in `OPENSEARCH_INITIAL_ADMIN_PASSWORD`\n\n### Data Collection\n\nTo collect network data, you'll need to configure either Packetbeat or Browserbeat:\n\n1. **Packetbeat**\n\n   Install and configure [Packetbeat](https://www.elastic.co/products/beats/packetbeat) on your network:\n\n   ```yaml\n   # packetbeat.yml example\n   packetbeat.protocols:\n     dns:\n       ports: [53]\n       include_authorities: true\n       include_additionals: true\n   \n   output.redis:\n     hosts: [\"valkey.domain.com:6379\"]\n     password: \"your_valkey_password\"\n     db: 0\n     key: \"turkeybite\"\n     data_type: \"list\"\n   ```\n\n2. **Browserbeat**\n\n   Follow the installation instructions for [Browserbeat](https://github.com/MelonSmasher/browserbeat) to collect browser history data.\n\n### Maintenance\n\n* **Logs**: Container logs are available in the `vols/logs/` directory\n* **Domain Lists**: Lists are stored in `vols/lists/` and updated according to the configured intervals\n\n### Troubleshooting\n\n* Check container logs: `docker compose logs -f [service_name]`\n* Restart services: `docker compose restart [service_name]`\n* Verify connectivity between containers: `docker compose exec turkeybite-core ping valkey`","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmelonsmasher%2Fturkeybite","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmelonsmasher%2Fturkeybite","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmelonsmasher%2Fturkeybite/lists"}