{"id":50766514,"url":"https://github.com/geeoon/engine-kibble","last_synced_at":"2026-06-11T14:01:32.456Z","repository":{"id":363181445,"uuid":"1133865285","full_name":"Geeoon/ENGINE-Kibble","owner":"Geeoon","description":null,"archived":false,"fork":false,"pushed_at":"2026-06-07T19:53:40.000Z","size":1276,"stargazers_count":1,"open_issues_count":1,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-07T20:21:07.092Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Geeoon.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS","dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-01-13T23:26:37.000Z","updated_at":"2026-06-07T19:53:44.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/Geeoon/ENGINE-Kibble","commit_stats":null,"previous_names":["geeoon/engine-kibble"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/Geeoon/ENGINE-Kibble","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Geeoon%2FENGINE-Kibble","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Geeoon%2FENGINE-Kibble/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Geeoon%2FENGINE-Kibble/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Geeoon%2FENGINE-Kibble/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Geeoon","download_url":"https://codeload.github.com/Geeoon/ENGINE-Kibble/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Geeoon%2FENGINE-Kibble/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34201842,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-11T02:00:06.485Z","response_time":57,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-06-11T14:01:31.561Z","updated_at":"2026-06-11T14:01:32.436Z","avatar_url":"https://github.com/Geeoon.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ENGINE Kibble project\nKibble probes devices with different protocols, stores status events in MongoDB, detects latency thresholds, and raises alerts.\n\n## Prerequisites\n\n- **Python 3.12+**\n- **Docker Desktop** (for the simulated environment and local MongoDB)\n- **sudo** on Linux/macOS\n\nOptional:\n\n- **MongoDB Compass** - GUI for browsing the `kibble` database\n- **mongosh** - MongoDB shell (included in the main simulator container)\n\n## Quick start (simulator)\nThis is the fastest way to try Kibble. **Docker Desktop must be running** before you begin. \n\n```bash\ngit clone https://github.com/Geeoon/ENGINE-Kibble.git\ncd ENGINE-Kibble\n\npython3 -m venv .venv\nsource .venv/bin/activate          # Windows: .venv\\Scripts\\activate\npip install -r src/Kibble/requirements.txt\n\ncd simulator\n./run.sh\n```\n`run.sh` opens a shell inside `kibble-main-container`. When you **exit that shell**, the whole stack is torn down \nInside the main container shell, start the monitor:\n\n```bash\ncd /tmp\n./start.sh\n```\nor  run\n\nOr run `python3 main.py` directly (equivalent inside the container).\n\nOn first run, `main.py` **automatically seeds** test devices (`simulator-secondary-1` … `5`, `simulator-scpi-1` … `5`, and a few others). No manual setup is required.\n\nIn **another terminal** on your host, open [MongoDB Compass](https://www.mongodb.com/try/download/compass) with:\n\n```\nmongodb://root:password@localhost:27017\n```\n\nIn MongoDB Compass devices and configurations can also be manually added/removed.\nOpen the **`kibble`** database and inspect **`timeseries_events`** for live probe results.\n\n## Main Monitor Installation\nUse this path when you want to run the monitor on your **host machine** (outside Docker) against a local MongoDB container.\n### Linux\n\n1. **Clone the repository**\n\n   ```bash\n   git clone https://github.com/Geeoon/ENGINE-Kibble.git\n   cd ENGINE-Kibble\n   ```\n\n2. **Create a virtual environment and install dependencies**\n\n   ```bash\n   python3 -m venv .venv\n   source .venv/bin/activate\n   pip install -r src/Kibble/requirements.txt\n   ```\n3. **Configure email alerts (optional)**\n4. \n   - Create a Gmail email account with an app password\n   - Modify the main.py args.sender_email and args.reciever_email\n   - Modify the .env to set the EMAIL_PASSWD to the app passwork\n\n\n5. **Start MongoDB**\n\n   Either run the full simulator:\n\n   ```bash\n   cd simulator\n   ./run.sh\n   ```\n\n   Or start only the database container:\n\n   ```bash\n   cd simulator\n   mkdir -p db \u0026\u0026 sudo chmod 777 db\n   docker compose up database -d\n   ```\n6. **Run the monitor**\n\n   **Inside the simulator container:**\n\n   ```bash\n   cd /tmp\n   ./start.sh\n   ```\n  **On the host machine:**\n\n   `src/main.py` defaults to `database.internal`. For host-side runs, either add `127.0.0.1 database.internal` to `/etc/hosts`, or change `mongo_host` in `src/main.py` to `'localhost'`.\n\n   ```bash\n   cd src\n   source ../.venv/bin/activate\n   ./start.sh\n   ```\n\n   `start.sh` loads `../.env` and runs `main.py` with `sudo` when available (needed for ICMP/Scapy).\n\n   Press `Ctrl+C` to stop the monitor. Logs are written to `kibble.log` and `kibble_status.log`.\n\n### Windows\n1. **Clone the repository**\n\n   ```powershell\n   git clone https://github.com/Geeoon/ENGINE-Kibble.git\n   cd ENGINE-Kibble\n   ```\n\n2. **Create a virtual environment and install dependencies**\n\n   ```powershell\n   python -m venv .venv\n   .venv\\Scripts\\activate\n   pip install -r src/Kibble/requirements.txt\n   ```\n3. **Configure email alerts (optional)**\n\n   - Create a Gmail email account with an app password\n   - Modify the main.py args.sender_email and args.reciever_email\n   - Modify the .env to set the EMAIL_PASSWD to the app passwork\n\n4. **Start MongoDB with Docker Desktop**\n\n   ```powershell\n   cd simulator\n   mkdir db\n   docker compose up database -d\n   ```\n\n   To run the full simulator without bash, build and start manually:\n\n   ```powershell\n   docker compose build\n   docker compose up --scale secondary=5 --scale scpi=5 -d\n   docker exec -it kibble-main-container bash\n   ```\n5. **Run the monitor**\n\n   **Inside the simulator container:**\n\n   ```bash\n   cd /tmp\n   python3 main.py\n   ```\n   or\n   ```bash\n   cd /tmp\n   ./start.sh\n   ```\n   (`./start.sh` and `python3 main.py` are equivalent inside the container)\n\n   **On the host machine:**\n\n   Change `mongo_host` in `src/main.py` to `'localhost'`, then:\n\n   ```powershell\n   cd src\n   ..\\.venv\\Scripts\\activate\n   python main.py\n   ```\n## Simulated environment (for development)\n\nThe `simulator/` directory runs a full test network in Docker: a monitoring host, MongoDB, secondary machines, and SCPI instrument simulators.\n### Start the stack\n\n```bash\ncd simulator\n./run.sh          # starts 5 secondary + 5 SCPI devices (default)\n./run.sh 3        # start 3 of each instead\n```\n\n`run.sh` will:\n\n1. Build all container images\n2. Start MongoDB and simulated devices\n3. Open a shell inside `kibble-main-container`\n4. Tear down the stack when you exit that shell (`docker compose down -v`)\n\nTo keep containers running in the background:\n\n```bash\ncd simulator\nmkdir -p db \u0026\u0026 sudo chmod 777 db\ndocker compose build\ndocker compose up --scale secondary=5 --scale scpi=5 -d\n```\n\nThe monitor connects to MongoDB at `database.internal:27017`, seeds test devices, and begins probing.\n\n## Configuration\n### Database\nKibble stores device configuration and monitoring events in MongoDB.\n\n| Setting  | Simulator (inside Docker) | Host machine / Compass |\n|----------|---------------------------|-------------------------|\n| Host     | `database.internal`       | `localhost`             |\n| Port     | `27017`                   | `27017`                 |\n| Username | `root`                    | `root`                  |\n| Password | `password`                | `password`              |\n| Database | `kibble`                  | `kibble`                |\n\nConnection string for Compass or `mongosh`:\n\n```\nmongodb://root:password@localhost:27017\n```\n\n\u003e These credentials are simulator defaults for local development only.\n\nStart the database container:\n\n```bash\ncd simulator \u0026\u0026 docker compose up database -d\n```\n#### Collections\n\n| Collection                  | Purpose                                      |\n|-----------------------------|----------------------------------------------|\n| `devices`                   | Device identity (`asset_tag`, `device_type_id`) |\n| `device_types`              | Device type definitions and supported protocols |\n| `device_configurations`     | Which interfaces are active for a device     |\n| `interface_configurations`  | IP, hostname, MAC, and other network details |\n| `timeseries_events`         | Monitoring events and latency readings       |\n\n\n#### Device configuration tool (`manage_devices.py`)\n\nUse `src/manage_devices.py` to add or remove monitored devices in MongoDB from the terminal. The monitor reads device configuration from the database on each scan cycle.\n\u003e **MongoDB must be running** before using this tool.\n\n**How device data is stored:**\n\n| What you set in the CLI | Where it goes in MongoDB |\n|-------------------------|--------------------------|\n| `--asset-tag`           | `devices.asset_tag` |\n| `--type`                | `devices.device_type_id` → `device_types` |\n| `--ip`, `--hostname`, `--mac` | `interface_configurations` and `device_configurations` |\n\n**Workflow (simulator - devices are auto-seeded):**\n\n1. Start the simulator (or at least the MongoDB container)\n2. Start the monitor - `main.py` seeds simulator devices on first run\n3. Inspect results in Compass (`timeseries_events`)\n4. Optionally add more devices with `manage_devices.py` while the monitor runs\n\n**Workflow (host / custom devices - manual setup):**\n1. Start MongoDB\n2. Add devices with `manage_devices.py`\n3. Verify in Compass (see below)\n4. Start the monitor\n5. Watch `timeseries_events` for probe results\n\n**Add a device**\n\nEach device needs a unique `--asset-tag`, a device type (`--type` or `--type-name` with `--protocols`), and at least one of `--ip` or `--hostname`.\n\n| `--type`  | Protocols \n|-----------|-----------\n| `icmp`    | ICMP      \n| `scpi`    | SCPI      \n| `snmp`    | SNMP      \n| `daemon`  | Daemon    \n\nExamples:\n\n```bash\n# Ping a device by IP\npython manage_devices.py add --asset-tag 1001 --type icmp --ip 192.168.1.10\n\n# Ping a device by hostname\npython manage_devices.py add --asset-tag 1002 --type icmp --hostname my-server.local\n\n# Add an SCPI instrument in the simulator (asset 3001 is auto-seeded; use a new tag to add another)\npython manage_devices.py add --asset-tag 9002 --type scpi --hostname simulator-scpi-1\n\n# Custom device type\npython manage_devices.py add --asset-tag 3001 --type-name \"custom probe\" --protocols ICMP --ip 10.0.0.5\n```\n\nInside the main simulator container (open a second shell with `docker exec -it kibble-main-container bash` while the monitor runs):\n\n```bash\ncd /tmp\npython manage_devices.py --mongo-host database.internal add \\\n  --asset-tag 9001 --type icmp --hostname simulator-secondary-1\n```\n\nRe-running `add` with the same `--asset-tag` updates the device. A new configuration snapshot is recorded only when IP, hostname, or MAC changes.\n\n**Remove a device**\n\n```bash\npython manage_devices.py remove --asset-tag 1001\n```\n\nThis deletes the device and its associated configuration records. Historical `timeseries_events` are not deleted.\n\n**CLI reference**\n\n```text\nusage: manage_devices.py [-h] [--mongo-host MONGO_HOST] [--mongo-port MONGO_PORT]\n                         {add,remove} ...\n\noptions:\n  --mongo-host MONGO_HOST   default: localhost (use database.internal in Docker)\n  --mongo-port MONGO_PORT   default: 27017\n\nadd:\n  --asset-tag ASSET_TAG     required\n  --type {daemon,icmp,scpi,snmp}\n  --type-name TYPE_NAME     requires --protocols\n  --protocols PROTOCOLS [PROTOCOLS ...]\n  --ip IP\n  --hostname HOSTNAME\n  --mac MAC\n\nremove:\n  --asset-tag ASSET_TAG     required\n```\n\n### Command-Line Arguments\n```\nusage: main.py [-h] [--low-thresh LOW_THRESH] [--medium-thresh MEDIUM_THRESH] [--high-thresh HIGH_THRESH] [--monitor-id MONITOR_ID] [--community-string COMMUNITY_STRING] [--device-timeout DEVICE_TIMEOUT] [--scan-period SCAN_PERIOD] [--threads THREADS]\n               [--sender-email SENDER_EMAIL] [--receiver-email RECEIVER_EMAIL] [--mongo-host MONGO_HOST] [--mongo-port MONGO_PORT] [--mongo-user MONGO_USER] [--mongo-pass MONGO_PASS]\n\noptions:\n  -h, --help            show this help message and exit\n  --low-thresh LOW_THRESH\n  --medium-thresh MEDIUM_THRESH\n  --high-thresh HIGH_THRESH\n  --monitor-id MONITOR_ID\n                        Unique integer ID for this monitoring node (lowest ID wins leader election)\n  --community-string COMMUNITY_STRING\n                        Community string for SNMP monitor\n  --device-timeout DEVICE_TIMEOUT\n                        Timeout for the device\n  --scan-period SCAN_PERIOD\n                        How often to scan the network. Should be at least double the device timeout\n  --threads THREADS     The number of threads to launch to do simultaneous device scans. Should scale with the number of devices.\n  --sender-email SENDER_EMAIL\n                        The email account to send alerts from\n  --receiver-email RECEIVER_EMAIL\n                        The email accoutn to send alerts to\n  --mongo-host MONGO_HOST\n                        The MongoDB hostname\n  --mongo-port MONGO_PORT\n                        The MongoDB port\n  --mongo-user MONGO_USER\n                        The MongoDB username\n  --mongo-pass MONGO_PASS\n                        The MongoDB password\n```\n\n\n#### MongoDB Compass\n\n[MongoDB Compass](https://www.mongodb.com/products/tools/compass) provides a GUI for inspecting the `kibble` database.\n**Connect:**\n\n1. Install Compass from the [MongoDB download page](https://www.mongodb.com/try/download/compass).\n2. Start the database container: `cd simulator \u0026\u0026 docker compose up database -d`\n3. Paste this connection string:\n\n   ```\n   mongodb://root:password@localhost:27017\n   ```\n\n4. Open the **`kibble`** database.\n\nIf Compass asks for an authentication database, use **`admin`**.\n\n\n\n## Custom Daemon Installation\n### Linux\n   - Copy edge_node directory onto device to be monitored\n\n   ```\n   cd edge_node/linux/\n   ./install.sh\n   ```\n\n### Windows\n   - Copy edge_node directory onto device to be monitored\n   - Open Powershell as administrator\n\n   ```\n   cd edge_node/windows/\n   ./install.ps1\n   ```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgeeoon%2Fengine-kibble","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgeeoon%2Fengine-kibble","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgeeoon%2Fengine-kibble/lists"}