{"id":30102873,"url":"https://github.com/nyu-mlab/inspector-core-library","last_synced_at":"2026-04-16T00:05:05.101Z","repository":{"id":280511443,"uuid":"937405844","full_name":"nyu-mlab/inspector-core-library","owner":"nyu-mlab","description":"Library for core functionalities of IoT Inspector","archived":false,"fork":false,"pushed_at":"2026-03-04T05:08:14.000Z","size":15314,"stargazers_count":1,"open_issues_count":2,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-03-04T08:35:11.778Z","etag":null,"topics":["internet-of-things","iot","iot-device","iot-platform","raspberry-pi","raspberrypi"],"latest_commit_sha":null,"homepage":"https://inspector.engineering.nyu.edu/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nyu-mlab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-02-23T00:54:48.000Z","updated_at":"2026-03-04T03:23:05.000Z","dependencies_parsed_at":"2026-02-16T03:03:58.999Z","dependency_job_id":null,"html_url":"https://github.com/nyu-mlab/inspector-core-library","commit_stats":null,"previous_names":["nyu-mlab/inspector-core-library"],"tags_count":23,"template":false,"template_full_name":null,"purl":"pkg:github/nyu-mlab/inspector-core-library","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nyu-mlab%2Finspector-core-library","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nyu-mlab%2Finspector-core-library/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nyu-mlab%2Finspector-core-library/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nyu-mlab%2Finspector-core-library/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nyu-mlab","download_url":"https://codeload.github.com/nyu-mlab/inspector-core-library/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nyu-mlab%2Finspector-core-library/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30375576,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-11T06:09:32.197Z","status":"ssl_error","status_checked_at":"2026-03-11T06:09:17.086Z","response_time":84,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["internet-of-things","iot","iot-device","iot-platform","raspberry-pi","raspberrypi"],"created_at":"2025-08-09T20:25:57.249Z","updated_at":"2026-04-16T00:05:05.090Z","avatar_url":"https://github.com/nyu-mlab.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# inspector-core-library\n[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)\n[![libinspector_test](https://github.com/nyu-mlab/inspector-core-library/actions/workflows/libinspector_test.yml/badge.svg)](https://github.com/nyu-mlab/inspector-core-library/actions/workflows/libinspector_test.yml)\n[![codecov](https://codecov.io/gh/nyu-mlab/inspector-core-library/graph/badge.svg?token=TBYF5KVBT7)](https://codecov.io/gh/nyu-mlab/inspector-core-library)\n\nLibrary for core functionalities of IoT Inspector\n\n## Installation\n\nTo install the `libinspector` module via pip, use the following command:\n\n```sh\npip install libinspector\n```\n\n\n## Usage\n\n### Running the Inspector\n\nFor debugging purposes, you can also set the following environment variables to control the behavior of the Inspector Core:\n\n| Variable           | Description                                                                                          | Default |\n|:-------------------|:-----------------------------------------------------------------------------------------------------|:--------|\n| `USE_IN_MEMORY_DB` | Set to `false` to use a physical `.db` file on disk. Useful for debugging the core library/database. | `true`  |\n| `SCAN_ALL_DEVICES` | Set to `true` to ARP-spoof all devices on the network BY DEFAULT. Disabled by default.               | `false` |\n| `ARP_SPOOF_ROUTER` | Set to `false` to NOT ARP-spoof the router.                                                          | `true`  |\n| `ARP_SPOOF_DEVICE` | Set to `false` to NOT ARP-spoof the device.                                                          | `true`  |\n\nTo run the Inspector, you need to activate the virtual environment first and then run the following command (You need to pass environment variables here too):\n\n```sh\nsudo USE_IN_MEMORY_DB=false SCAN_ALL_DEVICES=true PYTHONPATH=~/.local/lib/python3.11/site-packages python3 -m libinspector.core\n```\n\n#### How to set environment variables (Linux/macOS):\n```bash\nexport USE_IN_MEMORY_DB=false\nexport SCAN_ALL_DEVICES=true\nexport ARP_SPOOF_ROUTER=false\nexport ARP_SPOOF_DEVICE=false\n```\n\n#### How to set environment variables (Windows):\n```powershell\n$env:USE_IN_MEMORY_DB = \"false\"\n$env:SCAN_ALL_DEVICES = \"true\"\n$env:ARP_SPOOF_ROUTER = \"false\"\n$env:ARP_SPOOF_DEVICE = \"false\"\n```\n\n### Embedding in Your Own Python Application\n\nThe preferred way to use `libinspector` is to embed it within your own Python application. You can do this by importing `libinspector.core` and calling the `start_threads()` method, which returns almost instantaneously. Your Python script will then need to read the in-memory SQLite database for information about the devices and the network traffic flows.\n\n```python\nimport time\nimport libinspector.core\nimport libinspector.global_state\n\n# This method returns almost instantaneously\nlibinspector.core.start_threads()\n\n# Make sure to sleep and/or do other work here, such as analyzing the in-memory SQLite database. For example, you can keep printing the device list from the `devices` table.\ndb_conn, rwlock = libinspector.global_state.db_conn_and_lock\n\nwhile True:\n    with rwlock:\n        for device in db_conn.execute('SELECT mac_address, ip_address FROM devices').fetchall():\n            print(f'MAC: {device[\"mac_address\"]}, IP: {device[\"ip_address\"]}')\n    time.sleep(5)\n\n```\n\nIf you want to add additional packet parsing capabilities, you can specific a custom callback when you start Inspector. Here's an example that prints out the summary of each captured packet:\n\n```python\nimport libinspector\nlibinspector.core.start_threads(\n  custom_packet_callback_func=lambda pkt: print(f'Packet captured: {pkt.summary()}')\n)\n\n```\n\n### Data Schema\n\nThe data schema is defined in `mem_db.py` and includes the following tables:\n\n- `devices`: Stores information about devices on the network.\n  - `mac_address` (TEXT, PRIMARY KEY): The MAC address of the device.\n  - `ip_address` (TEXT, NOT NULL): The IP address assigned to the device.\n  - `is_inspected` (INTEGER, DEFAULT 0): Indicates whether the device is being inspected (1) or not (0).\n  - `is_gateway` (INTEGER, DEFAULT 0): Indicates whether the device is a gateway (1) or not (0).\n  - `updated_ts` (INTEGER, DEFAULT 0): The timestamp of the last update.\n  - `metadata_json` (TEXT, DEFAULT '{}'): Additional metadata in JSON format.\n\n- `hostnames`: Stores hostnames associated with IP addresses.\n  - `ip_address` (TEXT, PRIMARY KEY): The IP address associated with the hostname.\n  - `hostname` (TEXT, NOT NULL): The hostname of the device.\n  - `updated_ts` (INTEGER, DEFAULT 0): The timestamp of the last update.\n  - `data_source` (TEXT, NOT NULL): The source of the hostname data.\n  - `metadata_json` (TEXT, DEFAULT '{}'): Additional metadata in JSON format.\n\n- `network_flows`: Stores information about network flows.\n  - `timestamp` (INTEGER): The timestamp of the network flow.\n  - `src_ip_address` (TEXT): The source IP address of the flow.\n  - `dest_ip_address` (TEXT): The destination IP address of the flow.\n  - `src_hostname` (TEXT): The source hostname of the flow.\n  - `dest_hostname` (TEXT): The destination hostname of the flow.\n  - `src_mac_address` (TEXT): The source MAC address of the flow.\n  - `dest_mac_address` (TEXT): The destination MAC address of the flow.\n  - `src_port` (TEXT): The source port of the flow.\n  - `dest_port` (TEXT): The destination port of the flow.\n  - `protocol` (TEXT): The protocol used in the flow.\n  - `byte_count` (INTEGER, DEFAULT 0): The number of bytes transferred in the flow.\n  - `packet_count` (INTEGER, DEFAULT 0): The number of packets transferred in the flow.\n  - `metadata_json` (TEXT, DEFAULT '{}'): Additional metadata in JSON format.\n  - PRIMARY KEY (`timestamp`, `src_mac_address`, `dest_mac_address`, `src_ip_address`, `dest_ip_address`, `src_port`, `dest_port`, `protocol`): The composite primary key for the table.\n\n### How `libinspector` Works\n\nThe `libinspector` module works by starting various threads to monitor and inspect network traffic. Here is a high-level overview of the `start_threads` function in `core.py`:\n\n1. **Ensure Single Instance**: The function first ensures that only one instance of the Inspector core is running.\n2. **Initialize Database**: It initializes the database by calling `mem_db.initialize_db()`.\n3. **Initialize Networking Variables**: It enables IP forwarding and updates the network information.\n4. **Start Threads**: It starts several threads to perform various tasks:\n   - Update network info from the OS every 60 seconds.\n   - Discover devices on the network every 10 seconds.\n   - Collect and process packets from the network.\n   - Spoof internet traffic.\n   - Start the mDNS and UPnP scanner threads.\n\n\n### Testing and Development\n\nTo test locally, run these commands:\n\n```\npython3 -m venv .venv\nsource .venv/bin/activate\npip install --upgrade pip\npip install .\n```\n\n\n## Notes\n\nTODO:\n - Create more test cases to obtain higher code coverage.\n\n\n## Contributing\n\nContributions are welcome! Please feel free to submit a pull request or open an issue.\n\n## License\n\nThis project is licensed under the Apache 2.0 License. See the LICENSE file for details.\n\n## Contact\n\nAsk Prof. Danny Y. Huang (dhuang@nyu.edu) or Andrew Quijano (andrew.quijano@nyu.edu).\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnyu-mlab%2Finspector-core-library","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnyu-mlab%2Finspector-core-library","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnyu-mlab%2Finspector-core-library/lists"}