{"id":15629781,"url":"https://github.com/tobozo/esp32-blecollector","last_synced_at":"2025-04-06T03:07:23.631Z","repository":{"id":39838651,"uuid":"154091767","full_name":"tobozo/ESP32-BLECollector","owner":"tobozo","description":"ᛡᛒ BLE Scanner + Data persistence on SD Card for M5Stack, M5Core2, Odroid-Go, ESP32-Wrover-Kit and other models","archived":false,"fork":false,"pushed_at":"2024-09-04T10:51:05.000Z","size":3156,"stargazers_count":308,"open_issues_count":1,"forks_count":31,"subscribers_count":12,"default_branch":"master","last_synced_at":"2025-03-30T02:05:42.201Z","etag":null,"topics":["arduino","ble","ble-scanner","blescan","blescanner","bluetooth","esp-wrover-kit","esp32","esp32-chimera-core","esp32-wrover","ili9341","low-energy","m5stack","m5stack-sd-updater","odroid-go","scanner","sd-card","sqlite3","sqlite3-database","st7789"],"latest_commit_sha":null,"homepage":"https://youtu.be/G07loGS04Vg","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tobozo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-10-22T05:31:45.000Z","updated_at":"2025-03-29T05:36:23.000Z","dependencies_parsed_at":"2024-01-05T20:59:15.493Z","dependency_job_id":"a7e460b0-da73-45d9-abab-86f636dbc7d3","html_url":"https://github.com/tobozo/ESP32-BLECollector","commit_stats":null,"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tobozo%2FESP32-BLECollector","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tobozo%2FESP32-BLECollector/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tobozo%2FESP32-BLECollector/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tobozo%2FESP32-BLECollector/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tobozo","download_url":"https://codeload.github.com/tobozo/ESP32-BLECollector/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247427006,"owners_count":20937201,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["arduino","ble","ble-scanner","blescan","blescanner","bluetooth","esp-wrover-kit","esp32","esp32-chimera-core","esp32-wrover","ili9341","low-energy","m5stack","m5stack-sd-updater","odroid-go","scanner","sd-card","sqlite3","sqlite3-database","st7789"],"created_at":"2024-10-03T10:28:45.222Z","updated_at":"2025-04-06T03:07:23.611Z","avatar_url":"https://github.com/tobozo.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ESP32-BLECollector\n\n[![Join the chat at https://gitter.im/ESP32-BLECollector/ESP32-BLECollector](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/ESP32-BLECollector/ESP32-BLECollector)\n[![Build Status](https://travis-ci.com/tobozo/ESP32-BLECollector.svg?branch=master)](https://travis-ci.com/github/tobozo/ESP32-BLECollector)\n\nA BLE Scanner with persistence.\n\n  ![ESP32 BLECollector running on Wrover-Kit](https://user-images.githubusercontent.com/1893754/81865372-1b965680-956e-11ea-9f2d-448c2330f3d3.png) ![ESP32 BLECollector running on M5Stack](https://raw.githubusercontent.com/tobozo/ESP32-BLECollector/unstable/screenshots/BLECollector-M5Stack.jpeg)\n\n🎬 [Demo video](https://youtu.be/w5V80PobVWs)\n------------\n\nBLECollector is just a passive BLE scanner with a fancy UI.\nAll BLE data found by the BLE Scanner is collected into a [sqlite3](https://github.com/siara-cc/esp32_arduino_sqlite3_lib) format on the SD Card.\n\nPublic Mac addresses are compared against [OUI list](https://code.wireshark.org/review/gitweb?p=wireshark.git;a=blob_plain;f=manuf), while Vendor names are compared against [BLE Device list](https://www.bluetooth.com/specifications/assigned-numbers/company-identifiers).\n\nThose two database files are provided in a db format ([mac-oui-light.db](https://github.com/tobozo/ESP32-BLECollector/blob/master/SD/mac-oui-light.db) and [ble-oui.db](https://github.com/tobozo/ESP32-BLECollector/blob/master/SD/ble-oui.db)).\n\nOn first run, a default `blemacs.db` file is created, this is where BLE data will be stored.\nWhen a BLE device is found by the scanner, it is populated with the matching oui/vendor name (if any) and eventually inserted in the `blemasc.db` file.\n\n⚠️ This sketch is big! Use the \"No OTA (Large Apps)\" or \"Minimal SPIFFS (Large APPS with OTA)\" partition scheme to compile it.\nThe memory cost of using sqlite and BLE libraries is quite high.\n\n⚠️ Builds using ESP32-Wrover can eventually choose the 3.6MB SPIFFS partition scheme, and have the BLECollector working without the SD Card. Experimental support only since SPIFFS tends to get slower and buggy when the partition becomes full.\n\n\nHardware requirements\n---------------------\n  - [mandatory] ESP32-Wroom or ESP32-Wrover (Wrover is recommended)\n  - [mandatory] SD Card (breakout or bundled in Wrover-Kit, M5Stack, Odroid-Go, LoLinD32 Pro)\n  - [mandatory] Micro SD (FAT32 formatted, **max 4GB**)\n  - [mandatory] [mac-oui-light.db](https://github.com/tobozo/ESP32-BLECollector/blob/master/SD/mac-oui-light.db) and [ble-oui.db](https://github.com/tobozo/ESP32-BLECollector/blob/master/SD/ble-oui.db) files copied on the Micro SD Card root\n  - [mandatory] ST7789/ILI9341 320x240 TFT (or bundled in Wrover-Kit, M5Stack, Odroid-Go, LoLinD32 Pro, D-Duino32-XS)\n  - [optional] (but recommended) I2C RTC Module (see `#define HAS_EXTERNAL_RTC` in Settings.h)\n  - [optional] Serial GPS Module (see `#define HAS_GPS` in Settings.h)\n  - [⚠ NEW][optional] [XPad Buttons Shield](https://www.tindie.com/products/deshipu/x-pad-buttons-shield-for-d1-mini-version-60/) from [Radomir Dopieralski](https://github.com/deshipu)\n\nSoftware requirements (updated)\n---------------------\n  - [mandatory] Arduino IDE\n  - [mandatory] [ESP32-Chimera-Core](https://github.com/tobozo/ESP32-Chimera-Core/) (get it from the Arduino Library Manager)\n  - [mandatory] [LovyanGFX](https://github.com/Lovyan03/LovyanGFX/) (get it from the Arduino Library Manager)\n  - [mandatory] [M5Stack-SD-Updater](https://github.com/tobozo/M5Stack-SD-Updater) (get it from the Arduino Library Manager)\n  - [mandatory] [NimBLE Library](https://github.com/h2zero/NimBLE-Arduino/archive/master.zip) supersedes the BLE (legacy or custom) library versions, install it manually in the Arduino/Libraries folder.\n  - [mandatory] [PaulStoffregen's Time library](https://github.com/PaulStoffregen/Time) (get it from the Arduino Library Manager)\n  - [mandatory] [esp32_arduino_sqlite3_lib](https://github.com/siara-cc/esp32_arduino_sqlite3_lib) (get it from the Arduino Library Manager)\n  - [optional] [TinyGPSPlus](https://github.com/mikalhart/TinyGPSPlus)\n\nBehaviours (auto-selected except for WiFi):\n---------------------------\n  - **Hobo**: when no TinyRTC module exists in your build, only uptime will be displayed\n  - **Rogue**: TinyRTC module adjusted after flashing (build DateTime), shares time over BLE\n  - **Chronomaniac**: TinyRTC module adjusts itself via GPS, shares time over BLE\n  - **With WiFi**: Temporary dual BLE/WiFi mode to allow downloading or serving .db files, see `#define WITH_WIFI` in `Settings.h`\n\nOptional I2C RTC Module requirements\n------------------------------------\n  - Wire your TinyRTC to RTC_SDA/RTC_SCL (see `Settings.h` or `Display.h` to override)\n  - Insert the SD Card\n  - Set `#define HAS_EXTERNAL_RTC true` in [Settings.h](https://github.com/tobozo/ESP32-BLECollector/blob/master/Settings.h)\n  - Flash the ESP with partition scheme `Minimal SPIFFS (Large APPS with OTA)`\n\nOptional Serial GPS Module requirements\n---------------------------------------\n  - Wire your GPS module to TX1/RX1 (edit `GPS_RX` and `GPS_TX` in GPS.h\n  - Set `#define HAS_GPS true` in [Settings.h](https://github.com/tobozo/ESP32-BLECollector/blob/master/Settings.h)\n  - Flash the ESP with partition scheme `Minimal SPIFFS (Large APPS with OTA)`\n  - Wait for the GPS to find a fix\n  - issue the command `gpstime` in the serial console\n\nOptional XPad Buttons Shield requirements\n-----------------------------------------\n  - Wire your XPad Buttons Shield to XPAD_SDA/XPAD_SCL  (see `HID_XPad.h` to override)\n  - Enable the module in `Display.h` : `#define hasXPaxShield() (bool) true`\n  - Controls are:\n    - Down / Up : brightness\n    - Right / Left : unassigned (yet)\n    - A : start/stop scan\n    - B / C : toggle mac filter\n    - D : unassigned (yet)\n\nTime Sharing\n------------\n  - Once the time is set using RTC, GPS or NTP, the BLECollector may start the TimeSharing service and advertise a DateTime characteristic for other BLECollectors to sync with.\n  - Builds with no RTC/GPS will try to identify this service during their scan duty cycle and subscribe for notifications.\n\nFile Downloading (still experimental)\n------------\n\nSending the `DownloadDB` command will:\n\n  - Stop BLE\n  - Start WiFi\n  - Synchronize time to a nearby NTP server\n  - Download the latest oui/vendors database from github\n\n\nSerial command interface\n------------\n\n  Available Commands:\n\n    01)             help : Print this list\n    02)             halp : Same as help except it doesn't print anything\n    03)            start : Start/resume scan\n    04)             stop : Stop scan\n    05)     toggleFilter : Toggle vendor filter on the TFT (persistent)\n    06)       toggleEcho : Toggle BLECards in the Serial Console (persistent)\n    07)      setTimeZone : Set the timezone for next NTP Sync (persistent)\n    08)    setSummerTime : Toggle CEST / CET for next NTP Sync (persistent)\n    09)             dump : Dump returning BLE devices to the display and updates DB\n    10)    setBrightness : Set brightness to [value] (0-255) (persistent)\n    11)               ls : Show [dir] Content on the SD\n    12)               rm : Delete [file] from the SD\n    13)          restart : Restart BLECollector ('restart now' to skip replication)\n    14)       screenshot : Make a screenshot and save it on the SD\n    15)       screenshow : Show screenshot\n    16)           toggle : toggle a bool value\n    17)          resetDB : Hard Reset DB + forced restart\n    18)          pruneDB : Soft Reset DB without restarting (hopefully)\n    19)         bleclock : Broadcast time to another BLE Device (implicit)\n    20)          bletime : Get time from another BLE Device (explicit)\n    21)          gpstime : Sync time from GPS\n    22)           latlng : Print the GPS lat/lng\n    23)          stopBLE : Stop BLE (use 'restart' command to re-enable)\n    24)        startWiFi : Start WiFi (will stop BLE)\n    25)      setPoolZone : Set NTP Pool Zone for next NTP Sync (persistent)\n    26)          NTPSync : Update time from NTP (will start WiFi)\n    27)       DownloadDB : Download or update db files (will start WiFi and update NTP first)\n    28)      setWiFiSSID : Set WiFi SSID\n    29)      setWiFiPASS : Set WiFi Password\n\n\nContributions are welcome :-)\n\n\nKnown issues / Roadmap\n----------------------\n\nImplementing both [LovyanGFX](https://github.com/lovyan03/LovyanGFX) and [Nimble-Arduino](https://github.com/h2zero/NimBLE-Arduino) was such a huge optimization that none of the previous blockers exist any more!\n\nSome ideas I'll try to implement in the upcoming changes:\n\n- Add GPS Coords to entries for better pruning [as suggested by /u/playaspect](https://www.reddit.com/r/esp8266/comments/9s594c/esp32blecollector_ble_scanner_data_persistence_on/e8nipr6/?context=3)\n- Better Analysis of ServiceData (see @reelyactive's [advlib](https://github.com/reelyactive/advlib))\n- Extended logging (SDCard-less meshed builds)\n\n\nOther ESP32 security related tools:\n-----------------------------------\n\n  - https://github.com/cyberman54/ESP32-Paxcounter\n  - https://github.com/G4lile0/ESP32-WiFi-Hash-Monster\n  - https://github.com/justcallmekoko/ESP32Marauder\n\n\nCredits/requirements:\n---------------------\n\n- @Lovyan03 for integrating his [LovyanGFX](https://github.com/lovyan03/LovyanGFX) into the [ESP32-Chimera-Core](https://github.com/tobozo/ESP32-Chimera-Core/tree/lgfx_test) thus saving an enormous amount of sram and flash space\n- @h2zero for sharing [NimBLE Library](https://github.com/h2zero/NimBLE-Arduino/) and brillantly proving that BLE can work with WiFi on Arduino without eating all sram/flash space\n- https://github.com/siara-cc/esp32_arduino_sqlite3_lib\n- huge thanks to https://github.com/chegewara for maintaining the initial [BLE library](https://github.com/tobozo/ESP32-BLECollector/releases/download/1.2/BLE.zip) that made this project possible\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftobozo%2Fesp32-blecollector","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftobozo%2Fesp32-blecollector","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftobozo%2Fesp32-blecollector/lists"}