{"id":15732861,"url":"https://github.com/tobozo/esp32-flocker","last_synced_at":"2025-05-06T23:15:49.111Z","repository":{"id":237465993,"uuid":"707674713","full_name":"tobozo/ESP32-Flocker","owner":"tobozo","description":"🏗️ App-suite builder for ESP32","archived":false,"fork":false,"pushed_at":"2023-10-20T15:43:02.000Z","size":608,"stargazers_count":3,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-05-06T23:15:41.347Z","etag":null,"topics":["arduino","arduino-cli","arduino-ide","esp32","m5stack-sd-updater","m5stackupdater","platformio"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tobozo.png","metadata":{"files":{"readme":"ReadMe.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-10-20T12:12:59.000Z","updated_at":"2025-01-10T22:21:55.000Z","dependencies_parsed_at":"2024-05-02T04:30:09.544Z","dependency_job_id":"737327ed-8ae8-47b0-847b-50555537a50f","html_url":"https://github.com/tobozo/ESP32-Flocker","commit_stats":null,"previous_names":["tobozo/esp32-flocker"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tobozo%2FESP32-Flocker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tobozo%2FESP32-Flocker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tobozo%2FESP32-Flocker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tobozo%2FESP32-Flocker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tobozo","download_url":"https://codeload.github.com/tobozo/ESP32-Flocker/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252782835,"owners_count":21803410,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["arduino","arduino-cli","arduino-ide","esp32","m5stack-sd-updater","m5stackupdater","platformio"],"created_at":"2024-10-04T00:23:25.746Z","updated_at":"2025-05-06T23:15:49.071Z","avatar_url":"https://github.com/tobozo.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ESP32-Flocker\n\n![](assets/stacked-stones.png)\n\n\nESP32-Flocker is a shell script used to concatenate several esp32 binaries and their application launcher into a single firmware.\n\nThe goal is to ease the creation of all-in-one-firmware application suites without having to manually edit the partitions.csv file\nor requiring the creation of a custom bootloader.\n\nIn this scenario, the factory partition which holds the applications launcher is near the end of the flash space, just before\nspiffs, OTA0 is available for flashing, and all space in between is used to stack the applications.\n\nExample of generated partition scheme with 16MB flash size:\n\n```csv\n  # 9 Apps + Factory\n  # Name,   Type,  SubType,  Offset,   Size,     Flags, Comment\n  nvs,      data,  nvs,        0x9000,   0x5000,,\n  otadata,  data,  ota,        0xe000,   0x2000,,\n  ota_0,    0,     ota_0,     0x10000, 0x200000,,       OTA0 (available slot)\n  ota_1,    0,     ota_1,    0x210000, 0x090000,,       WiFiManager\n  ota_2,    0,     ota_2,    0x2a0000, 0x1f0000,,       BLEScanner\n  ota_3,    0,     ota_3,    0x490000, 0x0c0000,,       I2CScanner\n  ota_4,    0,     ota_4,    0x550000, 0x090000,,       SerialBridge\n  ota_5,    0,     ota_5,    0x5e0000, 0x0c0000,,       HomeAssistant\n  ota_6,    0,     ota_6,    0x6a0000, 0x090000,,       Camera\n  ota_7,    0,     ota_7,    0x730000, 0x090000,,       Tetris\n  ota_8,    0,     ota_8,    0x7c0000, 0x110000,,       Pacman\n  ota_9,    0,     ota_9,    0x8d0000, 0x420000,,       Leftover space (available slot)\n  firmware, app,   factory,  0xcf0000, 0x100000,,       Factory Launcher\n  spiffs,   data,  spiffs,   0xdf0000, 0x200000,,\n  coredump, data,  coredump, 0xff0000, 0x010000,,.\n```\n\n## Build process\n\nThe shell script will:\n\n- Read a [JSON settings](#json-settings-format) file\n- Gather/build the binaries for every applications, including the launcher\n- Create a custom partition scheme\n- Generate the NVS data for the launcher\n- Merge all binaries\n- Flash the ESP\n\n**⚠️ All included applications must meet the [application requirements](#application-requirements).**\n\nℹ️ `applications[]` items listed in the [JSON settings](#json-settings-format) file and can be either of:\n\n  - Path/git-url to .bin file\n  - Path/git-url to arduino project source\n  - Path/git-url to platformio project source\n\nℹ️ Project sources will be compiled with [`arduino-cli`](https://github.com/arduino/arduino-cli) and [`platformio`](https://github.com/platformio/platformio-core) according to their nature.\n\nℹ️ Although the recommended application launcher with this shell script is [M5Stack-SD-Updater](https://github.com/tobozo/M5Stack-SD-Updater)'s [M5Stack-FW-Menu](https://github.com/tobozo/M5Stack-SD-Updater/tree/master/examples/M5Stack-FW-Menu) example, any\ncustom launcher will work as long as it's capable of enumerating the ota partitions and setting any of them as bootable.\n\n\n## JSON settings format\n\nThe bash script needs a JSON settings file to work, here's an example:\n\n```json\n{\n  \"name\": \"MyApplicationSuite\",\n  \"fqbn\": \"esp32:esp32:m5stack-cores3:DebugLevel=debug\",\n  \"build_properties\": \"-DMY_CUSTOM_FLAG\",\n  \"factory\":\n  {\n    \"name\": \"M5Stack-FW-Menu\",\n    \"path\": \"~/Arduino/libraries/M5Stack-SD-Updater/examples/M5Stack-FW-Menu\",\n    \"depends\": [ \"M5Stack-SD-Updater\", \"ESP32-Chimera-Core\" ]\n  },\n  \"applications\":\n  [\n    {\n      \"name\":\"WiFiManager\",\n      \"path\": \"/path/to/Arduino_Projects/WiFiManager/examples/SmartConfig\"\n    },\n    {\n      \"name\":\"BLEScanner\",\n      \"path\": \"https://github.com/tobozo/ESP32-BLECollector.git\",\n      \"depends\": [ \"SQLiteEsp32\", \"ESP32-Chimera-Core\", \"NimBLE-Arduino\" ]\n    },\n    {\n      \"name\":\"SerialBridge\",\n      \"path\": \"/path/to/Arduino_Projects/SerialBridge\",\n      \"depends\": [ \"SoftwareSerial\" ]\n    },\n    {\n      \"name\":\"HomeAssistant\",\n      \"path\": \"/path/to/Platformio_Projects/SmartHass#env-name\"\n    },\n    {\n      \"name\":\"Camera\",\n      \"path\": \"/path/to/Precompiled_Projects/Camera.bin\"\n    }\n  ]\n}\n```\n\n## Optional members:\n\n  - `build_properties`: additional compilation flags e.g. `-DMY_CUSTOM_FLAG` for **arduino-cli**\n  - `flash_size`: defaults to `16MB`\n  - `flash_freq`: defaults to `80m`\n  - `flash_mode`: defaults to `dio`\n  - `target_port`: defaults to `/dev/ttyUSB0`\n  - `baud_rate`: defaults to `921600`\n  - `flashfs_type`: data partition type, defaults to `littlefs`\n  - `flashfs_size`: data partition size, defaults to `2MB`\n  - `factory_partsize`: factory partition size, defaults to `1MB`\n  - `applications[].depends`: list of **arduino-cli** library dependencies for a given app, assumed as all satisfied if empty or missing\n  - `verify`: grep sketches and launcher source to verify that M5StackUpdater library is used, defaults to `true`\n\n\n\n## CI vs desktop use\n\nUncommenting the `directories` entry in `arduino-cli.yml` file **only makes sense if** no local installation of Arduino IDE exists\ne.g. a CI is running the bash script and needs to cache downloads on a different volume to save bandwidth\n\nIn any other situation it is safe to keep the `directories` block commented out.\n\n\n## Application requirements\n\nEvery `/data/` folders from the listed applications will be merged into a single filesystem image, and only one of spiffs/littlefs/fatfs\ncan be used (default is littlefs), obviously every application using the flash filesystem will have to agree on this.\n\nEvery application must provide a way to load the factory partition, this can be achieved by using [`M5Stack-SD-Updater`](https://github.com/tobozo/M5Stack-SD-Updater) library:\n\n- With `M5Stack-SD-Updater` and through the lobby by calling `checkFWUpdater()` instead of the usual `checkSDUpdater()`\n- With `M5Stack-SD-Updater` outside the lobby by calling `Flash::loadFactory()`\n- Without `M5Stack-SD-Updater` by implementing this function:\n\n\n```cpp\n\n#include \"esp_ota_ops.h\"\n#include \"bootloader_common.h\"\n\nvoid loadFactory()\n{\n  auto factory = esp_partition_find( ESP_PARTITION_TYPE_APP,  ESP_PARTITION_SUBTYPE_APP_FACTORY, NULL );\n  if( !factory ) {\n    log_e( \"this partitions scheme has no factory partition\");\n    return;\n  }\n\n  auto factory_partition = esp_partition_get(factory);\n\n  if( !factory_partition ) {\n    log_e( \"Failed to find factory partition\" );\n    return;\n  }\n  // Set partition for boot\n  auto err = esp_ota_set_boot_partition ( factory_partition );\n  // restart on success\n  if ( err != ESP_OK ) {\n    log_e( \"Failed to set boot partition\" ) ;\n  } else {\n    log_i(\"Will reboot to factory partition\");\n    esp_restart() ;\n  }\n}\n```\n\n\n\n## System Prerequisites:\n\n  - linux\n  - bash\n  - git\n  - php\n  - platformio\n\n\n## Credits/thanks\n\n  - [Kongduino](https://github.com/Kongduino)\n  - [zeromem](https://twitter.com/zeromem0)\n  - [F4HWN](https://github.com/armel)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftobozo%2Fesp32-flocker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftobozo%2Fesp32-flocker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftobozo%2Fesp32-flocker/lists"}