{"id":14963142,"url":"https://github.com/espressif/esp-insights","last_synced_at":"2025-04-05T00:06:45.671Z","repository":{"id":39641979,"uuid":"381931544","full_name":"espressif/esp-insights","owner":"espressif","description":"ESP Insights: A remote diagnostics/observability framework for connected devices","archived":false,"fork":false,"pushed_at":"2025-03-03T07:51:02.000Z","size":3202,"stargazers_count":119,"open_issues_count":22,"forks_count":34,"subscribers_count":10,"default_branch":"main","last_synced_at":"2025-03-28T23:04:47.770Z","etag":null,"topics":["debugging","diagnostics","esp32","observability"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/espressif.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-07-01T06:24:40.000Z","updated_at":"2025-03-19T07:54:12.000Z","dependencies_parsed_at":"2023-01-30T00:31:00.503Z","dependency_job_id":"1eee3e61-85a9-4ff6-acfc-6bb4c074ec2f","html_url":"https://github.com/espressif/esp-insights","commit_stats":{"total_commits":148,"total_committers":11,"mean_commits":"13.454545454545455","dds":0.6081081081081081,"last_synced_commit":"6e3bca50a15dfb04fcd71151dede0dd63edcf609"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/espressif%2Fesp-insights","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/espressif%2Fesp-insights/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/espressif%2Fesp-insights/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/espressif%2Fesp-insights/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/espressif","download_url":"https://codeload.github.com/espressif/esp-insights/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247266563,"owners_count":20910836,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["debugging","diagnostics","esp32","observability"],"created_at":"2024-09-24T13:31:04.628Z","updated_at":"2025-04-05T00:06:45.654Z","avatar_url":"https://github.com/espressif.png","language":"C","funding_links":[],"categories":["C"],"sub_categories":[],"readme":"# ESP Insights (Beta)\n\nESP Insights is a remote diagnostics solution that allows users to remotely monitor the health of ESP devices in the field.\n\n## Introduction\n\nDevelopers normally prefer debugging issues by physically probing them using gdb or observing the logs. This surely helps debug issues, but there are often cases wherein issues are seen only in specific environments under specific conditions. Even things like casings and placement of the product can affect the behaviour. A few examples are\n\n- Wi-Fi disconnections for a smart switch concealed in a wall.\n- Smart speakers crashing during some specific usage pattern.\n- Appliance frequently rebooting due to power supply issues.\n\nHaving remote diagnostics facility helps in identifying such issues faster. ESP Insights includes a firmware agent (the Insights agent) that captures some of the vital pieces of diagnostics information from the device during runtime and uploads them to the ESP Insights cloud. The cloud then processes this data for easy visualisation. Developers can log in to a web-based dashboard to look at the health and issues reported by their devices in the field. A sample screen is shown here.\n\n![Insights Overview](docs/_static/overview.png)\n\nDevelopers can monitor the following information from the web-based dashboard:\n\n- Error logs: Anything that is logged on console with calls with ESP_LOGE by any component in the firmware\n- Warning logs: Anything that is logged on console with calls with ESP_LOGW by any component in the firmware\n- Custom Events: Application specific custom events that the firmware wishes to track via calls to ESP_DIAG_EVENT\n- Reset reason: The reason why the device was reset (power on, watchdog, brownout, etc.)\n- Coredump summary: In case of a crash, the register contents as well as the stack backtrace of the offending thread (wherever possible)\n- Metrics: Time-varying data like the free heap size, the Wi-Fi signal strength that is plotted over a period of time\n- Variables: Variable values like the IP Address or state variables that report their current value\n- Group analytics: Insights into how group of devices are performing\n\nAll of this information should help the developer understand better how their device is performing in the field.\n\nYou can find more details on [Insights Features](FEATURES.md) page.\n\n\u003e ESP Insights currently works with the ESP Insights cloud and ESP RainMaker cloud. Support for other cloud services will be available in a subsequent release.\n\n## Getting Started\nFollowing code should get you started, and your application can start reporting ESP Insights data to the Insights cloud.\n\n### Enabling ESP-Insights\nBy default, `ESP-Insights` is disabled. User need to enable it via menuconfig option.\n`(Component config → ESP Insights)`\n\nTo select transport to be used, please follow next step.\n\n### Enabling Insights with HTTPS\nFor Insights agent HTTPS is configure as the default transport.\n\n```c\n#include \u003cesp_insights.h\u003e\n#define ESP_INSIGHTS_AUTH_KEY \"\u003cPaste-Auth-Key-Here\u003e\"\n\n{\n    esp_insights_config_t config  = {\n        .log_type = ESP_DIAG_LOG_TYPE_ERROR,\n        .auth_key = ESP_INSIGHTS_AUTH_KEY,\n    };\n\n    esp_insights_init(\u0026config);\n\n    /* Rest of the application initialization */\n}\n```\n\nAs you may have noticed, all you will need is the unique ESP_INSIGHTS_AUTH_KEY to be embedded in your firmware.\nHere is how you can obtain the ESP Insights Auth Key:\n* Sign up or Sign in on [ESP Insights Dashboard](https://dashboard.insights.espressif.com/)\n* Visit [Manage Auth Keys](https://dashboard.insights.espressif.com/home/manage-auth-keys) and generate an Auth Key\n* Copy the Auth Key to your firmware\n\n\n### Enabling Insights with MQTT\nConfigure the default insights transport to MQTT (Component config → ESP Insights → Insights default transport → MQTT).\nAlternatively, you can add `CONFIG_ESP_INSIGHTS_TRANSPORT_MQTT=y` to `sdkconfig.defaults`.\n\n```c\n#include \u003cesp_insights.h\u003e\n\n{\n    esp_insights_config_t config  = {\n        .log_type = ESP_DIAG_LOG_TYPE_ERROR,\n    };\n\n    esp_insights_init(\u0026config);\n\n    /* Rest of the application initialization */\n}\n```\nYou will require the MQTT certs which you can obtain by performing [Claiming](examples/minimal_diagnostics#esp-insights-over-mqtt).\n\nFor more details please head over to [examples](examples).\n\n## Behind the Scenes\n\n  - As described in the overview section, ESP-Insights collects data of your choice and stores it in storage, the storage currently used is RTC store.\n  - The data is split into two sections, and the size is configurable via menuconfig:\n    - Critical: Errors, Warnings and Events\n    - Non-Critical: Metrics and Variables\n  - This data is then periodically encoded in CBOR format and is sent over the transport.\n  - The data send period is dynamic within a range. It adapts to the need of the reporting.\n  - \u003e **_Note_** Please note that the crashdump data is erased from the flash partition of the device once it is sent to the Insights cloud. \u003cbr\u003e\n  So, in case you need to debug the crash locally, in which case you have physical access to the device, you should disable crashdump data reporting.\n\n### RTC data store\nESP-Insight currently uses RTC memory to store the messages until it is sent to the cloud. Unlike, normal RAM, RTC store makes data available across soft resets. Thus, it brodens the usefulness of the data across reboot.\n\nMessages in Critical data take `121` bytes per message. Hence, one can calculate, say a `2048` bytes of critical section can hold `16` messages before it starts dropping new messages.\nLikewise Non-critical messages take `49` bytes per message and hence a `1024` bytes storage can hold `21` messages.\n\nThe RTC memory is limited and flooding too many messages, makes RTC storage full. Once full, the framework has to drop the messages.\nOne solution is to increase the data reporting frequency to cloud, and thereby emptying the RTC store frequently.\n\u003e **_Note_**\nThis however comes with the network cost. It is advisable for a developer to keep Insights logging concise and not overdo it.\n\n## Contributing\n\nIf you find an issue with ESP-Insights, or wish to submit an enhancement request, please use the Issues section on Github.\nFor ESP-IDF related issues please use [esp-idf](https://github.com/espressif/esp-idf) repo.\n\n## License\n\nESP-Insights code is covered under Apache2 license. Submodules and other third party repos have their own license.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fespressif%2Fesp-insights","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fespressif%2Fesp-insights","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fespressif%2Fesp-insights/lists"}