Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/klausj1/homeassistant-statistics

This HA integration allows to import and export long term statistics from/to a file like csv or tsv
https://github.com/klausj1/homeassistant-statistics

home-assistant-integration python statistics

Last synced: 3 months ago
JSON representation

This HA integration allows to import and export long term statistics from/to a file like csv or tsv

Awesome Lists containing this project

README 

          
# Import Statistics Integration



A Home Assistant custom integration to import and export long-term statistics from CSV, TSV, or JSON files.



[![GitHub Release][releases-shield]][releases]

[![GitHub Activity][commits-shield]][commits]

[![License][license-shield]](LICENSE)

[![Community Forum][forum-shield]][forum]



> **Note:** This integration provides actions only (no entities or dashboard cards). You call its actions from Developer Tools or automations.



## Quick Links



- [Installation](#installation) | [Importing](#importing-statistics) | [Exporting](#exporting-statistics) | [Inventory](#exporting-statistics-inventory) | [Troubleshooting Tips](./docs/user/troubleshooting-tips.md)

- [Counter Statistics Explained](./docs/user/counters.md#understanding-counter-statistics-sumstate) | [Delta Import](./docs/user/counters.md#delta-import) | [Inventory Categories](./docs/user/export_inventory_categories.md)

- [Debug Logging Guide](./docs/user/debug-logging.md) - How to enable debug logs for troubleshooting



This is the user guide. If you are a developer, check the [Developer Documentation](./docs/dev/README.md).



## Requirements



- Home Assistant 2026.1.0 or newer



## Installation



### Option 1: HACS (Recommended)



1. Install via HACS, or click: [![Open HACS Repository][hacs-repo-badge]][hacs-repo]

2. Restart Home Assistant

3. Add the integration: [![Add Integration][config-flow-badge]][config-flow]



### Option 2: Manual



1. Download all files from `custom_components/import_statistics/` in this repository

2. Copy them to `/custom_components/import_statistics/` (create folders if needed)

3. Add `import_statistics:` to your `configuration.yaml`

4. Restart Home Assistant



## Available Actions



| Action                                | Description                                    |

| ------------------------------------- | ---------------------------------------------- |

| `import_statistics.import_from_file`  | Import statistics from a CSV/TSV file          |

| `import_statistics.import_from_json`  | Import statistics from JSON (UI or API)        |

| `import_statistics.export_statistics` | Export statistics to CSV/TSV or JSON           |

| `import_statistics.export_inventory`  | Export metadata inventory of all statistics    |



> As this integration uses database-independent methods, it works with all databases supported by Home Assistant.



---



## Importing Statistics



Import your statistics from CSV, TSV, or JSON files to populate or update Home Assistant's long-term statistics database.



### How to Import



1. Copy your file to your Home Assistant config folder

2. Go to **Developer Tools → Actions**

3. Select `import_statistics: import_from_file` (or `import_from_json`)

4. Fill in the settings (from the UI or YAML)

5. Click `perform action` to start the import.



### Settings Description



- **`filename` (required)**

  - Input file name (relative to Home Assistant config directory).

  - Supported:

    - `.csv` or `.tsv` for CSV/TSV import

    - `.json` for JSON import (use `import_from_json` action)

- **`delimiter` (optional)**

  - Delimiter between columns for CSV/TSV import.

  - If omitted, it is inferred from the filename extension:

    - `.csv` uses comma (`,`) by default

    - `.tsv` uses tab (`\t`) by default

  - Options: `\t` (tab), `;` (semicolon), `,` (comma), `|` (pipe), or custom value.

- **`decimal` (required, default: `"."`)**

  - Decimal separator character: `"."` for dot or `","` for comma.

- **`datetime_format` (optional, default: `%d.%m.%Y %H:%M`)**

  - Input format of the `datetime` strings in the file.

  - Default: `DD.MM.YYYY HH:MM` (e.g., `17.03.2024 02:00`)

  - Common formats: `%d.%m.%Y %H:%M`, `%Y.%m.%d %H:%M`, `%Y-%m-%d %H:%M`, `%m/%d/%Y %H:%M`, `%d/%m/%Y %H:%M`

- **`timezone_identifier` (optional)**

  - Defaults to Home Assistant's configured timezone if omitted. Typically can be left empty.

  - Timezone identifier (check pytz timezones or ).



> **Important:** Input files **must** contain a `unit` column with the unit of measurement for each statistic. For existing statistics, the unit in the input file must match the unit already stored in Home Assistant's database, otherwise the import will fail with an error.

>

> **Note:** Timestamps must be full hours (minutes must be `:00`). Timestamps are interpreted in the specified timezone.

>

> **Performance note**: Importing large datasets may take time as the operation is synchronous. The action completes when all data is saved into the database.



#### Example using the UI



![import from file](assets/service_import_ui.png)



#### Examples using YAML



##### Import from TSV file



```yaml

action: import_statistics.import_from_file

data:

  filename: my_statistics.tsv

  delimiter: \t

  decimal: "."

```



##### Import from CSV file with custom format



```yaml

action: import_statistics.import_from_file

data:

  filename: my_statistics.csv

  delimiter: ","

  decimal: "."

  datetime_format: "%Y-%m-%d %H:%M"

  timezone_identifier: "Europe/Vienna"

```



##### Import from JSON



```yaml

action: import_statistics.import_from_json

data:

  statistics: 

```



### File Format Requirements



Your file must contain one type of statistics:



- **Measurements (state_class == measurement or measurement_angle)** (temperature, humidity, direction,etc.): columns `min`, `max`, `mean`

- **Counters (state_class == total or total_increasing)** (energy, water meters, etc.): columns `sum`, `state` (or `delta`)



> **Before importing counters, make sure to read** [Understanding counter statistics in Home Assistant](./docs/user/counters.md)

> For importing counters, it is **recommended to use the import with the delta column** instead of importing sum/state, see [Delta Import](./docs/user/counters.md#delta-import)



Example files:



- [Measurements (min/max/mean)](./assets/min_max_mean.tsv)

- [Counters (sum/state)](./assets/state_sum.tsv)

- [Counters (delta)](./assets/delta.tsv)



#### Required Columns



Only these columns are accepted (unknown columns cause an error):



| Column               | Required          | Description                          |

| -------------------- | ----------------- | ------------------------------------ |

| `statistic_id`       | Yes               | The entity identifier                |

| `start`              | Yes               | Timestamp                            |

| `unit`               | Sometimes         | Required for external statistics     |

| `min`, `max`, `mean` | For measurements  | Cannot mix with counter columns      |

| `sum`, `state`       | For counters      | Cannot mix with measurement columns  |

| `delta`              | For counters      | Alternative to sum/state (see below) |



#### Statistic ID Format



| Type         | Format                   | Example                  | When to use                                |

| ------------ | ------------------------ | ------------------------ | ------------------------------------------ |

| **Internal** | `domain.name` (with `.`) | `sensor.temperature`     | For existing Home Assistant entities       |

| **External** | `domain:name` (with `:`) | `sensor:imported_energy` | For external (custom/synthetic) statistics |



> Internal statistics must match an existing entity.



#### File Encoding



- **Encoding**: UTF-8 (required for special characters like m³ or °C)

- **Delimiter**: Tab (default), comma, semicolon, or pipe

- **Decimal separator**: `.` (default) or `,`



### Import Behavior



- **Overwrites existing data**: Importing the same timestamps replaces old values

- **Gaps are preserved**: Missing hours will show as gaps in graphs

- **Synchronous operation**: The action completes when all data is saved into the database. This can take a longer time for large input data

- **Validation errors**: Shown directly in the UI; check logs if import fails silently



> If importing does not work and you do not get an error directly in the GUI, but there is an error in the Home Assistant logs, then this is a bug. This happens if the integration misses some checks, which lead to import errors later. Please create an issue.



### Data Validation



The integration performs strict validation on all import data:



- **All rows must be valid** - If any row contains invalid data, the entire import fails

- **Clear error messages** - The error message will indicate which row failed and why

- **No partial imports** - Either all data is imported successfully, or none of it is

- **Common validation errors**:

  - Invalid timestamp format or non-full-hour timestamps

  - Invalid numeric values (non-numeric strings, NaN, empty values)

  - Constraint violations (e.g., min > max for measurement data)

  - Missing required columns



This strict validation ensures data integrity and helps you identify and fix data quality issues immediately.



### JSON Import



You can also import via JSON, either through the UI or the Home Assistant API.



Example format: [state_sum.json](./assets/state_sum.json)



**Via API:**



```http

POST https:///api/services/import_statistics/import_from_json

Content-Type: application/json



```



---



## Exporting Statistics



Export your statistics to a file e.g. for backup, analysis, preparing a counter import with delta, or transfer to another Home Assistant instance.



### How to Export



1. Go to **Developer Tools → Actions**

2. Select `import_statistics: export_statistics`

3. Fill in the settings (from the UI or YAML)

4. Click `perform action` to start the export.



### Settings Description



- **`filename` (required)**

  - Output file name (relative to Home Assistant config directory).

  - Supported:

    - `.json` for JSON export

    - `.csv` or `.tsv` for TSV/CSV export

- **`entities` (optional)**

  - List of statistic IDs or entity IDs to export. Make sure to use a YAML list with `-`

  - Leave empty to export all available statistics.

  - Supports wildcard patterns using `*` (glob matching) inside an entry:

    - `sensor.paris_*` (prefix)

    - `sensor.*_temperature` (suffix)

    - `sensor.*temp*` (contains)

  - Constraints:

    - `*` alone is not allowed

    - Broad patterns like `sensor.*` / `sensor:*` must not be mixed with other entries (omit `entities` to export all)

- **`start_time` (optional)**

  - Start of the export range format: `%Y-%m-%d %H:%M:%S` ( `YYYY-MM-DD HH:MM:SS` ). Make sure you use quotes around the string.

  - Must be a full hour (`MM:SS` must be `00:00`).

  - If omitted, export starts from the earliest available long-term (hourly) statistic.

- **`end_time` (optional)**

  - End of the export range format: `%Y-%m-%d %H:%M:%S` ( `YYYY-MM-DD HH:MM:SS` ). Make sure you use quotes around the string.

  - Must be a full hour (`MM:SS` must be `00:00`).

  - If omitted, export ends at the most recent available long-term (hourly) statistic.

- **`timezone_identifier` (optional)**

  - Defaults to Home Assistant's configured timezone if omitted. Typically can be left empty.

  - Timezone identifier (check pytz timezones or ).

- **`datetime_format` (optional, default: `%d.%m.%Y %H:%M`)**

  - Output format of the `datetime` strings in the exported file.

- **`delimiter` (optional)**

  - Delimiter between columns for TSV/CSV export.

  - If omitted, it is inferred from the filename extension:

    - `.csv` uses comma (`,`) by default

    - `.tsv` uses tab (`\t`) by default

- **`decimal` (required, default: `"."`)**

  - Decimal separator character: `"."` for dot or `","` for comma.

- **`split_by` (optional, default: `none`)**

  - Split output into multiple files by statistic type:

    - `none`: default; write a single combined file

    - `measurement`: write only measurements statistics (mean/min/max)

    - `counter`: write only counter statistics (sum/state/delta)

    - `both`: write both files

  - Output files use suffixes `_measurements` and `_counters` before the extension.

- **`counter_fields` (optional, default: `both`)**

  - Controls which counter columns are written in **CSV/TSV** exports:

    - `both`: include `state`, `sum`, and `delta`

    - `sum`: include `state` and `sum`

    - `delta`: include `delta` only

  - **Limitation**: this option is ignored for JSON export.



> **Note:** If you omit `start_time`/`end_time`, the action will auto-detect the time range from the recorder.

> This requires long-term (hourly) statistics to exist. On new Home Assistant instances you may only have short-term statistics at first;

> in that case, wait until long-term statistics are generated, or provide explicit `start_time` and `end_time`.

>

> **Performance note**: Exporting all statistics from large databases (450k+ records) may take 30-60 seconds depending on hardware.



#### Example using the UI



![export to file](assets/service_export_ui.png)



#### Examples using YAML



##### Export selected entities



```yaml

action: import_statistics.export_statistics

data:

  filename: exported_statistics.tsv

  entities:

    - sensor.temperature

    - sensor.energy_consumption

    - sensor:ext_value

  start_time: "2025-12-22 12:00:00"

  end_time: "2025-12-25 12:00:00"

  delimiter: \t

  decimal: "."

  # timezone_identifier: Europe/Vienna  # Optional - defaults to HA timezone

```



##### Export using wildcard patterns



```yaml

action: import_statistics.export_statistics

data:

  filename: temperatures.tsv

  entities:

    - sensor.*_temperature

  start_time: "2025-12-22 00:00:00"

  end_time: "2025-12-23 00:00:00"

  delimiter: \t

  decimal: "."

```



##### Export all statistics



```yaml

action: import_statistics.export_statistics

data:

  filename: exported_statistics.tsv

  delimiter: \t

  decimal: "."

```



##### Export all statistics into separate files (measurements + counters)



```yaml

action: import_statistics.export_statistics

data:

  filename: exported_statistics.tsv

  split_by: both

  delimiter: \t

  decimal: "."

```



##### Export only measurements



```yaml

action: import_statistics.export_statistics

data:

  filename: exported_statistics.tsv

  entities:

    - sensor.temperature

    - sensor.energy_consumption

  start_time: "2025-12-22 00:00:00"

  end_time: "2025-12-23 00:00:00"

  split_by: measurements

  delimiter: \t

  decimal: "."

```



### Export Output



The exported file contains:



| For Measurements     | For Counters            |

| -------------------- | ----------------------- |

| `min`, `max`, `mean` | `sum`, `state`, `delta` |



> **Note:** You can export measurements and counters together, but you'll need to split them into separate files before re-importing (import only accepts one type per file).



---



## Exporting Statistics Inventory



Export a metadata-only inventory of all long-term statistics. This is useful for:



- **Migration planning**: Compare statistics between old and new Home Assistant instances

- **Database analysis**: Understand what statistics exist and their sizes

- **Cleanup**: Identify deleted or orphaned statistics



### How to Export Inventory



1. Go to **Developer Tools → Actions**

2. Select `import_statistics: export_inventory`

3. Fill in the settings (from the UI or YAML)

4. Click `perform action` to start the export.



### Settings Description



- **`filename` (required)**

  - Output file name (relative to Home Assistant config directory).

  - Use `.tsv` or `.csv` extension.

- **`delimiter` (optional)**

  - Delimiter between columns.

  - If omitted, it is inferred from the filename extension:

    - `.csv` uses comma (`,`) by default

    - `.tsv` uses tab (`\t`) by default

- **`timezone_identifier` (optional)**

  - Defaults to Home Assistant's configured timezone if omitted.

  - Used for formatting `first_seen` and `last_seen` timestamps.



### Example YAML



```yaml

action: import_statistics.export_inventory

data:

  filename: statistics_inventory.csv

```



### Inventory Output



- Export creates **two files** using the same base name:

  - `.csv` or `.tsv`: table only (one row per statistic, no summary block)

  - `.txt`: summary only



- Example: if `filename: inventory.csv`, outputs are:

  - `inventory.csv` (table)

  - `inventory.txt` (summary)



- Summary file content:



```text

Total statistics: 257

Measurements: 194

Counters: 63

Total samples: 5038317

Global start: 2022-01-09 16:00:00

Global end: 2026-02-12 13:00:00

Active statistics: 224

Orphan statistics: 7

Deleted statistics: 26

External statistics: 0

```



- Table file content (one row per statistic):



| statistic_id | unit_of_measurement | source | category | type | samples_count | first_seen | last_seen | days_span |

| --- | --- | --- | --- | --- | --- | --- | --- | --- |

| sensor.button_master_power | % | recorder | Active | Measurement | 32949 | 2/12/2022 10:00 | 2/12/2026 13:00 | 1461.1 |

| sensor.disk_free | GiB | recorder | Active | Measurement | 35876 | 1/9/2022 16:00 | 2/12/2026 13:00 | 1494.9 |

| sensor.disk_use | GiB | recorder | Active | Measurement | 35876 | 1/9/2022 16:00 | 2/12/2026 13:00 | 1494.9 |

| sensor.disk_use_percent | % | recorder | Active | Measurement | 35876 | 1/9/2022 16:00 | 2/12/2026 13:00 | 1494.9 |

| sensor.e3_tcu10_x07_buffer_main_temperature | °C | recorder | Active | Measurement | 20326 | 9/29/2023 10:00 | 2/12/2026 13:00 | 867.2 |

| sensor.e3_tcu10_x07_compressor_hours | h | recorder | Active | Counter | 17638 | 1/20/2024 21:00 | 2/12/2026 13:00 | 753.7 |

| sensor.e3_tcu10_x07_compressor_starts |  | recorder | Active | Counter | 17638 | 1/20/2024 21:00 | 2/12/2026 13:00 | 753.7 |



#### Table Columns



| Column | Description |

| ------ | ----------- |

| `statistic_id` | The statistic ID (e.g., `sensor.temperature`) |

| `unit_of_measurement` | Unit (e.g., `°C`, `kWh`) |

| `source` | Source of the statistic (e.g., `recorder`) |

| `category` | Classification: `Active`, `Orphan`, `Deleted`, or `External` |

| `type` | `Measurement` (has mean/min/max) or `Counter` (has sum) |

| `samples_count` | Number of long-term (hourly) samples |

| `first_seen` | Timestamp of earliest sample |

| `last_seen` | Timestamp of most recent sample |

| `days_span` | Number of days covered by samples |



#### Category Classification



Category classification is based on the Home Assistant entity registry and does not use `states_meta`.



- **Active**: `statistic_id` exists in the entity registry active entities (`core.entity_registry.entities`).

- **Orphan**: `statistic_id` exists in the entity registry deleted entities (`core.entity_registry.deleted_entities`) and has a non-null `orphaned_timestamp`.

- **Deleted**:

  - `statistic_id` exists in `core.entity_registry.deleted_entities` and has a null/missing `orphaned_timestamp`, or

  - `statistic_id` is not found in the entity registry at all (neither active nor deleted).

- **External**: Statistic is external (either `source != "recorder"` or `statistic_id` uses the `domain:name` format with `:`).



For more details, see [Export Inventory Categories](./docs/user/export_inventory_categories.md).



---



## Best Practices



### Importing counters (state_class `total / total_increasing`)



- Import `delta`, not `sum` and `state`

- Export first

- Modify the exported file

- Import



See [Delta Import](./docs/user/counters.md#delta-import)



### Testing and Validation



- Always make a backup before importing

  - Consider using the export functionality for backup as well

- Test with a small dataset first (10-20 rows)

- Create a test entity for initial imports

- Verify data in History graph before checking Energy Dashboard

- Enable debug logging to see detailed error messages (see [Debug Logging Guide](./docs/user/debug-logging.md))

- Existing values can be overwritten - use this to correct mistakes



### Data Preparation



- Use Excel, Google Sheets or similar to prepare the data

- Verify column headers match exactly

- Check for consistent data types in each column

- Check for consistent usage of delimiters

- Validate timestamps are full hours



---



## Troubleshooting



For troubleshooting tips, see [Troubleshooting Tips for Import Statistics Integration](./docs/user/troubleshooting-tips.md).



---



## Additional Resources



- **[Community Guide: Loading, Manipulating, and Recovering Statistics](https://community.home-assistant.io/t/loading-manipulating-recovering-and-moving-long-term-statistics-in-home-assistant/953802)** — Detailed examples for fixing historical data (thanks to Geoffrey!)

- **[Jeedom Migration Guide](./misc/jeedom.md)** — How to import statistics from Jeedom



---



[commits-shield]: https://img.shields.io/github/commit-activity/y/klausj1/homeassistant-statistics.svg

[commits]: https://github.com/klausj1/homeassistant-statistics/commits/main

[forum-shield]: https://img.shields.io/badge/community-forum-brightgreen.svg

[forum]: https://community.home-assistant.io/t/custom-integration-to-import-long-term-statistics-from-a-file-like-csv-or-tsv

[license-shield]: https://img.shields.io/github/license/klausj1/homeassistant-statistics.svg

[releases-shield]: https://img.shields.io/github/v/release/klausj1/homeassistant-statistics?include_prereleases

[releases]: https://github.com/klausj1/homeassistant-statistics/releases

[hacs-repo-badge]: https://my.home-assistant.io/badges/hacs_repository.svg

[hacs-repo]: https://my.home-assistant.io/redirect/hacs_repository/?owner=klausj1&repository=homeassistant-statistics&category=integration

[config-flow-badge]: https://my.home-assistant.io/badges/config_flow_start.svg

[config-flow]: https://my.home-assistant.io/redirect/config_flow_start?domain=import_statistics