https://github.com/pierresouchay/data2notion
Export records in Notion Databases
https://github.com/pierresouchay/data2notion
asyncio csv json notion prometheus
Last synced: about 1 month ago
JSON representation
Export records in Notion Databases
- Host: GitHub
- URL: https://github.com/pierresouchay/data2notion
- Owner: pierresouchay
- License: mit
- Created: 2024-06-24T17:23:50.000Z (11 months ago)
- Default Branch: main
- Last Pushed: 2025-04-07T12:14:31.000Z (about 1 month ago)
- Last Synced: 2025-04-07T12:16:41.391Z (about 1 month ago)
- Topics: asyncio, csv, json, notion, prometheus
- Language: Python
- Homepage:
- Size: 110 KB
- Stars: 3
- Watchers: 1
- Forks: 2
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
# data2notion: export your data to Notion
[](https://pypi.org/project/data2notion/)
[](https://raw.githubusercontent.com/pierresouchay/data2notion/main/LICENSE)
This tool is intended for exporting various data from 3rd party systems
into Notion. If you tool can export some records, then you can make this data available to Notion!
## Installation
It requires Python 3.9+ and try not using too many dependencies.
to install it:
```bash
python3 install data2notion
```
This will install it and all plugins. You can now run it by typing `data2notion` in you terminal.
## Changes
[CHANGELOG.md](https://github.com/pierresouchay/data2notion/blob/main/CHANGELOG.md)
## Goals
When working as a CTO, I had the need to expose data from various systems (Applications, AWS, SIEM, Active Directory, users, groups...)
into Notion to generate automatic reports and select what I wanted to expose, so it made all systems transparent and open.
This was extremelly precious to automatically generate reports for audits.
One of the advantages of the tool is that:
- it creates entries in Notion database if entries do not exists
- it modifies the various fields of the Notion database if they have been modified
- it cleanups the old records when they don't exist anymore
So, if you run this tool everyday, you have a Notion database in sync with the data from all your third party systems.
And you can start commenting, taking decisions and documenting those entries within Notion: it gives you a centralized
way to discuss topics about all of your IT, provides comments, reminders (eg: reming me about renewing this certificate
in 2 years, check if the fix has been performed in 2 months...), and cleanup dead stuff automatically.
Initially, the tool was only dealing with CSV, but data2notion also deals with various formats and APIs, so you can
integrate faster and better!
# Features
data2notion supports plugins, so you can either work with JSON/CSV export or implement your own retrieval of data, for instance,
from a 3rd party API.
## Basic Usage
data2notion is build using commands. Each succession of command has its own help.
The first level is teh following:
```bash
data2notion --help
usage: data2notion [-h] [--version] [--log-level {CRITICAL,ERROR,WARNING,INFO,DEBUG}] [--notion-log-level {CRITICAL,ERROR,WARNING,INFO,DEBUG}]
[--notion-rate-limit NOTION_RATE_LIMIT] [--statistics {console,disabled}] [--notion-token NOTION_TOKEN]
{plugins,write-to-notion} ...
Export some data into a notion database
options:
-h, --help show this help message and exit
--version show program's version number and exit
--log-level {CRITICAL,ERROR,WARNING,INFO,DEBUG}
Set the default log level (default=WARNING)
--notion-log-level {CRITICAL,ERROR,WARNING,INFO,DEBUG}
Set the log level for data2notion (default=INFO)
--notion-rate-limit NOTION_RATE_LIMIT
Set the notion rate-limiter, by default {default_rate_limit} (3 requests/sec, 100 initial bucket size)
--statistics {console,disabled}
Display Statistics when program ends
--notion-token NOTION_TOKEN
Notion Token to use $NOTION_TOKEN by default
action to perform:
action to perform
{plugins,write-to-notion}
sub-command help
write-to-notion write to Notion Database
```
This is the level where you can specify using the command line the `NOTION_TOKEN` (or you can use the environemnt variable `$NOTION_TOKEN` instead).
You can also change the log-level (in case of issues) and enable/disable statistics.
### First level of help
How to get the _notion_database_id_?
From [official documentation](https://developers.notion.com/reference/retrieve-a-database), open the database in your brower (in a full page),
the URL should then be https://notion.so/my_workspace/**668d797c76fa49349b05ad288df2d136**v=... => in such an example, `668d797c76fa49349b05ad288df2d136`
would be the _notion_database_id_.
Be sure that you [granted access rights to this database](https://www.notion.so/help/add-and-manage-connections-with-the-api#add-connections-to-pages)
to the token you want to use in this app.
```bash
data2notion write-to-notion --help
```
Will list the arguments that are specific to a plugin.
### CSV plugin
Export CSV files arrays in Notion.
```bash
data2notion write-to-notion csv --help
usage: data2notion write-to-notion notion_database_id csv [-h] [--csv-dialect {excel,excel-tab,unix,excel_with_semi-colon}] csv_file
Write to Notion DB from a CSV file
positional arguments:
csv_file CSV file to inject in Notion, if '-' is set, read from stdin
options:
-h, --help show this help message and exit
--csv-dialect {excel,excel-tab,unix,excel_with_semi-colon}
CSV Dialect, excel by default
```
### JSON Plugin
Export some JSON arrays in Notion.
```bash
usage: data2notion write-to-notion json [-h] [--json-path PATH_IN_JSON] json_file
Write to Notion DB from a JSON file containing an array
positional arguments:
json_file JSON file to inject in Notion, if '-' is set, read from stdin
options:
-h, --help show this help message and exit
--json-path PATH_IN_JSON
JSON path separated by dots to look for the array, example: calendar.appointments
```
### Prometheus Plugin
Export metrics from prometheus (lastest values only) in Notion database.
Inspired by [prom2csv](https://pypi.org/project/prom2csv/), this plugin let you export your last prometheus metrics in Notion!
```bash
usage: data2notion [-h] [--version] [--log-level {CRITICAL,ERROR,WARNING,INFO,DEBUG}] [--notion-log-level {CRITICAL,ERROR,WARNING,INFO,DEBUG}] [--no-progress-bar] [--notion-rate-limit NOTION_RATE_LIMIT] [--statistics {console,disabled}]
[--notion-token NOTION_TOKEN] [--partition column_name=]
{plugins,export-from-notion,write-to-notion} ...
Export some data into a notion database
options:
-h, --help show this help message and exit
--version show program's version number and exit
--log-level {CRITICAL,ERROR,WARNING,INFO,DEBUG}
Set the default log level (default from $NOTION_LOG_LEVEL=WARNING)
--notion-log-level {CRITICAL,ERROR,WARNING,INFO,DEBUG}
Set the log level for data2notion (default=INFO)
--no-progress-bar Disable the progress bar
--notion-rate-limit NOTION_RATE_LIMIT
Set the notion rate-limiter, by default {default_rate_limit} (3 requests/sec, 100 initial bucket size)
--statistics {console,disabled}
Display Statistics when program ends
--notion-token NOTION_TOKEN
Notion Token to use $NOTION_TOKEN by default
--partition column_name=
Only synchronize records having a column matching given regexp
action to perform:
action to perform
{plugins,export-from-notion,write-to-notion}
sub-command help
export-from-notion export from Notion Database
write-to-notion write to Notion Database
```
## Adding other plugins
Adding plugin can be done using environment variable `$DATA2NOTION_ADDITIONAL_PLUGINS` which support the
following syntax `python_module.python_file:PluginClass`, thus to enable a new imaginary Bugzilla plugin, you might for instance
export the variable:
```bash
export DATA2NOTION_ADDITIONAL_PLUGINS="bugzilla_api.bugzilla_plugin:BugzillPlugin"
```
Then, the plugin should be visible in:
```bash
data2notion plugins
```
and you might use it as any other plugin.
## Writing new plugins
See [DESIGN.md](https://github.com/pierresouchay/data2notion/blob/main/DESIGN.md).