{"id":15432908,"url":"https://github.com/simonw/airtable-export","last_synced_at":"2025-04-09T09:08:31.008Z","repository":{"id":42066152,"uuid":"291339086","full_name":"simonw/airtable-export","owner":"simonw","description":"Export Airtable data to YAML, JSON or SQLite files on disk","archived":false,"fork":false,"pushed_at":"2024-04-26T01:24:23.000Z","size":34,"stargazers_count":115,"open_issues_count":13,"forks_count":15,"subscribers_count":6,"default_branch":"main","last_synced_at":"2024-10-30T00:30:28.285Z","etag":null,"topics":["airtable","airtable-api","datasette-io","datasette-tool","yaml"],"latest_commit_sha":null,"homepage":"https://datasette.io/tools/airtable-export","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/simonw.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-08-29T19:51:37.000Z","updated_at":"2024-10-18T14:27:20.000Z","dependencies_parsed_at":"2024-10-20T20:19:04.098Z","dependency_job_id":null,"html_url":"https://github.com/simonw/airtable-export","commit_stats":{"total_commits":29,"total_committers":3,"mean_commits":9.666666666666666,"dds":"0.10344827586206895","last_synced_commit":"74ba9314bc66f6fa1740d45b696e67dfba6b825e"},"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonw%2Fairtable-export","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonw%2Fairtable-export/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonw%2Fairtable-export/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonw%2Fairtable-export/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/simonw","download_url":"https://codeload.github.com/simonw/airtable-export/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248008630,"owners_count":21032556,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["airtable","airtable-api","datasette-io","datasette-tool","yaml"],"created_at":"2024-10-01T18:29:11.776Z","updated_at":"2025-04-09T09:08:30.986Z","avatar_url":"https://github.com/simonw.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# airtable-export\n\n[![PyPI](https://img.shields.io/pypi/v/airtable-export.svg)](https://pypi.org/project/airtable-export/)\n[![Changelog](https://img.shields.io/github/v/release/simonw/airtable-export?include_prereleases\u0026label=changelog)](https://github.com/simonw/airtable-export/releases)\n[![Tests](https://github.com/simonw/airtable-export/workflows/Test/badge.svg)](https://github.com/simonw/airtable-export/actions?query=workflow%3ATest)\n[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/airtable-export/blob/master/LICENSE)\n\nExport Airtable data to files on disk\n\n## Installation\n\nInstall this tool using `pip`:\n\n    $ pip install airtable-export\n\n## Usage\n\nYou will need to the following information:\n\n- Your Airtable base ID - this is a string starting with `app...`\n- Your Airtable personal access token - this is a string starting with `pat...`\n\nIf you just want to export a subset of your tables you also need to know the names of those tables.\n\nYou can export all of your data to a folder called `export/` by running the following:\n\n    airtable-export export base_id --key=key\n\nThis example would files for each of your tables, for example: `export/table1.yml` and `export/table2.yml`.\n\nRather than passing the API key using the `--key` option you can set it as an environment variable called `AIRTABLE_KEY`.\n\nTo export only specified tables, pass their names as additional arguments:\n\n    airtable-export export base_id table1 table2 --key=key\n\n## Export options\n\nBy default the tool exports your data as YAML.\n\nYou can also export as JSON or as [newline delimited JSON](http://ndjson.org/) using the `--json` or `--ndjson` options:\n\n    airtable-export export base_id --key=key --ndjson\n\nYou can pass multiple format options at once. This command will create a `.json`, `.yml` and `.ndjson` file for each exported table:\n\n    airtable-export export base_id \\\n        --key=key --ndjson --yaml --json\n\nIf you import all tables, or if you add the `--schema` option, a JSON schema for the base will be written to `output-dir/_schema.json`.\n\n### SQLite database export\n\nYou can export tables to a SQLite database file using the `--sqlite database.db` option:\n\n    airtable-export export base_id \\\n        --key=key --sqlite database.db\n\nThis can be combined with other format options. If you only specify `--sqlite` the export directory argument will be ignored.\n\nThe SQLite database will have a table created for each table you export. Those tables will have a primary key column called `airtable_id`.\n\nIf you run this command against an existing SQLite database records with matching primary keys will be over-written by new records from the export.\n\n## Request options\n\nBy default the tool uses [python-httpx](https://www.python-httpx.org)'s default configurations.\n\nYou can override the `user-agent` using the `--user-agent` option:\n\n    airtable-export export base_id table1 table2 --key=key --user-agent \"Airtable Export Robot\"\n\nYou can override the [timeout during a network read operation](https://www.python-httpx.org/advanced/#fine-tuning-the-configuration) using the `--http-read-timeout` option. If not set, this defaults to 5s.\n\n    airtable-export export base_id table1 table2 --key=key --http-read-timeout 60\n\n## Running this using GitHub Actions\n\n[GitHub Actions](https://github.com/features/actions) is GitHub's workflow automation product. You can use it to run `airtable-export` in order to back up your Airtable data to a GitHub repository. Doing this gives you a visible commit history of changes you make to your Airtable data - like [this one](https://github.com/natbat/rockybeaches/commits/main/airtable).\n\nTo run this for your own Airtable database you'll first need to add the following secrets to your GitHub repository:\n\n\u003cdl\u003e\n  \u003cdt\u003eAIRTABLE_BASE_ID\u003c/dt\u003e\n  \u003cdd\u003eThe base ID, a string beginning `app...`\u003c/dd\u003e\n  \u003cdt\u003eAIRTABLE_KEY\u003c/dt\u003e\n  \u003cdd\u003eYour Airtable API key\u003c/dd\u003e\n  \u003cdt\u003eAIRTABLE_TABLES\u003c/dt\u003e\n  \u003cdd\u003eA space separated list of the Airtable tables that you want to backup. If any of these contain spaces you will need to enclose them in single quotes, e.g. \u003csamp\u003e'My table with spaces in the name' OtherTableWithNoSpaces\u003c/samp\u003e\u003c/dd\u003e\n\u003c/dl\u003e\n\nOnce you have set those secrets, add the following as a file called `.github/workflows/backup-airtable.yml`:\n```yaml\nname: Backup Airtable\n\non:\n  workflow_dispatch:\n  schedule:\n  - cron: '32 0 * * *'\n\njobs:\n  build:\n    runs-on: ubuntu-latest\n    steps:\n    - name: Check out repo\n      uses: actions/checkout@v2\n    - name: Set up Python\n      uses: actions/setup-python@v2\n      with:\n        python-version: 3.8\n    - uses: actions/cache@v2\n      name: Configure pip caching\n      with:\n        path: ~/.cache/pip\n        key: ${{ runner.os }}-pip-\n        restore-keys: |\n          ${{ runner.os }}-pip-\n    - name: Install airtable-export\n      run: |\n        pip install airtable-export\n    - name: Backup Airtable to backups/\n      env:\n        AIRTABLE_BASE_ID: ${{ secrets.AIRTABLE_BASE_ID }}\n        AIRTABLE_KEY: ${{ secrets.AIRTABLE_KEY }}\n        AIRTABLE_TABLES: ${{ secrets.AIRTABLE_TABLES }}\n      run: |-\n        airtable-export backups $AIRTABLE_BASE_ID $AIRTABLE_TABLES -v\n    - name: Commit and push if it changed\n      run: |-\n        git config user.name \"Automated\"\n        git config user.email \"actions@users.noreply.github.com\"\n        git add -A\n        timestamp=$(date -u)\n        git commit -m \"Latest data: ${timestamp}\" || exit 0\n        git push\n```\nThis will run once a day (at 32 minutes past midnight UTC) and will also run if you manually click the \"Run workflow\" button, see [GitHub Actions: Manual triggers with workflow_dispatch](https://github.blog/changelog/2020-07-06-github-actions-manual-triggers-with-workflow_dispatch/).\n\n## Development\n\nTo contribute to this tool, first checkout the code. Then create a new virtual environment:\n\n    cd airtable-export\n    python -mvenv venv\n    source venv/bin/activate\n\nOr if you are using `pipenv`:\n\n    pipenv shell\n\nNow install the dependencies and tests:\n\n    pip install -e '.[test]'\n\nTo run the tests:\n\n    pytest\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimonw%2Fairtable-export","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsimonw%2Fairtable-export","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimonw%2Fairtable-export/lists"}