{"id":27042430,"url":"https://github.com/rickh94/airtable_local_backup","last_synced_at":"2025-04-05T04:28:32.985Z","repository":{"id":52704827,"uuid":"106216454","full_name":"rickh94/airtable_local_backup","owner":"rickh94","description":"Create local backups of airtable databases","archived":false,"fork":false,"pushed_at":"2022-12-08T01:14:48.000Z","size":359,"stargazers_count":33,"open_issues_count":7,"forks_count":6,"subscribers_count":3,"default_branch":"master","last_synced_at":"2023-03-02T07:46:51.740Z","etag":null,"topics":["airtable","backup","database","rest-client"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rickh94.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-10-08T23:12:33.000Z","updated_at":"2023-02-11T21:57:46.000Z","dependencies_parsed_at":"2023-01-25T02:31:04.845Z","dependency_job_id":null,"html_url":"https://github.com/rickh94/airtable_local_backup","commit_stats":null,"previous_names":[],"tags_count":null,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rickh94%2Fairtable_local_backup","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rickh94%2Fairtable_local_backup/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rickh94%2Fairtable_local_backup/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rickh94%2Fairtable_local_backup/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rickh94","download_url":"https://codeload.github.com/rickh94/airtable_local_backup/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247289410,"owners_count":20914463,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["airtable","backup","database","rest-client"],"created_at":"2025-04-05T04:28:32.454Z","updated_at":"2025-04-05T04:28:32.971Z","avatar_url":"https://github.com/rickh94.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"Airtable Local Backup - IN PROGRESS\n===================================\n.. image:: https://travis-ci.org/rickh94/airtable_local_backup.svg?branch=master\n    :target: https://travis-ci.org/rickh94/airtable_local_backup\n\n\n.. image:: https://codecov.io/gh/rickh94/airtable_local_backup/branch/master/graph/badge.svg\n  :target: https://codecov.io/gh/rickh94/airtable_local_backup\n\n\n.. image:: https://codebeat.co/badges/875ffd7a-5a0f-4ca2-89d2-a8e7037fff80\n  :target: https://codebeat.co/projects/github-com-rickh94-airtable_local_backup-master\n\n\n.. image:: https://img.shields.io/badge/python-3.6-blue.svg\n\n\nThe goal of this project is to provide an easy way to backup data from\nAirtable without too much user intervention. (The current recommended\nsolution is download individual csv files and attachments and correlate them\nmanually. yuck.)\n\nGoals\n=====\n\nWith a simple yaml configuration file and this library, it should be possible to\ndownload all table data (including attachments) and restore them to a new\nairtable database.\n\nA cli frontend is also possible once the library/api is complete.\n\nInternals\n=========\n\nAirtable does not offer an quick offline backup solution, but they do offer\nand decent REST API so that is what is used to download the table data.\nListing tables is not supported so they will need to be specified in a config\n(yaml) file. Attachment files are downloaded, their data compressed\n(optional), and base64 encoded and stored with their filename and a hash fo\nthe original data. The whole table is then serialized to json and dumped to a\nfile. *incomplete*\n\nThe json can be read and uploaded to a new database by the restore functions.\nAirtable requires a public endpoint for files so temporary storage is\nrequired. Currently s3 buckets are supported (NOTE: the bucket need not and should not be\npublic. A presigned url will be generated for airtable to download the file.)\nI hope to also support digitalocean spaces. If no backing store is provided,\nattachment data will be discarded (though still available in the backup).\n\nOnce json files are created for each table, they will be combined to a\nzipfile or tar archive for storage. (If this is not desired, scripts can be\nwritten to access other parts of the api.)\n\nTODO\n====\n\n- Finish core API: mainly files. Download and upload are mostly working.\n- Implement configuration file for database schema.\n- Help restore database schema in some way, possbily blank-ish csvs with data\n  type guesses based on downloaded data.\n\n\nLong-Term Goals\n===============\n\n- Linked Records: Record linking will be tricky because restored bases will\n  have different unique ids assigned to the records. It will require\n  something like creating the whole database and then doing a second pass to\n  create the links based on searches. Will probably be pretty hack-y.\n- Data migration: Because the data is being serialzed to json, it shouldn't be too\n  hard to go from that to loading it into another document database (tinydb,\n  mongodb, dynamodb, etc.) or maybe even a SQL database (not sure.)\n\nContributing\n============\n\nIf you are interested in using this, feel free to open an issue or a PR or\ngrab something off the todo and hack on it a little, or write a test for\nexisting code that isn't covered (please use pytest).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frickh94%2Fairtable_local_backup","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frickh94%2Fairtable_local_backup","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frickh94%2Fairtable_local_backup/lists"}