{"id":15181278,"url":"https://github.com/firebase/rtdb-to-csv","last_synced_at":"2025-10-01T22:31:46.671Z","repository":{"id":66068693,"uuid":"126098717","full_name":"firebase/rtdb-to-csv","owner":"firebase","description":null,"archived":true,"fork":false,"pushed_at":"2018-08-08T18:31:12.000Z","size":10,"stargazers_count":10,"open_issues_count":0,"forks_count":12,"subscribers_count":52,"default_branch":"master","last_synced_at":"2024-09-28T17:01:35.950Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/firebase.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-03-21T00:07:18.000Z","updated_at":"2024-03-12T12:43:04.000Z","dependencies_parsed_at":null,"dependency_job_id":"474fa3c9-621b-4b4c-bf30-c322a02b1be3","html_url":"https://github.com/firebase/rtdb-to-csv","commit_stats":{"total_commits":3,"total_committers":2,"mean_commits":1.5,"dds":"0.33333333333333337","last_synced_commit":"286da5b51313a9d1bc784435dc004523d81bae14"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/firebase%2Frtdb-to-csv","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/firebase%2Frtdb-to-csv/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/firebase%2Frtdb-to-csv/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/firebase%2Frtdb-to-csv/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/firebase","download_url":"https://codeload.github.com/firebase/rtdb-to-csv/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":234909085,"owners_count":18905503,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-09-27T17:00:35.397Z","updated_at":"2025-10-01T22:31:46.363Z","avatar_url":"https://github.com/firebase.png","language":"JavaScript","readme":"# Firebase Database Self-Serve Backup to BQ/CSV Exporter\n\nThis project takes a self-serve backup generated from the Firebase Realtime Database and outputs a flattened CSV as output.\n\nThe CSV is made to be uploaded to BigQuery but can be used for any number of purposes.\n\nThe term *flattened* is used to represent the following transformation\n\n```json\n{\n    \"a\": true,\n    \"b\": {\n        \"c\": true\n    }\n}\n```\n\nwill be converted to\n\n```text\na,true\nb,c,true\n```\n\n## Setup\n\nThis project was created and tested using Node v9.1.0.\n\nSetup using: `npm install`\n\n## Getting a Self-Serve Backup\n\nTo generate a self-serve backup:\n\n1. Go to the Firebase console (`https://console.firebase.google.com/project/\u003cYOUR_PROJECT\u003e/database/backups`).\n1. Click on a backup to download it from GCS. (or create a public link to it)\n\n## Exporting to a CSV\n\nRun this using: `node export.js --inputURL http://link/to/backup.gz --outputPath /tmp/output.gz`\n\n### Arguments\n\n#### `--inputURL http://path/to/input` OR `--inputPath /path/to/input`\n\n**required**\n\nThis flag indicates where the input resides.\n\n#### `--outputPath /path/to/output`\n\n**required**\n\nThis flag indicates where the output csv should reside.\n\n#### `--noCompressedInput`\n\n*optional*\n\nThis flag takes no parameters. By default, it is expected that the input is gzipped. If it is not, use this flag to indicate as such.\n\n#### `--noCompressedOutput`\n\n*optional*\n\nThis flag takes no parameters. By default, it is expected that the output will be gzipped. If it should not, use this flag to indicate as such.\n\n#### `--fillWithNulls`\n\n*optional*\n\nThis flag takes no parameters. By default, the output will contain rows of mixed-width depending on how deep the input data is. By using this flag, the output will be normalized to contain 32 columns.\n\n## Upload to BigQuery\n\n1. Navigate to the [BigQuery console](https://bigquery.cloud.google.com).\n1. Under the desired Data Set, click \"Create new table\".\n1. Use the following params:\n    1. Source Data: `Create from source`\n    1. Location: `Google Cloud Storage`, set the path (e.g. `gs://my-bucket/output.gz`)\n    1. File format: `CSV`\n    1. Table name: `your-dataset`.`your-table`\n    1. Schema: (If you used `--fillWithNulls`, you can set `Automatically detect`), otherwise click `Edit as Text` and input:\n    ```c1:STRING,c2:STRING,c3:STRING,c4:STRING,c5:STRING,c6:STRING,c7:STRING,c8:STRING,c9:STRING,c10:STRING,c11:STRING,c12:STRING,c13:STRING,c14:STRING,c15:STRING,c16:STRING,c17:STRING,c18:STRING,c19:STRING,c20:STRING,c21:STRING,c22:STRING,c23:STRING,c24:STRING,c25:STRING,c26:STRING,c27:STRING,c28:STRING,c29:STRING,c30:STRING,c31:STRING,c32:STRING```\n    1. Field delimiter: `Comma`\n    1. Allow jagged rows: `Check` (ONLY if you didn't use `--fillWithNulls`)\n1. Click `Create Table`.\n\n*This is not an official Google product*\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffirebase%2Frtdb-to-csv","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffirebase%2Frtdb-to-csv","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffirebase%2Frtdb-to-csv/lists"}