https://github.com/mathieu2301/airtable-to-json
Extract Airtable table to JSON
https://github.com/mathieu2301/airtable-to-json
airtable data-mining dataset json mongodb
Last synced: about 2 months ago
JSON representation
Extract Airtable table to JSON
- Host: GitHub
- URL: https://github.com/mathieu2301/airtable-to-json
- Owner: Mathieu2301
- Created: 2022-11-21T05:09:48.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2022-11-25T08:58:12.000Z (over 3 years ago)
- Last Synced: 2025-03-05T03:29:45.531Z (over 1 year ago)
- Topics: airtable, data-mining, dataset, json, mongodb
- Language: TypeScript
- Homepage:
- Size: 15.6 KB
- Stars: 1
- Watchers: 3
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Airtable to JSON
This project can be used to extract data from any public Airtable table and write it to a JSON file.
The project includes a table `fuser` that can be used to combine multiple tables into a single JSON file, with the ability to project columns and rename them, and group rows by a common column (Primary Key).
All possible merging errors are handled with error messages that make it easy to resolve them.
This project is designed to work perfectly with MongoDB.
## Install the dependencies
```bash
yarn
```
## Download data from Airtable
1. Go to the airtable page on your browser
2. Go to inspect element
3. Go to network tab
4. Reload the page (F5)
5. Search `readSharedViewData` or `read` in the network tab
6. Click the first or second result
7. Go to `Response` tab
8. Copy the content
9. Create a `input.json` file in the `./inputs` directory
10. Paste the content in the file
## Parse the JSON files
This will parse all the JSON files in the `./inputs` directory and create a file in the `./outputs` directory.
```bash
yarn start
```
## Format and merge the JSON files
This will merge all the JSON files in the `./outputs` directory and create a `merged.output.json` file.
```bash
yarn merge
```
### Fuser configuration
When you run the `yarn merge` command for the first time, it will create a `fuser.config.ts` file in the root directory. You can then edit this file to change the configuration.
```ts
{
/**
* The column name that will
* be used to merge the data.
*
* If not provided, there may
* be duplicates in the output.
*/
uniqueKey: 'theColumnId',
/**
* The projection that will be used
* to merge the data.
*
* The key is the column name of the
* Airtable table and the value is the
* column name wanted in the output file.
*/
projection: {
'The column id': 'theColumnId',
'The column name': 'theColumnName',
'the_column_type': 'theColumnType',
},
/**
* Set it to true if you want to remove
* the `_date` column after merging.
*
* This column is used to merge the data.
*
* Default: false
*/
removeDate: false,
}
```