Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/iragca/anilist-data-transfer
This repository is for transferring data from the node based AniList GraphQL database to a local DuckDB relational database. The anime entries retrieved are roughly ~13,000 rows as of 2025 January 10.
https://github.com/iragca/anilist-data-transfer
anilist anilist-api anime data-science dataset duckdb graphql python requests-library-python
Last synced: about 7 hours ago
JSON representation
This repository is for transferring data from the node based AniList GraphQL database to a local DuckDB relational database. The anime entries retrieved are roughly ~13,000 rows as of 2025 January 10.
- Host: GitHub
- URL: https://github.com/iragca/anilist-data-transfer
- Owner: iragca
- License: gpl-3.0
- Created: 2025-01-09T23:02:24.000Z (25 days ago)
- Default Branch: main
- Last Pushed: 2025-01-09T23:41:03.000Z (25 days ago)
- Last Synced: 2025-01-10T00:25:38.917Z (25 days ago)
- Topics: anilist, anilist-api, anime, data-science, dataset, duckdb, graphql, python, requests-library-python
- Language: HTML
- Homepage:
- Size: 4.95 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
Anilist Data Transfer
Part of Keikakku Dashboards
Quicklinks
- [Getting Started](#Getting-Started)
- [Exploratory Data Analysis](#EDA)
This repository is for transferring data from the node based [AniList GraphQL database](https://docs.anilist.co/) to
a local DuckDB relational database. The anime entries retrieved are roughly ~13,000 rows as of 2025 January 10.
> AniList database -> GraphQL API -> DuckDB
8 tables currently exist in the database:
- **Anime:** Stores detailed information about the anime shows.
- **Review:** Stores reviews of anime.
- **Status:** Stores statistics about anime user statuses.
- **User:** Stores user information.
- **WebAsset:** Stores web assets related to anime.
- **Studio:** Stores information about studios associated with anime.
- **Tag:** Stores tags associated with anime.
- **Genre:** Stores genres associated with anime.
> [!NOTE]
> Primary keys are for enforcing uniqueness. Foreign keys are not recommended as GraphQL is inherently node based and not relational.
#### Relevant / Similar Repositories
- https://github.com/manami-project/anime-offline-database
https://github.com/user-attachments/assets/7a25466b-d1a9-4c4f-8259-7d8bf6b89396
Requirements:
- Python
Linux
```bash
git clone https://github.com/iragca/anilist-data-transfer.git
cd anilist-data-transfer
bash setup.sh
source .venv/bin/activate
pip install -r requirements.txt
python src/init_db.py
python src/data_transfer.py 1940 2025 10 #
```
Powershell
```powershell
git clone https://github.com/iragca/anilist-data-transfer.git
cd anilist-data-transfer
pip install virtualenv
virtualenv .venv
source .\.venv\Scripts\activate
pip install -r requirements.txt
python .\src\init_db.py
python .\src\data_transfer.py 1940 2025 10 #
```
Basic reports are made for each table and are available on project folder [root/eda](https://github.com/iragca/Anilist-Data-Transfer/tree/main/eda)