{"id":27933477,"url":"https://github.com/timescale/nft-starter-kit","last_synced_at":"2025-05-07T04:59:00.112Z","repository":{"id":37538007,"uuid":"421962097","full_name":"timescale/nft-starter-kit","owner":"timescale","description":"Timescale NFT Starter Kit","archived":false,"fork":false,"pushed_at":"2024-02-21T15:58:12.000Z","size":21392,"stargazers_count":116,"open_issues_count":0,"forks_count":25,"subscribers_count":11,"default_branch":"main","last_synced_at":"2025-05-07T04:58:55.365Z","etag":null,"topics":["crypto","grafana","nft","postgresql","sql","superset","timescaledb"],"latest_commit_sha":null,"homepage":"https://docs.timescale.com/timescaledb/latest/tutorials/analyze-nft-data/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/timescale.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-10-27T20:19:29.000Z","updated_at":"2025-03-06T16:53:15.000Z","dependencies_parsed_at":"2022-09-13T09:03:29.346Z","dependency_job_id":null,"html_url":"https://github.com/timescale/nft-starter-kit","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timescale%2Fnft-starter-kit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timescale%2Fnft-starter-kit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timescale%2Fnft-starter-kit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timescale%2Fnft-starter-kit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/timescale","download_url":"https://codeload.github.com/timescale/nft-starter-kit/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252816946,"owners_count":21808704,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["crypto","grafana","nft","postgresql","sql","superset","timescaledb"],"created_at":"2025-05-07T04:58:59.617Z","updated_at":"2025-05-07T04:59:00.105Z","avatar_url":"https://github.com/timescale.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Timescale NFT Starter Kit\nThe Timescale NFT Starter Kit is a step-by-step guide to get up and running with collecting, storing, analyzing and visualizing NFT data from [OpenSea](https://opensea.io), using PostgreSQL and TimescaleDB.\n\nThe NFT Starter Kit will give you a foundation for analyzing NFT trends so that you can bring some data to your purchasing decisions, or just learn about the NFT space from a data-driven perspective. It also serves as a solid foundation for your more complex NFT analysis projects in the future.\n\nWe recommend following along with the [NFT Starter Kit tutorial](https://www.timescale.com/blog/analyzing-5-million-nft-sales-using-postgresql/) to get familiar with the contents of this repository.\n\nFor more information about the NFT Starter Kit, see the [announcement blog post](https://tsdb.co/nft-starter-kit-blog).\n\n## Project components\nWe provide multiple standalone components to help your data exploration journey at each level.\n\n### Design database schema\n* [Relational schema](/schema.sql) for storing NFT sales, assets, collections, and accounts.\n\n### Get data\n* [Data ingestion script](/opensea_ingest.py), that collects historical data from OpenSea and ingests it into TimescaleDB. [Read more!](#running-the-data-ingestion-script)\n* [Sample data][sample-dw], that you can download and ingest to get started quickly. [Read more!](#ingest-the-sample-data)\n\n### Build dashboards\n* Streamlit dashboard, to analyze collection sales. [Read more!](pre-built-dashboards/streamlit/README.md)\n* [Grafana dashboard](/pre-built-dashboards/grafana-collections.json) template file\n* [Dockerized TimescaleDB + Apache Superset](#setting-up-the-pre-built-superset-dashboards) with pre-loaded data, to store and analyze NFTs.\n\n### Analyze data\n* [Sample queries][queries] to use as a starting point for your own analysis.\n\n## Get started\nWhichever component you are most interested in, first clone the repository:\n```bash\ngit clone https://github.com/timescale/nft-starter-kit.git\ncd nft-starter-kit\n```\n\n## Setting up the pre-built Superset dashboards\nThis part of the project is fully Dockerized. TimescaleDB and the Superset dashboard \nis built out automatically using docker-compose. After completing the steps below, you \nwill have a local TimescaleDB and Superset instance running in \ncontainers - containing 500K+ NFT transactions from OpenSea.\n\n![superset dashboard](https://www.timescale.com/blog/content/images/2021/10/Superset_Dashboard--1-.png)\n\nThe Docker service uses port 8088 (for Superset) and 6543 (for TimescaleDB) so make sure \nthere's no other services using those ports before starting the installation process.\n\n### Prerequisites\n\n* [Docker](https://docs.docker.com/get-docker/)\n* [Docker compose](https://docs.docker.com/compose/install/)\n\n    Verify that both are installed:\n    ```bash\n    docker --version \u0026\u0026 docker-compose --version\n    ```\n\n### Instructions\n\n1. Run `docker-compose up --build` in the `/pre-built-dashboards` folder:\n    ```bash\n    cd pre-built-dashboards\n    docker-compose up --build\n    ```\n    See when the process is done (it could take a couple of minutes):\n    ```bash\n    timescaledb_1      | PostgreSQL init process complete; ready for start up.\n    ```\n1. Go to http://0.0.0.0:8088/ in your browser and login with these credentials:\n    ```txt\n    user: admin\n    password: admin\n    ```\n1. Open the `Databases` page inside Superset (http://0.0.0.0:8088/databaseview/list/). You will see exactly one item there\n    called `NFT Starter Kit`.\n1. Go check out your NFT dashboards! \n\n    Collections dashboard: http://0.0.0.0:8088/superset/dashboard/1\n\n    Assets dashboard: http://0.0.0.0:8088/superset/dashboard/2 \n\n## Running the data ingestion script\nIf you'd like to ingest data into your database (be it a local TimescaleDB, or in Timescale Cloud) \nstraight from the OpenSea API, follow these steps to configure the ingestion script:\n\n### Prerequisites\n* Python 3\n* [TimescaleDB installed][install-ts]\n* Schema has been set up using the [`schema.sql`][schema] script.\n\n### Instructions\n\n1. Go to the root folder of the project:\n    ```bash\n    cd nft-starter-kit\n    ```\n1.  Create a new Python virtual environment and install the requirements:\n    ```bash\n    virtualenv env \u0026\u0026 source env/bin/activate\n    pip install -r requirements.txt\n    ```\n1.  Replace the parameters in the [`config.py`][config] file:\n    ```python\n    DB_NAME=\"tsdb\"\n    HOST=\"YOUR_HOST_URL\"\n    USER=\"tsdbadmin\"\n    PASS=\"YOUR_PASSWORD_HERE\"\n    PORT=\"PORT_NUMBER\"\n    OPENSEA_START_DATE=\"2021-10-01T00:00:00\" # example start date (UTC)\n    OPENSEA_END_DATE=\"2021-10-06T23:59:59\" # example end date (UTC)\n    OPENSEA_APIKEY=\"YOUR_OPENSEA_APIKEY\" # need to request from OpenSea's docs\n    ```\n1.  Run the Python script:\n    ```python\n    python opensea_ingest.py\n    ```\n    This will start ingesting data in batches, ~300 rows at a time:\n    ```bash\n    Start ingesting data between 2021-10-01 00:00:00+00:00 and 2021-10-06 23:59:59+00:00\n    ---\n    Fetching transactions from OpenSea...\n    Data loaded into temp table!\n    Data ingested!\n    Data has been backfilled until this time: 2021-10-06 23:51:31.140126+00:00\n    ---\n    ```\n    You can stop the ingesting process anytime (Ctrl+C), otherwise the script will run until all \n    the transactions have been ingested from the given time period.\n\n\n## Ingest the sample data\nIf you don't want to spend time waiting until a decent amount of data is ingested, \nyou can just use our sample dataset which contains 500K+ sale transactions from \nOpenSea (this sample was used for the Superset dashboard as well) \n\n### Prerequisites\n* [TimescaleDB installed][install-ts]\n* PSQL ([installation guide](https://blog.timescale.com/blog/how-to-install-psql-on-mac-ubuntu-debian-windows/))\n\n### Instructions\n1.  Go to the folder with the sample CSV files (or you can also [download them from here][sample-dw]):\n    ```bash\n    cd pre-built-dashboards/database/data\n    ```\n1.  Connect to your database with PSQL:\n    ```bash\n    psql -x \"postgres://host:port/tsdb?sslmode=require\"\n    ```\n    If you're using Timescale Cloud, the instructions under `How to Connect` provide a \n    customized command to run to connect directly to your database.\n1.  Import the CSV files in this order (it can take a few minutes in total):\n    ```bash\n    \\copy accounts FROM 001_accounts.csv CSV HEADER;\n    \\copy collections FROM 002_collections.csv CSV HEADER;\n    \\copy assets FROM 003_assets.csv CSV HEADER;\n    \\copy nft_sales FROM 004_nft_sales.csv CSV HEADER;\n    ```  \n1.  Try running [some queries][queries] on your database:\n    ```sql\n    SELECT count(*), MIN(time) AS min_date, MAX(time) AS max_date FROM nft_sales \n    ```\n\n\n\n\n[schema]: https://github.com/timescale/nft-starter-kit/blob/master/schema.sql\n[install-ts]: https://docs.timescale.com/timescaledb/latest/how-to-guides/install-timescaledb/#install-timescaledb\n[ingest]: https://github.com/timescale/nft-starter-kit/blob/master/opensea_ingest.py\n[local-ts]: https://github.com/timescale/nft-starter-kit/tree/master/pre-built-dashboards/database\n[dash]: https://github.com/timescale/nft-starter-kit/tree/master/pre-built-dashboards/dashboard\n[queries]: https://github.com/timescale/nft-starter-kit/blob/master/queries.sql\n[config]: https://github.com/timescale/nft-starter-kit/blob/master/config.py\n[sample-dw]: https://assets.timescale.com/docs/downloads/nft_sample.zip\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftimescale%2Fnft-starter-kit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftimescale%2Fnft-starter-kit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftimescale%2Fnft-starter-kit/lists"}