{"id":25322403,"url":"https://github.com/checkernetwork/spark-stats","last_synced_at":"2025-04-07T22:22:29.137Z","repository":{"id":217326896,"uuid":"743594551","full_name":"CheckerNetwork/spark-stats","owner":"CheckerNetwork","description":"API exposing public statistics about Spark","archived":false,"fork":false,"pushed_at":"2025-01-29T08:12:38.000Z","size":1041,"stargazers_count":0,"open_issues_count":6,"forks_count":1,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-01-29T12:36:29.163Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CheckerNetwork.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-01-15T15:10:56.000Z","updated_at":"2025-01-29T08:12:42.000Z","dependencies_parsed_at":"2024-01-22T10:55:59.541Z","dependency_job_id":"ab6abf4a-756e-4787-a70f-e897cf85f0b1","html_url":"https://github.com/CheckerNetwork/spark-stats","commit_stats":null,"previous_names":["filecoin-station/spark-stats","checkernetwork/spark-stats"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CheckerNetwork%2Fspark-stats","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CheckerNetwork%2Fspark-stats/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CheckerNetwork%2Fspark-stats/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CheckerNetwork%2Fspark-stats/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CheckerNetwork","download_url":"https://codeload.github.com/CheckerNetwork/spark-stats/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247737931,"owners_count":20987763,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-02-13T23:40:42.173Z","updated_at":"2025-04-07T22:22:29.115Z","avatar_url":"https://github.com/CheckerNetwork.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# spark-stats\n\nAPI exposing public statistics about Spark\n\n## Public API\n\nBase URL: http://stats.filspark.com/\n\n- `GET /retrieval-success-rate?from=2024-01-01\u0026to=2024-01-31`\n\n  http://stats.filspark.com/retrieval-success-rate\n\n- `GET /retrieval-success-rate?from=2024-01-01\u0026to=2024-01-31\u0026nonZero=true`\n\n  _Miners with no successful retrievals are excluded from the RSR calculation._\n\n  http://stats.filspark.com/retrieval-success-rate?nonZero=true\n\n- `GET /miners/retrieval-success-rate/summary?from=\u003cday\u003e\u0026to=\u003cday\u003e`\n\n  http://stats.filspark.com/miners/retrieval-success-rate/summary\n\n- `GET /miners/retrieval-timings/summary?from=\u003cday\u003e\u0026to=\u003cday\u003e`\n\n  http://stats.filspark.com/miners/retrieval-timings/summary\n\n- `GET /miner/:id/retrieval-success-rate/summary?from=\u003cday\u003e\u0026to=\u003cday\u003e`\n\n  http://stats.filspark.com/miner/f0814049/retrieval-success-rate/summary\n\n- `GET /miner/:id/retrieval-timings/summary?from=\u003cday\u003e\u0026to=\u003cday\u003e`\n\n  http://stats.filspark.com/miner/f0814049/retrieval-timings/summary\n\n- `GET /miner/:id/deals/eligible/summary`\n\n  http://stats.filspark.com/miner/f0814049/deals/eligible/summary\n\n- `GET /client/:id/deals/eligible/summary`\n\n  http://stats.filspark.com/client/f0215074/deals/eligible/summary\n\n- `GET /clients/retrieval-success-rate/summary?from=\u003cday\u003e\u0026to=\u003cday\u003e`\n\n  http://stats.filspark.com/clients/retrieval-success-rate/summary\n\n\n- `GET /client/:id/retrieval-success-rate/summary?from=\u003cday\u003e\u0026to=\u003cday\u003e`\n\n  http://stats.filspark.com/client/f0215074/retrieval-success-rate/summary\n\n\n- `GET /allocator/:id/deals/eligible/summary`\n\n  http://stats.filspark.com/allocator/f03015751/deals/eligible/summary\n\n- `GET /allocators/retrieval-success-rate/summary?from=\u003cday\u003e\u0026to=\u003cday\u003e`\n\n  http://stats.filspark.com/allocators/retrieval-success-rate/summary\n\n\n- `GET /allocator/:id/retrieval-success-rate/summary?from=\u003cday\u003e\u0026to=\u003cday\u003e`\n\n  http://stats.filspark.com/allocator/f0215074/retrieval-success-rate/summary\n\n- `GET /participants/daily?from=\u003cday\u003e\u0026to=\u003cday\u003e`\n\n  http://stats.filspark.com/participants/daily\n\n- `GET /participants/monthly?from=\u003cday\u003e\u0026to=\u003cday\u003e`\n\n  http://stats.filspark.com/participants/monthly\n\n- `GET /participants/accumulative/daily?from=\u003cday\u003e\u0026to=\u003cday\u003e`\n\n  http://stats.filspark.com/participants/accumulative/daily\n\n- `GET /participants/change-rates?from=\u003cday\u003e\u0026to=\u003cday\u003e`\n\n  http://stats.filspark.com/participants/change-rates\n\n- `GET /participants/top-earning`\n\n  http://stats.filspark.com/participants/top-earning\n\n- `GET /participants/top-measurements`\n\n  http://stats.filspark.com/participants/top-measurements?from=yesterday\u0026to=yesterday\n\n- `GET /participants/summary`\n\n  http://stats.filspark.com/participants/summary\n\n- `GET /participant/:address/scheduled-rewards?address=\u003caddress\u003e\u0026from=\u003cday\u003e\u0026to=\u003cday\u003e`\n\n  http://stats.filspark.com/participant/0x000000000000000000000000000000000000dEaD/scheduled-rewards\n\n- `GET /participant/:address/reward-transfers?from=\u003cday\u003e\u0026to=\u003cday\u003e`\n\n  http://stats.filspark.com/participant//0x000000000000000000000000000000000000dEaD/reward-transfers\n\n- `GET /stations/daily?from=\u003cday\u003e\u0026to=\u003cday\u003e`\n\n  http://stats.filspark.com/stations/daily\n\n- `GET /stations/monthly?from=\u003cday\u003e\u0026to=\u003cday\u003e`\n\n  http://stats.filspark.com/stations/monthly\n\n- `GET /measurements/daily?from=\u003cday\u003e\u0026to=\u003cday\u003e`\n\n  http://stats.filspark.com/measurements/daily\n\n- `GET /transfers/daily?from=\u003cday\u003e\u0026to=\u003cday\u003e`\n\n  http://stats.filspark.com/transfers/daily\n\n- `GET /deals/daily?from=2024-01-01\u0026to=2024-01-31`\n\n  http://stats.filspark.com/deals/daily\n\n- `GET /deals/summary?from=2024-01-01\u0026to=2024-01-31`\n\n  http://stats.filspark.com/deals/summary\n\n- `GET /retrieval-result-codes/daily?from=2024-01-01\u0026to=2024-01-31`\n  \n  http://stats.filspark.com/retrieval-result-codes/daily\n\n- `GET /retrieval-timings/daily?from=2024-01-01\u0026to=2024-01-31`\n  \n  http://stats.filspark.com/retrieval-result-codes/daily\n\n## Development\n\n### Database\nFor running the tests you will need a postgresql database running on your machine. \nYou can either spin up your own postgres instance or use a prebuilt docker image. \n\n### Custom instance\nSet up [PostgreSQL](https://www.postgresql.org/) with default settings:\n - Port: 5432\n - User: _your system user name_\n - Password: _blank_\n - Database: spark_stats\n\nAlternatively, set the environment variable `$DATABASE_URL` with\n`postgres://${USER}:${PASS}@${HOST}:${POST}/${DATABASE}`.\n\nThe Postgres user and database need to exist already, and the user\nneeds full management permissions for the database.\n\nNext, you need to create `spark_evaluate` database.\n\n```bash\npsql postgres://localhost:5432/ -c \"CREATE DATABASE spark_evaluate\"\n```\n\n### Dockerized instance\nYou can also run the following command to set up the PostgreSQL server via Docker:\n\n```bash\ndocker run -d --name spark-db \\\n  -e POSTGRES_HOST_AUTH_METHOD=trust \\\n  -e POSTGRES_USER=$USER \\\n  -e POSTGRES_DB=$USER \\\n  -p 5432:5432 \\\n  postgres\n```\n\nAfterwards, you need to create the databases `spark_evaluate`and `spark_stats`\n```bash\npsql postgres://$USER@localhost/$USER -c 'CREATE DATABASE spark_stats'\n```\n```bash\npsql postgres://$USER@localhost/$USER -c 'CREATE DATABASE spark_evaluate'\n```\n### Run migration\n\nFinally, run database schema migration scripts.\n\n```bash\nnpm run migrate\n```\n\n### Run the test suite\n\n```sh\nnpm test\n```\n\n### Run the `spark-stats` service\n\n```sh\nnpm start -w stats\n```\n\nYou can also run the service against live data in Spark DB running on Fly.io.\n\n1. Set up a proxy to forward connections to Spark DB Postgres. Connect to the reader replica running\n  on port 5433 (not 5432).\n\n  The command below will forward connections to local post 5455 to Spark DB's reader replica.\n\n  ```\n  fly proxy 5455:5433 -a spark-db\n  ```\n\n2. Start the service and configure the database connection string to use the proxied connection.\n  Look up the user and the password in our shared 1Password vault.\n\n  ```bash\n  DATABASE_URL=\"postgres://user:password@localhost:5455/spark_stats\" \\\n    EVALUATE_DB_URL=\"postgres://user:password@localhost:5455/spark_evaluate\" \\\n    npm start -w stats\n  ```\n\n### Run the `spark-observer` service\n\n```sh\nnpm start -w observer\n```\n\n## Deployment\n\n```\ngit push\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcheckernetwork%2Fspark-stats","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcheckernetwork%2Fspark-stats","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcheckernetwork%2Fspark-stats/lists"}