{"id":28564713,"url":"https://github.com/cronian-tech/saturn-observatory","last_synced_at":"2025-06-10T13:36:47.821Z","repository":{"id":192406939,"uuid":"686300158","full_name":"cronian-tech/saturn-observatory","owner":"cronian-tech","description":"Bringing transparency to Filecoin Saturn by gathering and publishing open data about the network along with analytical reports every month.","archived":true,"fork":false,"pushed_at":"2024-06-01T05:57:25.000Z","size":16785,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-06-01T06:52:47.001Z","etag":null,"topics":["analytics","dcdn","duckdb","filecoin","plotly","saturn","sql"],"latest_commit_sha":null,"homepage":"https://observatory.cronian.tech/","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cronian-tech.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-09-02T10:35:31.000Z","updated_at":"2024-06-01T06:01:16.000Z","dependencies_parsed_at":"2023-10-01T18:26:49.718Z","dependency_job_id":"66f7b345-197d-445f-a4da-4dbc576f19e3","html_url":"https://github.com/cronian-tech/saturn-observatory","commit_stats":{"total_commits":71,"total_committers":1,"mean_commits":71.0,"dds":0.0,"last_synced_commit":"d45440815cea6b5b061b4ddcc0ccf26f8deefd6a"},"previous_names":["31z4/saturn-observatory","cronian-tech/saturn-observatory"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cronian-tech%2Fsaturn-observatory","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cronian-tech%2Fsaturn-observatory/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cronian-tech%2Fsaturn-observatory/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cronian-tech%2Fsaturn-observatory/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cronian-tech","download_url":"https://codeload.github.com/cronian-tech/saturn-observatory/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cronian-tech%2Fsaturn-observatory/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259085512,"owners_count":22803212,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["analytics","dcdn","duckdb","filecoin","plotly","saturn","sql"],"created_at":"2025-06-10T13:36:40.144Z","updated_at":"2025-06-10T13:36:47.801Z","avatar_url":"https://github.com/cronian-tech.png","language":"JavaScript","funding_links":[],"categories":["Use Cases"],"sub_categories":["Lilypad Community Modules"],"readme":"**On June 1st, 2024, the Saturn node provider program was concluded.**\n\n# 🔭 Saturn Observatory\n\nThe goal of this project is to increase the transparency of 🪐 [Filecoin Saturn](https://saturn.tech) network – the fastest growing, community-run distributed content delivery network (CDN) for Web3.\n\nWe use historical data about the network gathered by 🌖 [Saturn Moonlet](https://github.com/cronian-tech/saturn-moonlet) to prepare and publish analytical reports on Saturn Observatory [website](https://observatory.cronian.tech) every month. Raw data that is used to generate the reports [is available](https://gateway.ipfs.io/ipfs/bafybeibrmxyaolu226uqj3bv7qqjtjmmzugigsq6n3qzx74xpz2yt73yha) on IPFS and Filecoin.\n\nSaturn Observatory compliments official tools like [Saturn Node Dashboard](https://dashboard.saturn.tech) and [Saturn Explorer](https://explorer.saturn.tech), aiming to provide better insights into the network state and performance. To highlight a few:\n\n#### [Network Size \u0026 Traffic](https://observatory.cronian.tech/#network-size-and-traffic) shows the number of Saturn nodes and network traffic over time.\n\n\u003cimg width=\"931\" alt=\"image\" src=\"https://github.com/cronian-tech/saturn-observatory/assets/3657959/0d89ae2b-6f8c-4227-a425-e75aa83e5737\"\u003e\n\n#### [Earnings \u0026 Traffic by Country](https://observatory.cronian.tech/#earnings-and-traffic-by-country) can help node operators identify saturated (in terms of node count) regions and find the most advantageous geographical locations.\n\n\u003cimg width=\"917\" alt=\"image\" src=\"https://github.com/cronian-tech/saturn-observatory/assets/3657959/2abdeaa8-019f-46c1-bc59-23988102cb21\"\u003e\n\n#### [Nodes Without Traffic](https://observatory.cronian.tech/#nodes-without-traffic) may highlight issues with network traffic distribution.\n\n\u003cimg width=\"887\" alt=\"image\" src=\"https://github.com/cronian-tech/saturn-observatory/assets/3657959/d09b73d8-b6d3-4040-98c4-fe4146d5eebb\"\u003e\n\nHead over to Saturn Observatory [website](https://observatory.cronian.tech) to see the full report. If you're curious, you can see [how the project is made](#how-its-made).\n\nSaturn Observatory is neither affiliated with the [Filecoin Saturn](https://github.com/filecoin-saturn) project nor the [Protocol Labs](https://protocol.ai) organization.\nHowever, it was built during the [Open Data Hack](https://www.blog.encode.club/open-data-hack-powered-by-filecoin-prizewinners-and-summary-dcdf52059867) powered by Filecoin and received a Grand Prize from Saturn 🏆 and Honorable mention from Lilypad 🏅\n\n## 🛠 How it's made\n\nTo create reports we export historical data about the Saturn network from a running [Saturn Moonlet](https://moonlet.cronian.tech) long-term storage into a set of [CSV](https://en.wikipedia.org/wiki/Comma-separated_values) files. We run a bunch of analytical [SQL queries](analytics.sql) against these files and output results into CSVs. Finally, we use the results to generate plots and publish them on the Saturn Observatory website.\n\nThe following is a more detailed explanation of each step.\n\n1.  Everything starts with exporting Saturn metrics into CSV:\n\n        EXPORTER_PASSWORD=secret ./moonlet/export-csv.sh 2023-09-01 2023-10-01\n\n    This [shell script](moonlet/export-csv.sh) uses VictoriaMetrics export API for some metrics and a small [Python script](moonlet/export.py) for others. This is because we need to apply a rollup function to some metrics (e.g., `increase(saturn_node_retrievals_total)`).\n\n2.  Now we have raw CSV data that we upload and pin to IPFS and store on Filecoin using [web3.storage](https://web3.storage):\n\n        make web3-storage-upload-inputs\n\n    Before running this command for the first time, we need to set our web3.storage token: `make web3-storage-token`.\n\n3.  To perform the actual analysis we run a [Bacalhau]() job that uses [DuckDB]() to execute a bunch of [SQL queries](analytics.sql) on the input CSV data that we previously pinned to IPFS:\n\n        make bacalhau-analytics cid=bafybeiarymtc6w32n2ud6w27vbhjqf2seax65l2rrfsfyucxc4gjutugni\n\n    We wanted to use [Lilypad](https://docs.lilypadnetwork.org) to run the analysis and made a couple of PRs ([one](https://github.com/bacalhau-project/lilypad-modicum/pull/80), [two](https://github.com/bacalhau-project/lilypad-docs/pull/9), [three](https://github.com/31z4/lilypad-duckdb)) with a custom DuckDB module. But, by the time of writing the module is not yet available on Lilypad testnet. This was mostly blocked on Lilypad's team.\n\n4.  Once the Bacalhau job finishes we download its results and make sure that these results are adequate (check for outliers, empty values, compare with the previous month, etc.). Then store the results in IPFS and Filecoin using web3.storage:\n\n        make web3-storage-upload-outputs\n\n5.  Finally, we plug the CID that we get from the previous step into `dataUrl` function of [`web/main.js`](web/main.js). We commit and push this change to the project's repo and the Saturn Observatory website gets published using GitHub pages.\n\n6.  When you open the Saturn Observatory website, CSV data from step 4 is fetched using the [Saturn browser client](https://github.com/filecoin-saturn/browser-client) and then plotted using [PlotlyJS](https://plotly.com/javascript).\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcronian-tech%2Fsaturn-observatory","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcronian-tech%2Fsaturn-observatory","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcronian-tech%2Fsaturn-observatory/lists"}