{"id":13448981,"url":"https://github.com/HFAiLab/OpenCastKit","last_synced_at":"2025-03-22T18:32:13.346Z","repository":{"id":38110469,"uuid":"501622279","full_name":"HFAiLab/OpenCastKit","owner":"HFAiLab","description":"The open-source solutions of FourCastNet and GraphCast","archived":false,"fork":false,"pushed_at":"2023-04-18T02:57:00.000Z","size":3547,"stargazers_count":301,"open_issues_count":24,"forks_count":75,"subscribers_count":6,"default_branch":"master","last_synced_at":"2024-10-29T19:39:51.387Z","etag":null,"topics":["foucastnet","weather-forecast"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/HFAiLab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2022-06-09T11:21:34.000Z","updated_at":"2024-10-14T01:47:17.000Z","dependencies_parsed_at":"2024-01-14T22:03:36.649Z","dependency_job_id":"9529e7d2-8c3b-4f04-a635-9dfe72b858c9","html_url":"https://github.com/HFAiLab/OpenCastKit","commit_stats":{"total_commits":15,"total_committers":1,"mean_commits":15.0,"dds":0.0,"last_synced_commit":"6e3f934533e1aeb6446058a19eb3057b1a304ad2"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HFAiLab%2FOpenCastKit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HFAiLab%2FOpenCastKit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HFAiLab%2FOpenCastKit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HFAiLab%2FOpenCastKit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/HFAiLab","download_url":"https://codeload.github.com/HFAiLab/OpenCastKit/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245002910,"owners_count":20545511,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["foucastnet","weather-forecast"],"created_at":"2024-07-31T06:00:26.919Z","updated_at":"2025-03-22T18:32:08.321Z","avatar_url":"https://github.com/HFAiLab.png","language":"Python","readme":"# OpenCastKit: an open-source solutions of global data-driven high-resolution weather forecasting\n\n简体中文 | [English](README_en.md)\n\n本项目是由幻方AI团队复现优化，并开源的全球AI气象预报模型工具库。基于 [FourCastNet](https://arxiv.org/abs/2202.11214) 和 [GraphCast](https://arxiv.org/abs/2212.12794) 的论文，我们构建了一个新的全球AI气象预报项目——**OpenCastKit**，它能够与欧洲中期天气预报中心（ECMWF）的传统物理模型——高分辨率综合预测系统（IFS），进行直接比较。\n\n我们将基于1979年1月到2022年12月的ERA5数据训练出来的模型参数开源到 [Hugging Face 仓库](https://huggingface.co/hf-ai/OpenCastKit)中，并上线了一个每日更新的 [HF-Earth](https://www.high-flyer.cn/hf-earth/)，展示模型的预测效果。\n\n下面是一些预测案例：\n\n![台风路径预测与真实路径比较](./img/wind_small.gif)\n\n![汽水浓度预测与真实情况比较](./img/precipitation_small.gif)\n\n\n## 依赖\n\n- [hfai](https://doc.hfai.high-flyer.cn/index.html) \u003e= 7.9.5\n- torch \u003e=1.8\n\n\n## 训练\n原始数据来自欧洲中期天气预报中心（ECMWF）提供的一个公开可用的综合数据集 [ERA5](https://www.ecmwf.int/en/forecasts/datasets/reanalysis-datasets/era5) ，需要通过 `data_factory/convert_ear5_hourly.py` 脚本提出数据特征，转化为[高性能训练样本格式 FFRecord](https://www.high-flyer.cn/blog/ffrecord/) 下的样本数据。\n\n\n### 训练 FourCastNet\n\n本地运行：\n```shell\n   python train_fourcastnet.py --pretrain-epochs 100 --fintune-epochs 40 --batch-size 4\n```\n\n也可以提交任务至幻方萤火集群，使用96张A100进行数据并行训练\n```shell\n   hfai python train_fourcastnet.py --pretrain-epochs 100 --fintune-epochs 40 --batch-size 4 -- -n 12 --name train_fourcastnet\n```\n\n### 训练 GraphCast\n\n本地运行：\n```shell\n   python train_graphcast.py --epochs 200 --batch-size 2\n```\n\n也可以提交任务至幻方萤火集群，使用256张A100进行流水线并行训练\n```shell\n   hfai python train_graphcast.py --epochs 200 --batch-size 2 -- -n 32 --name train_graphcast\n```\n\n\n## 引用\n\n```bibtex\n@article{pathak2022fourcastnet,\n  title={Fourcastnet: A global data-driven high-resolution weather model using adaptive fourier neural operators},\n  author={Pathak, Jaideep and Subramanian, Shashank and Harrington, Peter and Raja, Sanjeev and Chattopadhyay, Ashesh and Mardani, Morteza and Kurth, Thorsten and Hall, David and Li, Zongyi and Azizzadenesheli, Kamyar and others},\n  journal={arXiv preprint arXiv:2202.11214},\n  year={2022}\n}\n```\n\n```bibtex\n@article{remi2022graphcast,\n  title={GraphCast: Learning skillful medium-range global weather forecasting},\n  author={Remi Lam, Alvaro Sanchez-Gonzalez, Matthew Willson, Peter Wirnsberger, Meire Fortunato, Alexander Pritzel, Suman Ravuri, Timo Ewalds, Ferran Alet, Zach Eaton-Rosen, Weihua Hu, Alexander Merose, Stephan Hoyer, George Holland, Jacklynn Stott, Oriol Vinyals, Shakir Mohamed, Peter Battaglia},\n  journal={arXiv preprint arXiv:2212.12794},\n  year={2022}\n}\n```","funding_links":[],"categories":["Models","Atmosphere"],"sub_categories":["✨Other kits","Meteorological Observation and Forecast"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FHFAiLab%2FOpenCastKit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FHFAiLab%2FOpenCastKit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FHFAiLab%2FOpenCastKit/lists"}