{"id":28366981,"url":"https://github.com/thuml/sundial","last_synced_at":"2025-10-28T03:36:09.084Z","repository":{"id":294880699,"uuid":"983283637","full_name":"thuml/Sundial","owner":"thuml","description":"About model release for \"Sundial: A Family of Highly Capable Time Series Foundation Models\" (ICML 2025 Oral)","archived":false,"fork":false,"pushed_at":"2025-06-19T11:24:11.000Z","size":15362,"stargazers_count":35,"open_issues_count":3,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-19T12:31:03.746Z","etag":null,"topics":["generative-models","large-time-series-model","time-series","time-series-foundation-models","transformer"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2502.00816","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/thuml.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-14T06:45:25.000Z","updated_at":"2025-06-19T11:24:14.000Z","dependencies_parsed_at":"2025-06-19T12:34:09.218Z","dependency_job_id":null,"html_url":"https://github.com/thuml/Sundial","commit_stats":null,"previous_names":["thuml/sundial"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/thuml/Sundial","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thuml%2FSundial","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thuml%2FSundial/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thuml%2FSundial/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thuml%2FSundial/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/thuml","download_url":"https://codeload.github.com/thuml/Sundial/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thuml%2FSundial/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260901621,"owners_count":23079797,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["generative-models","large-time-series-model","time-series","time-series-foundation-models","transformer"],"created_at":"2025-05-29T00:13:48.195Z","updated_at":"2025-10-28T03:36:09.078Z","avatar_url":"https://github.com/thuml.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# Sundial\n\nThis is the official repository of [Sundial: A Family of Highly Capable  Time Series Foundation Models](https://arxiv.org/abs/2502.00816) [[Slides]](https://cloud.tsinghua.edu.cn/f/8d526337afde465e87c9/) [[Poster]](https://cloud.tsinghua.edu.cn/f/cc2a156315e9453f99b3/) [[Intro (CN)]](https://mp.weixin.qq.com/s/y3sc2e2lmW1sqfnoK-ZdDA).\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"./figures/cover.png\" alt=\"\" align=center /\u003e\n\u003c/p\u003e\n\n## Updates\n\n:triangular_flag_on_post: **News** (2025.06) Sundial has been accepted as **ICML 2025 Oral** (Top 1%). See you at Vancouver :)\n\n:triangular_flag_on_post: **News** (2025.05) Get **1st MASE** on the [GIFT-Eval](https://huggingface.co/spaces/Salesforce/GIFT-Eval) Benchmark.\n\n:triangular_flag_on_post: **News** (2025.05) Released a **trillion-scale** pre-trained model on [HuggingFace](https://huggingface.co/thuml/sundial-base-128m). A quickstart is provided [here](./examples/quickstart_zero_shot_generation.ipynb).\n\n:triangular_flag_on_post: **News** (2025.02) Get **1st MSE/MAE** zero-shot performance on [Time-Series-Library](https://github.com/thuml/Time-Series-Library) datasets.\n\n## Introduction\n\nSundial is a family of **generative** time series foundation models, which is pre-trained on TimeBench (**10^12** time points). The model can be applied for both **point** / **probabilistic** **zero-shot** forecasting.\n\nNot only the mean or quantiles, you can get any statistical predictions with a set of generated samples.\n\nWe propose **TimeFlow Loss** to predict next-patch’s distribution, allowing Transformers to be trained **without discrete tokenization** and make **non-deterministic predictions**.\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"./figures/compare.png\" alt=\"\" align=center /\u003e\n\u003c/p\u003e\n\n## Quickstart\n\nWe release a [HuggingFace model](https://huggingface.co/thuml/sundial-base-128m), which can make zero-shot predictions on CPU within seconds! 🚀\n\n\u003e Inference Time on Apple M1 Pro CPU (16 GB)\n\n| Lookback | Forcast | # Generated | Wall-Clock Time | Accelerate By         |\n| --------------- | ----------------- | ------------------- | -------------- | --------------         |\n| 672             | 16                | 1                   | 249ms          |       -                |\n| 2880            | 16                | 1                   | 510ms          | FlashAttention         |\n| 2880            | 720               | 1                   | 510ms          | Multi-Patch Prediction |\n| 2880            | 1440              | 1                   | 789ms          | KV Cache               |\n| 2880            | 720               | 20                  | 949ms          | Shared Condition       |\n\nAll you need is a network and a HuggingFace account!\n\n```\npip install transformers==4.40.1\n```\n\n```\nimport torch\nfrom transformers import AutoModelForCausalLM\n\n# load pretrain model\n# supports different lookback/forecast lengths\nmodel = AutoModelForCausalLM.from_pretrained('thuml/sundial-base-128m', trust_remote_code=True) \n\n# prepare input\nbatch_size, lookback_length = 1, 2880 \nseqs = torch.randn(batch_size, lookback_length)\n\n# Note that Sundial can generate multiple probable predictions\nforecast_length = 96 \nnum_samples = 20\n\noutput = model.generate(seqs, max_new_tokens=forecast_length, num_samples=num_samples)\n\n# use raw predictions for mean/quantiles/confidence-interval estimation\nprint(output.shape) \n```\n\nMore examples of predicting quantiles or confidence intervals are provided in this [notebook](https://github.com/thuml/Sundial/blob/main/examples/quickstart_zero_shot_generation.ipynb). Please raise your valuable suggestions [here](https://huggingface.co/thuml/sundial-base-128m/discussions/new), we 'd like to solve it ASAP 🤗.\n\n\n\n## Architecture\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"./figures/arch.png\" alt=\"\" align=center /\u003e\n\u003c/p\u003e\n\n\u003e Intuitively, Sundial can be viewed as an **ARMA** model (Auto-Regression and Moving-Average). Transformer learns auto-regressive token representations. Conditioned on them, TimeFlow transforms random noises into non-deterministic predictions.\n\n## Model Configurations\n\nWe have currently built three different sizes of Sundial. Model configurations are provided here:\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"./figures/config.png\" alt=\"\" align=center /\u003e\n\u003c/p\u003e\n\n## Evaluation\n\nWe evaluate Sundial (Base) with advanced time series foundation models on well-recognized benchmarks:\n\n- [GIFT-Eval (1st MASE)](https://cdn-uploads.huggingface.co/production/uploads/64fbe24a2d20ced4e91de38a/3BxatwayhK5GAoqMf1oHv.png) [[Leaderboard]](https://huggingface.co/spaces/Salesforce/GIFT-Eval).\n- [Time-Series-Library (1st MSE/MAE)](https://cdn-uploads.huggingface.co/production/uploads/64fbe24a2d20ced4e91de38a/5VqnFwWTWoYz877Zkluiw.png).\n- [FEV Leaderboard](https://cdn-uploads.huggingface.co/production/uploads/64fbe24a2d20ced4e91de38a/mrKL9QmX-aX8rCiwxKgmA.png).\n\n## Exciting News ✨\n\n Code for fine-tuning is on its way and will be available soon! Stay tuned for updates!\n\n## Citation\n\nIf you find this repo helpful, please cite our paper. \n\n\n```\n@article{liu2025sundial,\n  title={Sundial: A Family of Highly Capable Time Series Foundation Models},\n  author={Liu, Yong and Qin, Guo and Shi, Zhiyuan and Chen, Zhi and Yang, Caiyin and Huang, Xiangdong and Wang, Jianmin and Long, Mingsheng},\n  journal={arXiv preprint arXiv:2502.00816},\n  year={2025}\n}\n```\n\n## Acknowledgment\n\nWe appreciate the following resources a lot for their valuable code and datasets:\n\n- Time-Series-Library (https://github.com/thuml/Time-Series-Library)\n- Large-Time-Series-Model \u0026 UTSD (https://github.com/thuml/Large-Time-Series-Model)\n- Timer-XL (https://github.com/thuml/Timer-XL)\n- LoTSA Data (https://huggingface.co/datasets/Salesforce/lotsa_data)\n- Chronos Datasets (https://huggingface.co/datasets/autogluon/chronos_datasets)\n\n## Contact\n\nIf you have any questions or want to use the code, feel free to contact:\n\n* Yong Liu (liuyong21@mails.tsinghua.edu.cn)\n* Guo Qin (qinguo24@mails.tsinghua.edu.cn)\n\n## License\n\nThis model is licensed under the Apache-2.0 License.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthuml%2Fsundial","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthuml%2Fsundial","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthuml%2Fsundial/lists"}