{"id":13510411,"url":"https://github.com/thuml/Autoformer","last_synced_at":"2025-03-30T15:31:27.015Z","repository":{"id":37099709,"uuid":"420947391","full_name":"thuml/Autoformer","owner":"thuml","description":"About Code release for \"Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting\" (NeurIPS 2021), https://arxiv.org/abs/2106.13008","archived":false,"fork":false,"pushed_at":"2025-02-28T01:14:54.000Z","size":2229,"stargazers_count":2098,"open_issues_count":1,"forks_count":444,"subscribers_count":14,"default_branch":"main","last_synced_at":"2025-03-23T23:01:54.631Z","etag":null,"topics":["deep-learning","time-series"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/thuml.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-10-25T08:52:35.000Z","updated_at":"2025-03-23T07:44:22.000Z","dependencies_parsed_at":"2023-02-16T05:15:35.543Z","dependency_job_id":"55f0bfee-795d-428e-bf02-bcb6b4ffb0aa","html_url":"https://github.com/thuml/Autoformer","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thuml%2FAutoformer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thuml%2FAutoformer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thuml%2FAutoformer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thuml%2FAutoformer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/thuml","download_url":"https://codeload.github.com/thuml/Autoformer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246339001,"owners_count":20761473,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","time-series"],"created_at":"2024-08-01T02:01:37.799Z","updated_at":"2025-03-30T15:31:22.006Z","avatar_url":"https://github.com/thuml.png","language":"Jupyter Notebook","funding_links":[],"categories":["Jupyter Notebook","时间序列","time-series"],"sub_categories":["网络服务_其他"],"readme":"# Autoformer (NeurIPS 2021)\n\nAutoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting\n\nTime series forecasting is a critical demand for real applications. Enlighted by the classic time series analysis and stochastic process theory, we propose the Autoformer as a general series forecasting model [[paper](https://arxiv.org/abs/2106.13008)]. **Autoformer goes beyond the Transformer family and achieves the series-wise connection for the first time.**\n\nIn long-term forecasting, Autoformer achieves SOTA, with a **38% relative improvement** on six benchmarks, covering five practical applications: **energy, traffic, economics, weather and disease**.\n\n:triangular_flag_on_post:**News** (2023.08) Autoformer has been included in [Hugging Face](https://huggingface.co/models?search=autoformer). See [blog](https://huggingface.co/blog/autoformer).\n\n:triangular_flag_on_post:**News** (2023.06) The extension version of Autoformer ([Interpretable weather forecasting for worldwide stations with a unified deep model](https://www.nature.com/articles/s42256-023-00667-9)) has been published in Nature Machine Intelligence as the [Cover Article](https://www.nature.com/natmachintell/volumes/5/issues/6).\n\n:triangular_flag_on_post:**News** (2023.02) Autoformer has been included in our [[Time-Series-Library]](https://github.com/thuml/Time-Series-Library), which covers long- and short-term forecasting, imputation, anomaly detection, and classification.\n\n:triangular_flag_on_post:**News** (2022.02-2022.03) Autoformer has been deployed in [2022 Winter Olympics](https://en.wikipedia.org/wiki/2022_Winter_Olympics) to provide weather forecasting for competition venues, including wind speed and temperature.\n\n## Autoformer vs. Transformers\n\n**1. Deep decomposition architecture**\n\nWe renovate the Transformer as a deep decomposition architecture, which can progressively decompose the trend and seasonal components during the forecasting process.\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\".\\pic\\Autoformer.png\" height = \"250\" alt=\"\" align=center /\u003e\n\u003cbr\u003e\u003cbr\u003e\n\u003cb\u003eFigure 1.\u003c/b\u003e Overall architecture of Autoformer.\n\u003c/p\u003e\n\n**2. Series-wise Auto-Correlation mechanism**\n\nInspired by the stochastic process theory, we design the Auto-Correlation mechanism, which can discover period-based dependencies and aggregate the information at the series level. This empowers the model with inherent log-linear complexity. This series-wise connection contrasts clearly from the previous self-attention family.\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\".\\pic\\Auto-Correlation.png\" height = \"250\" alt=\"\" align=center /\u003e\n\u003cbr\u003e\u003cbr\u003e\n\u003cb\u003eFigure 2.\u003c/b\u003e Auto-Correlation mechansim.\n\u003c/p\u003e\n\n## Get Started\n\n1. Install Python 3.6, PyTorch 1.9.0.\n2. Download data. You can obtain all the six benchmarks from [Google Drive](https://drive.google.com/drive/folders/1ZOYpTUa82_jCcxIdTmyr0LXQfvaM9vIy?usp=sharing). **All the datasets are well pre-processed** and can be used easily.\n3. Train the model. We provide the experiment scripts of all benchmarks under the folder `./scripts`. You can reproduce the experiment results by:\n\n```bash\nbash ./scripts/ETT_script/Autoformer_ETTm1.sh\nbash ./scripts/ECL_script/Autoformer.sh\nbash ./scripts/Exchange_script/Autoformer.sh\nbash ./scripts/Traffic_script/Autoformer.sh\nbash ./scripts/Weather_script/Autoformer.sh\nbash ./scripts/ILI_script/Autoformer.sh\n```\n\n4. Special-designed implementation\n\n- **Speedup Auto-Correlation:** We built the Auto-Correlation mechanism as a batch-normalization-style block to make it more memory-access friendly. See the [paper](https://arxiv.org/abs/2106.13008) for details.\n\n- **Without the position embedding:** Since the series-wise connection will inherently keep the sequential information, Autoformer does not need the position embedding, which is different from Transformers.\n\n### Reproduce with Docker\n\nTo easily reproduce the results using Docker, conda and Make,  you can follow the next steps:\n1. Initialize the docker image using: `make init`. \n2. Download the datasets using: `make get_dataset`.\n3. Run each script in `scripts/` using `make run_module module=\"bash scripts/ETT_script/Autoformer_ETTm1.sh\"` for each script.\n4. Alternatively, run all the scripts at once:\n```\nfor file in `ls scripts`; do make run_module module=\"bash scripts/$script\"; done\n```\n### A Simple Example\nSee `predict.ipynb` for workflow (in Chinese).\n\n## Main Results\n\nWe experiment on six benchmarks, covering five main-stream applications. We compare our model with ten baselines, including Informer, N-BEATS, etc. Generally, for the long-term forecasting setting, Autoformer achieves SOTA, with a **38% relative improvement** over previous baselines.\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\".\\pic\\results.png\" height = \"550\" alt=\"\" align=center /\u003e\n\u003c/p\u003e\n\n## Baselines\n\nWe will keep adding series forecasting models to expand this repo:\n\n- [x] Autoformer\n- [x] Informer\n- [x] Transformer\n- [x] Reformer\n- [ ] LogTrans\n- [ ] N-BEATS\n\n## Citation\n\nIf you find this repo useful, please cite our paper. \n\n```\n@inproceedings{wu2021autoformer,\n  title={Autoformer: Decomposition Transformers with {Auto-Correlation} for Long-Term Series Forecasting},\n  author={Haixu Wu and Jiehui Xu and Jianmin Wang and Mingsheng Long},\n  booktitle={Advances in Neural Information Processing Systems},\n  year={2021}\n}\n```\n\n## Contact\n\nIf you have any questions or want to use the code, please contact wuhx23@mails.tsinghua.edu.cn.\n\n## Acknowledgement\n\nWe appreciate the following github repos a lot for their valuable code base or datasets:\n\nhttps://github.com/zhouhaoyi/Informer2020\n\nhttps://github.com/zhouhaoyi/ETDataset\n\nhttps://github.com/laiguokun/multivariate-time-series-data\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthuml%2FAutoformer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthuml%2FAutoformer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthuml%2FAutoformer/lists"}