{"id":15030615,"url":"https://github.com/cure-lab/ltsf-linear","last_synced_at":"2025-05-15T09:08:29.174Z","repository":{"id":37433580,"uuid":"496687161","full_name":"cure-lab/LTSF-Linear","owner":"cure-lab","description":"[AAAI-23 Oral] Official implementation of the paper \"Are Transformers Effective for Time Series Forecasting?\"","archived":false,"fork":false,"pushed_at":"2024-01-27T16:45:10.000Z","size":5943,"stargazers_count":2205,"open_issues_count":26,"forks_count":474,"subscribers_count":17,"default_branch":"main","last_synced_at":"2025-05-15T09:08:19.670Z","etag":null,"topics":["aaai","aaai2023","deep-learning","forecasting","forecasting-model","linear-models","pytorch","time-series","time-series-forecasting","time-series-prediction"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cure-lab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2022-05-26T16:09:29.000Z","updated_at":"2025-05-13T06:25:03.000Z","dependencies_parsed_at":"2024-01-13T07:33:57.757Z","dependency_job_id":"23664794-01c1-431c-b962-f81c9e017577","html_url":"https://github.com/cure-lab/LTSF-Linear","commit_stats":{"total_commits":56,"total_committers":4,"mean_commits":14.0,"dds":0.4107142857142857,"last_synced_commit":"0c113668a3b88c4c4ee586b8c5ec3e539c4de5a6"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cure-lab%2FLTSF-Linear","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cure-lab%2FLTSF-Linear/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cure-lab%2FLTSF-Linear/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cure-lab%2FLTSF-Linear/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cure-lab","download_url":"https://codeload.github.com/cure-lab/LTSF-Linear/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254310520,"owners_count":22049470,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aaai","aaai2023","deep-learning","forecasting","forecasting-model","linear-models","pytorch","time-series","time-series-forecasting","time-series-prediction"],"created_at":"2024-09-24T20:13:52.093Z","updated_at":"2025-05-15T09:08:24.165Z","avatar_url":"https://github.com/cure-lab.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Are Transformers Effective for Time Series Forecasting? (AAAI 2023)\r\n\r\nThis repo is the official Pytorch implementation of LTSF-Linear: \"[Are Transformers Effective for Time Series Forecasting?](https://arxiv.org/pdf/2205.13504.pdf)\". \r\n\r\n\r\n## Updates\r\n- [2024/01/28] Our model has been included in [NeuralForecast](https://github.com/Nixtla/neuralforecast). Special thanks to the contributor [@cchallu](https://github.com/cchallu)!\r\n- [2022/11/23] Accepted to AAAI 2023 with three strong accept! We also release a **[benchmark for long-term time series forecasting](LTSF-Benchmark.md)** for further research.\r\n- [2022/08/25] We update our [paper](https://arxiv.org/pdf/2205.13504.pdf) with comprehensive analyses on why existing LTSF-Transformers do not work well on the LTSF problem!\r\n- [2022/08/25] Besides DLinear, we're excited to add two Linear models to the paper and this repo. Now we have an LTSF-Linear family!\r\n  - Linear: Just one linear layer.\r\n  - DLinear: Decomposition Linear to handle data with trend and seasonality patterns.\r\n  - NLinear: A Normalized Linear to deal with train-test set distribution shifts. See section 'LTSF-Linear' for more details. \r\n\r\n- [2022/08/25] We update some scripts of LTSF-Linear. \r\n  - Linear, NLinear, and DLinear use the same scripts.\r\n  - Some results of DLinear are slightly different now.\r\n\r\n\r\n\r\n## Features\r\n- [x] Add a [benchmark](LTSF-Benchmark.md) for long-term time series forecasting.\r\n- [x] Support both [Univariate](https://github.com/cure-lab/DLinear/tree/main/scripts/EXP-LongForecasting/DLinear/univariate) and [Multivariate](https://github.com/cure-lab/DLinear/tree/main/scripts/EXP-LongForecasting/DLinear) long-term time series forecasting.\r\n- [x] Support visualization of weights.\r\n- [x] Support scripts on different [look-back window size](https://github.com/cure-lab/DLinear/tree/main/scripts/EXP-LookBackWindow).\r\n\r\nBesides LTSF-Linear, we provide five significant forecasting Transformers to re-implement the results in the paper.\r\n- [x] [Transformer](https://arxiv.org/abs/1706.03762) (NeuIPS 2017)\r\n- [x] [Informer](https://arxiv.org/abs/2012.07436) (AAAI 2021 Best paper)\r\n- [x] [Autoformer](https://arxiv.org/abs/2106.13008) (NeuIPS 2021)\r\n- [x] [Pyraformer](https://openreview.net/pdf?id=0EXmFzUn5I) (ICLR 2022 Oral)\r\n- [x] [FEDformer](https://arxiv.org/abs/2201.12740) (ICML 2022)\r\n\r\n\r\n## Detailed Description\r\nWe provide all experiment script files in `./scripts`:\r\n| Files      |                              Interpretation                          |\r\n| ------------- | -------------------------------------------------------| \r\n| EXP-LongForecasting      | Long-term Time Series Forecasting Task                    |\r\n| EXP-LookBackWindow      | Study the impact of different look-back window sizes   | \r\n| EXP-Embedding        | Study the effects of different embedding strategies      |\r\n\r\n\r\nThis code is simply built on the code base of Autoformer. We appreciate the following GitHub repos a lot for their valuable code base or datasets:\r\n\r\nThe implementation of Autoformer, Informer, and Transformer is from https://github.com/thuml/Autoformer\r\n\r\nThe implementation of FEDformer is from https://github.com/MAZiqing/FEDformer\r\n\r\nThe implementation of Pyraformer is from https://github.com/alipay/Pyraformer\r\n\r\n## LTSF-Linear\r\n### LTSF-Linear family\r\n![image](pics/Linear.png)\r\nLTSF-Linear is a set of linear models. \r\n- Linear: It is just a one-layer linear model, but it outperforms Transformers.\r\n- NLinear: **To boost the performance of Linear when there is a distribution shift in the dataset**, NLinear first subtracts the input by the last value of the sequence. Then, the input goes through a linear layer, and the subtracted part is added back before making the final prediction. The subtraction and addition in NLinear are a simple normalization for the input sequence.\r\n- DLinear: It is a combination of a Decomposition scheme used in Autoformer and FEDformer with linear layers. It first decomposes a raw data input into a trend component by a moving average kernel and a remainder (seasonal) component. Then, two one-layer linear layers are applied to each component and we sum up the two features to get the final prediction. By explicitly handling trend, **DLinear enhances the performance of a vanilla linear when there is a clear trend in the data.** \r\n\r\nAlthough LTSF-Linear is simple, it has some compelling characteristics:\r\n- An O(1) maximum signal traversing path length: The shorter the path, the better the dependencies are captured, making LTSF-Linear capable of capturing both short-range and long-range temporal relations.\r\n- High-efficiency: As each branch has only one linear layer, it costs much lower memory and fewer parameters and has a faster inference speed than existing Transformers.\r\n- Interpretability: After training, we can visualize weights to have some insights into the predicted values.\r\n- Easy-to-use: LTSF-Linear can be obtained easily without tuning model hyper-parameters.\r\n\r\n### Comparison with Transformers\r\nUnivariate Forecasting:\r\n![image](pics/Uni-results.png)\r\nMultivariate Forecasting:\r\n![image](pics/Mul-results.png)\r\nLTSF-Linear outperforms all transformer-based methods by a large margin.\r\n\r\n### Efficiency\r\n![image](pics/efficiency.png)\r\nComparison of method efficiency with Look-back window size 96 and Forecasting steps 720 on Electricity. MACs are the number of multiply-accumulate operations. We use DLinear for comparison since it has the double cost in LTSF-Linear. The inference time averages 5 runs.\r\n\r\n## Getting Started\r\n### Environment Requirements\r\n\r\nFirst, please make sure you have installed Conda. Then, our environment can be installed by:\r\n```\r\nconda create -n LTSF_Linear python=3.6.9\r\nconda activate LTSF_Linear\r\npip install -r requirements.txt\r\n```\r\n\r\n### Data Preparation\r\n\r\nYou can obtain all the nine benchmarks from [Google Drive](https://drive.google.com/drive/folders/1ZOYpTUa82_jCcxIdTmyr0LXQfvaM9vIy) provided in Autoformer. All the datasets are well pre-processed and can be used easily.\r\n\r\n```\r\nmkdir dataset\r\n```\r\n**Please put them in the `./dataset` directory**\r\n\r\n### Training Example\r\n- In `scripts/ `, we provide the model implementation *Dlinear/Autoformer/Informer/Transformer*\r\n- In `FEDformer/scripts/`, we provide the *FEDformer* implementation\r\n- In `Pyraformer/scripts/`, we provide the *Pyraformer* implementation\r\n\r\nFor example:\r\n\r\nTo train the **LTSF-Linear** on **Exchange-Rate dataset**, you can use the script `scripts/EXP-LongForecasting/Linear/exchange_rate.sh`:\r\n```\r\nsh scripts/EXP-LongForecasting/Linear/exchange_rate.sh\r\n```\r\nIt will start to train DLinear by default, the results will be shown in `logs/LongForecasting`. You can specify the name of the model in the script. (Linear, DLinear, NLinear)\r\n\r\nAll scripts about using LTSF-Linear on long forecasting task is in `scripts/EXP-LongForecasting/Linear/`, you can run them in a similar way. The default look-back window in scripts is 336, LTSF-Linear generally achieves better results with longer look-back window as dicussed in the paper. \r\n\r\nScripts about look-back window size and long forecasting of FEDformer and Pyraformer are in `FEDformer/scripts` and `Pyraformer/scripts`, respectively. To run them, you need to first `cd FEDformer` or `cd Pyraformer`. Then, you can use sh to run them in a similar way. Logs will be stored in `logs/`.\r\n\r\nEach experiment in `scripts/EXP-LongForecasting/Linear/` takes 5min-20min. For other Transformer scripts, since we put all related experiments in one script file, directly running them will take 8 hours per day. You can keep the experiments you are interested in and comment on the others. \r\n\r\n### Weights Visualization\r\nAs shown in our paper, the weights of LTSF-Linear can reveal some characteristics of the data, i.e., the periodicity. As an example, we provide the weight visualization of DLinear in `weight_plot.py`. To run the visualization, you need to input the model path (model_name) of DLinear (the model directory in `./checkpoint` by default). To obtain smooth and clear patterns, you can use the initialization we provided in the file of linear models.  \r\n\r\n![image](pics/Visualization_DLinear.png)\r\n## Citing\r\n\r\nIf you find this repository useful for your work, please consider citing it as follows:\r\n\r\n```BibTeX\r\n@inproceedings{Zeng2022AreTE,\r\n  title={Are Transformers Effective for Time Series Forecasting?},\r\n  author={Ailing Zeng and Muxi Chen and Lei Zhang and Qiang Xu},\r\n  journal={Proceedings of the AAAI Conference on Artificial Intelligence},\r\n  year={2023}\r\n}\r\n```\r\n\r\nPlease remember to cite all the datasets and compared methods if you use them in your experiments.\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcure-lab%2Fltsf-linear","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcure-lab%2Fltsf-linear","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcure-lab%2Fltsf-linear/lists"}