{"id":18401692,"url":"https://github.com/borealisai/scaleformer","last_synced_at":"2025-04-07T07:31:36.651Z","repository":{"id":75695714,"uuid":"499646004","full_name":"BorealisAI/scaleformer","owner":"BorealisAI","description":null,"archived":false,"fork":false,"pushed_at":"2023-02-07T20:44:51.000Z","size":3485,"stargazers_count":130,"open_issues_count":3,"forks_count":13,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-03-22T14:11:51.154Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/BorealisAI.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-06-03T20:41:38.000Z","updated_at":"2025-03-12T15:30:23.000Z","dependencies_parsed_at":"2023-06-07T09:45:22.710Z","dependency_job_id":null,"html_url":"https://github.com/BorealisAI/scaleformer","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BorealisAI%2Fscaleformer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BorealisAI%2Fscaleformer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BorealisAI%2Fscaleformer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BorealisAI%2Fscaleformer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/BorealisAI","download_url":"https://codeload.github.com/BorealisAI/scaleformer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247612085,"owners_count":20966663,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-06T02:39:38.926Z","updated_at":"2025-04-07T07:31:36.641Z","avatar_url":"https://github.com/BorealisAI.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n\n# Scaleformer\n**[Scaleformer: Iterative Multi-scale Refining Transformers for Time Series Forecasting](https://arxiv.org/abs/2206.04038), ICLR 2023**\n\nThis repo is a modified version of the public implementation of [Autoformer paper](https://arxiv.org/abs/2106.13008) which can be find in this [repository](https://github.com/thuml/Autoformer). We also use the related parts of [FEDformer](https://github.com/MAZiqing/FEDformer), [FiLM](https://github.com/tianzhou2011/FiLM/), and [NHits](https://github.com/Nixtla/neuralforecast).\n\n## Why Scaleformer?\nUsing iteratively refining a forecasted time series at multiple scales with shared weights, architecture adaptations and a specially-designed normalization scheme, we are able to achieve significant performance improvements with minimal additional computational overhead.\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"figs\\teaser.png\" width=90% alt=\"\" align=center /\u003e\n\u003cbr\u003e\u003cbr\u003e\n\u003cb\u003eFigure 1.\u003c/b\u003e Overview of the proposed framework. (\u003cb\u003eLeft\u003c/b\u003e) Representation of a single scaling block. In each step, we pass the normalized upsampled version of the output from previous step along with the normalized downsampled version of encoder as the input. (\u003cb\u003eRight\u003c/b\u003e) Representation of the full architecture. We process the input in a multi-scale manner iteratively from the smallest scale to the original scale.\n\u003c/p\u003e\n\n\nOur experiments on various public datasets demonstrate that the proposed method outperforms the corresponding baselines. Depending on the choice of transformer architecture, our mutli-scale framework results in mean squared error reductions ranging from 5.5% to 38.5%.\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"figs\\table.png\" width=90% alt=\"\" align=center /\u003e\n\u003cbr\u003e\u003cbr\u003e\n\u003cb\u003eTable 1.\u003c/b\u003e \nComparison of the MSE and MAE results for our proposed multi-scale framework version of different methods (\u003cb\u003e-MSA\u003c/b\u003e) with respective baselines. Results are given in the multi-variate setting, for different lenghts of the horizon window. The best results are shown in \u003cb\u003eBold\u003c/b\u003e. Our method outperforms vanilla version of the baselines over almost all datasets and settings. The average improvement (error reduction) is shown in Green numbers at the bottom with respect the base models.\n\u003c/p\u003e\n\n## Installation\n**1. Clone our repo and install the requirements:**\n```\ngit clone https://github.com/BorealisAI/scaleformer.git\ncd scaleformer\npip install -r requirements.txt\n```\n**2. Download datasets and create the dataset directory**\nYou can download the datasets from [Tsinghua Cloud](https://cloud.tsinghua.edu.cn/d/e1ccfff39ad541908bae/) or [Google Drive](https://drive.google.com/drive/folders/1ZOYpTUa82_jCcxIdTmyr0LXQfvaM9vIy?usp=sharing) links provided by [Autoformer](https://github.com/thuml/Autoformer) repository. For more information, please visit the repository.\n\nPut all of the downloaded datasets in a `dataset` folder in the current directory:\n\n```\nscaleformer\n├── dataset\n│   ├── exchange_rate\n|   |   └── exchange_rate.csv\n│   ├── traffic\n|   |   └── traffic.csv\n|   └── ...\n└── data_provider\n└── exp\n└── ...\n```\n\n## Running the code\n\n**1. running a single experiment**\n\nYou can run a single experiment using the following command:\n```\npython -u run.py --data_path {DATASET} --model {MODEL} --pred_len {L} --loss {LOSS_FUNC}\n```\nfor example, for using **Informer-MSA** as the model for **traffic** dataset with an output length of **192** and **adaptive** loss function, you can run:\n```\npython -u run.py --data_path traffic.csv --model InformerMS --pred_len 192 --loss adaptive\n```\nTo see more examples and parameters, please see `run_all.sh`. \n\n**2. Running all of the experiments**\n\nTo run all of the experiments using slurm, you can use `run_all.sh` which uses `run_single.sh` to submit jobs with different parameters. The final errors of experiments will be available in `results.txt` and you can check `slurm` directory for the log of each experiment.\n\n\n## Contact\n\nIf you have any question regarding the ScaleFormer, please contact aminshabaany@gmail.com.\n\n## Citation\n\n```\n@article{shabani2022scaleformer,\n  title={Scaleformer: Iterative Multi-scale Refining Transformers for Time Series Forecasting},\n  author={Shabani, Amin and Abdi, Amir and Meng, Lili and Sylvain, Tristan},\n  journal={arXiv preprint arXiv:2206.04038},\n  year={2022}\n}\n```\n\n## Acknowledgement\n\nWe acknowledge the following github repositories that made the base of our work:\n\nhttps://github.com/thuml/Autoformer\n\nhttps://github.com/MAZiqing/FEDformer\n\nhttps://github.com/jonbarron/robust_loss_pytorch.git\n\nhttps://github.com/Nixtla/neuralforecast\n\nhttps://github.com/tianzhou2011/FiLM/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fborealisai%2Fscaleformer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fborealisai%2Fscaleformer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fborealisai%2Fscaleformer/lists"}