{"id":42416703,"url":"https://github.com/automl/tempopfn","last_synced_at":"2026-01-28T02:02:08.837Z","repository":{"id":320831928,"uuid":"1082531513","full_name":"automl/TempoPFN","owner":"automl","description":"Official code release for the paper \"TempoPFN: Synthetic Pre-training of Linear RNNs for Zero-shot Time Series Forecasting\"","archived":false,"fork":false,"pushed_at":"2025-11-04T15:05:25.000Z","size":271,"stargazers_count":12,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-11-04T16:13:45.825Z","etag":null,"topics":["foundation-model","synthetic-data-generation","time-series-forecasting"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2510.25502","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/automl.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-10-24T11:34:11.000Z","updated_at":"2025-11-04T15:05:31.000Z","dependencies_parsed_at":"2025-10-26T07:39:41.878Z","dependency_job_id":"1d50e33e-fd46-42cc-a25c-b944c91ff083","html_url":"https://github.com/automl/TempoPFN","commit_stats":null,"previous_names":["automl/tempopfn"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/automl/TempoPFN","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/automl%2FTempoPFN","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/automl%2FTempoPFN/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/automl%2FTempoPFN/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/automl%2FTempoPFN/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/automl","download_url":"https://codeload.github.com/automl/TempoPFN/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/automl%2FTempoPFN/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28833355,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-27T23:29:49.665Z","status":"online","status_checked_at":"2026-01-28T02:00:06.943Z","response_time":57,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["foundation-model","synthetic-data-generation","time-series-forecasting"],"created_at":"2026-01-28T02:01:45.051Z","updated_at":"2026-01-28T02:02:08.832Z","avatar_url":"https://github.com/automl.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# TempoPFN: Synthetic Pre-Training of Linear RNNs for Zero-Shot Time Series Forecasting\n\n[![preprint](https://img.shields.io/static/v1?label=Paper\u0026message=2509.26468\u0026color=B31B1B\u0026logo=arXiv)](https://arxiv.org/abs/2510.25502) [![GIFT-Eval](https://img.shields.io/badge/%F0%9F%8F%86%20GIFT--Eval-Leaderboard-0078D4)](https://huggingface.co/spaces/Salesforce/GIFT-Eval) [![huggingface](https://img.shields.io/badge/%F0%9F%A4%97%20HF-Model_Repo-FFD21E)](https://huggingface.co/AutoML-org/TempoPFN) [![License: Apache 2.0](https://img.shields.io/badge/License-Apache%202.0-green.svg)](https://github.com/automl/TempoPFN/blob/main/LICENSE)\n\n---\n\n**TempoPFN** introduced in [TempoPFN: Synthetic Pre-Training of Linear RNNs for Zero-Shot Time Series Forecasting](https://arxiv.org/abs/2510.25502), is a univariate time series foundation model pretrained **entirely on synthetic data**. It delivers top-tier zero-shot forecasting accuracy while remaining fully reproducible and free from real-data leakage.\n\nBuilt on a **Linear RNN (GatedDeltaProduct)** backbone, TempoPFN performs end-to-end forecasting without patching or windowing. Its design enables fully parallelizable training and inference while maintaining stable temporal state-tracking across long sequences. The GatedDeltaProduct architecture is based on [DeltaProduct](https://arxiv.org/html/2502.10297v3), extended with state-weaving for time series forecasting. For detailed information about the architecture and custom modifications, see [`src/models/gated_deltaproduct/README.md`](src/models/gated_deltaproduct/README.md).\n\nThis repository includes the [**pretrained 38M parameter model**](https://www.dropbox.com/scl/fi/mqsni5lehooyaw93y3uzq/checkpoint_38M.pth?rlkey=3uyehvmtted02xkha24zgpzb6\u0026st=seevsbkn\u0026dl=0), all training and inference code, and the **complete synthetic data generation pipeline** used for pretraining.\n\n## ✨ Why TempoPFN?\n\n* **High Performance, No Real Data:** Achieves top-tier competitive results on **GIFT-Eval, outperforming all existing synthetic-only approaches** and **surpassing the vast majority of models trained on real-world data**. This ensures full reproducibility and eliminates benchmark leakage.\n* **Parallel and Efficient:** The linear recurrence design enables full-sequence parallelization. This gives us the best of both worlds: the linear efficiency of an RNN, but with the training parallelism of a Transformer.\n* **Open and Reproducible:** Includes the full synthetic data pipeline, configurations, and scripts to reproduce training from scratch.  \n* **State-Tracking Stability:** The GatedDeltaProduct recurrence and *state-weaving* mechanism preserve temporal continuity and information flow across long horizons, improving robustness without non-linear recurrence.\n\n![TempoPFN Overview](https://iili.io/KDCHpou.png)\n\n## ⚙️ Installation\n\nThis repository includes all training and inference code and the **complete synthetic data generation pipeline** used for pretraining.\n\nThe **pretrained 38M parameter model** is hosted on our **[Hugging Face repository](https://huggingface.co/AutoML-org/TempoPFN)**.\n\n## 🚀 Get the Model \u0026 Quick Start\n\nThe easiest and recommended way to get the model, inference code, and weights is to clone our **[Hugging Face repository](https://huggingface.co/AutoML-org/TempoPFN)**.\n\n```bash\n# 1. Install Git LFS (if you haven't already)\n# On Ubuntu: sudo apt-get install git-lfs\n# On macOS: brew install git-lfs\ngit lfs install\n\n# 2. Clone the Hugging Face repository\ngit clone https://huggingface.co/AutoML-org/TempoPFN\ncd TempoPFN\n\n# 3. Set up the environment\npython3.12 -m venv venv \u0026 source venv/bin/activate\nexport PYTHONPATH=$PWD\n\n# 4. Install PyTorch version matching your CUDA version\npip install torch --index-url https://download.pytorch.org/whl/cu128\n\n# 5. Install dependencies\npip install .\npip install .[dev]\n\n# 4. Run the Quick Start Script \npython examples/quick_start_tempo_pfn.py\n\n# 5. Alternatively, you can run the Notebook version\njupyter notebook examples/quick_start_tempo_pfn.ipynb\n```\n\n### Hardware \u0026 Performance Tips\n\n**GPU Required:** Inference requires a CUDA-capable GPU with a matching PyTorch version installed. Tested on NVIDIA A100/H100.\n\n**First Run:** The first inference for a new sequence length will be slow due to Triton kernel compilation. Subsequent runs will be fast.\n\n**Cache Tip:** If using a network filesystem, prevent slowdowns by routing caches to a local directory (like `/tmp`) *before* running:\n```bash\nLOCAL_CACHE_BASE=\"${TMPDIR:-/tmp}/tsf-$(date +%s)\"\nmkdir -p \"${LOCAL_CACHE_BASE}/triton\" \"${LOCAL_CACHE_BASE}/torchinductor\"\nexport TRITON_CACHE_DIR=\"${LOCAL_CACHE_BASE}/triton\"\nexport TORCHINDUCTOR_CACHE_DIR=\"${LOCAL_CACHE_BASE}/torchinductor\"\n\npython examples/quick_start_tempo_pfn.py\n```\n\n## 🚂 Training\n\nAll training and model parameters are controlled via YAML files in `configs/`.  \n\n```bash\n# Single-GPU (Debug)\ntorchrun --standalone --nproc_per_node=1 src/training/trainer_dist.py --config ./configs/train.yaml\n\n# Multi-GPU (e.g., 8 GPUs)\ntorchrun --standalone --nproc_per_node=8 src/training/trainer_dist.py --config ./configs/train.yaml\n```\n\n## 💾 Synthetic Data Generation\n\nA core contribution of this work is our open-source synthetic data pipeline, located in `src/synthetic_generation/`. It combines diverse generators with a powerful augmentation cascade.\n\n**Generators Used:**\n\n* **Adapted Priors:** ForecastPFN, KernelSynth, GaussianProcess (GP), and CauKer (Structural Causal Models).\n* **Novel Priors:** SDE (a flexible regime-switching Ornstein-Uhlenbeck process), Sawtooth, StepFunction, Anomaly, Spikes, SineWave, and Audio-Inspired generators (Stochastic Rhythms, Financial Volatility, Network Topology, Multi-Scale Fractals).\n\nYou can easily generate your own data by installing the development dependencies and instantiating a generator wrapper. See `examples/generate_synthetic_data.py` for a minimal script, or inspect the generator code in `src/synthetic_generation/`.\n\n## 🤝 License\n\nThis project is licensed under the Apache 2.0 License. See the LICENSE file for details. This permissive license allows for both academic and commercial use.\n\n## 📚 Citation\n\nIf you find TempoPFN useful in your research, please consider citing our paper:\n```bibtex\n@misc{moroshan2025tempopfn,\n  title={TempoPFN: Synthetic Pre-Training of Linear RNNs for Zero-Shot Time Series Forecasting},\n  author={Vladyslav Moroshan and Julien Siems and Arber Zela and Timur Carstensen and Frank Hutter},\n  year={2025},\n  eprint={2510.25502},\n  archivePrefix={arXiv},\n  primaryClass={cs.LG}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fautoml%2Ftempopfn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fautoml%2Ftempopfn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fautoml%2Ftempopfn/lists"}