https://github.com/statmixedml/convts-mixer

Time Series Extension of "Patches Are All You Need"
https://github.com/statmixedml/convts-mixer

Last synced: 6 months ago
JSON representation

Time Series Extension of "Patches Are All You Need"

• Host: GitHub
• URL: https://github.com/statmixedml/convts-mixer
• Owner: StatMixedML
• License: apache-2.0
• Created: 2023-04-05T12:24:56.000Z (about 2 years ago)
• Default Branch: main
• Last Pushed: 2023-10-29T09:27:32.000Z (over 1 year ago)
• Last Synced: 2024-05-15T09:36:26.560Z (about 1 year ago)
• Language: Jupyter Notebook
• Size: 18.7 MB
• Stars: 6
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# ConvTSMixer, MlpTSMixer and TsT

Time Series Extension of "Patches Are All You Need"

Implementations of some prototypical time series mixers based on Conv, MLP, and ViT archs. modified for the probabilistic multivariate forecasting use case, where the emission head is currently an "independent same-family" distribution, e.g., diagonal Student-T.

In everything that follows, the inputs are typically 4-Tensors of shape `[Batch, Variate-dim, Context-length, 1+Features]`, and during training, the subsequent prediction window values are given `[B, Variate-dim, Pred-length]`. The inputs are embedded via 2d-conv to obtain patch embeddings:

![Screenshot 2024-10-25 at 09 29 21](https://github.com/user-attachments/assets/5ecc92c5-a115-44a1-95de-971d2d34ed58)

## ConvTSMixer

![Screenshot 2024-10-25 at 09 29 55](https://github.com/user-attachments/assets/2033b75b-6105-4c5d-bfc6-aaf9c8330d5d)

## MlpTSMixer

![Screenshot 2024-10-25 at 09 30 09](https://github.com/user-attachments/assets/14db4edd-2004-4152-acc7-3c926c97c306)

## TsT (ViT style)

![Screenshot 2024-10-25 at 09 30 40](https://github.com/user-attachments/assets/6a5f88d4-8ded-41fe-9cc5-3ce1ab9df5b9)

## Output head

![Screenshot 2024-10-25 at 09 31 12](https://github.com/user-attachments/assets/324bec4c-d4bf-40ea-9084-413d1c647870)