https://github.com/soniccodes/integrated_dit

minibatch ode with block causal transformers :00:
https://github.com/soniccodes/integrated_dit

Last synced: 8 months ago
JSON representation

minibatch ode with block causal transformers :00:

• Host: GitHub
• URL: https://github.com/soniccodes/integrated_dit
• Owner: SonicCodes
• Created: 2025-02-04T20:15:28.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2025-02-04T20:56:46.000Z (over 1 year ago)
• Last Synced: 2025-02-04T21:39:02.781Z (over 1 year ago)
• Language: Python
• Size: 0 Bytes
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: Readme.md

Awesome Lists containing this project

README

          
is interesting idea, still on progress, dis just copy paste of past codes :333

So basically the idea is that instead of learning fixed trajectory from N(0,1) to target distribution , why don't we do soemthing close to integration (atleast) using block causal transformers, that way we can let the model loose abt the timestep and let it figure it out by it self, 

Implementation will basically be like, 10 different discrete points on the entire trajectory, the model sees different levels of noise and it'll predict what "v" needs to be predicted to reach the next denoising, so technically its autoregressive on the absolute sense...