Projects in Awesome Lists tagged with learned-tokenization
A curated list of projects in awesome lists tagged with learned-tokenization .
https://github.com/lucidrains/MEGABYTE-pytorch
Implementation of MEGABYTE, Predicting Million-byte Sequences with Multiscale Transformers, in Pytorch
artificial-intelligence attention-mechanisms deep-learning learned-tokenization long-context transformers
Last synced: 09 May 2025
https://github.com/lucidrains/megabyte-pytorch
Implementation of MEGABYTE, Predicting Million-byte Sequences with Multiscale Transformers, in Pytorch
artificial-intelligence attention-mechanisms deep-learning learned-tokenization long-context transformers
Last synced: 14 May 2025
https://github.com/lucidrains/rvq-vae-gpt
My attempts at applying Soundstream design on learned tokenization of text and then applying hierarchical attention to text generation
artificial-intelligence deep-learning gpt learned-tokenization text-autoencoder
Last synced: 05 Apr 2025