{"id":15600945,"url":"https://github.com/lucidrains/titok-pytorch","last_synced_at":"2025-04-07T17:08:44.485Z","repository":{"id":244845312,"uuid":"816441549","full_name":"lucidrains/titok-pytorch","owner":"lucidrains","description":"Implementation of TiTok, proposed by Bytedance in \"An Image is Worth 32 Tokens for Reconstruction and Generation\"","archived":false,"fork":false,"pushed_at":"2024-06-20T13:41:52.000Z","size":1310,"stargazers_count":170,"open_issues_count":1,"forks_count":5,"subscribers_count":9,"default_branch":"main","last_synced_at":"2025-03-31T16:14:23.252Z","etag":null,"topics":["artificial-intelligence","deep-learning","image-generation","vector-quantization"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lucidrains.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-17T18:59:25.000Z","updated_at":"2025-03-22T00:34:22.000Z","dependencies_parsed_at":"2024-06-17T21:08:52.431Z","dependency_job_id":"34f226b0-5415-4286-9570-3be544d5be74","html_url":"https://github.com/lucidrains/titok-pytorch","commit_stats":{"total_commits":16,"total_committers":1,"mean_commits":16.0,"dds":0.0,"last_synced_commit":"2f95258f27797f142df0d5dc2388e034285ab7e5"},"previous_names":["lucidrains/titok-pytorch"],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucidrains%2Ftitok-pytorch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucidrains%2Ftitok-pytorch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucidrains%2Ftitok-pytorch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucidrains%2Ftitok-pytorch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lucidrains","download_url":"https://codeload.github.com/lucidrains/titok-pytorch/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247694876,"owners_count":20980733,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-intelligence","deep-learning","image-generation","vector-quantization"],"created_at":"2024-10-03T02:09:59.487Z","updated_at":"2025-04-07T17:08:44.467Z","avatar_url":"https://github.com/lucidrains.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cimg src=\"./titok.png\" width=\"400px\"\u003e\u003c/img\u003e\n\n## TiTok - Pytorch (wip)\n\nImplementation of TiTok, proposed by Bytedance in \u003ca href=\"https://yucornetto.github.io/projects/titok.html\"\u003eAn Image is Worth 32 Tokens for Reconstruction and Generation\u003c/a\u003e\n\n## Install\n\n```bash\n$ pip install titok-pytorch\n```\n\n## Usage\n\n```python\nimport torch\nfrom titok_pytorch import TiTokTokenizer\n\nimages = torch.randn(2, 3, 256, 256)\n\ntitok = TiTokTokenizer(\n    dim = 1024,\n    patch_size = 32,\n    num_latent_tokens = 32,   # they claim only 32 tokens needed\n    codebook_size = 4096      # codebook size 4096\n)\n\nloss = titok(images)\nloss.backward()\n\n# after much training\n# extract codes for gpt, maskgit, whatever\n\ncodes = titok.tokenize(images) # (2, 32)\n\n# reconstructing images from codes\n\nrecon_images = titok.codebook_ids_to_images(codes)\n\nassert recon_images.shape == images.shape\n```\n\n## Todo\n\n- [ ] add multi-resolution patches\n\n## Citations\n\n```bibtex\n@article{yu2024an,\n  author    = {Qihang Yu and Mark Weber and Xueqing Deng and Xiaohui Shen and Daniel Cremers and Liang-Chieh Chen},\n  title     = {An Image is Worth 32 Tokens for Reconstruction and Generation},\n  journal   = {arxiv: 2406.07550},\n  year      = {2024}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flucidrains%2Ftitok-pytorch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flucidrains%2Ftitok-pytorch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flucidrains%2Ftitok-pytorch/lists"}