{"id":19156189,"url":"https://github.com/kyegomez/tinygptv","last_synced_at":"2025-05-07T07:34:36.645Z","repository":{"id":214583762,"uuid":"736886780","full_name":"kyegomez/TinyGPTV","owner":"kyegomez","description":"Simple Implementation of TinyGPTV in super simple Zeta lego blocks","archived":false,"fork":false,"pushed_at":"2024-11-11T17:02:59.000Z","size":2277,"stargazers_count":16,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-31T07:41:37.375Z","etag":null,"topics":["artificial-intelligence","attention","attention-is-all-you-need","deep-learning","multi-modal","multi-modality","transformers"],"latest_commit_sha":null,"homepage":"https://discord.gg/GYbXvDGevY","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kyegomez.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["kyegomez"],"patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"lfx_crowdfunding":null,"custom":null}},"created_at":"2023-12-29T06:52:36.000Z","updated_at":"2025-03-29T15:26:37.000Z","dependencies_parsed_at":"2024-11-09T08:35:38.812Z","dependency_job_id":"ff0d4e3f-a6e0-40ae-8b20-4dd8e7314a0e","html_url":"https://github.com/kyegomez/TinyGPTV","commit_stats":{"total_commits":15,"total_committers":3,"mean_commits":5.0,"dds":"0.19999999999999996","last_synced_commit":"487f9dfad82eb6466291d08f88873e4e00d4eb2f"},"previous_names":["kyegomez/tinygptv"],"tags_count":0,"template":false,"template_full_name":"kyegomez/Python-Package-Template","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kyegomez%2FTinyGPTV","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kyegomez%2FTinyGPTV/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kyegomez%2FTinyGPTV/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kyegomez%2FTinyGPTV/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kyegomez","download_url":"https://codeload.github.com/kyegomez/TinyGPTV/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249764810,"owners_count":21322303,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-intelligence","attention","attention-is-all-you-need","deep-learning","multi-modal","multi-modality","transformers"],"created_at":"2024-11-09T08:33:32.769Z","updated_at":"2025-04-19T18:31:14.829Z","avatar_url":"https://github.com/kyegomez.png","language":"Python","funding_links":["https://github.com/sponsors/kyegomez"],"categories":[],"sub_categories":[],"readme":"[![Multi-Modality](agorabanner.png)](https://discord.gg/qUtxnK2NMf)\n\n# TinyGPTV\nSimple Implementation of TinyGPTV in super simple Zeta lego blocks. Here all the modules from figure 2 are implemented in Zeta and Pytorch.\n\nThe flow is the following:\nx -\u003e skip connection -\u003e layer norm -\u003e lora -\u003e mha + lora -\u003e residual_rms_norm -\u003e original_skip_connection -\u003e mlp + rms norm\n\n\n## Install\n`pip3 install tiny-gptv`\n\n\n## Usage\n\n### TinyGPTVBlock, Figure3 (c):\n- Layernorm\n- MHA\n- Lora\n- QK Norm\n- RMS Norm\n- MLP\n\n\n```python\nimport torch\nfrom tiny_gptv.blocks import TinyGPTVBlock\n\n# Random tensor, replace with your input data\nx = torch.rand(2, 8, 512)\n\n# TinyGPTVBlock\nblock = TinyGPTVBlock(512, 8, depth=10)\n\n# Print the block\nprint(block)\n\n# Forward pass\nout = block(x)\n\n# Print the output shape\nprint(out.shape)\n\n\n```\n\n### Figure3 (b) Lora Module for LLMS Block\n- MHA,\n- Lora,\n- Normalization,\n- MLP\n- Skip connection\n- Split then add\n\n```python\nimport torch\nfrom tiny_gptv import LoraMHA\n\nx = torch.rand(2, 8, 512)\nblock = LoraMHA(512, 8)\nout = block(x)\nprint(out.shape)\n\n```\n\n\n# Citation\n\n```bibtex\n@misc{yuan2023tinygptv,\n    title={TinyGPT-V: Efficient Multimodal Large Language Model via Small Backbones}, \n    author={Zhengqing Yuan and Zhaoxu Li and Lichao Sun},\n    year={2023},\n    eprint={2312.16862},\n    archivePrefix={arXiv},\n    primaryClass={cs.CV}\n}\n\n```\n\n# License\nMIT","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkyegomez%2Ftinygptv","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkyegomez%2Ftinygptv","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkyegomez%2Ftinygptv/lists"}