{"id":20925291,"url":"https://github.com/apehex/llaminate","last_synced_at":"2025-03-13T01:13:56.476Z","repository":{"id":240869854,"uuid":"801042950","full_name":"apehex/llaminate","owner":"apehex","description":"Optimized llama3 using tokun","archived":false,"fork":false,"pushed_at":"2025-01-31T18:17:20.000Z","size":4533,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-31T19:23:43.966Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/apehex.png","metadata":{"files":{"readme":".github/README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-05-15T13:36:08.000Z","updated_at":"2025-01-31T18:17:24.000Z","dependencies_parsed_at":"2024-08-05T18:54:59.132Z","dependency_job_id":"812f4ba3-2e05-4ca7-9b40-2d7bf0fa76e5","html_url":"https://github.com/apehex/llaminate","commit_stats":null,"previous_names":["apehex/llaminate"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apehex%2Fllaminate","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apehex%2Fllaminate/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apehex%2Fllaminate/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apehex%2Fllaminate/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/apehex","download_url":"https://codeload.github.com/apehex/llaminate/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243318758,"owners_count":20272144,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-18T20:30:32.238Z","updated_at":"2025-03-13T01:13:56.450Z","avatar_url":"https://github.com/apehex.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# llaminate\n\n\u003e Optimized version of [llama3][github-llama3], using [tokun][github-tokun].\n\n\u003cimg src=\"../.github/header.png\" alt=\"Neural tokenization\" title=\"Source: Image by Author and generated with MidJourney\" width=\"100%\" style=\"margin: auto;\"/\u003e\n\nThis project is a showcase for a neural tokenization technique.\nSince the inputs are compressed and have a smaller shape, the LLM is downsized accordingly.\n\nFor example, llama3-8b is brought down to 34 million parameters instead of 8 billion.\n\n## Installation\n\n## Usage\n\n## Resources\n\n### Models\n\n### Notebooks\n\nFinal model:\n\n- pretraining: [file][notebook-github-pretrain] / [Google Colab][notebook-colab-pretrain]\n- fine-tuning: file / Google Colab\n\n## TODO\n\nSee [TODO](TODO.md).\n\n## Credits\n\nThis project winks at [llama3 from Meta][github-llama3], but doesn't actually its weights nor code.\n\n## License\n\nLicensed under the [aGPLv3](LICENSE.md).\n\n[github-llama3]: https://github.com/meta-llama/llama3\n[github-tokun]: https://github.com/apehex/tokun\n\n[notebook-colab-pretrain]: https://colab.research.google.com/github/apehex/llaminate/blob/main/notebooks/llaminate.student.pretrain.ipynb\n[notebook-github-pretrain]: ../notebooks/llaminate.student.pretrain.ipynb\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapehex%2Fllaminate","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fapehex%2Fllaminate","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapehex%2Fllaminate/lists"}