{"id":19156294,"url":"https://github.com/kyegomez/multi-model-training","last_synced_at":"2026-04-15T15:32:31.024Z","repository":{"id":247323512,"uuid":"825542910","full_name":"kyegomez/Multi-Model-Training","owner":"kyegomez","description":"An experimental repository on research for training multiple models all at once in an evolutionary capacity!","archived":false,"fork":false,"pushed_at":"2025-09-22T12:13:54.000Z","size":2287,"stargazers_count":2,"open_issues_count":8,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-10-04T20:59:09.570Z","etag":null,"topics":["ai","cats","mamba","ml","pytorch","ssms","tensorflow","training","transformers"],"latest_commit_sha":null,"homepage":"https://discord.com/servers/agora-999382051935506503","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kyegomez.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null},"funding":{"github":["kyegomez"],"patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"lfx_crowdfunding":null,"custom":null}},"created_at":"2024-07-08T03:38:16.000Z","updated_at":"2024-07-08T21:21:50.000Z","dependencies_parsed_at":"2024-11-09T08:35:50.985Z","dependency_job_id":"09da9db2-0b3f-4abe-aa6a-b11c03099e5f","html_url":"https://github.com/kyegomez/Multi-Model-Training","commit_stats":null,"previous_names":["kyegomez/multi-model-training"],"tags_count":0,"template":false,"template_full_name":"kyegomez/Python-Package-Template","purl":"pkg:github/kyegomez/Multi-Model-Training","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kyegomez%2FMulti-Model-Training","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kyegomez%2FMulti-Model-Training/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kyegomez%2FMulti-Model-Training/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kyegomez%2FMulti-Model-Training/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kyegomez","download_url":"https://codeload.github.com/kyegomez/Multi-Model-Training/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kyegomez%2FMulti-Model-Training/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31847668,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-15T15:24:51.572Z","status":"ssl_error","status_checked_at":"2026-04-15T15:24:39.138Z","response_time":63,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","cats","mamba","ml","pytorch","ssms","tensorflow","training","transformers"],"created_at":"2024-11-09T08:34:01.707Z","updated_at":"2026-04-15T15:32:30.999Z","avatar_url":"https://github.com/kyegomez.png","language":"Python","funding_links":["https://github.com/sponsors/kyegomez"],"categories":[],"sub_categories":[],"readme":"[![Multi-Modality](agorabanner.png)](https://discord.com/servers/agora-999382051935506503)\n\n# Multi-Model Trainers\nOver the past year, Agora has implemented and created thousands of models all with slightly different variations and architectures. So a question arose, which was: How can we train multiple models all at once and then evaluate them and then distribute more GPU memory to the models that are learning the fastest with the lowest loss. There are many configurations we'll attempt in the future, but this is super experimental! If you want to hack on this, Join us at Agora and let's accelerate!\n\n## Install\n```bash\n$ pip install multi-model-trainers\n```\n\n## Usage\n\n```python\nimport torch\nimport torch.nn as nn\nfrom loguru import logger\n\nfrom multi_model_trainers.main import MultiModelMemoryTrainer\n\n# Example usage\nif __name__ == \"__main__\":\n    # Create some dummy models\n    models = [\n        nn.Sequential(nn.Linear(10, 50), nn.ReLU(), nn.Linear(50, 1))\n        for _ in range(3)\n    ]\n    initial_allocation = [1 / 3, 1 / 3, 1 / 3]\n    total_memory = 4 * 1024 * 1024 * 1024  # 4 GB\n\n    gpu_allocator = MultiModelMemoryTrainer(\n        models, initial_allocation, total_memory\n    )\n\n    # Simulate a few training steps\n    for step in range(5):\n        logger.info(f\"Training step {step}\")\n\n        # Generate dummy data\n        train_data = {\n            \"inputs\": torch.rand(32, 10),\n            \"targets\": torch.rand(32, 1),\n        }\n\n        losses = gpu_allocator.train_step(train_data)\n\n        # Update learning rates based on losses (this is a simplistic approach)\n        learning_rates = [1 / (loss + 1e-5) for loss in losses]\n        gpu_allocator.update_learning_rates(learning_rates)\n\n        # Reallocate GPU memory\n        gpu_allocator.reallocate_gpu_memory()\n\n        # Validation step\n        val_data = {\n            \"inputs\": torch.rand(64, 10),\n            \"targets\": torch.rand(64, 1),\n        }\n        val_losses = gpu_allocator.validate(val_data)\n\n    logger.info(\"Training complete\")\n```\n\n\n# License\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkyegomez%2Fmulti-model-training","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkyegomez%2Fmulti-model-training","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkyegomez%2Fmulti-model-training/lists"}