{"id":20674889,"url":"https://github.com/visionscaper/stateful_multi_gpu","last_synced_at":"2025-07-22T12:34:07.202Z","repository":{"id":95250018,"uuid":"113490529","full_name":"visionscaper/stateful_multi_gpu","owner":"visionscaper","description":"Experimental utility to build stateful RNN models for multi GPU training.","archived":false,"fork":false,"pushed_at":"2018-04-05T19:38:49.000Z","size":6,"stargazers_count":4,"open_issues_count":0,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-19T21:24:32.538Z","etag":null,"topics":["gru","keras","keras-models","keras-tensorflow","lstm","multi-gpu","rnn"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/visionscaper.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2017-12-07T19:32:37.000Z","updated_at":"2019-06-19T07:49:31.000Z","dependencies_parsed_at":null,"dependency_job_id":"94c6ef54-c6fa-49bf-bc5d-0f1f05a5b630","html_url":"https://github.com/visionscaper/stateful_multi_gpu","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/visionscaper/stateful_multi_gpu","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/visionscaper%2Fstateful_multi_gpu","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/visionscaper%2Fstateful_multi_gpu/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/visionscaper%2Fstateful_multi_gpu/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/visionscaper%2Fstateful_multi_gpu/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/visionscaper","download_url":"https://codeload.github.com/visionscaper/stateful_multi_gpu/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/visionscaper%2Fstateful_multi_gpu/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266496399,"owners_count":23938711,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-22T02:00:09.085Z","response_time":66,"last_error":null,"robots_txt_status":null,"robots_txt_updated_at":null,"robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gru","keras","keras-models","keras-tensorflow","lstm","multi-gpu","rnn"],"created_at":"2024-11-16T21:07:55.166Z","updated_at":"2025-07-22T12:34:07.186Z","avatar_url":"https://github.com/visionscaper.png","language":"Python","readme":"# stateful_multi_gpu\nExperimental utility in development to build stateful RNN models for multi GPU training.\n\n## How to use stateful_multi_gpu()\nSee `example.py` for a toy example of how the utility function can be used.\n\nTo make stateful models in Keras you need to provide the batch size you are using.  \n`stateful_multi_gpu()` ensures that the right batch size is used throughout the model.\n\nTo use this utility function you need to create a `inputs_generator` method that can make \ninputs for your model, for any batch size, and a `model_generator` method that can make\nyour model for any batch size.\n\nFor instance, as follows:\n\n```python\ndef inputs_generator(batch_size):\n    rnn_input = Input(\n        name=\"rnn-input-%d\" % batch_size,\n        batch_shape=(batch_size, seq_len, num_symbols))\n\n    return rnn_input\n```\n\n```python\ndef model_generator(batch_size):\n    inputs = inputs_generator(batch_size)\n\n    layer_output = RNNLayer(\n        state_size,\n        stateful=stateful_model,\n        return_sequences=True)(inputs)\n    outputs = TimeDistributed(Dense(num_classes))(layer_output)\n\n    return Model(inputs=inputs, outputs=outputs)\n```\n\n`stateful_multi_gpu` uses these generator methods to \ncreate model inputs that expect batch size `batch_size` and model replica's (one per GPU) that expect \nbatch size `batch_size` // `num_gpus`. E.g.:\n\n```python\nparallel_model = stateful_multi_gpu(inputs_generator, model_generator, training_batch_size, num_gpus)\n```\n\nIt is important that `batch_size` is wholly dividable by `num_gpus`.","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvisionscaper%2Fstateful_multi_gpu","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvisionscaper%2Fstateful_multi_gpu","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvisionscaper%2Fstateful_multi_gpu/lists"}