{"id":20699669,"url":"https://github.com/hyeonsangjeon/colab-tensorflow-tpu-example","last_synced_at":"2025-04-22T22:22:27.434Z","repository":{"id":57819818,"uuid":"528197407","full_name":"hyeonsangjeon/colab-tensorflow-tpu-example","owner":"hyeonsangjeon","description":"colab-tensorflow-tpu-example","archived":false,"fork":false,"pushed_at":"2023-04-10T19:54:06.000Z","size":460,"stargazers_count":6,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-29T19:22:40.974Z","etag":null,"topics":["bert","colab","colab-notebook","confusion-matrix","histogram","kobert","nlp","resolver-library","sagemaker","sentence","sentence-embeddings","sentence-transformers","sentiment","tpu","transformers"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hyeonsangjeon.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-08-23T23:37:55.000Z","updated_at":"2023-05-24T14:22:12.000Z","dependencies_parsed_at":"2024-11-17T00:31:29.568Z","dependency_job_id":"327ec3b1-0d3c-426c-9b13-471e70e6c841","html_url":"https://github.com/hyeonsangjeon/colab-tensorflow-tpu-example","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hyeonsangjeon%2Fcolab-tensorflow-tpu-example","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hyeonsangjeon%2Fcolab-tensorflow-tpu-example/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hyeonsangjeon%2Fcolab-tensorflow-tpu-example/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hyeonsangjeon%2Fcolab-tensorflow-tpu-example/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hyeonsangjeon","download_url":"https://codeload.github.com/hyeonsangjeon/colab-tensorflow-tpu-example/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250332195,"owners_count":21413166,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bert","colab","colab-notebook","confusion-matrix","histogram","kobert","nlp","resolver-library","sagemaker","sentence","sentence-embeddings","sentence-transformers","sentiment","tpu","transformers"],"created_at":"2024-11-17T00:31:21.272Z","updated_at":"2025-04-22T22:22:27.403Z","avatar_url":"https://github.com/hyeonsangjeon.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# colab-tensorflow-tpu-example\ncolab-tensorflow-tpu-example\n\nTensorflow v2와 Bert 모델을 이용한 colab GPU와 TPU로 학습하는 예제입니다.\n두개의 ipynb는 동일한 코드이고, TPU와 GPU 사용 부분만 변경되어져 있습니다. \n\n### 1. example, method to change GPU mode and TPU mode in one code :\n\n##### GPU\n```python\nstrategy = tf.distribute.MirroredStrategy(devices=[\"/gpu:0\"],cross_device_ops=tf.distribute.HierarchicalCopyAllReduce()) #GPU모드 \nprint('Number of devices: {}'.format(strategy.num_replicas_in_sync)) # GPU\nwith strategy.scope():\n    sentiment_model = create_sentiment_bert() \n```\n\n##### TPU\n```python\nresolver = tf.distribute.cluster_resolver.TPUClusterResolver(tpu='grpc://' + os.environ['COLAB_TPU_ADDR'])\ntf.config.experimental_connect_to_cluster(resolver)\ntf.tpu.experimental.initialize_tpu_system(resolver)\nstrategy = tf.distribute.experimental.TPUStrategy(resolver) # TPU모드 \nwith strategy.scope():\n    sentiment_model = create_sentiment_bert() \n```\n\n\n### 2. model training speed\n\n##### GPU\nGPU was randomly selected for P100, k80, and T4. The picture below is T4\nphysical_device_desc: \"device: 0, name: Tesla T4, pci bus id: 0000:00:04.0, compute capability: 7.5\"\n![screenshot1](https://github.com/hyeonsangjeon/colab-tensorflow-tpu-example/blob/main/pic/T4_GPU_time_per_epoch.png?raw=true)\n\n\n##### TPU\n![screenshot1](https://github.com/hyeonsangjeon/colab-tensorflow-tpu-example/blob/main/pic/TPUs_time_per_epoch.png?raw=true)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhyeonsangjeon%2Fcolab-tensorflow-tpu-example","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhyeonsangjeon%2Fcolab-tensorflow-tpu-example","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhyeonsangjeon%2Fcolab-tensorflow-tpu-example/lists"}