{"id":19460083,"url":"https://github.com/nghuyong/bert-classification-tf-serving","last_synced_at":"2025-04-25T07:32:39.842Z","repository":{"id":54177468,"uuid":"210143799","full_name":"nghuyong/bert-classification-tf-serving","owner":"nghuyong","description":"Use BERT to train a classification model and deploy the model by tensorflow serving ","archived":false,"fork":false,"pushed_at":"2021-03-05T03:45:43.000Z","size":1819,"stargazers_count":48,"open_issues_count":1,"forks_count":23,"subscribers_count":3,"default_branch":"master","last_synced_at":"2023-03-03T06:14:59.495Z","etag":null,"topics":["bert","classification-model","tensorflow","tensorflow-serving"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nghuyong.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-09-22T12:30:23.000Z","updated_at":"2023-02-14T23:02:21.000Z","dependencies_parsed_at":"2022-08-13T08:21:02.671Z","dependency_job_id":null,"html_url":"https://github.com/nghuyong/bert-classification-tf-serving","commit_stats":null,"previous_names":[],"tags_count":null,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nghuyong%2Fbert-classification-tf-serving","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nghuyong%2Fbert-classification-tf-serving/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nghuyong%2Fbert-classification-tf-serving/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nghuyong%2Fbert-classification-tf-serving/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nghuyong","download_url":"https://codeload.github.com/nghuyong/bert-classification-tf-serving/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223989445,"owners_count":17237079,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bert","classification-model","tensorflow","tensorflow-serving"],"created_at":"2024-11-10T17:35:32.797Z","updated_at":"2024-11-10T17:35:33.500Z","avatar_url":"https://github.com/nghuyong.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# bert-classification-tf-serving\n\n## Introduction\nUse [BERT](https://github.com/google-research/bert/) to train a classification model and deploy the model by [tensorflow serving](https://www.tensorflow.org/tfx/serving/serving_basic). \nThen we can use REST API to do online prediction.\n\n## Get Started\n\nThe whole experiment is based on `Chnsenticorp` data set, which is a two-class data set of positive and negative emotions.\n\n### 0. Prepare Pre-train model\n\nDownload the Chinese bert model `chinese_L-12_H-768_A-12`, then unzip and move to the `models` dir.\n\n### 1. Train a model\n\n```bash\nsh fine-tuning.sh\n```\n\n### 2. Do prediction and export model\n\nWe need to change the checkpoint to the tensorflow serving's format.\n\n```bash\nsh export-model.sh\n```\n\nThen the structure of `export-model` will be:\n```bash\n.\n└── 1569141360\n    ├── saved_model.pb\n    └── variables\n        ├── variables.data-00000-of-00001\n        └── variables.index\n```\n\n### 3. Deploy model\nWe use tensorflow serving docker version to deploy the model on CPU OR GPU.\n\na. On CPU\n\n`docker run -p 8500:8500 -p 8501:8501 --mount type=bind,source=$(pwd)/models/export-model,target=/models/chnsenticorp -e MODEL_NAME=chnsenticorp -t tensorflow/serving`\n\nb. On GPU\n\nYou need to install nvidia-docker environment first!\n\n`docker run --runtime=nvidia -e CUDA_VISIBLE_DEVICES=0 -p 8500:8500 -p 8501:8501 --mount type=bind,source=$(pwd)/models/export-model,target=/models/chnsenticorp -e MODEL_NAME=chnsenticorp -t tensorflow/serving:1.12.0-gpu`\n\n### 4. Predict online\n\n```\nexport PYTHONPATH=$(pwd)/google-bert\npython predict_online.py\n```\n\na. On CPU\n```\nInput test sentence:\n跟住招待所没什么太大区别。 绝对不会再住第2次的酒店！\nnegative pro:0.96344769 positive pro:0.0365523435 time consuming:198ms\nInput test sentence:\nLED屏就是爽，基本硬件配置都很均衡，镜面考漆不错，小黑，我喜欢。\nnegative pro:0.0291572567 positive pro:0.970842779 time consuming:195ms\n```\n\nb. On GPU\n```\nInput test sentence:\n跟住招待所没什么太大区别。 绝对不会再住第2次的酒店！\nnegative pro:0.963448 positive pro:0.0365524 time consuming:92ms\nInput test sentence:\nLED屏就是爽，基本硬件配置都很均衡，镜面考漆不错，小黑，我喜欢。\nnegative pro:0.0291573 positive pro:0.970843 time consuming:87ms\n```\n\nThe probability on GPU and GPU is the same, but the speed on GPU is faster **50%+** than on CPU!\n\nThe max token length is 128 and the CPU and GPU configuration is as follows:\n\n|CPU|GPU|\n|:---:|:---:|\n|Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz|Tesla V100|\n\n## Conclusion\nThis is a demo project for classification by BERT and deploy the model by tensorflow serving.\n\nWe use tensorflow serving docker version, so we can use docker stack technology to easily expand for load balance in production environment.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnghuyong%2Fbert-classification-tf-serving","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnghuyong%2Fbert-classification-tf-serving","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnghuyong%2Fbert-classification-tf-serving/lists"}