{"id":13534926,"url":"https://github.com/innodatalabs/tbert","last_synced_at":"2026-01-14T07:54:25.409Z","repository":{"id":57473552,"uuid":"156621649","full_name":"innodatalabs/tbert","owner":"innodatalabs","description":"PyTorch port of BERT ML model","archived":false,"fork":false,"pushed_at":"2019-02-19T00:33:04.000Z","size":85,"stargazers_count":17,"open_issues_count":0,"forks_count":5,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-09-29T19:38:45.966Z","etag":null,"topics":["bert-model","natural-language-processing","neural-network","nlp","pytorch"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/innodatalabs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-11-07T23:23:34.000Z","updated_at":"2025-08-07T12:23:19.000Z","dependencies_parsed_at":"2022-09-26T17:40:47.445Z","dependency_job_id":null,"html_url":"https://github.com/innodatalabs/tbert","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/innodatalabs/tbert","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/innodatalabs%2Ftbert","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/innodatalabs%2Ftbert/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/innodatalabs%2Ftbert/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/innodatalabs%2Ftbert/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/innodatalabs","download_url":"https://codeload.github.com/innodatalabs/tbert/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/innodatalabs%2Ftbert/sbom","scorecard":{"id":489187,"data":{"date":"2025-08-11","repo":{"name":"github.com/innodatalabs/tbert","commit":"84c1c9507b3b1bffd2a08a86efaf9bc9955271e0"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":1.7,"checks":[{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"SAST","score":0,"reason":"no SAST tool detected","details":["Warn: no pull requests merged into dev branch"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Code-Review","score":0,"reason":"Found 0/30 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Vulnerabilities","score":0,"reason":"420 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: GHSA-23hm-7w47-xw72","Warn: Project is vulnerable to: GHSA-2475-53vw-vp25","Warn: Project is vulnerable to: GHSA-247x-2f9f-5wp7","Warn: Project is vulnerable to: GHSA-24x4-6qmh-88qg","Warn: Project is vulnerable to: PYSEC-2021-227 / GHSA-24x6-8c7m-hv3f","Warn: Project is vulnerable to: PYSEC-2021-223 / GHSA-26j7-6w8w-7922","Warn: Project is vulnerable to: GHSA-278g-rq84-9hmg","Warn: Project is vulnerable to: GHSA-27j5-4p9v-pp67","Warn: Project is vulnerable to: GHSA-27qf-jwm8-g7f3","Warn: Project is vulnerable to: GHSA-27rc-728f-x5w2","Warn: Project is vulnerable to: PYSEC-2021-160 / GHSA-2cpx-427x-q2c6","Warn: Project is vulnerable to: PYSEC-2021-173 / GHSA-2gfx-95x2-5v3x","Warn: Project is vulnerable to: PYSEC-2021-391 / GHSA-2p25-55c9-h58q","Warn: Project is vulnerable to: GHSA-2p9q-h29j-3f5v","Warn: Project is vulnerable to: GHSA-2r2f-g8mw-9gvr","Warn: Project is vulnerable to: GHSA-2r8p-fg3c-wcj4","Warn: Project is vulnerable to: GHSA-2vv3-56qg-g2cf","Warn: Project is vulnerable to: GHSA-2wmv-37vq-52g5","Warn: Project is vulnerable to: PYSEC-2021-249 / GHSA-2xgj-xhgf-ggjv","Warn: Project is vulnerable to: GHSA-34f9-hjfq-rr8j","Warn: Project is vulnerable to: GHSA-368v-7v32-52fx","Warn: Project is vulnerable to: PYSEC-2021-199 / GHSA-36vm-xw34-x4pj","Warn: Project is vulnerable to: PYSEC-2021-418 / GHSA-374m-jm66-3vj8","Warn: Project is vulnerable to: GHSA-37jf-mjv6-xfqw","Warn: Project is vulnerable to: PYSEC-2021-170 / GHSA-393f-2jr3-cp69","Warn: Project is vulnerable to: GHSA-397c-5g2j-qxpv","Warn: Project is vulnerable to: PYSEC-2021-408 / GHSA-3ff2-r28g-w7h9","Warn: Project is vulnerable to: PYSEC-2021-206 / GHSA-3h8m-483j-7xxm","Warn: Project is vulnerable to: GHSA-3hxh-8cp2-g4hg","Warn: Project is vulnerable to: GHSA-3mw4-6rj6-74g5","Warn: Project is vulnerable to: PYSEC-2021-231 / GHSA-3qgw-p4fm-x7gf","Warn: Project is vulnerable to: PYSEC-2021-168 / GHSA-3qxp-qjq7-w4hf","Warn: Project is vulnerable to: PYSEC-2021-420 / GHSA-3rcw-9p9x-582v","Warn: Project is vulnerable to: PYSEC-2021-226 / GHSA-3w67-q784-6w7c","Warn: Project is vulnerable to: GHSA-428x-9xc2-m8mj","Warn: Project is vulnerable to: GHSA-43jf-985q-588j","Warn: Project is vulnerable to: GHSA-43q8-3fv7-pr5x","Warn: Project is vulnerable to: GHSA-44qp-9wwf-734r","Warn: Project is vulnerable to: PYSEC-2021-150 / GHSA-452g-f7fp-9jf7","Warn: Project is vulnerable to: GHSA-49rq-hwc3-x77w","Warn: Project is vulnerable to: PYSEC-2021-398 / GHSA-49rx-x2rw-pc6f","Warn: Project is vulnerable to: GHSA-4c4g-crqm-xrxw","Warn: Project is vulnerable to: PYSEC-2021-411 / GHSA-4f99-p9c2-3j8x","Warn: Project is vulnerable to: PYSEC-2021-184 / GHSA-4fg4-p75j-w5xj","Warn: Project is vulnerable to: PYSEC-2020-113 / GHSA-4g9f-63rx-5cw4","Warn: Project is vulnerable to: PYSEC-2021-179 / GHSA-4hrh-9vmp-2jgg","Warn: Project is vulnerable to: GHSA-4hvf-hxvg-f67v","Warn: Project is vulnerable to: PYSEC-2021-253 / GHSA-4hvv-7x94-7vq8","Warn: Project is vulnerable to: GHSA-4j82-5ccr-4r8v","Warn: Project is vulnerable to: PYSEC-2021-205 / GHSA-4p4p-www8-8fv9","Warn: Project is vulnerable to: GHSA-4pc4-m9mj-v2r9","Warn: Project is vulnerable to: GHSA-4v5p-v5h9-6xjx","Warn: Project is vulnerable to: PYSEC-2021-163 / GHSA-4vf2-4xcg-65cx","Warn: Project is vulnerable to: PYSEC-2021-233 / GHSA-4vrf-ff7v-hpgr","Warn: Project is vulnerable to: GHSA-4w68-4x85-mjj9","Warn: Project is vulnerable to: GHSA-4xfp-4pfp-89wg","Warn: Project is vulnerable to: PYSEC-2021-207 / GHSA-545v-42p7-98fq","Warn: Project is vulnerable to: GHSA-54ch-gjq5-4976","Warn: Project is vulnerable to: GHSA-54pp-c6pp-7fpx","Warn: Project is vulnerable to: GHSA-558h-mq8x-7q9g","Warn: Project is vulnerable to: PYSEC-2021-400 / GHSA-57wx-m983-2f88","Warn: Project is vulnerable to: GHSA-5889-7v45-q28m","Warn: Project is vulnerable to: PYSEC-2021-196 / GHSA-59q2-x2qc-4c97","Warn: Project is vulnerable to: PYSEC-2021-409 / GHSA-5crj-c72x-m7gq","Warn: Project is vulnerable to: GHSA-5f2r-qp73-37mr","Warn: Project is vulnerable to: PYSEC-2021-209 / GHSA-5gqf-456p-4836","Warn: Project is vulnerable to: GHSA-5hj3-vjjf-f5m7","Warn: Project is vulnerable to: PYSEC-2021-392 / GHSA-5hx2-qx8j-qjqm","Warn: Project is vulnerable to: GHSA-5qw5-89mw-wcg2","Warn: Project is vulnerable to: GHSA-5v77-j66x-4c4g","Warn: Project is vulnerable to: GHSA-5w96-866f-6rm8","Warn: Project is vulnerable to: GHSA-5wpj-c6f7-24x8","Warn: Project is vulnerable to: GHSA-5xwc-mrhx-5g3m","Warn: Project is vulnerable to: GHSA-627q-g293-49q7","Warn: Project is vulnerable to: PYSEC-2021-155 / GHSA-62gx-355r-9fhg","Warn: Project is vulnerable to: PYSEC-2020-118 / GHSA-63xm-rx5p-xvqr","Warn: Project is vulnerable to: GHSA-6445-fm66-fvq2","Warn: Project is vulnerable to: GHSA-647v-r7qq-24fh","Warn: Project is vulnerable to: GHSA-64jg-wjww-7c5w","Warn: Project is vulnerable to: GHSA-66vq-54fq-6jvv","Warn: Project is vulnerable to: GHSA-67pf-62xr-q35m","Warn: Project is vulnerable to: GHSA-689c-r7h2-fv9v","Warn: Project is vulnerable to: GHSA-68v3-g9cm-rmm6","Warn: Project is vulnerable to: PYSEC-2021-165 / GHSA-6f84-42vf-ppwp","Warn: Project is vulnerable to: PYSEC-2021-215 / GHSA-6f89-8j54-29xf","Warn: Project is vulnerable to: GHSA-6gmv-pjp9-p8w8","Warn: Project is vulnerable to: GHSA-6gv8-p3vj-pxvr","Warn: Project is vulnerable to: GHSA-6hg6-5c2q-7rcr","Warn: Project is vulnerable to: PYSEC-2021-401 / GHSA-6hpv-v2rx-c5g6","Warn: Project is vulnerable to: PYSEC-2021-171 / GHSA-6j9c-grc6-5m6g","Warn: Project is vulnerable to: GHSA-6p5r-g9mq-ggh2","Warn: Project is vulnerable to: PYSEC-2021-212 / GHSA-6qgm-fv6v-rfpv","Warn: Project is vulnerable to: GHSA-6wfh-89q8-44jq","Warn: Project is vulnerable to: GHSA-6x99-gv2v-q76v","Warn: Project is vulnerable to: GHSA-75c9-jrh4-79mc","Warn: Project is vulnerable to: PYSEC-2021-201 / GHSA-75f6-78jr-4656","Warn: Project is vulnerable to: GHSA-762h-vpvw-3rcx","Warn: Project is vulnerable to: PYSEC-2021-156 / GHSA-772j-h9xw-ffp5","Warn: Project is vulnerable to: PYSEC-2021-154 / GHSA-772p-x54p-hjrv","Warn: Project is vulnerable to: GHSA-77gp-3h4r-6428","Warn: Project is vulnerable to: PYSEC-2021-397 / GHSA-786j-5qwq-r36x","Warn: Project is vulnerable to: PYSEC-2021-216 / GHSA-79fv-9865-4qcv","Warn: Project is vulnerable to: GHSA-79h2-q768-fpxr","Warn: Project is vulnerable to: PYSEC-2021-213 / GHSA-7cqx-92hp-x6wh","Warn: Project is vulnerable to: GHSA-7fvx-3jfc-2cpc","Warn: Project is vulnerable to: GHSA-7ghq-fvr3-pj2x","Warn: Project is vulnerable to: GHSA-7j3m-8g3c-9qqq","Warn: Project is vulnerable to: GHSA-7jvm-xxmr-v5cw","Warn: Project is vulnerable to: PYSEC-2021-396 / GHSA-7pxj-m4jf-r6h2","Warn: Project is vulnerable to: PYSEC-2021-417 / GHSA-7r94-xv9v-63jw","Warn: Project is vulnerable to: PYSEC-2021-399 / GHSA-7v94-64hj-m82h","Warn: Project is vulnerable to: GHSA-7x4v-9gxg-9hwj","Warn: Project is vulnerable to: GHSA-7xwj-5r4v-429p","Warn: Project is vulnerable to: GHSA-828c-5j5q-vrjq","Warn: Project is vulnerable to: PYSEC-2021-211 / GHSA-828x-qc2p-wprq","Warn: Project is vulnerable to: PYSEC-2019-209 / GHSA-844w-j86r-4x2j","Warn: Project is vulnerable to: GHSA-84jm-4cf3-9jfm","Warn: Project is vulnerable to: PYSEC-2021-153 / GHSA-84mw-34w6-2q43","Warn: Project is vulnerable to: GHSA-87v6-crgm-2gfj","Warn: Project is vulnerable to: PYSEC-2021-174 / GHSA-8c89-2vwr-chcq","Warn: Project is vulnerable to: GHSA-8cxv-76p7-jxwr","Warn: Project is vulnerable to: GHSA-8fvv-46hw-vpg3","Warn: Project is vulnerable to: PYSEC-2021-197 / GHSA-8gv3-57p6-g35r","Warn: Project is vulnerable to: GHSA-8jj7-5vxc-pg2q","Warn: Project is vulnerable to: PYSEC-2021-251 / GHSA-8pmx-p244-g88h","Warn: Project is vulnerable to: GHSA-8r7c-3cm2-3h8f","Warn: Project is vulnerable to: GHSA-8rcj-c8pj-v3m3","Warn: Project is vulnerable to: PYSEC-2021-241 / GHSA-8rm6-75mf-7r7r","Warn: Project is vulnerable to: GHSA-8w5g-3wcv-9g2j","Warn: Project is vulnerable to: GHSA-8wwm-6264-x792","Warn: Project is vulnerable to: GHSA-93vr-9q9m-pj8p","Warn: Project is vulnerable to: GHSA-94mm-g2mv-8p7r","Warn: Project is vulnerable to: GHSA-95xm-g58g-3p88","Warn: Project is vulnerable to: GHSA-9697-98pf-4rw7","Warn: Project is vulnerable to: PYSEC-2020-258 / GHSA-977j-xj7q-2jr9","Warn: Project is vulnerable to: GHSA-97p7-w86h-vcf9","Warn: Project is vulnerable to: PYSEC-2021-236 / GHSA-97wf-p777-86jq","Warn: Project is vulnerable to: GHSA-98j8-c9q4-r38g","Warn: Project is vulnerable to: GHSA-98p5-x8x4-c9m5","Warn: Project is vulnerable to: GHSA-9942-r22v-78cp","Warn: Project is vulnerable to: GHSA-9c78-vcq7-7vxq","Warn: Project is vulnerable to: PYSEC-2021-238 / GHSA-9c84-4hx6-xmm4","Warn: Project is vulnerable to: GHSA-9c8h-2mv3-49ww","Warn: Project is vulnerable to: GHSA-9c8h-vvrj-w2p8","Warn: Project is vulnerable to: GHSA-9cr2-8pwr-fhfq","Warn: Project is vulnerable to: PYSEC-2021-410 / GHSA-9crf-c6qr-r273","Warn: Project is vulnerable to: GHSA-9fpg-838v-wpv7","Warn: Project is vulnerable to: GHSA-9gwq-6cwj-47h3","Warn: Project is vulnerable to: GHSA-9j4v-pp28-mxv7","Warn: Project is vulnerable to: PYSEC-2020-117 / GHSA-9mqp-7v2h-2382","Warn: Project is vulnerable to: GHSA-9p77-mmrw-69c7","Warn: Project is vulnerable to: GHSA-9px9-73fg-3fqp","Warn: Project is vulnerable to: GHSA-9v8w-xmr4-wgxp","Warn: Project is vulnerable to: PYSEC-2021-210 / GHSA-9vpm-rcf4-9wqw","Warn: Project is vulnerable to: GHSA-9vqj-64pv-w55c","Warn: Project is vulnerable to: GHSA-9w2p-5mgw-p94c","Warn: Project is vulnerable to: PYSEC-2021-220 / GHSA-9xh4-23q4-v6wr","Warn: Project is vulnerable to: PYSEC-2021-219 / GHSA-c45w-2wxr-pp53","Warn: Project is vulnerable to: GHSA-c545-c4f9-rf6v","Warn: Project is vulnerable to: GHSA-c582-c96p-r5cq","Warn: Project is vulnerable to: GHSA-c5x2-p679-95wc","Warn: Project is vulnerable to: GHSA-c6fh-56w7-fvjw","Warn: Project is vulnerable to: GHSA-c94w-c95p-phf8","Warn: Project is vulnerable to: PYSEC-2021-159 / GHSA-c968-pq7h-7fxv","Warn: Project is vulnerable to: PYSEC-2020-140 / GHSA-c9f3-9wfr-wgh7","Warn: Project is vulnerable to: GHSA-c9qf-r67m-p7cg","Warn: Project is vulnerable to: GHSA-cfpj-3q4c-jhvr","Warn: Project is vulnerable to: PYSEC-2021-230 / GHSA-cfx7-2xpc-8w4h","Warn: Project is vulnerable to: GHSA-cg88-rpvp-cjv5","Warn: Project is vulnerable to: GHSA-cgfm-62j4-v4rf","Warn: Project is vulnerable to: GHSA-ch4f-829c-v5pw","Warn: Project is vulnerable to: PYSEC-2021-246 / GHSA-cjc7-49v2-jp64","Warn: Project is vulnerable to: GHSA-cm5x-837x-jf3c","Warn: Project is vulnerable to: PYSEC-2021-414 / GHSA-cpf4-wx82-gxp6","Warn: Project is vulnerable to: PYSEC-2021-842 / GHSA-cq76-mxrc-vchh","Warn: Project is vulnerable to: PYSEC-2021-413 / GHSA-cqv6-3phm-hcwx","Warn: Project is vulnerable to: GHSA-cqvq-fvhr-v6hc","Warn: Project is vulnerable to: PYSEC-2021-240 / GHSA-crch-j389-5f84","Warn: Project is vulnerable to: GHSA-cv2p-32v3-vhwq","Warn: Project is vulnerable to: PYSEC-2020-134 / GHSA-cvpc-8phh-8f45","Warn: Project is vulnerable to: GHSA-cwpm-f78v-7m5c","Warn: Project is vulnerable to: PYSEC-2021-228 / GHSA-cwv3-863g-39vx","Warn: Project is vulnerable to: GHSA-f2vv-v9cg-qhh7","Warn: Project is vulnerable to: GHSA-f2w8-jw48-fr7j","Warn: Project is vulnerable to: GHSA-f49c-87jh-g47q","Warn: Project is vulnerable to: GHSA-f4rr-5m7v-wxcw","Warn: Project is vulnerable to: GHSA-f4w6-h4f5-wx45","Warn: Project is vulnerable to: PYSEC-2021-415 / GHSA-f54p-f6jp-4rhr","Warn: Project is vulnerable to: GHSA-f5cx-5wr3-5qrc","Warn: Project is vulnerable to: GHSA-f637-vh3r-vfh2","Warn: Project is vulnerable to: PYSEC-2021-187 / GHSA-f78g-q7r4-9wcv","Warn: Project is vulnerable to: GHSA-f7r5-q7cx-h668","Warn: Project is vulnerable to: GHSA-f8h4-7rgh-q2gm","Warn: Project is vulnerable to: GHSA-fcwc-p4fc-c5cc","Warn: Project is vulnerable to: GHSA-ffjm-4qwc-7cmf","Warn: Project is vulnerable to: GHSA-fhfc-2q7x-929f","Warn: Project is vulnerable to: GHSA-fpcp-9h7m-ffpx","Warn: Project is vulnerable to: PYSEC-2021-180 / GHSA-fphq-gw9m-ghrv","Warn: Project is vulnerable to: GHSA-fq6p-6334-8gr4","Warn: Project is vulnerable to: GHSA-fq86-3f29-px2c","Warn: Project is vulnerable to: GHSA-fqm2-gh8w-gr68","Warn: Project is vulnerable to: GHSA-fqxc-pvf8-2w9v","Warn: Project is vulnerable to: PYSEC-2021-404 / GHSA-fr77-rrx3-cp7g","Warn: Project is vulnerable to: GHSA-frqp-wp83-qggv","Warn: Project is vulnerable to: GHSA-fv25-wrff-wf86","Warn: Project is vulnerable to: GHSA-fv43-93gv-vm8f","Warn: Project is vulnerable to: GHSA-fx5c-h9f6-rv7c","Warn: Project is vulnerable to: GHSA-fxgc-95xx-grvq","Warn: Project is vulnerable to: PYSEC-2021-193 / GHSA-fxqh-cfjm-fp93","Warn: Project is vulnerable to: GHSA-g25h-jr74-qp5j","Warn: Project is vulnerable to: GHSA-g35r-369w-3fqp","Warn: Project is vulnerable to: GHSA-g468-qj8g-vcjc","Warn: Project is vulnerable to: PYSEC-2021-176 / GHSA-g4h2-gqm3-c9wq","Warn: Project is vulnerable to: PYSEC-2020-128 / GHSA-g7p5-5759-qv46","Warn: Project is vulnerable to: GHSA-g8wg-cjwc-xhhp","Warn: Project is vulnerable to: GHSA-g9fm-r5mm-rf9f","Warn: Project is vulnerable to: GHSA-g9h5-vr8m-x2h4","Warn: Project is vulnerable to: GHSA-gcvh-66ff-4mwm","Warn: Project is vulnerable to: GHSA-gf2j-f278-xh4v","Warn: Project is vulnerable to: GHSA-gf88-j2mg-cc82","Warn: Project is vulnerable to: GHSA-gf97-q72m-7579","Warn: Project is vulnerable to: GHSA-gh6x-4whr-2qv4","Warn: Project is vulnerable to: PYSEC-2021-393 / GHSA-gh8h-7j2j-qv4f","Warn: Project is vulnerable to: GHSA-gjh7-xx4r-x345","Warn: Project is vulnerable to: GHSA-gjqc-q9g6-q2j3","Warn: Project is vulnerable to: GHSA-gq2j-cr96-gvqx","Warn: Project is vulnerable to: PYSEC-2021-244 / GHSA-gv26-jpj9-c8gq","Warn: Project is vulnerable to: PYSEC-2021-198 / GHSA-gvm4-h8j3-rjrq","Warn: Project is vulnerable to: GHSA-gw97-ff7c-9v96","Warn: Project is vulnerable to: GHSA-h246-cgh4-7475","Warn: Project is vulnerable to: GHSA-h2wq-prv9-2f56","Warn: Project is vulnerable to: GHSA-h48f-q7rw-hvr7","Warn: Project is vulnerable to: PYSEC-2021-243 / GHSA-h4pc-gx2w-f2xv","Warn: Project is vulnerable to: GHSA-h5g4-ppwx-48q2","Warn: Project is vulnerable to: GHSA-h5vq-gw2c-pq47","Warn: Project is vulnerable to: PYSEC-2021-405 / GHSA-h67m-xg8f-fxcf","Warn: Project is vulnerable to: PYSEC-2020-125 / GHSA-h6fg-mjxg-hqq4","Warn: Project is vulnerable to: GHSA-h6gw-r52c-724r","Warn: Project is vulnerable to: GHSA-h6jh-7gv5-28vg","Warn: Project is vulnerable to: GHSA-h6q3-vv32-2cq5","Warn: Project is vulnerable to: GHSA-h7ff-cfc9-wmmh","Warn: Project is vulnerable to: PYSEC-2021-190 / GHSA-h9px-9vqg-222h","Warn: Project is vulnerable to: PYSEC-2021-152 / GHSA-hc6c-75p4-hmq4","Warn: Project is vulnerable to: PYSEC-2020-255 / GHSA-hhvc-g5hv-48c6","Warn: Project is vulnerable to: PYSEC-2021-182 / GHSA-hmg3-c7xj-6qwm","Warn: Project is vulnerable to: GHSA-hp4c-x6r7-6555","Warn: Project is vulnerable to: GHSA-hpv4-7p9c-mvfr","Warn: Project is vulnerable to: GHSA-hq7g-wwwp-q46h","Warn: Project is vulnerable to: GHSA-hrg5-737c-2p56","Warn: Project is vulnerable to: GHSA-hwr7-8gxx-fj5p","Warn: Project is vulnerable to: GHSA-hx9q-2mx4-m4pg","Warn: Project is vulnerable to: GHSA-j3mj-fhpq-qqjj","Warn: Project is vulnerable to: GHSA-j43h-pgmg-5hjq","Warn: Project is vulnerable to: PYSEC-2021-169 / GHSA-j47f-4232-hvv8","Warn: Project is vulnerable to: GHSA-j5w9-hmfh-4cr6","Warn: Project is vulnerable to: PYSEC-2021-224 / GHSA-j7rm-8ww4-xx2g","Warn: Project is vulnerable to: PYSEC-2021-394 / GHSA-j86v-p27c-73fm","Warn: Project is vulnerable to: PYSEC-2021-419 / GHSA-j8c8-67vp-6mx7","Warn: Project is vulnerable to: PYSEC-2021-175 / GHSA-j8qc-5fqr-52fp","Warn: Project is vulnerable to: PYSEC-2021-237 / GHSA-j8qh-3xrq-c825","Warn: Project is vulnerable to: PYSEC-2021-242 / GHSA-jf7h-7m85-w2v2","Warn: Project is vulnerable to: PYSEC-2021-166 / GHSA-jfp7-4j67-8r3q","Warn: Project is vulnerable to: PYSEC-2021-189 / GHSA-jhq9-wm9m-cf89","Warn: Project is vulnerable to: GHSA-jjm6-4vf7-cjh4","Warn: Project is vulnerable to: PYSEC-2021-229 / GHSA-jjr8-m8g8-p6wv","Warn: Project is vulnerable to: GHSA-jq6x-99hj-q636","Warn: Project is vulnerable to: GHSA-jqm7-m5q7-3hm5","Warn: Project is vulnerable to: GHSA-jvhc-5hhr-w3v5","Warn: Project is vulnerable to: GHSA-jwf9-w5xm-f437","Warn: Project is vulnerable to: PYSEC-2021-402 / GHSA-m342-ff57-4jcc","Warn: Project is vulnerable to: PYSEC-2021-183 / GHSA-m34j-p8rj-wjxq","Warn: Project is vulnerable to: PYSEC-2021-172 / GHSA-m3f9-w3p3-p669","Warn: Project is vulnerable to: GHSA-m4hf-j54p-p353","Warn: Project is vulnerable to: PYSEC-2021-389 / GHSA-m539-j985-hcr8","Warn: Project is vulnerable to: PYSEC-2020-256 / GHSA-m648-33qf-v3gp","Warn: Project is vulnerable to: GHSA-m6cv-4fmf-66xf","Warn: Project is vulnerable to: GHSA-m6vp-8q9j-whx4","Warn: Project is vulnerable to: GHSA-m7fm-4jfh-jrg6","Warn: Project is vulnerable to: PYSEC-2019-210 / GHSA-mfg7-x5m7-6p8w","Warn: Project is vulnerable to: GHSA-mg66-qvc5-rm93","Warn: Project is vulnerable to: GHSA-mgmh-g2v6-mqw5","Warn: Project is vulnerable to: GHSA-mh3m-62v7-68xg","Warn: Project is vulnerable to: PYSEC-2021-254 / GHSA-mmq6-q8r3-48fm","Warn: Project is vulnerable to: PYSEC-2021-247 / GHSA-mq5c-prh3-3f3h","Warn: Project is vulnerable to: PYSEC-2021-195 / GHSA-mqh2-9wrp-vx84","Warn: Project is vulnerable to: GHSA-mv77-9g28-cwg3","Warn: Project is vulnerable to: PYSEC-2021-222 / GHSA-mv78-g7wq-mhp4","Warn: Project is vulnerable to: GHSA-mv8m-8x97-937q","Warn: Project is vulnerable to: GHSA-mw6j-hh29-h379","Warn: Project is vulnerable to: PYSEC-2020-131 / GHSA-mxjj-953w-2c2v","Warn: Project is vulnerable to: GHSA-p2xf-8hgm-hpw5","Warn: Project is vulnerable to: PYSEC-2021-185 / GHSA-p45v-v4pw-77jr","Warn: Project is vulnerable to: GHSA-p7hr-f446-x6qf","Warn: Project is vulnerable to: GHSA-p9rc-rmr5-529j","Warn: Project is vulnerable to: GHSA-pf36-r9c6-h97j","Warn: Project is vulnerable to: GHSA-pfjj-m3jj-9jc9","Warn: Project is vulnerable to: PYSEC-2021-843 / GHSA-pgcq-h79j-2f69","Warn: Project is vulnerable to: PYSEC-2021-200 / GHSA-ph87-fvjr-v33w","Warn: Project is vulnerable to: PYSEC-2021-235 / GHSA-pmpr-55fj-r229","Warn: Project is vulnerable to: GHSA-pqhm-4wvf-2jg8","Warn: Project is vulnerable to: GHSA-pqrv-8r2f-7278","Warn: Project is vulnerable to: PYSEC-2021-390 / GHSA-prcg-wp5q-rv7p","Warn: Project is vulnerable to: PYSEC-2021-203 / GHSA-pvrc-hg3f-58r6","Warn: Project is vulnerable to: GHSA-pxrw-j2fv-hx3h","Warn: Project is vulnerable to: PYSEC-2020-257 / GHSA-q263-fvxm-m5mw","Warn: Project is vulnerable to: GHSA-q2c3-jpmc-gfjx","Warn: Project is vulnerable to: GHSA-q3g3-h9r4-prrc","Warn: Project is vulnerable to: PYSEC-2020-130 / GHSA-q4qf-3fc6-8x34","Warn: Project is vulnerable to: GHSA-q5jv-m6qw-5g37","Warn: Project is vulnerable to: GHSA-q7f7-544h-67h9","Warn: Project is vulnerable to: GHSA-q85f-69q7-55h2","Warn: Project is vulnerable to: PYSEC-2020-127 / GHSA-q8gv-q7wr-9jf8","Warn: Project is vulnerable to: GHSA-qfpc-5pjr-mh26","Warn: Project is vulnerable to: PYSEC-2020-132 / GHSA-qh32-6jjc-qprm","Warn: Project is vulnerable to: GHSA-qhw4-wwr7-gjc5","Warn: Project is vulnerable to: PYSEC-2020-254 / GHSA-qhxx-j73r-qpm2","Warn: Project is vulnerable to: GHSA-qj5r-f9mv-rffh","Warn: Project is vulnerable to: GHSA-qjj8-32p7-h289","Warn: Project is vulnerable to: GHSA-qjqc-vqcf-5qvj","Warn: Project is vulnerable to: GHSA-qr82-2c78-4m8h","Warn: Project is vulnerable to: PYSEC-2021-252 / GHSA-qw5h-7f53-xrp6","Warn: Project is vulnerable to: GHSA-qx3f-p745-w4hr","Warn: Project is vulnerable to: GHSA-qxpx-j395-pw36","Warn: Project is vulnerable to: GHSA-r26c-679w-mrjm","Warn: Project is vulnerable to: PYSEC-2021-192 / GHSA-r35g-4525-29fq","Warn: Project is vulnerable to: GHSA-r4c4-5fpq-56wg","Warn: Project is vulnerable to: PYSEC-2021-161 / GHSA-r4pj-74mg-8868","Warn: Project is vulnerable to: GHSA-r6jx-9g48-2r5r","Warn: Project is vulnerable to: PYSEC-2021-202 / GHSA-r6pg-pjwc-j585","Warn: Project is vulnerable to: GHSA-rc9w-5c64-9vqq","Warn: Project is vulnerable to: GHSA-rcf8-g8jv-vg6p","Warn: Project is vulnerable to: PYSEC-2021-239 / GHSA-rf3h-xgv5-2q39","Warn: Project is vulnerable to: PYSEC-2021-416 / GHSA-rg3m-hqc5-344v","Warn: Project is vulnerable to: PYSEC-2021-245 / GHSA-rgvq-pcvf-hx75","Warn: Project is vulnerable to: GHSA-rh87-q4vg-m45j","Warn: Project is vulnerable to: GHSA-rhrq-64mq-hf9h","Warn: Project is vulnerable to: GHSA-rjx6-v474-2ch9","Warn: Project is vulnerable to: GHSA-rmg2-f698-wq35","Warn: Project is vulnerable to: PYSEC-2020-138 / GHSA-rrfp-j2mp-hq9c","Warn: Project is vulnerable to: GHSA-rrx2-r989-2c43","Warn: Project is vulnerable to: GHSA-rww7-2gpw-fv6j","Warn: Project is vulnerable to: GHSA-v3f7-j968-4h5f","Warn: Project is vulnerable to: PYSEC-2021-234 / GHSA-v52p-hfjf-wg88","Warn: Project is vulnerable to: GHSA-v5xg-3q2c-c2r4","Warn: Project is vulnerable to: GHSA-v6h3-348g-6h5x","Warn: Project is vulnerable to: PYSEC-2021-214 / GHSA-v6r6-84gr-92rm","Warn: Project is vulnerable to: GHSA-v768-w7m9-2vmm","Warn: Project is vulnerable to: GHSA-v7vw-577f-vp8x","Warn: Project is vulnerable to: GHSA-v82p-hv3v-p6qp","Warn: Project is vulnerable to: GHSA-vcjj-9vg7-vf68","Warn: Project is vulnerable to: PYSEC-2021-232 / GHSA-vf94-36g5-69v8","Warn: Project is vulnerable to: PYSEC-2021-225 / GHSA-vfr4-x8j2-3rf9","Warn: Project is vulnerable to: GHSA-vgvh-2pf4-jr2x","Warn: Project is vulnerable to: GHSA-vjg4-v33c-ggc4","Warn: Project is vulnerable to: GHSA-vm7x-4qhj-rrcq","Warn: Project is vulnerable to: GHSA-vmjw-c2vp-p33c","Warn: Project is vulnerable to: PYSEC-2021-218 / GHSA-vq2r-5xvm-3hc3","Warn: Project is vulnerable to: PYSEC-2021-188 / GHSA-vqw6-72r7-fgw7","Warn: Project is vulnerable to: PYSEC-2021-250 / GHSA-vvg4-vgrv-xfr7","Warn: Project is vulnerable to: PYSEC-2021-406 / GHSA-vwhq-49r4-gj9v","Warn: Project is vulnerable to: GHSA-vxv8-r8q2-63xw","Warn: Project is vulnerable to: GHSA-w4xf-2pqw-5mq7","Warn: Project is vulnerable to: PYSEC-2020-129 / GHSA-w5gh-2wr2-pm6g","Warn: Project is vulnerable to: GHSA-w62h-8xjm-fv49","Warn: Project is vulnerable to: GHSA-w74j-v8xh-3w5h","Warn: Project is vulnerable to: GHSA-wc4g-r73w-x8mm","Warn: Project is vulnerable to: PYSEC-2021-157 / GHSA-wcv5-qrj6-9pfm","Warn: Project is vulnerable to: GHSA-wcv5-vrvr-3rx2","Warn: Project is vulnerable to: GHSA-wf5p-c75w-w3wh","Warn: Project is vulnerable to: PYSEC-2021-208 / GHSA-whr9-vfh2-7hm6","Warn: Project is vulnerable to: GHSA-wm93-f238-7v37","Warn: Project is vulnerable to: PYSEC-2021-204 / GHSA-wp3c-xw9g-gpcg","Warn: Project is vulnerable to: GHSA-wp77-4gmm-7cq8","Warn: Project is vulnerable to: GHSA-wq6q-6m32-9rv9","Warn: Project is vulnerable to: GHSA-wqmc-pm8c-2jhc","Warn: Project is vulnerable to: GHSA-wr9v-g9vf-c74v","Warn: Project is vulnerable to: PYSEC-2021-256 / GHSA-wvjw-p9f5-vq28","Warn: Project is vulnerable to: GHSA-wxjj-cgcx-r3vq","Warn: Project is vulnerable to: PYSEC-2021-407 / GHSA-x3v8-c8qx-3j3r","Warn: Project is vulnerable to: PYSEC-2021-164 / GHSA-x4g7-fvjj-prg8","Warn: Project is vulnerable to: GHSA-x4qx-4fjv-hmw6","Warn: Project is vulnerable to: PYSEC-2021-186 / GHSA-x83m-p7pv-ch8v","Warn: Project is vulnerable to: PYSEC-2021-217 / GHSA-x8h6-xgqx-jqgp","Warn: Project is vulnerable to: GHSA-x989-q2pq-4q5x","Warn: Project is vulnerable to: PYSEC-2020-133 / GHSA-x9j7-x98r-r4w2","Warn: Project is vulnerable to: PYSEC-2021-167 / GHSA-xcwj-wfcm-m23c","Warn: Project is vulnerable to: GHSA-xf83-q765-xm6m","Warn: Project is vulnerable to: PYSEC-2021-177 / GHSA-xgc3-m89p-vr3x","Warn: Project is vulnerable to: PYSEC-2021-162 / GHSA-xm2v-8rrw-w9pm","Warn: Project is vulnerable to: PYSEC-2020-126 / GHSA-xmq7-7fxm-rr79","Warn: Project is vulnerable to: PYSEC-2021-178 / GHSA-xqfj-35wv-m3cr","Warn: Project is vulnerable to: PYSEC-2021-255 / GHSA-xqfj-cr6q-pc8w","Warn: Project is vulnerable to: GHSA-xrp2-fhq4-4q3w","Warn: Project is vulnerable to: PYSEC-2021-395 / GHSA-xrqm-fpgr-6hhx","Warn: Project is vulnerable to: PYSEC-2021-221 / GHSA-xvjm-fvxx-q3hv","Warn: Project is vulnerable to: GHSA-xvwp-h6jv-7472","Warn: Project is vulnerable to: PYSEC-2021-194 / GHSA-xw93-v57j-fcgh","Warn: Project is vulnerable to: PYSEC-2020-139 / GHSA-xwhf-g6j5-j5gc","Warn: Project is vulnerable to: GHSA-xxcj-rhqg-m46g","Warn: Project is vulnerable to: PYSEC-2020-114","Warn: Project is vulnerable to: PYSEC-2020-115","Warn: Project is vulnerable to: PYSEC-2020-116","Warn: Project is vulnerable to: PYSEC-2020-119","Warn: Project is vulnerable to: PYSEC-2020-120","Warn: Project is vulnerable to: PYSEC-2020-121","Warn: Project is vulnerable to: PYSEC-2020-122","Warn: Project is vulnerable to: PYSEC-2020-123","Warn: Project is vulnerable to: PYSEC-2020-124","Warn: Project is vulnerable to: PYSEC-2021-151","Warn: Project is vulnerable to: PYSEC-2021-181","Warn: Project is vulnerable to: PYSEC-2021-191","Warn: Project is vulnerable to: PYSEC-2021-248","Warn: Project is vulnerable to: GHSA-3749-ghw9-m3mg","Warn: Project is vulnerable to: PYSEC-2022-43015 / GHSA-47fc-vmwq-366v","Warn: Project is vulnerable to: PYSEC-2025-41 / GHSA-53q9-r3pm-6pq6","Warn: Project is vulnerable to: PYSEC-2024-252 / GHSA-5pcm-hx3q-hm94","Warn: Project is vulnerable to: GHSA-887c-mr87-cxwp","Warn: Project is vulnerable to: PYSEC-2024-251 / GHSA-pg7h-5qx3-wjr3","Warn: Project is vulnerable to: PYSEC-2024-250","Warn: Project is vulnerable to: PYSEC-2024-259"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}}]},"last_synced_at":"2025-08-19T18:36:46.558Z","repository_id":57473552,"created_at":"2025-08-19T18:36:46.558Z","updated_at":"2025-08-19T18:36:46.558Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28413527,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T05:26:33.345Z","status":"ssl_error","status_checked_at":"2026-01-14T05:21:57.251Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bert-model","natural-language-processing","neural-network","nlp","pytorch"],"created_at":"2024-08-01T08:00:46.843Z","updated_at":"2026-01-14T07:54:25.388Z","avatar_url":"https://github.com/innodatalabs.png","language":"Python","readme":"# tBERT\n[![PyPI version](https://badge.fury.io/py/tbert.svg)](https://badge.fury.io/py/tbert)\n[![Build Status](https://travis-ci.com/innodatalabs/tbert.svg?branch=master)](https://travis-ci.com/innodatalabs/tbert)\n\nBERT model converted to PyTorch.\n\nPlease, **do NOT use this repo**, instead use the (better) library from\nHuggingFace: https://github.com/huggingface/pytorch-pretrained-BERT.git\n\nThis repo is kept as an example of converting TF model to PyTorch (utilis may be handy in case I need\nto do some thing like this again).\n\nThis is a literal port of BERT code from TensorFlow to PyTorch.\nSee the [original TF BERT repo here](https://github.com/google-research/bert).\n\nWe provide a script to convert TF BERT pre-trained checkpoint to tBERT: `tbert.cli.convert`\n\nTesting is done to ensure that tBERT code behaves exactly as TF BERT.\n\n## License\nThis work uses MIT license.\n\nOriginal code is covered by Apache 2.0 License.\n\n## Installation\n\nPython 3.6 or better is required.\n\nEasiest way to install is with the `pip`:\n```\npip install tbert\n```\nNow you can start using tBERT models in your code!\n\n## Pre-trained models\nGoogle-trained models, converted to tBERT format. For description of models, see\nthe [original TF BERT repo here](https://github.com/google-research/bert#pre-trained-models):\n\n* [Base, Uncased](https://storage.googleapis.com/public.innodatalabs.com/tbert-uncased_L-12_H-768_A-12.zip)\n* [Large, Uncased](https://storage.googleapis.com/public.innodatalabs.com/tbert-uncased_L-24_H-1024_A-16.zip)\n* [Base, Cased](https://storage.googleapis.com/public.innodatalabs.com/tbert-cased_L-12_H-768_A-12.zip)\n* [Large, Cased](https://storage.googleapis.com/public.innodatalabs.com/tbert-cased_L-24_H-1024_A-16.zip)\n* [Base, Multilingual Cased (New, recommended)](https://storage.googleapis.com/public.innodatalabs.com/tbert-multi_cased_L-12_H-768_A-12.zip)\n* [Base, Multilingual Uncased (Not recommended)](https://storage.googleapis.com/public.innodatalabs.com/tbert-multilingual_L-12_H-768_A-12.zip)\n* [Base, Chinese](https://storage.googleapis.com/public.innodatalabs.com/tbert-chinese_L-12_H-768_A-12.zip)\n\n## Using tBERT model in your PyTorch code\n\n### tbert.bert.Bert\nThis is the main juice - the Bert transformer. It is a normal PyTorch module.\nYou can use it stand-alone or in combination with other PyTorch modules.\n\n```python\nfrom tbert.bert import Bert\n\nconfig = dict(\n    attention_probs_dropout_prob=0.1,\n    directionality=\"bidi\",\n    hidden_act=\"gelu\",\n    hidden_dropout_prob=0.1,\n    hidden_size=768,\n    initializer_range=0.02,\n    intermediate_size=3072,\n    max_position_embeddings=512,\n    num_attention_heads=12,\n    num_hidden_layers=12,\n    type_vocab_size=2,\n    vocab_size=105879\n)\n\nbert = Bert(config)\n# ... should load trained parameters (see below)\n\ninput_ids      = torch.LongTensor([[1, 2, 3, 4, 5, 0]])\ninput_type_ids = torch.LongTensor([[0, 0, 1, 1, 1, 0]])\ninput_mask     = torch.LongTensor([[1, 1, 1, 1, 1, 0]])\n\nactivations = bert(input_ids, input_type_ids, input_mask)\n```\nReturns an array of activations (for each hidden layer).\nTypically only the topmost, or few top layers are used.\nEach element in the array is a Tensor of shape [B*S, H]\nwhere B is the batch size, S is the sequence length, and H is the\nsize of the hidden layer.\n\n### tbert.bert.BertPooler\nThis is the Bert transformer with pooling layer on the top.\nConvenient for sequence classification tasks. Use is very similar to\nthat of `tbert.bert.Bert` module:\n```python\nfrom tbert.bert import Bert\n\nconfig = dict(\n    attention_probs_dropout_prob=0.1,\n    directionality=\"bidi\",\n    hidden_act=\"gelu\",\n    hidden_dropout_prob=0.1,\n    hidden_size=768,\n    initializer_range=0.02,\n    intermediate_size=3072,\n    max_position_embeddings=512,\n    num_attention_heads=12,\n    num_hidden_layers=12,\n    type_vocab_size=2,\n    vocab_size=105879\n)\n\nbert_pooler = BertPooler(config)\n# ... should load trained parameters (see below)\n\ninput_ids      = torch.LongTensor([[1, 2, 3, 4, 5, 0]])\ninput_type_ids = torch.LongTensor([[0, 0, 1, 1, 1, 0]])\ninput_mask     = torch.LongTensor([[1, 1, 1, 1, 1, 0]])\n\nactivation = bert_pooler(input_ids, input_type_ids, input_mask)\n```\nReturns a single tensor of size [B, H], where\nB is the batch size, and H is the size of the hidden layer.\n\n### Programmatically loading pre-trained weights\nTo initialize `tbert.bert.Bert` or `tbert.bert.BertPooler` from pre-trained\nsaved checkpoint:\n```\n...\nbert = Bert(config)\nbert.load_pretrained(dir_name)\n```\nHere, `dir_name` should be a directory containing pre-trained tBIRT model,\nwith `bert_model.pickle` and `pooler_model.pickle` files. See below to learn how\nto convert published TF BERT pre-trained models to tBERT format.\n\nSimilarly, `load_pretrained` method can be used on `tbert.bert.BertPooler`\ninstance.\n\n## Installing optional dependencies\nOptional deps are needed to use CLI utilities:\n* to convert TF BERT checkpoint to tBERT format\n* to extract features from a sequence\n* to run training of a classifier\n\n```\npip install -r requirements.txt\nmkdir tf\ncd tf\ngit clone https://github.com/google-research/bert\ncd ..\nexport PYTHONPATH=.:tf/bert\n```\n\nNow all is set up:\n```\npython -m tbert.cli.extract_features --help\npython -m tbert.cli.convert --help\npython -m tbert.cli.run_classifier --help\n```\n\n## Running unit tests\n```\npip install pytest\npytest tbert/test\n```\n\n## Converting TF BERT pre-trained checkpoint to tBERT\n\n* Download TF BERT checkpoint and unzip it\n  ```\n  mkdir data\n  cd data\n  wget https://storage.googleapis.com/bert_models/2018_11_03/multilingual_L-12_H-768_A-12.zip\n  unzip multilingual_L-12_H-768_A-12.zip\n  cd ..\n  ```\n* Run the converter\n  ```\n  python -m tbert.cli.convert \\\n    data/multilingual_L-12_H-768_A-12 \\\n    data/tbert-multilingual_L-12_H-768_A-12\n  ```\n\n## Extracting features\n\nMake sure that you have pre-trained tBERT model (see section above).\n\n```\necho \"Who was Jim Henson ? ||| Jim Henson was a puppeteer\" \u003e /tmp/input.txt\necho \"Empty answer is cool!\" \u003e\u003e /tmp/input.txt\n\npython -m tbert.cli.extract_features \\\n    /tmp/input.txt \\\n    /tmp/output-tbert.jsonl \\\n    data/tbert-multilingual_L-12_H-768_A-12\n```\n\n## Comparing TF BERT and tBERT results\n\nRun TF BERT `extract_features`:\n```\necho \"Who was Jim Henson ? ||| Jim Henson was a puppeteer\" \u003e /tmp/input.txt\necho \"Empty answer is cool!\" \u003e\u003e /tmp/input.txt\n\nexport BERT_BASE_DIR=data/multilingual_L-12_H-768_A-12\n\npython -m extract_features \\\n  --input_file=/tmp/input.txt \\\n  --output_file=/tmp/output-tf.jsonl \\\n  --vocab_file=$BERT_BASE_DIR/vocab.txt \\\n  --bert_config_file=$BERT_BASE_DIR/bert_config.json \\\n  --init_checkpoint=$BERT_BASE_DIR/bert_model.ckpt \\\n  --layers=-1,-2,-3,-4 \\\n  --max_seq_length=128 \\\n  --batch_size=8\n```\n\nThis creates file `/tmp/output-tf.jsonl`. Now, compare this to the JSON-L file created\nby tBERT:\n\n```\npython -m tbert.cli.cmp_jsonl \\\n    --tolerance 5e-5 \\\n    /tmp/output-tbert.jsonl \\\n    /tmp/output-tf.jsonl\n```\n\nExpect output similar to this:\n```\nMax float values delta: 3.6e-05\nStructure is identical\n```\n\n## Fine-tuning a classifier\n\nDownload GLUE datasets, as explained\n[here](https://github.com/google-research/bert#sentence-and-sentence-pair-classification-tasks).\nIn the following we assume that\nGLUE datasets are in the `glue_data` directory.\n\nTo train MRPC task, do this:\n```\npython -m tbert.cli.run_classifier \\\n    data/tbert-multilingual_L-12_H-768_A-12 \\\n    /tmp \\\n    --problem mrpc \\\n    --data_dir glue_data/MRPC \\\n    --do_train \\\n    --num_train_steps 600 \\\n    --num_warmup_steps 60 \\\n    --do_eval\n```\n\nExpect to see something similar to that:\n```\n...\nStep:        550, loss:  0.039, learning rates: 1.888888888888889e-06\nStep:        560, loss:  0.014, learning rates: 1.5185185185185186e-06\nStep:        570, loss:  0.017, learning rates: 1.1481481481481482e-06\nStep:        580, loss:  0.021, learning rates: 7.777777777777779e-07\nStep:        590, loss:  0.053, learning rates: 4.074074074074075e-07\nStep:        600, loss:  0.061, learning rates: 3.703703703703704e-08\nSaved trained model\n*** Evaluating ***\nNumber of samples evaluated: 408\nAverage per-sample loss: 0.4922609218195373\nAccuracy: 0.8504901960784313\n```\n","funding_links":[],"categories":["implement of BERT besides tensorflow:","Transformer Implementations By Communities"],"sub_categories":["PyTorch"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finnodatalabs%2Ftbert","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Finnodatalabs%2Ftbert","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finnodatalabs%2Ftbert/lists"}