{"id":17595360,"url":"https://github.com/anasfik/gpt_3_encoder","last_synced_at":"2025-04-10T23:21:29.684Z","repository":{"id":163697289,"uuid":"637567677","full_name":"anasfik/gpt_3_encoder","owner":"anasfik","description":"A pure Dart implementation of OpenAI's original Python encoder/decoder for GPT models.","archived":false,"fork":false,"pushed_at":"2023-05-10T22:17:13.000Z","size":573,"stargazers_count":7,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-03T08:59:55.177Z","etag":null,"topics":["dart","got","gpt","gpt-2","gpt-3","gpt-4","gpt3","openai","openai-api"],"latest_commit_sha":null,"homepage":"https://pub.dev/packages/gpt_3_encoder/","language":"Dart","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/anasfik.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-05-07T23:57:40.000Z","updated_at":"2024-08-24T13:51:30.000Z","dependencies_parsed_at":null,"dependency_job_id":"bcabc22e-308a-4c01-8994-1e4dc8d289f5","html_url":"https://github.com/anasfik/gpt_3_encoder","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anasfik%2Fgpt_3_encoder","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anasfik%2Fgpt_3_encoder/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anasfik%2Fgpt_3_encoder/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anasfik%2Fgpt_3_encoder/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/anasfik","download_url":"https://codeload.github.com/anasfik/gpt_3_encoder/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248313171,"owners_count":21082812,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dart","got","gpt","gpt-2","gpt-3","gpt-4","gpt3","openai","openai-api"],"created_at":"2024-10-22T07:54:57.783Z","updated_at":"2025-04-10T23:21:29.666Z","avatar_url":"https://github.com/anasfik.png","language":"Dart","funding_links":[],"categories":[],"sub_categories":[],"readme":"# GPT 3 Encoder\n\nThis package aims to provide a simple interface for encoding and decoding text same as GPT-3, GPT-2 that uses byte pair encoding (BPE) to turn text into a series of integers to feed into the models.\n\nThis package is a pure Dart implementation of OpenAI's original Python encoder/decoder.\n\n## Usage\n\n```dart\nimport 'package:gpt_3_encoder/gpt_3_encoder.dart';\n\nvoid main() {\n  // This is the text we want to encode and decode.\n  final text = \"Hello World!\";\n\n  // Encode the text.\n  final encoded = GPT3Encoder.instance.encode(text);\n\n  // Print the encoded text and its token length. \n  print(\n    \"Your text contains ${encoded.length} tokens, encoded as follows: $encoded\",\n  );\n\n  // Decode back the encoded text token by token and print the results.\n  encoded.forEach((token) {\n    final decoded = GPT3Encoder.instance.decode([token]);\n    print(\"Token: $token, decoded as: $decoded\");\n  });\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanasfik%2Fgpt_3_encoder","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fanasfik%2Fgpt_3_encoder","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanasfik%2Fgpt_3_encoder/lists"}