{"id":13677904,"url":"https://github.com/latitudegames/GPT-3-Encoder","last_synced_at":"2025-04-29T12:32:21.882Z","repository":{"id":39643823,"uuid":"292921177","full_name":"latitudegames/GPT-3-Encoder","owner":"latitudegames","description":"Javascript BPE Encoder Decoder for GPT-2 / GPT-3","archived":false,"fork":false,"pushed_at":"2023-04-16T00:44:43.000Z","size":667,"stargazers_count":715,"open_issues_count":32,"forks_count":193,"subscribers_count":17,"default_branch":"master","last_synced_at":"2024-11-01T15:41:58.910Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/latitudegames.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2020-09-04T18:36:31.000Z","updated_at":"2024-11-01T15:24:30.000Z","dependencies_parsed_at":"2024-01-13T17:10:57.912Z","dependency_job_id":"62d17f99-63af-4b6c-90e9-9fcd8740c0a3","html_url":"https://github.com/latitudegames/GPT-3-Encoder","commit_stats":null,"previous_names":["aidungeon/gpt-3-encoder"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/latitudegames%2FGPT-3-Encoder","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/latitudegames%2FGPT-3-Encoder/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/latitudegames%2FGPT-3-Encoder/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/latitudegames%2FGPT-3-Encoder/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/latitudegames","download_url":"https://codeload.github.com/latitudegames/GPT-3-Encoder/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223927075,"owners_count":17226451,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T13:00:48.378Z","updated_at":"2024-11-11T20:30:36.379Z","avatar_url":"https://github.com/latitudegames.png","language":"JavaScript","funding_links":[],"categories":["JavaScript","📦 Legacy \u0026 Inactive Projects"],"sub_categories":[],"readme":"# GPT-3-Encoder\nJavascript BPE Encoder Decoder for GPT-2 / GPT-3\n\n## About\nGPT-2 and GPT-3 use byte pair encoding to turn text into a series of integers to feed into the model. This is a javascript implementation of OpenAI's original python encoder/decoder which can be found [here](https://github.com/openai/gpt-2)\n\n## Install with npm\n\n```\nnpm install gpt-3-encoder\n```\n\n## Usage\n\nCompatible with Node \u003e= 12\n\n```js\nconst {encode, decode} = require('gpt-3-encoder')\n\nconst str = 'This is an example sentence to try encoding out on!'\nconst encoded = encode(str)\nconsole.log('Encoded this string looks like: ', encoded)\n\nconsole.log('We can look at each token and what it represents')\nfor(let token of encoded){\n  console.log({token, string: decode([token])})\n}\n\nconst decoded = decode(encoded)\nconsole.log('We can decode it back into:\\n', decoded)\n\n```\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flatitudegames%2FGPT-3-Encoder","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flatitudegames%2FGPT-3-Encoder","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flatitudegames%2FGPT-3-Encoder/lists"}