{"id":13723571,"url":"https://github.com/discord/erlpack","last_synced_at":"2025-05-15T04:04:23.757Z","repository":{"id":43350481,"uuid":"48080973","full_name":"discord/erlpack","owner":"discord","description":"High Performance Erlang Term Format Packer","archived":false,"fork":false,"pushed_at":"2025-01-30T23:56:13.000Z","size":1658,"stargazers_count":234,"open_issues_count":24,"forks_count":66,"subscribers_count":150,"default_branch":"master","last_synced_at":"2025-05-11T18:58:55.357Z","etag":null,"topics":["electron","elixir","erlang","etf","nodejs","python"],"latest_commit_sha":null,"homepage":null,"language":"Cython","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/discord.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-12-16T02:03:44.000Z","updated_at":"2025-05-02T03:52:22.000Z","dependencies_parsed_at":"2024-05-10T06:42:25.964Z","dependency_job_id":"58bba2d0-b73d-42ed-8fc4-5175ccc37e72","html_url":"https://github.com/discord/erlpack","commit_stats":{"total_commits":96,"total_committers":19,"mean_commits":5.052631578947368,"dds":0.625,"last_synced_commit":"cbe76be04c2210fc9cb6ff95910f0937c1011d04"},"previous_names":["hammerandchisel/erlpack"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/discord%2Ferlpack","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/discord%2Ferlpack/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/discord%2Ferlpack/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/discord%2Ferlpack/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/discord","download_url":"https://codeload.github.com/discord/erlpack/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254270641,"owners_count":22042858,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["electron","elixir","erlang","etf","nodejs","python"],"created_at":"2024-08-03T01:01:42.981Z","updated_at":"2025-05-15T04:04:23.727Z","avatar_url":"https://github.com/discord.png","language":"Cython","funding_links":[],"categories":["Languages Integration"],"sub_categories":[],"readme":"# Erlpack\n\nErlpack is a fast encoder and decoder for the Erlang Term Format (version 131) for Python and JavaScript.\n\n# JavaScript\n\n## Things that can be packed:\n\n- [X] Null\n- [X] Booleans\n- [X] Strings\n- [ ] Atoms\n- [X] Unicode Strings\n- [X] Floats\n- [X] Integers\n- [ ] Longs\n- [ ] Longs over 64 bits\n- [X] Objects\n- [X] Arrays\n- [ ] Tuples\n- [ ] PIDs\n- [ ] Ports\n- [ ] Exports\n- [ ] References\n\n## How to pack:\n```js\nlet erlpack = require(\"erlpack\");\n\npacked = erlpack.pack({'a': true, 'list': ['of', 3, 'things', 'to', 'pack']});\n```\n\n## How to unpack:\nNote: Unpacking requires the binary data be a Uint8Array or Buffer. For those using electron/libchromium see the gotcha below. \n```js\nlet erlpack = require(\"erlpack\");\n\nlet unpacked = null;\nlet packed = new Buffer('', 'binary');\ntry  {\n    unpacked = erlpack.unpack(packed);\n}\ncatch (e) {\n    // got an exception parsing\n}\n```\n\n## Libchromium / Electron Gotcha\nSome versions of libchromium replace the native data type backing TypedArrays with a custom data type called \nblink::WebArrayBuffer. To keep erlpack' dependencies simple this data type is not supported directly. If you're using\nElectron / Libchromium you need to convert the blink::WebArrayBuffer into a node::Buffer before passing to erlpack. You will\nneed to add this code into your native package somewhere:\n```cpp\nv8::Local\u003cv8::Value\u003e ConvertToNodeBuffer(const v8::Local\u003cv8::Object\u003e\u0026 blinkArray)\n{\n    if (node::Buffer::HasInstance(blinkArray)) {\n        return blinkArray;\n    }\n    else if (blinkArray-\u003eIsArrayBufferView()) {\n        auto byteArray = v8::ArrayBufferView::Cast(*blinkArray);\n        return node::Buffer::Copy(v8::Isolate::GetCurrent(), (const char*)byteArray-\u003eBuffer()-\u003eGetContents().Data(), byteArray-\u003eByteLength()).ToLocalChecked();\n    }\n    \n    return v8::Local\u003cv8::Primitive\u003e(v8::Null(v8::Isolate::GetCurrent()));\n}\n```\n\nThen in JavaScript something like:\n\n```js\nlet packed = NativeUtils.convertToNodeBuffer(new Uint8Array(binaryPayload));\n// unpack now using erlpack.unpack(packed)\n```\n\n# Python\n\n## Things that can be packed:\n\n- [X] None\n- [X] Booleans\n- [X] Strings\n- [X] Atoms\n- [X] Unicode Strings\n- [X] Floats\n- [X] Integers\n- [X] Longs\n- [ ] Longs over 64 bits\n- [X] Dictionaries\n- [X] Lists\n- [X] Tuples\n- [X] User Types (via an encode hook)\n- [ ] PIDs\n- [ ] Ports\n- [ ] Exports\n- [ ] References\n\n## How to pack:\n```py\nfrom erlpack import pack\n\npacked = pack([\"thing\", \"to\", \"pack\"])\n```\n\n## How to unpack:\n```py\nfrom erlpack import unpack\n\nunpacked = unpack(packed)\n```\n\n## How to pack an atom:\n\n```py\nfrom erlpack import Atom, pack\n\npacked = pack(Atom('hello'))\n```\n\n## How to use an encode hook.\n\n```py\nfrom erlpack import ErlangTermEncoder\n\ndef encode_hook(obj):\n    if isinstance(obj, datetime.datetime):\n        return obj.isoformat()\n\nencoder = ErlangTermEncoder(encode_hook=encode_hook)\npacked = encoder.pack(datetime.datetime(2015, 12, 25, 12, 23, 55))\n\n```\n\n## How to make custom types packable.\n\n```py\nfrom erlpack import pack, Atom\n\nclass User(object):\n    def __init__(self, name, age):\n        self.name = name\n        self.age = age\n\n    def __erlpack__(self):\n        return {\n            Atom('name'): self.name,\n            Atom('age'): self.age\n        }\n\nu = User(name='Jake', age=23)\npacked = pack(u)\n```\n\n# Go (golang)\n\nDiscord has moved away from Go internally and so we do not maintain a version of erlpack in Go ourselves. However, all is\nnot lost!, please check out: https://github.com/JakeMakesStuff/go-erlpack\n\n# Building\n\n## Python\nGenerating the new `.cpp` files can be accomplished by running `python setup.py --use-cython build_ext --inplace`.\n\n`cython` must be installed for this to work.\n\n### Testing\n\n1. Install the development version of erlpack with `python setup.py develop`.\n2. Install `pytest`.\n3. Execute `pytest py/tests`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdiscord%2Ferlpack","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdiscord%2Ferlpack","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdiscord%2Ferlpack/lists"}