{"id":15797260,"url":"https://github.com/mosecorg/numbin","last_synced_at":"2025-05-07T15:01:14.722Z","repository":{"id":104339187,"uuid":"545829154","full_name":"mosecorg/numbin","owner":"mosecorg","description":"An efficient binary serialization format for numerical data.","archived":false,"fork":false,"pushed_at":"2024-12-28T01:00:24.000Z","size":53,"stargazers_count":16,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-26T00:05:41.390Z","etag":null,"topics":["deep-learning","hacktoberfest","machine-learning","numpy","python","scientific-computing","serialization"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mosecorg.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-10-05T03:33:43.000Z","updated_at":"2024-12-28T01:00:32.000Z","dependencies_parsed_at":null,"dependency_job_id":"2af4382f-67bd-4a86-bdf4-ee8192772ffd","html_url":"https://github.com/mosecorg/numbin","commit_stats":{"total_commits":26,"total_committers":1,"mean_commits":26.0,"dds":0.0,"last_synced_commit":"dab6d2fc944d2a49a172930e423b908c72e56933"},"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mosecorg%2Fnumbin","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mosecorg%2Fnumbin/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mosecorg%2Fnumbin/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mosecorg%2Fnumbin/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mosecorg","download_url":"https://codeload.github.com/mosecorg/numbin/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252902604,"owners_count":21822259,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","hacktoberfest","machine-learning","numpy","python","scientific-computing","serialization"],"created_at":"2024-10-05T00:05:49.668Z","updated_at":"2025-05-07T15:01:14.555Z","avatar_url":"https://github.com/mosecorg.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# NumBin\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://pypi.org/project/numbin/\"\u003e\n    \u003cimg src=\"https://badge.fury.io/py/numbin.svg\" alt=\"PyPI version\" height=\"20\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://pypi.org/project/numbin\"\u003e\n    \u003cimg src=\"https://img.shields.io/pypi/pyversions/numbin\" alt=\"Python Version\" /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://tldrlegal.com/license/apache-license-2.0-(apache-2.0)\"\u003e\n    \u003cimg src=\"https://img.shields.io/github/license/mosecorg/numbin\" alt=\"License\" height=\"20\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/mosecorg/numbin/actions/workflows/python-check.yml\"\u003e\n    \u003cimg src=\"https://github.com/mosecorg/numbin/actions/workflows/python-check.yml/badge.svg\" alt=\"Check status\" height=\"20\"\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\nAn efficient binary serialization format for numerical data.\n\n## Install\n\n```sh\npip install numbin\n```\n\n## Usage\n\nWork with pure NumPy data:\n\n```python\nimport numbin as nb\nimport numpy as np\n\n\narr = np.random.rand(5, 3)\n\n# in memory\nbinary = nb.dumps(arr)\nprint(nb.loads(binary))\n\n# file\nwith open(\"num.bin\", \"wb\") as f:\n    nb.dump(arr, f)\n\nwith open(\"num.bin\", \"rb\") as f:\n    print(nb.load(f))\n```\n\nWork with complex data:\n\n```python\nfrom numbin.msg_ext import NumBinMessage\n\n\nnbm = NumBinMessage()\ndata = {\"tensor\": arr, \"labels\": [\"dog\", \"cat\"], \"safe\": True}\n\n# in memory\nbinary = nbm.dumps(data)\nprint(nbm.loads(binary))\n\n# file\nwith open(\"data.bin\", \"wb\") as f:\n    nbm.dump(data, f)\n\nwith open(\"data.bin\", \"rb\") as f:\n    print(nbm.load(f))\n```\n\n## Benchmark\n\nThe code can be found in [bench.py](benchmark/bench.py)\n\nTested with Intel(R) Core(TM) i7-13700K Python 3.11.0.\n\n```shell\npip install .[bench]\npython benchmark/bench.py\n```\n\n```console\n\u003e\u003e\u003e benchmark for numpy array\n========================================================================================================================\npickle_serde\tsize:         1\ttimes: min(3.33e-06)\tmid(3.782e-06)\tmax(5.6893e-05)\t95%(3.491e-06)\tStd.(2.1728e-07)\nnumbin_serde\tsize:         1\ttimes: min(9.9101e-07)\tmid(1.106e-06)\tmax(0.00016518)\t95%(1.032e-06)\tStd.(1.9601e-07)\nnumpy_serde\tsize:         1\ttimes: min(4.9589e-05)\tmid(5.2873e-05)\tmax(0.0010263)\t95%(5.0937e-05)\tStd.(7.0191e-06)\nsafets_serde\tsize:         1\ttimes: min(3.558e-06)\tmid(4.141e-06)\tmax(0.00016262)\t95%(3.841e-06)\tStd.(3.9577e-07)\nmsg_np_serde\tsize:         1\ttimes: min(1.743e-06)\tmid(1.937e-06)\tmax(3.4253e-05)\t95%(1.83e-06)\tStd.(1.3042e-07)\n========================================================================================================================\npickle_serde\tsize:      1024\ttimes: min(3.555e-06)\tmid(4.158e-06)\tmax(9.9592e-05)\t95%(3.813e-06)\tStd.(5.7795e-07)\nnumbin_serde\tsize:      1024\ttimes: min(1.204e-06)\tmid(1.355e-06)\tmax(2.9116e-05)\t95%(1.256e-06)\tStd.(1.668e-07)\nnumpy_serde\tsize:      1024\ttimes: min(5.0394e-05)\tmid(5.4297e-05)\tmax(0.00019953)\t95%(5.2156e-05)\tStd.(2.0507e-06)\nsafets_serde\tsize:      1024\ttimes: min(4.08e-06)\tmid(4.667e-06)\tmax(4.5634e-05)\t95%(4.342e-06)\tStd.(2.6851e-07)\nmsg_np_serde\tsize:      1024\ttimes: min(2.081e-06)\tmid(2.339e-06)\tmax(3.0831e-05)\t95%(2.194e-06)\tStd.(2.1181e-07)\n========================================================================================================================\npickle_serde\tsize:     65536\ttimes: min(1.9884e-05)\tmid(2.1078e-05)\tmax(9.3203e-05)\t95%(2.024e-05)\tStd.(1.2878e-06)\nnumbin_serde\tsize:     65536\ttimes: min(1.6847e-05)\tmid(1.7845e-05)\tmax(5.5421e-05)\t95%(1.7083e-05)\tStd.(1.0197e-06)\nnumpy_serde\tsize:     65536\ttimes: min(0.00010117)\tmid(0.00010785)\tmax(0.00022275)\t95%(0.00010237)\tStd.(4.3429e-06)\nsafets_serde\tsize:     65536\ttimes: min(4.0613e-05)\tmid(4.2319e-05)\tmax(0.00010681)\t95%(4.0948e-05)\tStd.(2.1643e-06)\nmsg_np_serde\tsize:     65536\ttimes: min(2.4801e-05)\tmid(2.6234e-05)\tmax(7.0627e-05)\t95%(2.5042e-05)\tStd.(1.2072e-06)\n========================================================================================================================\npickle_serde\tsize:   3145728\ttimes: min(0.0077576)\tmid(0.0080867)\tmax(0.016288)\t95%(0.0077705)\tStd.(0.00068357)\nnumbin_serde\tsize:   3145728\ttimes: min(0.0093903)\tmid(0.013968)\tmax(0.014586)\t95%(0.013006)\tStd.(0.00054932)\nnumpy_serde\tsize:   3145728\ttimes: min(0.016239)\tmid(0.017057)\tmax(0.017629)\t95%(0.01627)\tStd.(0.00038771)\nsafets_serde\tsize:   3145728\ttimes: min(0.01532)\tmid(0.016254)\tmax(0.022971)\t95%(0.015348)\tStd.(0.00083347)\nmsg_np_serde\tsize:   3145728\ttimes: min(0.016298)\tmid(0.021077)\tmax(0.021851)\t95%(0.019673)\tStd.(0.00062183)\n========================================================================================================================\npickle_serde\tsize: 201326592\ttimes: min(0.89339)\tmid(0.89483)\tmax(0.89901)\t95%(0.89343)\tStd.(0.0020278)\nnumbin_serde\tsize: 201326592\ttimes: min(0.87285)\tmid(0.87507)\tmax(0.87934)\t95%(0.87292)\tStd.(0.0021327)\nnumpy_serde\tsize: 201326592\ttimes: min(0.76402)\tmid(0.76939)\tmax(0.8509)\t95%(0.76415)\tStd.(0.032678)\nsafets_serde\tsize: 201326592\ttimes: min(1.7488)\tmid(1.7555)\tmax(1.8294)\t95%(1.7489)\tStd.(0.030627)\nmsg_np_serde\tsize: 201326592\ttimes: min(1.3325)\tmid(1.3386)\tmax(1.343)\t95%(1.3325)\tStd.(0.004391)\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmosecorg%2Fnumbin","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmosecorg%2Fnumbin","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmosecorg%2Fnumbin/lists"}