{"id":14421435,"url":"https://github.com/jonnor/micropython-npyfile","last_synced_at":"2025-10-11T22:32:15.608Z","repository":{"id":252653549,"uuid":"841054541","full_name":"jonnor/micropython-npyfile","owner":"jonnor","description":"Numpy .npy file support for MicroPython - read/write/streaming","archived":false,"fork":false,"pushed_at":"2024-10-02T20:16:57.000Z","size":197,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-10-17T13:17:47.379Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jonnor.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["jonnor"]}},"created_at":"2024-08-11T14:03:18.000Z","updated_at":"2024-10-14T01:06:47.000Z","dependencies_parsed_at":"2024-08-11T15:25:46.958Z","dependency_job_id":"d92fd1e1-b5d5-408b-bdd9-f5425a48675c","html_url":"https://github.com/jonnor/micropython-npyfile","commit_stats":null,"previous_names":["jonnor/micropython-npyfile"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonnor%2Fmicropython-npyfile","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonnor%2Fmicropython-npyfile/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonnor%2Fmicropython-npyfile/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonnor%2Fmicropython-npyfile/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jonnor","download_url":"https://codeload.github.com/jonnor/micropython-npyfile/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":221808856,"owners_count":16883800,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-29T22:01:27.806Z","updated_at":"2025-10-11T22:32:10.569Z","avatar_url":"https://github.com/jonnor.png","language":"Python","readme":"\n[![Tests](https://github.com/jonnor/micropython-npyfile/actions/workflows/tests.yaml/badge.svg?branch=master)](https://github.com/jonnor/micropython-npyfile/actions/workflows/tests.yaml)\n\n# micropython-npyfile\n\nSupport for [Numpy files (.npy)](https://numpy.org/doc/stable/reference/generated/numpy.lib.format.html) for [MicroPython](https://micropython.org/).\nSimple persistence of multi-dimensional numeric array data, and interoperability with Numpy/CPython et.c.\n\nWas initially written to be used with [emlearn-micropython](https://github.com/emlearn/emlearn-micropython),\na Machine Learning and Digital Signal Processing library for MicroPython.\n\n#### Features\n\n- Reading \u0026 writing .npy files with numeric data (see below for Limitations)\n- Streaming/chunked reading \u0026 writing\n- No external dependencies. Uses [array.array](https://docs.micropython.org/en/latest/library/array.html)\n- Written in pure Python. Compatible with CPython, CircuitPython, et.c.\n\n\n## Installing\n\nThis package can be installed using [mip](https://docs.micropython.org/en/latest/reference/packages.html#installing-packages-with-mip).\n\nFor example:\n\n```bash\nmpremote mip install github:jonnor/micropython-npyfile\n```\n\nOr just copy the `npyfile.py` file to your MicroPython device.\n\n## Usage\n\n#### Save a file (simple)\n\n```python\n\nimport array\nimport npyfile\n\nshape = (10, 4)\ndata = array.array('f', (1.0 for _ in range(shape[0]*shape[1])))\n\nnpyfile.save('mydata.npy', data, shape)\n```\n\n#### Load a file (simple)\n\n```python\n\nimport npyfile\nshape, data = npyfile.load('mydata.npy')\n\nprint(shape)\nprint(data)\n```\n\n#### Streaming read\n\nStreaming/chunked reading can be used to keep memory usage low.\n\n```python\nimport npyfile\n\nwith npyfile.Reader('mydata.npy') as reader:\n\n    # Metadata available on the reader object\n    print(reader.shape, reader.typecode, reader.itemsize)\n\n    # NOTE: assumes input is 2d. Pick chunksize in another way if not\n    chunksize = reader.shape[1]\n    for chunk in reader.read_data_chunks(chunksize):\n        print(len(chunk), chunk)\n```\n\nMore examples:\n\n- Streaming matching data from two files: [two_streams.py](./examples/digits/two_streams.py)\n\n#### Streaming write\n\nStreaming/chunked writing can be used to keep memory usage low.\n\nSee implementation of `npyfile.save()`, in [npyfile.py](./npyfile.py)\n\n#### Reading .npz files\n\n.npz files are ZIP archives.\nThey can be read using an implementation of the zipfile module,\nsuch as [jonnor/micropython-zipfile](https://github.com/jonnor/micropython-zipfile).\nBoth uncompressed and DEFLATE compressed files are supported.\n\nFor example code, see [read_npz.py](./examples/read_npz.py).\n\n#### Saving portable files with numpy\n\nnumpy.save defaults to allowing pickle support.\nThis makes is very easy to accidentially save numpy.array\nobjects that use pickle for serialization, instead of just numeric data.\nSuch files will not be loadable by `npyfile` (or most other .npy readers).\n\nTo avoid this, when saving numeric arrays, always use:\n\n```python\nnumpy.save(PATH, DATA, allow_pickle=False)\n```\n\n\n## Limitations\n\n- Only little-endian is supported, not big-endian\n- Only C data order is supported, not Fortran\n- Strings are not supported\n- Complex numbers not supported\n- Pickled data is not supported\n\nNOT TESTED on unsupported/malformed/malicious inputs.\n\n## TODO \nContributions welcomed!\n\nTODO:\n\n- Add tests for unsupported/malformed inputs\n\n## Developing\n\n#### Running tests on host\n\nInstall the Unix/Window port of MicroPython. Then run:\n\n```\nMICROPYPATH=./ micropython tests/test_npyfile.py\n```\n\nThe tests can also be ran under CPython\n```\nPYTHONPATH=./ python tests/test_npyfile.py\n```\n\n#### Running tests on device\n\nConnect a MicroPython device via USB.\n\nCopy over the data\n```\nmpremote cp npyfile.py :\nmpremote -r cp tests/ :\nmpremote run tests/test_npyfile.py\n```\n\n","funding_links":["https://github.com/sponsors/jonnor"],"categories":["Libraries"],"sub_categories":["Mathematics"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjonnor%2Fmicropython-npyfile","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjonnor%2Fmicropython-npyfile","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjonnor%2Fmicropython-npyfile/lists"}