{"id":26844314,"url":"https://github.com/usnistgov/h5wasm","last_synced_at":"2026-01-12T09:57:29.108Z","repository":{"id":39794749,"uuid":"401916491","full_name":"usnistgov/h5wasm","owner":"usnistgov","description":"A WebAssembly HDF5 reader/writer library ","archived":false,"fork":false,"pushed_at":"2025-03-03T17:08:47.000Z","size":42412,"stargazers_count":103,"open_issues_count":23,"forks_count":16,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-03-30T19:19:26.365Z","etag":null,"topics":["browser","hdf5","javascript","nodejs","wasm","webassembly","zero-dependency"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/usnistgov.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-09-01T03:16:09.000Z","updated_at":"2025-03-27T01:45:15.000Z","dependencies_parsed_at":"2023-02-09T13:16:33.076Z","dependency_job_id":"8e4be72d-2d85-48ff-9437-5bcd4431c4a7","html_url":"https://github.com/usnistgov/h5wasm","commit_stats":{"total_commits":334,"total_committers":4,"mean_commits":83.5,"dds":"0.050898203592814384","last_synced_commit":"7e435b782cb39ce71b37f67d4c4c8a522c01988f"},"previous_names":[],"tags_count":45,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/usnistgov%2Fh5wasm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/usnistgov%2Fh5wasm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/usnistgov%2Fh5wasm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/usnistgov%2Fh5wasm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/usnistgov","download_url":"https://codeload.github.com/usnistgov/h5wasm/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247543587,"owners_count":20955865,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["browser","hdf5","javascript","nodejs","wasm","webassembly","zero-dependency"],"created_at":"2025-03-30T19:19:51.910Z","updated_at":"2026-01-12T09:57:29.102Z","avatar_url":"https://github.com/usnistgov.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# h5wasm\na zero-dependency WebAssembly-powered library for [reading](#reading) and [writing](#writing) HDF5 files from javascript\n\n(built on the [HDF5 C API](http://portal.hdfgroup.org/pages/viewpage.action?pageId=50073943))\n\nThe built binaries (esm and node) will be attached to the [latest release](https://github.com/usnistgov/h5wasm/releases/latest/) as h5wasm-{version}.tgz\n  \nThe wasm-compiled libraries `libhdf5.a`, `libhdf5_cpp.a` ... and the related `include/` folder are retrieved from [libhdf5-wasm](https://github.com/usnistgov/libhdf5-wasm) during the build.\n\nInstead of importing a namespace \"*\", it is now possible to import the important h5wasm components in an object, from the default export:\n```js\n// in hdf5_hl.ts:\nexport const h5wasm = {\n    File,\n    Group,\n    Dataset,\n    ready,\n    ACCESS_MODES\n}\n```\n\nThe Emscripten filesystem is important for operations, and it can be accessed after the WASM is loaded as below.\n\n## Browser (no-build)\n```js\nimport h5wasm from \"https://cdn.jsdelivr.net/npm/h5wasm@0latest/dist/esm/hdf5_hl.js\";\n\n// the WASM loads asychronously, and you can get the module like this:\nconst Module = await h5wasm.ready;\n\n// then you can get the FileSystem object from the Module:\nconst { FS } = Module;\n\n// Or, you can directly get the FS if you don't care about the rest \n// of the module:\n// const { FS } = await h5wasm.ready;\n\nlet response = await fetch(\"https://ncnr.nist.gov/pub/ncnrdata/vsans/202003/24845/data/sans59510.nxs.ngv\");\nlet ab = await response.arrayBuffer();\n\nFS.writeFile(\"sans59510.nxs.ngv\", new Uint8Array(ab));\n\n// use mode \"r\" for reading.  All modes can be found in h5wasm.ACCESS_MODES\nlet f = new h5wasm.File(\"sans59510.nxs.ngv\", \"r\");\n// File {path: \"/\", file_id: 72057594037927936n, filename: \"data.h5\", mode: \"r\"}\n```\n\n### Worker usage\nSince ESM is not supported in all web worker contexts (e.g. Firefox), an additional  ```./dist/iife/h5wasm.js``` is provided in the package for `h5wasm\u003e=0.4.8`; it can be loaded in a worker and used as in the example below (which uses the WORKERFS file system for random access on local files):\n```js\n// worker.js\nonmessage = async function(e) {\n    const { FS } = await h5wasm.ready;\n    \n    // send in a file opened from an \u003cinput type=\"file\" /\u003e\n    const f_in = e.data[0];\n\n    FS.mkdir('/work');\n    FS.mount(FS.filesystems.WORKERFS, { files: [f_in] }, '/work');\n\n    const f = new h5wasm.File(`/work/${f_in.name}`, 'r');\n    console.log(f);\n}\n\nself.importScripts('../dist/iife/h5wasm.js');\n```\n\n## Browser target (build system)\n```npm i h5wasm``` or ```yarn add h5wasm```\nthen in your file\n```js\n// index.js\nimport h5wasm from \"h5wasm\";\nconst { FS } = await h5wasm.ready;\n\nlet f = new h5wasm.File(\"test.h5\", \"w\");\nf.create_dataset({name: \"text_data\", data: [\"this\", \"that\"]});\n// ...\n```\n__note__: you must configure your build system to target \u003e= ES2020 (for bigint support)\n\n## nodejs\nThe host filesystem is made available through Emscripten \"NODERAWFS=1\".\n\nEnabling BigInt support may be required for nodejs \u003c 16\n```bash\nnpm i h5wasm\nnode --experimental-wasm-bigint\n\n```\n\n```js\nconst h5wasm = await import(\"h5wasm/node\");\nawait h5wasm.ready;\n\nlet f = new h5wasm.File(\"/home/brian/Downloads/sans59510.nxs.ngv\", \"r\");\n/*\nFile {\n  path: '/',\n  file_id: 72057594037927936n,\n  filename: '/home/brian/Downloads/sans59510.nxs.ngv',\n  mode: 'r'\n} \n*/\n```\n\n## Usage\n_(all examples are written in ESM - for Typescript some type casting is probably required, as `get` returns either Group or Dataset)_\n### Reading\n```js\nlet f = new h5wasm.File(\"sans59510.nxs.ngv\", \"r\");\n\n// list keys:\nf.keys()\n// [\"entry\"]\n\nf.get(\"entry/instrument\").keys()\n// [\"attenuator\",\"beam\",\"beam_monitor_low\",\"beam_monitor_norm\",\"beam_stop_C2\",\"beam_stop_C3\",\"collimator\",\"converging_pinholes\",\"detector_B\",\"detector_FB\",\"detector_FL\",\"detector_FR\",\"detector_FT\",\"detector_MB\",\"detector_ML\",\"detector_MR\",\"detector_MT\",\"lenses\",\"local_contact\",\"name\",\"sample_aperture\",\"sample_aperture_2\",\"sample_table\",\"source\",\"source_aperture\",\"type\"]\n\nlet data = f.get(\"entry/instrument/detector_MR/data\")\n// Dataset {path: \"/entry/instrument/detector_MR/data\", file_id: 72057594037927936n}\n\ndata.metadata\n/* \n{\n    \"signed\": true,\n    \"vlen\": false,\n    \"littleEndian\": true,\n    \"type\": 0,\n    \"size\": 4,\n    \"shape\": [\n        48,\n        128\n    ],\n    \"total_size\": 6144\n}\n*/\n\n// for convenience, these are extracted from metadata:\ndata.dtype\n// \"\u003ci\"\ndata.shape\n// (2) [48, 128]\n\n// data are loaded into a matching TypedArray in javascript if one exists, otherwise raw bytes are returned (there is no Float16Array, for instance).  In this case the matching type is Int32Array\ndata.value\n/*\nInt32Array(6144) [0, 0, 0, 2, 2, 2, 3, 1, 1, 7, 3, 5, 7, 8, 9, 21, 43, 38, 47, 8, 8, 7, 3, 6, 1, 7, 3, 7, 47, 94, 91, 99, 76, 81, 86, 112, 98, 103, 85, 100, 83, 122, 111, 123, 136, 129, 134, 164, 130, 164, 176, 191, 200, 211, 237, 260, 304, 198, 32, 9, 5, 2, 6, 5, 8, 6, 25, 219, 341, 275, 69, 11, 4, 5, 5, 45, 151, 154, 141, 146, 108, 107, 105, 113, 99, 101, 96, 84, 86, 77, 78, 107, 73, 80, 105, 65, 75, 79, 62, 31, …]\n*/\n\n// take a slice from 0:10 on axis 0, keeping all of axis 1:\n// (slicing is done through libhdf5 instead of in the javascript library - should be very efficient)\ndata.slice([[0,10],[]])\n/*\nInt32Array(1280) [0, 0, 0, 2, 2, 2, 3, 1, 1, 7, 3, 5, 7, 8, 9, 21, 43, 38, 47, 8, 8, 7, 3, 6, 1, 7, 3, 7, 47, 94, 91, 99, 76, 81, 86, 112, 98, 103, 85, 100, 83, 122, 111, 123, 136, 129, 134, 164, 130, 164, 176, 191, 200, 211, 237, 260, 304, 198, 32, 9, 5, 2, 6, 5, 8, 6, 25, 219, 341, 275, 69, 11, 4, 5, 5, 45, 151, 154, 141, 146, 108, 107, 105, 113, 99, 101, 96, 84, 86, 77, 78, 107, 73, 80, 105, 65, 75, 79, 62, 31, …]\n*/\n\n// Convert to nested Array, with JSON-compatible elements:\ndata.to_array()\n/*\n[\n  [\n      0,   0,   0,   2,   2,   2,   3,   1,   1,   7,   3,   5,\n      7,   8,   9,  21,  43,  38,  47,   8,   8,   7,   3,   6,\n      1,   7,   3,   7,  47,  94,  91,  99,  76,  81,  86, 112,\n     98, 103,  85, 100,  83, 122, 111, 123, 136, 129, 134, 164,\n    130, 164, 176, 191, 200, 211, 237, 260, 304, 198,  32,   9,\n      5,   2,   6,   5,   8,   6,  25, 219, 341, 275,  69,  11,\n      4,   5,   5,  45, 151, 154, 141, 146, 108, 107, 105, 113,\n     99, 101,  96,  84,  86,  77,  78, 107,  73,  80, 105,  65,\n     75,  79,  62,  31,\n    ... 28 more items\n  ],\n  [\n      0,   0,   2,   2,   4,   1,   2,   7,   2,   3,   2,   5,\n      6,   3,   6,  24,  37,  42,  25,   8,   3,   5,   4,   8,\n      2,   6,   7,   9,  61,  81,  81,  89, 104, 110,  82,  82,\n    104,  92,  97,  99, 104, 115, 106, 128, 134, 111, 125, 123,\n    159, 155, 182, 228, 227, 242, 283, 290, 295, 114,  11,   6,\n      5,   6,   8,   4,   4,  10,  59, 401, 401, 168,  10,   6,\n      6,   4,  10,  37, 150, 152, 146, 121, 125, 117, 122,  88,\n    100,  97,  86,  79,  90,  87,  78,  87,  87,  87,  84,  76,\n     76,  66,  51,  11,\n    ... 28 more items\n  ],\n  ... 46 more items\n*/\n```\n\n### SWMR Read\n(single writer multiple readers)\n```js\nconst swmr_file = new h5wasm.File(\"swmr.h5\", \"Sr\");\nlet dset = swmr_file.get(\"data\");\ndset.shape;\n// 12\n// ...later\ndset.refresh();\ndset.shape;\n// 16\n```\n\n### Writing\n```js\nlet new_file = new h5wasm.File(\"myfile.h5\", \"w\");\n\nnew_file.create_group(\"entry\");\n\n// shape and dtype will match input if omitted\nnew_file.get(\"entry\").create_dataset({name: \"auto\", data: [3.1, 4.1, 0.0, -1.0]});\nnew_file.get(\"entry/auto\").shape\n// [4]\nnew_file.get(\"entry/auto\").dtype\n// \"\u003cd\"\nnew_file.get(\"entry/auto\").value\n// Float64Array(4) [3.1, 4.1, 0, -1]\n\n// make float array instead of double (shape will still match input if it is set to null)\nnew_file.get(\"entry\").create_dataset({name: \"data\", data: [3.1, 4.1, 0.0, -1.0], shape: null, dtype: '\u003cf'});\nnew_file.get(\"entry/data\").shape\n// [4]\nnew_file.get(\"entry/data\").value\n//Float32Array(4) [3.0999999046325684, 4.099999904632568, 0, -1]\n\n// create a dataset with shape=[2,2]\n// The dataset stored in the HDF5 file with the correct shape, \n// but no attempt is made to make a 2x2 array out of it in javascript\nnew_file.get(\"entry\").create_dataset({name: \"square_data\", data: [3.1, 4.1, 0.0, -1.0], shape: [2,2], dtype: '\u003cd'});\nnew_file.get(\"entry/square_data\").shape\n// (2) [2, 2]\nnew_file.get(\"entry/square_data\").value\n//Float64Array(4) [3.1, 4.1, 0, -1]\n\n// create a dataset with compression\nconst long_data = [...new Array(1000000)].map((_, i) =\u003e i);\nnew_file.get(\"entry\").create_dataset({name: \"compressed\", data: long_data, shape: [1000, 1000], dtype: '\u003cf', chunks: [100,100], compression: 9});\n// equivalent to:\n// new_file.get(\"entry\").create_dataset({name: \"compressed\", data: long_data, shape: [1000, 1000], dtype: '\u003cf', chunks=[100,100], compression='gzip', compression_opts=[9]});\nnew_file.get(\"entry/compressed\").filters\n// [{id: 1, name: 'deflate'}]);\nnew_file.get(\"entry/compressed\").slice([[2,3]]);\n// Float32Array(1000) [ 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, … ]\n\n\n// create an attribute (creates a VLEN string by default for a string)\nnew_file.get(\"entry\").create_attribute(\"myattr\", \"a string\");\nObject.keys(new_file.get(\"entry\").attrs)\n// [\"myattr\"]\nnew_file.get(\"entry\").attrs[\"myattr\"]\n// {value: \"a string\", shape: Array(0), dtype: \"S\"}\n\nnew_file.get(\"entry\").create_attribute(\"fixed\", [\"hello\", \"you\"], null, \"S5\")\nnew_file.get(\"entry\").attrs[\"fixed\"]\n/*\n{\n    \"value\": [\n        \"hello\",\n        \"you\"\n    ],\n    \"shape\": [\n        2\n    ],\n    \"dtype\": \"S5\"\n}\n*/\n\n// close the file - reading and writing will no longer work.\n// calls H5Fclose on the file_id.\nnew_file.close()\n\n```\n\n### Links\n```js\nlet new_file = new h5wasm.File(\"myfile.h5\", \"w\");\nnew_file.create_group(\"entry\");\nnew_file.get(\"entry\").create_dataset({name: \"auto\", data: [3.1, 4.1, 0.0, -1.0]});\n\n// create a soft link in root:\nnew_file.create_soft_link(\"/entry/auto\", \"my_soft_link\");\nnew_file.get(\"my_soft_link\").value;\n// Float64Array(4) [3.1, 4.1, 0, -1]\n\n// create a hard link:\nnew_file.create_hard_link(\"/entry/auto\", \"my_hard_link\");\nnew_file.get(\"my_hard_link\").value;\n// Float64Array(4) [3.1, 4.1, 0, -1]\n\n// create an external link:\nnew_file.create_external_link(\"other_file.h5\", \"other_dataset\", \"my_external_link\");\nnew_file.get_external_link(\"my_external_link\");\n// {filename: \"other_file.h5\", obj_path: \"other_dataset\"}\n\n// create a soft link in a group:\nnew_file.create_group(\"links\");\nconst links_group = new_file.get(\"links\");\nlinks_group.create_soft_link(\"/entry/auto\", \"soft_link\");\nnew_file.get(\"/links/soft_link\").value;\n// Float64Array(4) [3.1, 4.1, 0, -1]\nnew_file.get_link(\"/links/soft_link\");\n// \"/entry/auto\"\nnew_file.get_link(\"/entry/auto\");\n// null // (null is returned if the path is not a symbolic link);\n\nnew_file.close()\n```\n\n### Edit\nOne can also open an existing file and write to it:\n```js\nlet f = new h5wasm.File(\"myfile.h5\", \"a\");\n\nf.create_attribute(\"new_attr\", \"something wicked this way comes\");\nf.close()\n```\n## Web Helpers\nOptional, to support uploads and downloads\n\n```js\nimport {uploader, download, UPLOADED_FILES} from \"https://cdn.jsdelivr.net/npm/h5wasm@latest/dist/esm/file_handlers.js\";\n// \n// Attach to a file input element:\n// will save to Module.FS (memfs) with the name of the uploaded file\ndocument.getElementById(\"upload_selector\").onchange = uploader;\n// file can be found with \nlet f = new h5wasm.File(UPLOADED_FILES[UPLOADED_FILES.length -1], \"r\");\n\nlet new_file = new h5wasm.File(\"myfile.h5\", \"w\");\n\nnew_file.create_group(\"entry\");\n\n// shape and dtype will match input if omitted\nnew_file.get(\"entry\").create_dataset({name: \"auto\", data: [3.1, 4.1, 0.0, -1.0]});\n\n// this will download a snapshot of the HDF5 in its current state, with the same name\n// (in this case, a file named \"myfile.h5\" would be downloaded)\ndownload(new_file);\n```\n\n## Persistent file store (web)\nTo persist the emscripten virtual filesystem between sessions, use IDBFS (syncs with browser IndexedDB), e.g.\n```js\n// create a local mount of the IndexedDB filesystem:\nFS.mount(FS.filesystems.IDBFS, {}, \"/home/web_user\")\n\n// to read from the browser IndexedDB into the active filesystem:\nFS.syncfs(true, (e) =\u003e {console.log(e)});\n\n// to push all current files in /home/web_user to IndexedDB, e.g. when closing your application:\nFS.syncfs(false, (e) =\u003e {console.log(e)})\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fusnistgov%2Fh5wasm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fusnistgov%2Fh5wasm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fusnistgov%2Fh5wasm/lists"}