{"id":23278481,"url":"https://github.com/hdfgroup/h5pyd","last_synced_at":"2026-02-25T06:52:00.412Z","repository":{"id":3428622,"uuid":"49459089","full_name":"HDFGroup/h5pyd","owner":"HDFGroup","description":"h5py  distributed - Python client library for HDF Rest API","archived":false,"fork":false,"pushed_at":"2025-03-19T16:54:19.000Z","size":2471,"stargazers_count":121,"open_issues_count":22,"forks_count":42,"subscribers_count":10,"default_branch":"master","last_synced_at":"2025-04-01T07:49:51.758Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/HDFGroup.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"hdfgroup"}},"created_at":"2016-01-11T22:41:46.000Z","updated_at":"2025-03-19T16:54:23.000Z","dependencies_parsed_at":"2023-12-11T16:29:51.666Z","dependency_job_id":"0a516159-a70b-4040-96d0-5822897ad3c5","html_url":"https://github.com/HDFGroup/h5pyd","commit_stats":{"total_commits":757,"total_committers":14,"mean_commits":54.07142857142857,"dds":0.08586525759577279,"last_synced_commit":"6d4c5f0582891b8e70db2283c74411400f938dfa"},"previous_names":[],"tags_count":33,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HDFGroup%2Fh5pyd","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HDFGroup%2Fh5pyd/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HDFGroup%2Fh5pyd/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HDFGroup%2Fh5pyd/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/HDFGroup","download_url":"https://codeload.github.com/HDFGroup/h5pyd/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247809964,"owners_count":20999816,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-19T22:31:54.840Z","updated_at":"2026-02-25T06:52:00.372Z","avatar_url":"https://github.com/HDFGroup.png","language":"Python","funding_links":["https://github.com/sponsors/hdfgroup"],"categories":[],"sub_categories":[],"readme":"[![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/hdfgroup/h5pyd)\n\nh5pyd\n=====\n\nPython client library for HSDS\n\n\nIntroduction\n------------\nThis repository contains library, test, and examples of h5pyd - a Python package for HSDS \n(Highly Scalable Data Service), or other HDF REST compatible web services.\n\nThe h5pyd library provides a high-level interface to the REST specification that is generally\neasier to use than invoking http calls directly.\n\nThis package is based on the popular h5py package and aims to be source compatible with\nthe h5py high level interface.\n\n\nWebsites\n--------\n\n* Main website: http://www.hdfgroup.org\n* Source code: https://github.com/HDFGroup/h5pyd\n* Forum: https://forum.hdfgroup.org/c/hsds\n* Documentation: TBD (but http://docs.h5py.org/en/latest/ should be helpful)\n\nRelated Projects\n----------------\n\n* HSDS: https://github.com/HDFGroup/hsds\n* HDF5-JSON: https://github.com/HDFGroup/hdf5-json\n* h5py: https://github.com/h5py/h5py\n* REST API Documentation: https://github.com/HDFGroup/hdf-rest-api\n\nInstalling\n-----------\n\nVia pip::\n\n   pip install h5pyd\n\nFrom a release tarball or Git checkout::\n\n   pip install .\n\nRun `hsconfigure` to setup the connection info (endpoint, username, and password) to HSDS.  \nIf you don't have access to an HSDS instance, you can easily setup your own HSDS instance.\nSee  https://github.com/HDFGroup/hsds for instructions on installing and running HSDS\non locally or in the cloud.\n\nDirect Mode\n-----------\nThe h5pyd package can be used without an explicit HSDS connection.  Rather, the storage system will\nbe accessed directly.\n  \nTo use in direct mode, set the HS_ENDPOINT to \"local\" (or \"local[n]\" where n is the number of desired\nsub-processes).  \n\nFor direct mode, some additional environment\nvariables are needed to be defined:\n\n* ``BUCKET_NAME`` - name of the S3 Bucket, Azure Container, or Posix top level folder\n\nTo use \"local\" mode with S3, define these variables:\n\n* ``AWS_S3_GATEWAY`` - AWS S3 endpoint, e.g.: ``https://s3.us-west-2.amazonaws.com``\n* ``AWS_REGION`` - Region where the Lambda function is installed, e.g.: ``us-west-2``\n* ``AWS_SECRET_ACCESS_KEY`` - Your AWS secret access AWS_SECRET_ACCESS_KEY\n* ``AWS_ACCESS_KEY_ID`` - Your AWS access key ID\n\nTo use \"local\" mode with Azure, defined these variables:\n\n* ``AZURE_CONNECTION_STRING`` - The connection string for your Azure storage account\n\nTo use \"local\" with Posix storage, define these variables:\n\n* ``ROOT_DIR`` - The top level directory used for storage (i.e. the parent directory of \"buckets\")\n\n\nH5PYD Command Line Apps\n-----------------------\n\nSeveral utility applications are included with this package:\n\n* ``hsconfigure`` - save endpoint, username, and password in config files\n* ``hsacl`` - read/update ACL (access control list) for a given folder or domain\n* ``hscopy`` - copy a domain\n* ``hsrm`` - delete a domain or folder\n* ``hsdiff`` - compare HDF5 file with HSDS domain\n* ``hsget`` - create an HDF5 file from HSDS domain\n* ``hsinfo`` - get server status or domain details\n* ``hsload`` - upload an HDF5 file to an HSDS domain\n* ``hsls`` - list contents of domain or folder\n* ``hsmv`` - change the naame of a domain\n* ``hstouch`` - create new domain or folder\n\nUse the ``--help`` option to get usage information for each command.\n\nTesting\n-------\n\nBy default the test suite will attempt to connect to a local instance of HSDS with the \n`http://localhost:5101` endpoint.\n\nUse the following environment variables as needed to modify the default configuration\nfor the test suite:\n\n* ``HS_ENDPOINT`` - \"http://127.0.0.1:5000\" for HSDS installed locally or appropriate remote endpoint\n* ``HS_USERNAME`` - \"test_user1\" or your preferred username\n* ``HS_PASSWORD`` - password for above username - \"test\" for test_user1 with local HSDS install\n* ``TEST2_USERNAME`` - \"test_user2\" or your preffered username\n* ``TEST2_PASSWORD`` - password for above username - \"test\" for test_user2 with local HSDS install\n* ``H5PYD_TEST_FOLDER`` - server folder that will be used for generated files.  Example: \"/home/test_user1/h5pyd_test/\". Use hstouch command to create the folder before running the test, e.g.: ``$ hstouch /home/test_user1/h5pyd_test/``\n\nRun: ``$python testall.py`` to execute the test suite.\n\nUninstalling\n-------------\n\nJust remove the install directory and all contents to uninstall.\n\n\nReporting bugs (and general feedback)\n-------------------------------------\n\nCreate new issues at http://github.com/HDFGroup/h5pyd/issues for any problems you find.\n\nFor general questions/feedback, please use the HSDS forum (https://forum.hdfgroup.org/c/hsds).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhdfgroup%2Fh5pyd","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhdfgroup%2Fh5pyd","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhdfgroup%2Fh5pyd/lists"}