{"id":20358165,"url":"https://github.com/irods/irods_resource_plugin_rados","last_synced_at":"2025-04-12T03:15:18.112Z","repository":{"id":28214396,"uuid":"31718388","full_name":"irods/irods_resource_plugin_rados","owner":"irods","description":"Cacheless Ceph/rados resource plugin for iRODS","archived":false,"fork":false,"pushed_at":"2019-03-15T03:24:04.000Z","size":178,"stargazers_count":7,"open_issues_count":9,"forks_count":6,"subscribers_count":9,"default_branch":"master","last_synced_at":"2025-04-12T03:15:13.533Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/irods.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-03-05T14:42:46.000Z","updated_at":"2022-05-06T04:16:34.000Z","dependencies_parsed_at":"2022-07-27T12:47:57.489Z","dependency_job_id":null,"html_url":"https://github.com/irods/irods_resource_plugin_rados","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/irods%2Firods_resource_plugin_rados","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/irods%2Firods_resource_plugin_rados/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/irods%2Firods_resource_plugin_rados/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/irods%2Firods_resource_plugin_rados/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/irods","download_url":"https://codeload.github.com/irods/irods_resource_plugin_rados/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248510001,"owners_count":21116130,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-14T23:25:37.718Z","updated_at":"2025-04-12T03:15:18.085Z","avatar_url":"https://github.com/irods.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# irods_resource_plugin_rados\n\nCacheless, direct Ceph/rados resource plugin for iRODS.\n\nFirst presentation at iRODS user group meeting 2014 / Boston, MA - http://www.slideshare.net/mgrawinkel/development-of-the-irods-rados-plugin\n\n## TL;DR\n\n- `iadmin mkresc radosResc irados rs-host:/path \"cluster_name|pool_name|client_name\"`\n- No superfluous cache/archive tier\n- Parallel, direct, high performance access to your data!\n- Multiple rados pools from one resource server\n\n## Introduction\n\nThis iRODS plugin implements a direct access to Ceph/rados in the most efficient manner.\nFiles in the iRODS namespace are mapped to objects in the rados key-blob store.\nIn contrast to other plugins, the irados resource plugin does not need to cache or stage files, but gives you direct and parallel access to data.\nInternally, the plugin maps the POSIX like open, read, write, seek, unlink, stat, and close calls to the librados client's operations.\nTo fully use the inherent rados cluster parallelity, irods files are split to multiple 4 MB files and uploads of large files open multiple parallel transfer threads.\n\nThe plugin assumes that file's ACLs as well as its namespace and metadata is fully managed by iRODS.\nRados stores the bytes of the file with the options of the target pool.\n\nFor every new file, a unique uuid is generated as the primary access key in rados. This uuid is set as the physical path of the file in the iRODS icat.\nFiles are spread to 4 MB blobs that are named by an incrementing identifier suffix.\nThe first object is called by its uuid and contains extended attributes to store the actual size of the file and the number of blobs that make it up.\nAll following files are named as uuid-1, uuid-2, ...\n\n## Requirements\n\n- Tested on Ubuntu / CentOS\n- Requires iRODS \u003e= 4.0.3\n\n## Installation \n\nCurrently, there are no prebuilt packages, but Ubuntu 12.04 and CentOS6.5 have been successfully tested.\n\nPrerequisites for Ubuntu:\n\nFollow the steps at http://docs.ceph.com/docs/master/start/quick-start-preflight/#advanced-package-tool-apt to add the official ceph repositories that match your running cluster's version.\n\n```\nsudo apt-get install uuid-dev libssl-dev build-essential\nsudo apt-get install librados2 librados-dev\nsudo apt-get install -f\n```\nPrerequisites for CentOS:\n\nFollow the steps at http://docs.ceph.com/docs/master/start/quick-start-preflight/#rhel-centos to add the official ceph repositories that match your running cluster's version.\n\n```\nyum install librados2 librados2-devel libuuid-devel openssl-devel cmake3 irods-devel irods-externals-clang3.8-0\nyum groups install \"Development Tools\"\n```\n\nThen checkout and build the plugin package on the resource server:\n\n```\ngit clone https://github.com/irods/irods_resource_plugin_rados.git\nmkdir build_irods_resource_plugin_rados\ncd build_irods_resource_plugin_rados\ncmake ../irods_resource_plugin_rados # or cmake3 on CentOS\nmake package\n```\n\nThen install the newly created package via `dpkg/gdebi` or `yum`.\n\n## Setup\n\nCreate an irods pool on ceph, i.e.\n\n```\nceph osd pool create irods 128 \nceph auth get-or-create client.irods osd 'allow rw pool=irods' mon 'allow r' \u003e /etc/ceph/client.irods.keyring\n```\n\nN.B: 128 is the \"Placement Group\", see http://docs.ceph.com/docs/mimic/rados/operations/placement-groups/\n\nCopy the key from the newly created keyring and create the ceph config files on the resource server.\nYou can have multiple pools with different clients \u0026 capabilities.\n\n`touch /etc/irods/irados.config \u0026\u0026 chown irods: /etc/irods/irados.config \u0026\u0026 chmod 600 /etc/irods/irados.config`\n\n```\n[global]\n    mon host = ceph-mon\n\n[client.irods]\n        key = AQD7pVhUSMx1JRAA1eqDfSynx4qQBe9DHt79Ow==\n\n[client.irods2]\n        key = AQB3xHVUAPS+HxAA6PlML8jmcDMkX+5SP7Y6lw==\n```\n\nThe cluster_name, pool_name, and user_name to connect to a rados pool are configured in the resource context on resource creation.\n\nIf no context like :/tmp/ is provided, the plugin does not work correctly. Nevertheless, the context is not used at all.\n```\niadmin mkresc radosResc irados rs-host:/path \"cluster_name|pool_name|client_name\"\n```\n\nThen upload files with:\n\n```\niput -R radosResc files/\n```\n\n## Scale out\n\nAll traffic from clients to rados is routed through the resource server. If it becomes a bottleneck, just add more!\n\n```\niadmin mkresc radosRandomResc random\niadmin mkresc child_01 irados rs-01.local:/path \"ceph|poolname|client.irods\"\niadmin mkresc child_02 irados rs-02.local:/path \"ceph|poolname|client.irods\"\n...\niadmin addchildtoresc radosRandomResc child_01\niadmin addchildtoresc radosRandomResc child_02\n...\n```\n\nThe `radosRandomResc` will then distribute the load over all resource servers.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Firods%2Firods_resource_plugin_rados","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Firods%2Firods_resource_plugin_rados","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Firods%2Firods_resource_plugin_rados/lists"}