{"id":15009273,"url":"https://github.com/irods/python-irodsclient","last_synced_at":"2025-05-15T12:06:12.997Z","repository":{"id":19625948,"uuid":"22877727","full_name":"irods/python-irodsclient","owner":"irods","description":"A Python API for iRODS","archived":false,"fork":false,"pushed_at":"2025-04-30T17:18:42.000Z","size":1450,"stargazers_count":67,"open_issues_count":113,"forks_count":74,"subscribers_count":24,"default_branch":"main","last_synced_at":"2025-04-30T18:29:50.555Z","etag":null,"topics":["irods","python3"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/irods.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2014-08-12T13:12:28.000Z","updated_at":"2025-04-30T17:18:46.000Z","dependencies_parsed_at":"2023-10-14T16:13:33.970Z","dependency_job_id":"3fc17745-1e11-496c-9f45-5c4336604899","html_url":"https://github.com/irods/python-irodsclient","commit_stats":{"total_commits":745,"total_committers":57,"mean_commits":"13.070175438596491","dds":0.7476510067114094,"last_synced_commit":"5e7334dde5404168bde75d2267d825328740e244"},"previous_names":[],"tags_count":29,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/irods%2Fpython-irodsclient","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/irods%2Fpython-irodsclient/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/irods%2Fpython-irodsclient/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/irods%2Fpython-irodsclient/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/irods","download_url":"https://codeload.github.com/irods/python-irodsclient/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254337613,"owners_count":22054253,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["irods","python3"],"created_at":"2024-09-24T19:24:11.861Z","updated_at":"2025-05-15T12:06:09.581Z","avatar_url":"https://github.com/irods.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"Python iRODS Client (PRC)\n=========================\n\n[iRODS](https://www.irods.org) is an open source distributed data management system. This is a client API implemented in Python.\n\nCurrently supported:\n\n-   Python 3.9 or later\n-   Establish a (secure) connection to iRODS\n-   Authenticate via password, GSI, PAM\n-   GenQuery and Specific Queries\n-   GenQuery2\n-   Manage collections, data objects, and permissions\n    -   Checksum data objects\n    -   Replicate data objects\n    -   Parallel PUT/GET data objects\n    -   Read, write, and seek operations\n    -   Register files and directories\n-   Manage users and groups\n-   Manage resources\n-   Manage and execute iRODS rules\n-   Manage metadata\n-   Manage ticket-based access\n\nInstalling\n----------\n\nInstall via pip:\n\n    pip install python-irodsclient\n\nor:\n\n    pip install git+https://github.com/irods/python-irodsclient.git[@branch|@commit|@tag]\n\nUninstalling\n------------\n\n    pip uninstall python-irodsclient\n\nEstablishing a (secure) connection\n----------------------------------\n\nOne way of starting a session is to pass iRODS credentials as keyword\narguments:\n\n```python\n\u003e\u003e\u003e from irods.session import iRODSSession\n\u003e\u003e\u003e with iRODSSession(host='localhost', port=1247, user='bob', password='1234', zone='tempZone') as session:\n...      # workload\n...\n\u003e\u003e\u003e\n```\n\nIf you're an administrator acting on behalf of another user:\n\n```python\n\u003e\u003e\u003e from irods.session import iRODSSession\n\u003e\u003e\u003e with iRODSSession(host='localhost', port=1247, user='rods', password='1234', zone='tempZone', client_user='bob',\n           client_zone='possibly_another_zone') as session:\n...      # workload\n...\n\u003e\u003e\u003e\n```\n\nIf no `client_zone` is provided, the `zone` parameter is used in its place.\n\nUsing environment files (including any SSL settings) in `~/.irods/`:\n\n```python\n\u003e\u003e\u003e import os\n\u003e\u003e\u003e import ssl\n\u003e\u003e\u003e from irods.session import iRODSSession\n\u003e\u003e\u003e try:\n...     env_file = os.environ['IRODS_ENVIRONMENT_FILE']\n... except KeyError:\n...     env_file = os.path.expanduser('~/.irods/irods_environment.json')\n...\n\u003e\u003e\u003e ssl_settings = {} # Or, optionally: {'ssl_context': \u003cuser_customized_SSLContext\u003e}\n\u003e\u003e\u003e with iRODSSession(irods_env_file=env_file, **ssl_settings) as session:\n...     # workload\n...\n\u003e\u003e\u003e\n```\n\nIn the above example, an SSL connection can be made even if no\n'ssl_context' argument is specified, in which case the Python client\ninternally generates its own SSLContext object to best match the iRODS\nSSL configuration parameters (such as\n\"irods_ssl_ca_certificate_file\", etc.) used to initialize the\niRODSSession. Those parameters can be given either in the environment\nfile, or in the iRODSSession constructor call as shown by the next\nexample.\n\nA pure Python SSL session (without a local `env_file` requires a few more things defined:\n\n```python\n\u003e\u003e\u003e import ssl\n\u003e\u003e\u003e from irods.session import iRODSSession\n\u003e\u003e\u003e ssl_settings = {'client_server_negotiation': 'request_server_negotiation',\n...                'client_server_policy': 'CS_NEG_REQUIRE',\n...                'encryption_algorithm': 'AES-256-CBC',\n...                'encryption_key_size': 32,\n...                'encryption_num_hash_rounds': 16,\n...                'encryption_salt_size': 8,\n...                'ssl_context': ssl_context\n...                'ssl_verify_server': 'cert',\n...                'ssl_ca_certificate_file': '/etc/irods/ssl/irods.crt'\n... }\n```\n\nIf necessary, a user may provide a custom SSLContext object; although,\nas of release v1.1.6, this will rarely be required:\n\n```python\n\u003e\u003e\u003e ssl_settings ['ssl_context'] = ssl.create_default_context(purpose=ssl.Purpose.SERVER_AUTH, # ... other options\n... )\n```\n\nAt this point, we are ready to instantiate and use the session:\n\n```python\n\u003e\u003e\u003e with iRODSSession(host='irods-provider', port=1247, user='bob', password='1234', zone='tempZone', **ssl_settings) as session:\n...\t# workload\n```\n\nNote that the `irods_` prefix is unnecessary when providing\nthe `encryption_*` and `ssl_*` options\ndirectly to the constructor as keyword arguments, even though it is\nrequired when they are placed in the environment file.\n\nCreating a PAM or Native Authentication File\n--------------------------------------------\n\nThe following free functions may be used to create the authentication secrets files (called\n`.irodsA` per the convention of iRODS's iCommands):\n   - `irods.client_init.write_native_irodsA_file`\n   - `irods.client_init.write_pam_irodsA_file`\n\nThese functions can roughly be described as duplicating the \"authentication\" functionality of `iinit`,\nprovided that a valid `irods_environment.json` has already been created.\n\nEach of the above functions can take a cleartext password and write an appropriately encoded\nversion of it into an authentication file in the appropriate location.  That location is\n`~/.irods/.irodsA` unless the environment variable IRODS_AUTHENTICATION_FILE has been set\nin the command shell to dictate an alternative file path.\n\nAs an example, here we write a native `.irodsA` file using the first of the two functions.  We\nprovide the one required argument, a password string which is entered interactively at the\nterminal.\n\n```bash\n$ echo '{ \"irods_user_name\":\"rods\",\n          ... # other parameters as needed\n        }'\u003e ~/.irods/irods_environment.json\n$ python -c \"import irods.client_init, getpass\nirods.client_init.write_native_irodsA_file(getpass.getpass('Enter iRODS password -\u003e '))\"\n```\n\nBy default, when an `.irodsA` file already exists, it will be overwritten. If however the\n`overwrite` parameter is set to `False`, an exception of type `irods.client_init.irodsA_already_exists`\nis raised to warn of any older `.irodsA` file that might otherwise have been overwritten.\n\nEquivalently to the above, we can issue the following command.\n\n```bash\n$ prc_write_irodsA.py native \u003c\u003c\u003c\"${MY_CURRENT_IRODS_PASSWORD}\"\n```\n\nThe redirect may of course be left off, in which case the user is prompted for the iRODS password\nand echo of the keyboard input will be suppressed, in the style of `iinit`.  Regardless of\nwhich technique is used, no password will be visible on the terminal during or after input.\n\nFor the `pam_password` scheme, typically SSL/TLS must first be enabled to avoid sending data related\nto the password - or even sending the raw password itself - over a network connection in the clear.\n\nThus, for `pam_password` authentication to work well, we should first ensure, when setting up the\nclient environment, to include within `irods_environment.json` the appropriate SSL/TLS connection\nparameters.  In a pinch, `iinit` can be used to verify this prerequisite is fulfilled,\nas its invocation would then create a valid `.irodsA` from merely prompting the user for their PAM password.\n\nOnce again, this can also be done using the free function directly:\n\n```python\nirods.client_init.write_pam_irodsA_file(getpass.getpass('Enter current PAM password -\u003e '))\n```\n\nor from the Bash command shell:\n\n```bash\n$ prc_write_irodsA.py pam_password \u003c\u003c\u003c\"${MY_CURRENT_PAM_PASSWORD}\"\n```\n\nAs a final note, in the `pam_password` scheme, the default SSL requirement can be disabled.\n**Warning:** Disabling the SSL requirement may cause user passwords to be sent over the network\nin the clear. This should only be done for purposes of testing. Here's how to do it:\n\n```python\nfrom irods.auth.pam_password import ENSURE_SSL_IS_ACTIVE\n\nsession = irods.session.iRODSSession(host = \"localhost\", port = 1247,\n                                     user = \"alice\", password = \"test123\", zone=\"tempZone\",\n                                     authentication_scheme = \"pam_password\")\n\nsession.set_auth_option_for_scheme('pam_password', ENSURE_SSL_IS_ACTIVE, False)\n\n# Do something with the session:\nhome = session.collections.get('/tempZone/home/alice')\n```\n\nNote, however, in future releases of iRODS it is possible that extra SSL checking could be\nimplemented server-side, at which point the above code could not be guaranteed to work.\n\nLegacy (iRODS 4.2-compatible) PAM authentication\n------------------------------------------------\n\nSince v2.0.0, the Python iRODS Client is able to authenticate via PAM using the same file-based client environment as the\niCommands.\n\nCaveat for iRODS 4.3+: when upgrading from 4.2, the \"irods_authentication_scheme\" setting must be changed from \"pam\" to \"pam_password\" in\n`~/.irods/irods_environment.json` for all file-based client environments.\n\nTo use the PRC PAM login credentials update function for the client login environment, we can set these two configuration variables:\n\n```\nlegacy_auth.pam.password_for_auto_renew \"my_pam_password\"\nlegacy_auth.pam.store_password_to_environment True\n```\n\nOptionally, the `legacy_auth.pam.time_to_live_in_hours` may also be set to determine the time-to-live for the new password.\nLeaving it at the default value defers this decision to the server.\n\nMaintaining a connection\n------------------------\n\nThe default library timeout for a connection to an iRODS Server is 120 seconds.\n\nThis can be overridden by changing the session `connection_timeout` immediately after creation of the\nsession object:\n\n```python\n\u003e\u003e\u003e session.connection_timeout = 300\n```\n\nThis will set the timeout to five minutes for any associated connections.\n\nTimeouts are either a positive `int` or `float` with units of seconds, or `None`, all in accordance with their\nmeaning in the context of the socket module.  A value of `None` indicates timeouts are effectively\ninfinite in value, i.e. turned off.  Setting a session's `connection_timeout` value to 0 is disallowed\nbecause this would cause the socket to enter non-blocking mode.\n\nSession objects and cleanup\n---------------------------\n\nWhen iRODSSession objects are kept as state in an application, spurious\nSYS_HEADER_READ_LEN_ERR errors can sometimes be seen in the\nconnected iRODS server's log file. This is frequently seen at program\nexit because socket connections are terminated without having been\nclosed out by the session object's cleanup() method.\n\nSince v0.9.0, code has been included in the session\nobject's `__del__` method to call cleanup(), properly closing out\nnetwork connections. However, `__del__`  is not guaranteed to run as\nexpected, so an alternative may be to call `session.cleanup()`\non any session variable which will not be used again.\n\nSimple PUTs and GETs\n--------------------\n\nWe can use the just-created session object to put files to (or get them\nfrom) iRODS.\n\n```python\n\u003e\u003e\u003e logical_path = \"/{0.zone}/home/{0.username}/{1}\".format(session,\"myfile.dat\")\n\u003e\u003e\u003e session.data_objects.put(\"myfile.dat\", logical_path)\n\u003e\u003e\u003e session.data_objects.get(logical_path, \"/tmp/myfile.dat.copy\")\n```\n\nNote that local file paths may be relative, but iRODS data objects must\nalways be referred to by their absolute paths. This is in contrast to\nthe `iput` and `iget` icommands, which keep track of the current working\ncollection (as modified by `icd`) for the unix shell.\n\nNote also that PRC `put()` is actually using the `open`, `write`, and `close` APIs, rather than the\niRODS PUT API directly.  This is transparent to the caller, but an administrator\nshould take note as this affects which policy enforcement points (PEPs) are executed\non the iRODS server.\n\nParallel Transfer\n-----------------\n\nSince v0.9.0, data object transfers using `put()` and `get()`\nwill spawn a number of threads in order to optimize performance for\niRODS server versions 4.2.9+ and file sizes larger than a default\nthreshold value of 32 Megabytes.\n\nProgress bars\n-------------\n\nThe PRC supports progress bars which function on the basis of\nan \"update\" callback function.  In the case of a tqdm progress bar (see https://github.com/tqdm/tqdm), you can always just\npass the update method of the progress bar instance directly to the data_object\n`put` or `get` method:\n\n```python\n   pbar = tqdm.tqdm(total = data_obj.size)\n   session.data_objects.get(file_name, data_obj.path, updatables = pbar.update)\n```\n\nThe updatables parameter can be a list or tuple of update-enabling objects and/or bound methods.\n\nAlternatively, the tqdm progress bar object itself can be passed in, if an adapting\nfunction such as the following is first registered:\n\n```python\n    def adapt_tqdm(pbar, l = threading.Lock()):\n        def _update(n):\n            with l:\n                pbar.update(n)\n        return _update\n    irods.manager.data_objects_manager.register_update_type( adapt_tqdm )\n    session.data_objects.put( file, logical_path, updatables = [tqdm_1,tqdm_2] ) # update two tqdm's simultaneously\n```\n\nOther progress bars may be included in an updatables parameter, but may require more extensive adaptation.\nFor example, the ProgressBar object (from the progressbar module) also has an update method, but it\ntakes an up-to-date cumulative byte-count, instead of the size of an individual transfer in bytes,\nas its sole parameter.  There can be other complications:  e.g. a ProgressBar instance does not allow a weak\nreference to itself to be formed, which interferes with the Python iRODS Client's internal scheme of accounting\nfor progress bar instances \"still in progress\" while also preventing resource leaks.\n\nIn such cases, it is probably best to implement a wrapper for the progress\nbar in question, and submit the wrapper instance as the updatable parameter.  Whether\na wrapper or the progress bar object itself is thus employed, it is recommended that the user take steps to\nensure the lifetime of the updatable instance extends beyond the time needed for the transfer to complete.\n\nSee `irods/test/data_obj_test.py` for examples of these and other subtleties of progress bar usage.\n\nWorking with collections (directories)\n--------------------------------------\n\n```python\n\u003e\u003e\u003e coll = session.collections.get(\"/tempZone/home/rods\")\n\n\u003e\u003e\u003e coll.id\n45798\n\n\u003e\u003e\u003e coll.path\n/tempZone/home/rods\n\n\u003e\u003e\u003e for col in coll.subcollections:\n\u003e\u003e\u003e   print(col)\n\u003ciRODSCollection /tempZone/home/rods/subcol1\u003e\n\u003ciRODSCollection /tempZone/home/rods/subcol2\u003e\n\n\u003e\u003e\u003e for obj in coll.data_objects:\n\u003e\u003e\u003e   print(obj)\n\u003ciRODSDataObject /tempZone/home/rods/file.txt\u003e\n\u003ciRODSDataObject /tempZone/home/rods/file2.txt\u003e\n```\n\nCreate a new collection:\n\n```python\n\u003e\u003e\u003e coll = session.collections.create(\"/tempZone/home/rods/testdir\")\n\u003e\u003e\u003e coll.id\n45799\n```\n\nWorking with data objects (files)\n---------------------------------\n\nCreate a new data object:\n\n```python\n\u003e\u003e\u003e obj = session.data_objects.create(\"/tempZone/home/rods/test1\")\n\u003ciRODSDataObject /tempZone/home/rods/test1\u003e\n```\n\nGet an existing data object:\n\n```python\n\u003e\u003e\u003e obj = session.data_objects.get(\"/tempZone/home/rods/test1\")\n\u003e\u003e\u003e obj.id 12345\n\n\u003e\u003e\u003e obj.name\ntest1\n\u003e\u003e\u003e obj.collection\n\u003ciRODSCollection /tempZone/home/rods\u003e\n\n\u003e\u003e\u003e for replica in obj.replicas:\n...     print(replica.resource_name)\n...     print(replica.number)\n...     print(replica.path)\n...     print(replica.status)\n...\ndemoResc\n0\n/var/lib/irods/Vault/home/rods/test1\n1\n```\n\nUsing the `put()` method rather than the `create()` method will trigger different policy enforcement points (PEPs) on the server.\n\nPut an existing file as a new data object:\n\n```python\n\u003e\u003e\u003e session.data_objects.put(\"test.txt\", \"/tempZone/home/rods/test2\")\n\u003e\u003e\u003e obj2 = session.data_objects.get(\"/tempZone/home/rods/test2\")\n\u003e\u003e\u003e obj2.id\n56789\n```\n\nSpecifying paths\n----------------\n\nPath strings for collection and data objects are usually expected to be\nabsolute in most contexts in the PRC. They must also be normalized to a\nform including single slashes separating path elements and no slashes at\nthe string's end. If there is any doubt that a path string fulfills\nthis requirement, the wrapper class `irods.path.iRODSPath` (a subclass of `str`) may be used to normalize it:\n\n    if not session.collections.exists( iRODSPath( potentially_unnormalized_path )): #....\n\nThe wrapper serves also as a path joiner; thus:\n\n    iRODSPath( zone, \"home\", user )\n\nmay replace:\n\n    \"/\".join([\"\", zone, \"home\", user])\n\n`iRODSPath` has been available since v1.1.2.\n\nReading and writing files\n-------------------------\n\nPRC provides [file-like\nobjects](https://docs.python.org/3/glossary.html#term-file-object) for reading and writing files.\n\n```python\n\u003e\u003e\u003e obj = session.data_objects.get(\"/tempZone/home/rods/test1\")\n\u003e\u003e\u003e with obj.open('r+') as f:\n...   f.write('foonbarn')\n...   f.seek(0,0)\n...   for line in f:\n...      print(line)\n...\nfoo\nbar\n```\n\nSince v1.1.9, there is also an auto-close configuration setting for data\nobjects, set to `False` by default, which may be assigned\nthe value `True` for guaranteed auto-closing of open data\nobject handles at the proper time.\n\nIn a small but illustrative example, the following Python session does\nnot require an explicit call to `f.close()`:\n\n```python\n\u003e\u003e\u003e import irods.client_configuration as config, irods.test.helpers as helpers\n\u003e\u003e\u003e config.data_objects.auto_close = True\n\u003e\u003e\u003e session = helpers.make_session()\n\u003e\u003e\u003e f = session.data_objects.open('/{0.zone}/home/{0.username}/new_object.txt'.format(session),'w')\n\u003e\u003e\u003e f.write(b'new content.')\n```\n\nThis may be useful for Python programs in which frequent flushing of\nwrite updates to data objects is undesirable -- with descriptors on such\nobjects possibly being held open for indeterminately long lifetimes --\nyet the eventual application of those updates prior to the teardown of\nthe Python interpreter is required.\n\nThe current value of the setting is global in scope (i.e. applies to all\nsessions, whenever created) and is always consulted for the creation of\nany data object handle to govern that handle's cleanup behavior.\n\nAlso, alternatively, the client may opt into a special \"redirect\" behavior\nin which data objects' `open()` method makes a new connection directly to whichever\niRODS server is found to host the selected replica.  Data reads and\nwrites will therefore happen on that alternate network route, instead of\nthrough the originally-connected server.  Though not the client's default\nbehavior, this approach can turn out to be more efficient, particularly\nif several concurrent data uploads (\"puts\") and downloads (\"gets\") are \nhappening which might increase traffic on the client's main communication\nroute with the server.  (See, in [Python iRODS Client Settings File](#python-irods-client-settings-file),\nthe client configuration setting `data_objects.allow_redirect`, which may be\nset to True to designate the opt-in.)\n\nPython iRODS Client Settings File\n---------------------------------\n\nSince v1.1.9, Python iRODS client configuration can be saved in, and\nloaded from, a settings file.\n\nIf the settings file exists, each of its lines contains (a) a dotted\nname identifying a particular configuration setting to be assigned\nwithin the PRC, potentially changing its runtime behavior; and (b) the\nspecific value, in Python \"repr\"-style format, that should be assigned\ninto it.\n\nAn example follows:\n\n```\ndata_objects.auto_close True\n```\n\nNew dotted names may be created following the example of the one valid\nexample created thus far, `data_objects.auto_close]`,\ninitialized in `irods/client_configuration/__init__.py`.\nEach such name should correspond to a globally set value which the PRC\nroutinely checks when performing the affected library function.\n\nThe use of a settings file can be indicated, and the path to that file\ndetermined, by setting the environment variable:\n`PYTHON_IRODSCLIENT_CONFIGURATION_PATH`. If this variable\nis present but empty, this denotes use of a default settings file path\nof `~/.python-irodsclient`; if the variable's value is of\nnon-zero length, the value should be an absolute path to the desired settings\nfile location. Also, if the variable is set, auto-load of\nsettings will be performed, meaning that the act of importing\n`irods` or any of its submodules will cause the automatic\nloading of the settings from the settings file, assuming it exists.\n(Failure to find the file at the indicated path will be logged as a\nwarning.)\n\nSettings can also be saved and loaded manually using the `save()` and\n`load()` functions in the `irods.client_configuration`\nmodule. Each of these functions accepts an optional `file`\nparameter which, if set to a non-empty string, will override the\nsettings file path currently \"in force\" (i.e., the\nCONFIG_DEFAULT_PATH, as optionally overridden by the environment\nvariable PYTHON_IRODSCLIENT_CONFIGURATION_PATH).\n\nConfiguration settings may also be individually overridden by defining\ncertain environment variables:\n\n-   Setting: Auto-close option for all data objects.\n    -   Dotted Name: `data_objects.auto_close`\n    -   Type: `bool`\n    -   Default Value: `False`\n    -   Environment Variable Override: `PYTHON_IRODSCLIENT_CONFIG__DATA_OBJECTS__AUTO_CLOSE`\n\n-   Setting: Let a call to data object open() redirect to the server whose storage resource hosts the given object's preferred replica.\n    -   Dotted Name: `data_objects.allow_redirect`\n    -   Type: `bool`\n    -   Default Value: `False`\n    -   Environment Variable Override: `PYTHON_IRODSCLIENT_CONFIG__DATA_OBJECTS__ALLOW_REDIRECT`\n\n-   Setting: Number of hours to request for the new password entry's TTL (Time To Live) when auto-renewing PAM-authenticated sessions.\n    - Dotted Name: `legacy_auth.pam.time_to_live_in_hours`\n    - Type: `int`\n    - Default Value: `0` (Meaning: conform to server's default TTL value.)\n    - Environment Variable Override: `PYTHON_IRODSCLIENT_CONFIG__LEGACY_AUTH__PAM__TIME_TO_LIVE_IN_HOURS`\n\n-   Setting: Plaintext PAM password value, to be used when auto-renewing PAM-authenticated sessions because TTL has expired.\n    - Dotted Name: `legacy_auth.pam.password_for_auto_renew`\n    - Type: `str`\n    - Default Value: `\"\"` (Meaning: no password is set, and thus no automatic attempts will be made at auto-renewing PAM authentication.)\n    - Environment Variable Override: `PYTHON_IRODSCLIENT_CONFIG__LEGACY_AUTH__PAM__PASSWORD_FOR_AUTO_RENEW`.  (But note that use of the environment variable could pose a threat to password security.)\n\n-   Setting: Whether to write the (native encoded) new hashed password to the iRODS password file.  This step is only performed while auto-renewing PAM authenticated sessions.\n    - Dotted Name: `legacy_auth.pam.store_password_to_environment`\n    - Type: `bool`\n    - Default Value: `False`\n    - Environment Variable Override: `PYTHON_IRODSCLIENT_CONFIG__LEGACY_AUTH__PAM__STORE_PASSWORD_TO_ENVIRONMENT`\n\n-   Setting: Force the use of PAM_AUTH_REQUEST_AN API for entering a new PAM password into the catalog.  This API accommodates longer passwords and avoids the step of parsing a semicolon-delimited\n    \"context\" parameter.\n    - Dotted Name: `legacy_auth.pam.force_use_of_dedicated_pam_api`\n    - Type: `bool`\n    - Default Value: `False`\n    - Environment Variable Override: `PYTHON_IRODSCLIENT_CONFIG__LEGACY_AUTH__PAM__FORCE_USE_OF_DEDICATED_PAM_API`\n\n-   Setting: Default choice of XML parser for all new threads.\n    -   Dotted Name: `connections.xml_parser_default`\n    -   Type: `str`\n    -   Default Value: `\"STANDARD_XML\"`\n    -   Possible Values: Any of `[\"STANDARD_XML\", \"QUASI_XML\", \"SECURE_XML\"]`\n    -   Environment Variable Override: `PYTHON_IRODSCLIENT_CONFIG__CONNECTIONS__XML_PARSER_DEFAULT`\n\nFor example, if `~/.python_irodsclient` contains the line :\n\n```\nconnections.xml_parser_default        \"QUASI_XML\"\n```\n\nthen the session below illustrates the effect of defining the\nappropriate environment variable. Note the value stored in the variable\nmust be a valid input for `ast.literal_eval()`; that is, a\nprimitive Pythonic value - and quoted, for instance, if a string.\n\n```bash\n$ PYTHON_IRODSCLIENT_CONFIGURATION_PATH=\"\" \\\n  PYTHON_IRODSCLIENT_CONFIG__CONNECTIONS__XML_PARSER_DEFAULT=\"'SECURE_XML'\" \\\n  python -c \"import irods.message, irods.client_configuration as c; print (irods.message.default_XML_parser())\"\nXML_Parser_Type.SECURE_XML\n$ PYTHON_IRODSCLIENT_CONFIGURATION_PATH=\"\" \\\n  python -c \"import irods.message, irods.client_configuration as c; print (irods.message.default_XML_parser())\"\nXML_Parser_Type.QUASI_XML\n```\n\nComputing and Retrieving Checksums\n----------------------------------\n\nEach data object may be associated with a checksum by calling `chksum()`\non the object in question. Various behaviors can be elicited by passing\nin combinations of keywords (for a description of which, please consult\nthe [header documentation](https://github.com/irods/irods/blob/4-3-stable/lib/api/include/irods/dataObjChksum.h).)\n\nAs with most other iRODS APIs, it is straightforward to specify keywords\nby adding them to an options dictionary:\n\n```python\n\u003e\u003e\u003e data_object_1.chksum() # - computes the checksum if already in the catalog, otherwise computes and stores it\n...                        # (i.e. default behavior with no keywords passed in.)\n\u003e\u003e\u003e from irods.manager.data_object_manager import Server_Checksum_Warning\n\u003e\u003e\u003e import irods.keywords as kw\n\u003e\u003e\u003e opts = { kw.VERIFY_CHKSUM_KW:'' }\n\u003e\u003e\u003e try:\n...     data_object_2.chksum( **opts ) # - Uses verification option. (Does not create or save a checksum in the catalog).\n...     # or:\n...     opts[ kw.NO_COMPUTE_KW ] = ''\n...     data_object_2.chksum( **opts ) # - Uses both verification and no-compute options. (Like `ichksum -K --no-compute`)\n... except Server_Checksum_Warning:\n...     print('some checksums are missing or wrong')\n```\n\nAdditionally, if a freshly created `irods.message.RErrorStack` instance is\ngiven, information can be returned and read by the client:\n\n```python\n\u003e\u003e\u003e from irods.message import RErrorStack\n\u003e\u003e\u003e r_err_stk = RErrorStack()\n\u003e\u003e\u003e warn = None\n\u003e\u003e\u003e try:   # Here, data_obj has one replica, not yet checksummed.\n...     data_obj.chksum( r_error = r_err_stk , **{kw.VERIFY_CHKSUM_KW:''} )\n... except Server_Checksum_Warning as exc:\n...     warn = exc\n\u003e\u003e\u003e print(r_err_stk)\n[RError\u003cmessage = u'WARNING: No checksum available for replica [0].', status = -862000 CAT_NO_CHECKSUM_FOR_REPLICA\u003e]\n```\n\nWorking with metadata\n---------------------\n\nShowing the Attribute-Value-Units (AVUs) on an object with no metadata attached shows an empty list:\n\n```python\n\u003e\u003e\u003e from irods.meta import iRODSMeta\n\u003e\u003e\u003e obj = session.data_objects.get(\"/tempZone/home/rods/test1\")\n\u003e\u003e\u003e print(obj.metadata.items())\n[]\n```\n\nAdding multiple AVUs with the same name field:\n\n```python\n\u003e\u003e\u003e obj.metadata.add('key1', 'value1', 'units1')\n\u003e\u003e\u003e obj.metadata.add('key1', 'value2')\n\u003e\u003e\u003e obj.metadata.add('key2', 'value3')\n\u003e\u003e\u003e obj.metadata.add('key2', 'value4')\n\u003e\u003e\u003e print(obj.metadata.items())\n[\u003ciRODSMeta 13182 key1 value1 units1\u003e, \u003ciRODSMeta 13185 key2 value4 None\u003e,\n\u003ciRODSMeta 13183 key1 value2 None\u003e, \u003ciRODSMeta 13184 key2 value3 None\u003e]\n```\n\nWe can also use Python's item indexing syntax to perform the equivalent\nof an \"imeta set \\...\", e.g. overwriting all AVUs with a name field\nof \"key2\" in a single update:\n\n```python\n\u003e\u003e\u003e new_meta = iRODSMeta('key2','value5','units2')\n\u003e\u003e\u003e obj.metadata\\[new_meta.name\\] = new_meta\n\u003e\u003e\u003e print(obj.metadata.items())\n[\u003ciRODSMeta 13182 key1 value1 units1\u003e, \u003ciRODSMeta 13183 key1 value2 None\u003e,\n\u003ciRODSMeta 13186 key2 value5 units2\u003e]\n```\n\nWith only one AVU on the object with a name of \"key2\", *get_one*\nis assured of not throwing an exception:\n\n```python\n\u003e\u003e\u003e print(obj.metadata.get_one('key2'))\n\u003ciRODSMeta 13186 key2 value5 units2\u003e\n```\n\nHowever, the same is not true of \"key1\":\n\n```python\n\u003e\u003e\u003e print(obj.metadata.get_one('key1'))\nTraceback (most recent call last):\n  File \"\u003cstdin\u003e\", line 1, in \u003cmodule\u003e\n  File \"/[...]/python-irodsclient/irods/meta.py\", line 41, in get_one\n    raise KeyError\nKeyError\n```\n\nFinally, to remove a specific AVU from an object:\n\n```python\n\u003e\u003e\u003e obj.metadata.remove('key1', 'value1', 'units1')\n\u003e\u003e\u003e print(obj.metadata.items())\n[\u003ciRODSMeta 13186 key2 value5 units2\u003e, \u003ciRODSMeta 13183 key1 value2 None\u003e]\n```\n\nAlternately, this form of the `remove()` method can also be useful:\n\n```python\n\u003e\u003e\u003e for avu in obj.metadata.items():\n...    obj.metadata.remove(avu)\n\u003e\u003e\u003e print(obj.metadata.items())\n[]\n```\n\nIf we intended on deleting the data object anyway, we could have just\ndone this instead:\n\n```\n\u003e\u003e\u003e obj.unlink(force=True)\n```\n\nBut notice that the force option is important, since a data object in\nthe trash may still have AVUs attached.\n\nAt the end of a long session of AVU add/manipulate/delete operations,\none should make sure to delete all unused AVUs. We can in fact use any\n`*Meta` data model in the queries below, since unattached AVUs are\nnot aware of the (type of) catalog object they once annotated:\n\n```python\n\u003e\u003e\u003e from irods.models import (DataObjectMeta, ResourceMeta)\n\u003e\u003e\u003e len(list( session.query(ResourceMeta) ))\n4\n\u003e\u003e\u003e from irods.test.helpers import remove_unused_metadata\n\u003e\u003e\u003e remove_unused_metadata(session)\n\u003e\u003e\u003e len(list( session.query(ResourceMeta) ))\n0\n```\n\nWhen altering a fetched iRODSMeta, we must copy it first to avoid\nerrors, due to the fact the reference is cached by the iRODS object\nreference. A shallow copy is sufficient:\n\n```python\n\u003e\u003e\u003e meta = album.metadata.items()[0]\n\u003e\u003e\u003e meta.units\n'quid'\n\u003e\u003e\u003e import copy; meta = copy.copy(meta); meta.units = 'pounds sterling'\n\u003e\u003e\u003e album.metadata[ meta.name ] = meta\n```\n\nSince v1.1.4, `set()` can be used instead:\n\n```python\n\u003e\u003e\u003e album.metadata.set( meta )\n```\n\nIn versions of iRODS 4.2.12 and later, we can also do:\n\n```python\n\u003e\u003e\u003e album.metadata.set( meta, \\*\\*{kw.ADMIN_KW: ''} )\n```\n\nor even:\n\n```python\n\u003e\u003e\u003e album.metadata(admin = True)\\[meta.name\\] = meta\n```\n\nSince v1.1.5, the \"timestamps\" keyword is provided to enable the loading\nof create and modify timestamps for every AVU returned from the server:\n\n```python\n\u003e\u003e\u003e avus = album.metadata(timestamps = True).items()\n\u003e\u003e\u003e avus[0].create_time\ndatetime.datetime(2022, 9, 19, 15, 26, 7)\n```\n\nAtomic operations on metadata\n-----------------------------\n\nSince iRODS 4.2.8, the atomic metadata API\nallows a group of metadata add and remove operations to be performed\ntransactionally, within a single call to the server. This capability is available\nsince PRC v0.8.6.\n\nFor example, if 'obj' is a handle to an object in the iRODS\ncatalog (whether a data object, collection, user, or storage resource),\nwe can send an arbitrary number of AVUOperation instances to be executed\ntogether as one indivisible operation on that object:\n\n```python\n\u003e\u003e\u003e from irods.meta import iRODSMeta, AVUOperation\n\u003e\u003e\u003e obj.metadata.apply_atomic_operations( AVUOperation(operation='remove', avu=iRODSMeta('a1','v1','these_units')),\n...                                       AVUOperation(operation='add', avu=iRODSMeta('a2','v2','those_units')),\n...                                       AVUOperation(operation='remove', avu=iRODSMeta('a3','v3')) \\# , ...\n... )\n```\n\nThe list of operations are applied in the order given, so that a\n\"remove\" followed by an \"add\" of the same AVU is, in effect, a\nmetadata \"set\" operation. Also note that a \"remove\" operation will\nbe ignored if the AVU value given does not exist on the target object at\nthat point in the sequence of operations.\n\nWe can also source from a pre-built list of AVUOperations using\nPython's `f(*args_list)` syntax. For example, this\nfunction uses the atomic metadata API to very quickly remove all AVUs\nfrom an object:\n\n```python\n\u003e\u003e\u003e def remove_all_avus( Object ):\n...     avus_on_Object = Object.metadata.items()\n...     Object.metadata.apply_atomic_operations( *[AVUOperation(operation='remove', avu=i) for i in avus_on_Object] )\n```\n\nExtracting JSON encoded server information in case of error\n-----------------------------------------------------------\n\nSome server APIs, including atomic metadata and replica truncation, can fail for various reasons and generate an\nexception.  In these cases the message object returned from the server is made available in the 'server_msg' attribute\nof the iRODSException object.\n\nThis enables an approach like the following, which logs server information possibly underlying the error that was evoked:\n\n```python\n    try:\n        Object.metadata.apply_atomic_operations( ops )\n        # or:\n        DataObject.replica_truncate( size )\n    except iRODSException as exc:\n        log.error('Server API call failure. Traceback = %r ; iRODS Server info = %r',\n            traceback.extract_tb(sys.exc_info()[2]),\n            exc.server_msg.get_json_encoded_struct())\n```\n\nFor `DataObject.replica_truncate(...)`, note that `exc.server_msg.get_json_encoded_struct()` can be used in the exception-handling\ncode path to retrieve the same information that would have been routinely returned from the truncate call itself, had it actually\ncompleted without error.\n\nSpecial Characters\n------------------\n\niRODS supports Unicode characters into collection and\ndata object names. However, certain non-printable ASCII characters, in addition to\nthe backquote character, have historically presented problems\n- especially for clients using the iRODS human readable XML protocol.\nConsider this small, contrived application:\n\n```python\n    from irods.test.helpers import make_session\n\n    def create_notes( session, obj_name, content = u'' ):\n        get_home_coll = lambda ses: \"/{0.zone}/home/{0.username}\".format(ses)\n        path = get_home_coll(session) + \"/\" + obj_name\n        with session.data_objects.open(path,\"a\") as f:\n            f.seek(0, 2) # SEEK_END\n            f.write(content.encode('utf8'))\n        return session.data_objects.get(path)\n\n    with make_session() as session:\n\n        # Example 1 : exception thrown when name has non-printable character\n        try:\n            create_notes( session, \"lucky\\033.dat\", content = u'test' )\n        except:\n            pass\n\n        # Example 2 (Ref. issue: irods/irods #4132, fixed for 4.2.9 release of iRODS)\n        print(\n            create_notes( session, \"Alice's diary\").name  # note diff (' != ') in printed name\n        )\n```\n\nThis creates two data objects, but with less than optimal success. The\nfirst example object is created but receives no content because an\nexception is thrown trying to query its name after creation. In the\nsecond example, for iRODS 4.2.8 and before, a deficiency in packStruct\nXML protocol causes the backtick to be read back as an apostrophe, which\ncould create problems manipulating or deleting the object later.\n\nSince v1.1.0, both problems can be mitigated by switching in the\nQUASI_XML parser for the default one:\n\n```\n    from irods.message import (XML_Parser_Type, ET)\n    ET( XML_Parser_Type.QUASI_XML,\n        server_version = session.server_version\n    )\n```\n\nThe server_version parameter can be used independently to change the\ncurrent thread's choice of entities during QUASI_XML transactions with the server.\n(This is only a concern when interacting with servers before iRODS 4.2.9.)\n\n```\n    ET(server_version = (4,2,8))\n```\n\nTwo dedicated environment variables may also be used to customize the\nPython client's XML parsing behavior via the setting of global defaults\nduring start-up.\n\nFor example, we can set the default parser to QUASI_XML, optimized for\nuse with version 4.2.8 of the iRODS server, in the following manner:\n\n```\n    Bash-Shell\u003e export PYTHON_IRODSCLIENT_DEFAULT_XML=QUASI_XML PYTHON_IRODSCLIENT_QUASI_XML_SERVER_VERSION=4,2,8\n```\n\nOther alternatives for PYTHON_IRODSCLIENT_DEFAULT_XML are\n\"STANDARD_XML\" and \"SECURE_XML\". These two latter options denote\nuse of the xml.etree and defusedxml modules, respectively.\n\nOnly the choice of \"QUASI_XML\" is affected by the specification of a\nparticular server version.\n\nThese global defaults, once set, may be overridden on\na per-thread basis using `ET(parser_type, server_version)`.\n\nThe current thread's XML parser can always be reverted to the global default by the\nexplicit use of `ET(None)`.  However, when frequently switching back and forth between\nparsers, it may be more convenient to use the `xml_mode` context manager:\n\n```\n    # ... Interactions with the server now use the default XML parser.\n\n    from irods.helpers import xml_mode\n    with xml_mode('QUASI_XML'):\n        # ... Interactions with the server, in the current thread, temporarily use QUASI_XML\n\n    # ... We have now returned to using the default XML parser.\n```\n\nApplication Cleanup\n-------------------\n\nUsing the `irods.at_client_exit` module, we may register user-defined functions to be executed at or around the\ntime when the Python iRODS Client is engaged in object teardown (also called \"cleanup\") operations.\nThis is analogous to Python's [atexit module](https://docs.python.org/3/library/atexit.html#module-atexit),\nexcept that here we have the extra resolution to specify that a function or callable object be expressly before,\nor expressly after, aforementioned object teardown stage:\n\n```python\n    from irods import at_client_exit\n    at_client_exit.register_for_execution_after_prc_cleanup(lambda: print(\"PRC cleanup has completed.\"))\n    at_client_exit.register_for_execution_before_prc_cleanup(lambda: print(\"PRC cleanup is about to start.\"))\n```\n\nA function normally cannot be registered multiple times to run in the same stage, but we may overcome this limitation\n(and, optionally, arguments set for the invocation) by wrapping the same function into two different callable objects:\n\n```python\n    def print_exit_message(n):\n        print(f\"Called just after PRC cleanup - iteration {n}\")\n\n    for n_iter in (1,2):\n        at_client_exit.register_for_execution_after_prc_cleanup(\n            at_client_exit.unique_function_invocation(print_exit_message, tup_args = (n_iter,))\n            )\n```\n\nThe output of the above, upon script exit, will be:\n\n```\nCalled just after PRC cleanup - iteration 2\nCalled just after PRC cleanup - iteration 1\n```\n\nwhich may be reversed from the order that one might expect.  This is because -- similarly as with Python atexit module, and\nconsistently with the teardown of higher abstractions before lower ones -- functions _registered_ later within a given cleanup\nstage will actually be _executed_ sooner (i.e. in \"LIFO\" order).\n\nRule Execution\n--------------\n\nThe following example shows how to execute an iRODS rule from the Python iRODS client.\n\nA rule file `native1.r` contains a rule in the native iRODS Rule Language:\n\n```\n    main() {\n        writeLine(\"*stream\",\n                  *X ++ \" squared is \" ++ str(double(*X)^2) )\n    }\n\n    INPUT *X=\"3\", *stream=\"serverLog\"\n    OUTPUT null\n```\n\nThe following Python client code will run the rule and produce the\nappropriate output in the iRODS server log:\n\n```\n    r = irods.rule.Rule( session, rule_file = 'native1.r')\n    r.execute()\n```\n\nSince v1.1.1, not only can we target a specific rule engine\ninstance by name (which is useful when more than one is present), but we\ncan also use a file-like object for the `rule_file`\nparameter:\n\n```\n    Rule( session, rule_file = io.StringIO(u'''mainRule() { anotherRule(*x); writeLine('stdout',*x) }\\n'''\n                                           u'''anotherRule(*OUT) {*OUT='hello world!'}\\n\\n'''\n                                           u'''OUTPUT ruleExecOut\\n'''),\n          instance_name = 'irods_rule_engine_plugin-irods_rule_language-instance' )\n```\n\nIf we wanted to change the `native1.r` rule\ncode print to `stdout`, we could set the `INPUT`\nparameter, `*stream`, using the Rule constructor's\n`params` keyword argument. Similarly, we can change the\n`OUTPUT` parameter from `null` to\n`ruleExecOut`, to accommodate the output stream, via the\n`output` argument:\n\n```\n    r = irods.rule.Rule( session, rule_file = 'native1.r',\n               instance_name = 'irods_rule_engine_plugin-irods_rule_language-instance',\n               params={'*stream':'\"stdout\"'} , output = 'ruleExecOut' )\n    output = r.execute( )\n    if output and len(output.MsParam_PI):\n        buf = output.MsParam_PI[0].inOutStruct.stdoutBuf.buf\n        if buf: print(buf.rstrip(b'\\0').decode('utf8'))\n```\n\nTo deal with errors resulting from rule execution failure, two\napproaches can be taken. Suppose we have defined this in the\n`/etc/irods/core.re` rule base:\n\n```\n    rule_that_fails_with_error_code(*x) {\n      *y = (if (*x!=\"\") then int(*x) else 0)\n    # if (SOME_PROCEDURE_GOES_WRONG) {\n        if (*y \u003c 0) { failmsg(*y,\"-- my error message --\"); }  #-\u003e throws an error code of int(*x) in REPF\n        else { fail(); }                                       #-\u003e throws FAIL_ACTION_ENCOUNTERED_ERR in REPF\n    # }\n    }\n```\n\nWe can run the rule thus:\n\n```python\n\u003e\u003e\u003e Rule( session, body='rule_that_fails_with_error_code(\"\"), instance_name = 'irods_rule_engine_plugin-irods_rule_language-instance',\n...     ).execute( r_error = (r_errs:= irods.message.RErrorStack()) )\n```\n\nWhere we've used the Python 3.8+ \"walrus operator\" for brevity. The\nerror will automatically be caught and translated to a returned-error\nstack:\n\n```python\n\u003e\u003e\u003e pprint.pprint([vars(r) for r in r_errs])\n[{'raw_msg_': 'DEBUG: fail action encountered\\n'\n              'line 14, col 15, rule base core\\n'\n              '        else { fail(); }\\n'\n              '               ^\\n'\n              '\\n',\n  'status_': -1220000}]\n```\n\nNote, if a stringized negative integer is given , i.e. as a special fail\ncode to be thrown within the rule, we must add this code into the `acceptable_errors`\nparameter to have this automatically caught as well:\n\n```python\n\u003e\u003e\u003e Rule( session, body='rule_that_fails_with_error_code(\"-2\")',instance_name = 'irods_rule_engine_plugin-irods_rule_language-instance'\n...     ).execute( acceptable_errors = ( FAIL_ACTION_ENCOUNTERED_ERR, -2),\n...                r_error = (r_errs := irods.message.RErrorStack()) )\n```\n\nBecause the rule is written to emit a custom error message via `failmsg()`,\nthe resulting r_error stack will now include that custom\nerror message as a substring:\n\n```python\n\u003e\u003e\u003e pprint.pprint([vars(r) for r in r_errs])\n[{'raw_msg_': 'DEBUG: -- my error message --\\n'\n              'line 21, col 20, rule base core\\n'\n              '      if (*y \u003c 0) { failmsg(*y,\"-- my error message --\"); }  '\n              '#-\u003e throws an error code of int(*x) in REPF\\n'\n              '                    ^\\n'\n              '\\n',\n  'status_': -1220000}]\n```\n\nAlternatively, or in combination with the automatic catching of errors,\nwe may also catch errors as exceptions on the client side. For example,\nif the Python rule engine is configured, and the following rule is\nplaced in `/etc/irods/core.py`:\n\n```python\ndef python_rule(rule_args, callback, rei):\n#   if some operation fails():\n        raise RuntimeError\n```\n\nwe can trap the error thus:\n\n```python\ntry:\n    Rule( session, body = 'python_rule', instance_name = 'irods_rule_engine_plugin-python-instance' ).execute()\nexcept irods.exception.RULE_ENGINE_ERROR:\n    print('Rule execution failed!')\n    exit(1)\nprint('Rule execution succeeded!')\n```\n\nAs fail actions from native rules are not thrown by default (refer to\nthe help text for `Rule.execute`), if we anticipate these\nand prefer to catch them as exceptions, we can do it this way:\n\n```python\ntry:\n    Rule( session, body = 'python_rule', instance_name = 'irods_rule_engine_plugin-python-instance'\n         ).execute( acceptable_errors = () )\nexcept (irods.exception.RULE_ENGINE_ERROR,\n        irods.exception.FAIL_ACTION_ENCOUNTERED_ERR) as e:\n    print('Rule execution failed!')\n    exit(1)\nprint('Rule execution succeeded!')\n```\n\nFinally, keep in mind that rule code submitted through an\n`irods.rule.Rule` object is processed by the\nexec_rule_text function in the targeted plugin instance in the server.\nThis may be a\nlimitation for plugins not equipped to handle rule code in this way. In\na sort of middle-ground case, the iRODS Python Rule Engine Plugin is not\ncurrently able to handle simple rule calls and the manipulation of iRODS\ncore primitives (like simple parameter passing and variable expansion')\nas flexibly as the iRODS Rule Language.\n\nAlso, core.py rules may only be run directly by a rodsadmin, currently.\n[See this issue for discussion](https://github.com/irods/irods_rule_engine_plugin_python/issues/105).\n\n\nGeneral Queries\n---------------\n\n```python\n\u003e\u003e\u003e import os\n\u003e\u003e\u003e from irods.session import iRODSSession\n\u003e\u003e\u003e from irods.models import Collection, DataObject\n\u003e\u003e\u003e\n\u003e\u003e\u003e env_file = os.path.expanduser('~/.irods/irods_environment.json')\n\u003e\u003e\u003e with iRODSSession(irods_env_file=env_file) as session:\n...     query = session.query(Collection.name, DataObject.id, DataObject.name, DataObject.size)\n...\n...     for result in query:\n...             print('{}/{} id={} size={}'.format(result[Collection.name], result[DataObject.name], result[DataObject.id], result[DataObject.size]))\n...\n/tempZone/home/rods/manager/access_manager.py id=212665 size=2164\n/tempZone/home/rods/manager/access_manager.pyc id=212668 size=2554\n/tempZone/home/rods/manager/collection_manager.py id=212663 size=4472\n/tempZone/home/rods/manager/collection_manager.pyc id=212664 size=4464\n/tempZone/home/rods/manager/data_object_manager.py id=212662 size=10291\n/tempZone/home/rods/manager/data_object_manager.pyc id=212667 size=8772\n/tempZone/home/rods/manager/__init__.py id=212670 size=79\n/tempZone/home/rods/manager/__init__.pyc id=212671 size=443\n/tempZone/home/rods/manager/metadata_manager.py id=212660 size=4263\n/tempZone/home/rods/manager/metadata_manager.pyc id=212659 size=4119\n/tempZone/home/rods/manager/resource_manager.py id=212666 size=5329\n/tempZone/home/rods/manager/resource_manager.pyc id=212661 size=4570\n/tempZone/home/rods/manager/user_manager.py id=212669 size=5509\n/tempZone/home/rods/manager/user_manager.pyc id=212658 size=5233\n```\n\nQuery using other models:\n\n```python\n\u003e\u003e\u003e from irods.column import Criterion\n\u003e\u003e\u003e from irods.models import DataObject, DataObjectMeta, Collection, CollectionMeta\n\u003e\u003e\u003e from irods.session import iRODSSession\n\u003e\u003e\u003e import os\n\u003e\u003e\u003e env_file = os.path.expanduser('~/.irods/irods_environment.json')\n\u003e\u003e\u003e with iRODSSession(irods_env_file=env_file) as session:\n...    # by metadata\n...    # equivalent to 'imeta qu -C type like Project'\n...    results = session.query(Collection, CollectionMeta).filter( \\\n...        Criterion('=', CollectionMeta.name, 'type')).filter( \\\n...        Criterion('like', CollectionMeta.value, '%Project%'))\n...    for r in results:\n...        print(r[Collection.name], r[CollectionMeta.name], r[CollectionMeta.value], r[CollectionMeta.units])\n...\n('/tempZone/home/rods', 'type', 'Project', None)\n```\n\nSince v0.8.3, the 'In()' operator is available:\n\n```python\n\u003e\u003e\u003e from irods.models import Resource\n\u003e\u003e\u003e from irods.column import In\n\u003e\u003e\u003e [ resc[Resource.id]for resc in session.query(Resource).filter(In(Resource.name, ['thisResc','thatResc'])) ]\n[10037,10038]\n```\n\nQuery with aggregation(min, max, sum, avg, count):\n\n```python\n\u003e\u003e\u003e with iRODSSession(irods_env_file=env_file) as session:\n...     query = session.query(DataObject.owner_name).count(DataObject.id).sum(DataObject.size)\n...     print(next(query.get_results()))\n{\u003cirods.column.Column 411 D_OWNER_NAME\u003e: 'rods', \u003cirods.column.Column 407 DATA_SIZE\u003e: 62262, \u003cirods.column.Column 401 D_DATA_ID\u003e: 14}\n```\n\nIn this case since we are expecting only one row we can directly call\n`query.execute()`:\n\n```python\n\u003e\u003e\u003e with iRODSSession(irods_env_file=env_file) as session:\n...     query = session.query(DataObject.owner_name).count(DataObject.id).sum(DataObject.size)\n...     print(query.execute())\n+--------------+-----------+-----------+\n| D_OWNER_NAME | D_DATA_ID | DATA_SIZE |\n+--------------+-----------+-----------+\n| rods         | 14        | 62262     |\n+--------------+-----------+-----------+\n```\n\nFor a case-insensitive query, add a `case_sensitive=False`\nparameter to the query:\n\n```python\n\u003e\u003e\u003e with iRODSSession(irods_env_file=env_file) as session:\n...     query = session.query(DataObject.name, case_sensitive=False).filter(Like(DataObject.name, \"%oBjEcT\"))\n...     print(query.all())\n+---------------------+\n| DATA_NAME           |\n+---------------------+\n| caseSENSITIVEobject |\n+---------------------+\n```\n\nSpecific Queries\n----------------\n\n```python\n\u003e\u003e\u003e import os\n\u003e\u003e\u003e from irods.session import iRODSSession\n\u003e\u003e\u003e from irods.models import Collection, DataObject\n\u003e\u003e\u003e from irods.query import SpecificQuery\n\u003e\u003e\u003e\n\u003e\u003e\u003e env_file = os.path.expanduser('~/.irods/irods_environment.json')\n\u003e\u003e\u003e with iRODSSession(irods_env_file=env_file) as session:\n...     # define our query\n...     sql = \"select data_name, data_id from r_data_main join r_coll_main using (coll_id) where coll_name = '/tempZone/home/rods/manager'\"\n...     alias = 'list_data_name_id'\n...     columns = [DataObject.name, DataObject.id] # optional, if we want to get results by key\n...     query = SpecificQuery(session, sql, alias, columns)\n...\n...     # register specific query in iCAT\n...     _ = query.register()\n...\n...     for result in query:\n...             print('{} {}'.format(result[DataObject.name], result[DataObject.id]))\n...\n...     # delete specific query\n...     _ = query.remove()\n...\nuser_manager.pyc 212658\nmetadata_manager.pyc 212659\nmetadata_manager.py 212660\nresource_manager.pyc 212661\ndata_object_manager.py 212662\ncollection_manager.py 212663\ncollection_manager.pyc 212664\naccess_manager.py 212665\nresource_manager.py 212666\ndata_object_manager.pyc 212667\naccess_manager.pyc 212668\nuser_manager.py 212669\n__init__.py 212670\n__init__.pyc 212671\n```\n\nRecherché Queries\n-----------------\n\nIn some cases you might like to use a GenQuery operator not directly\noffered by this Python library, or even combine query filters in ways\nGenQuery may not directly support.\n\nAs an example, the code below finds metadata value fields\nlexicographically outside the range of decimal integers, while also\nrequiring that the data objects to which they are attached do not reside\nin the trash.\n\n```python\n\u003e\u003e\u003e search_tuple = (DataObject.name , Collection.name ,\n...                 DataObjectMeta.name , DataObjectMeta.value)\n\n\u003e\u003e\u003e # \"not like\" : direct instantiation of Criterion (operator in literal string)\n\u003e\u003e\u003e not_in_trash = Criterion ('not like', Collection.name , '%/trash/%')\n\n\u003e\u003e\u003e # \"not between\"( column, X, Y) := column \u003c X OR column \u003e Y (\"OR\" done via chained iterators)\n\u003e\u003e\u003e res1 = session.query (* search_tuple).filter(not_in_trash).filter(DataObjectMeta.value \u003c '0')\n\u003e\u003e\u003e res2 = session.query (* search_tuple).filter(not_in_trash).filter(DataObjectMeta.value \u003e '9' * 9999 )\n\n\u003e\u003e\u003e chained_results = itertools.chain ( res1.get_results(), res2.get_results() )\n\u003e\u003e\u003e pprint( list( chained_results ) )\n```\n\nInstantiating iRODS objects from query results\n----------------------------------------------\n\nThe General query works well for getting information out of the ICAT if\nall we're interested in is information representable with primitive\ntypes (i.e. object names, paths, and ID's, as strings or integers). But\nPython's object orientation also allows us to create object references\nto mirror the persistent entities (instances of *Collection*,\n*DataObject*, *User*, or *Resource*, etc.) inhabiting the ICAT.\n\n**Background:**\n\nCertain iRODS object types can be instantiated easily\nusing the session object's custom type managers, particularly if some\nparameter (often just the name or path) of the object is already known:\n\n```python\n\u003e\u003e\u003e type(session.users)\n\u003cclass 'irods.manager.user_manager.UserManager'\u003e\n\u003e\u003e\u003e u = session.users.get('rods')\n\u003e\u003e\u003e u.id\n10003\n```\n\nType managers are good for specific operations, including object\ncreation and removal:\n\n```python\n\u003e\u003e\u003e session.collections.create('/tempZone/home/rods/subColln')\n\u003e\u003e\u003e session.collections.remove('/tempZone/home/rods/subColln')\n\u003e\u003e\u003e session.data_objects.create('/tempZone/home/rods/dataObj')\n\u003e\u003e\u003e session.data_objects.unlink('/tempZone/home/rods/dataObj')\n```\n\nWhen we retrieve a reference to an existing collection using *get* :\n\n```python\n\u003e\u003e\u003e c = session.collections.get('/tempZone/home/rods')\n\u003e\u003e\u003e c\n\u003ciRODSCollection 10011 rods\u003e\n```\n\nwe have, in that variable *c*, a reference to an iRODS *Collection*\nobject whose properties provide useful information:\n\n```python\n\u003e\u003e\u003e [ x for x in dir(c) if not x.startswith('__') ]\n['_meta', 'data_objects', 'id', 'manager', 'metadata', 'move', 'name', 'path', 'remove', 'subcollections', 'unregister', 'walk']\n\u003e\u003e\u003e c.name\n'rods'\n\u003e\u003e\u003e c.path\n'/tempZone/home/rods'\n\u003e\u003e\u003e c.data_objects\n[\u003ciRODSDataObject 10019 test1\u003e]\n\u003e\u003e\u003e c.metadata.items()\n[ \u003c... list of AVUs attached to Collection c ... \u003e ]\n```\n\nor whose methods can do useful things:\n\n```python\n\u003e\u003e\u003e for sub_coll in c.walk(): print('---'); pprint( sub_coll )\n[ ...\u003c series of Python data structures giving the complete tree structure below collection 'c'\u003e ...]\n```\n\nThis approach of finding objects by name, or via their relations with\nother objects (ie \"contained by\", or in the case of metadata,\n\"attached to\"), is helpful if we know something about the location or\nidentity of what we're searching for, but we don't always have that\nkind of a-priori knowledge.\n\nSo, although we can (as seen in the last example) walk an\n*iRODSCollection* recursively to discover all subordinate collections\nand their data objects, this approach will not always be best for a\ngiven type of application or data discovery, especially in more advanced\nuse cases.\n\n**A Different Approach:**\n\nFor the PRC to be sufficiently powerful for general use, we'll often need at least:\n\n-   general queries, and\n-   the capabilities afforded by the PRC's object-relational mapping.\n\nSuppose, for example, we wish to enumerate all collections in the iRODS\ncatalog.\n\nAgain, the object managers are the answer, but they are now invoked\nusing a different scheme:\n\n```python\n\u003e\u003e\u003e from irods.collection import iRODSCollection; from irods.models import Collection\n\u003e\u003e\u003e all_collns = [ iRODSCollection(session.collections, result) for result in session.query(Collection) ]\n```\n\nFrom there, we have the ability to do useful work, or filtering based on\nthe results of the enumeration. And, because *all_collns* is an\niterable of true objects, we can either use Python's list\ncomprehensions or execute more catalog queries to achieve further aims.\n\nNote that, for similar system-wide queries of Data Objects (which, as it\nhappens, are inextricably joined to their parent Collection objects), a\nbit more finesse is required. Let us query, for example, to find all\ndata objects in a particular zone with an AVU that matches the following\ncondition:\n\n```\n    META_DATA_ATTR_NAME = \"irods::alert_time\" and META_DATA_ATTR_VALUE like '+0%'\n```\n\n```python\n\u003e\u003e\u003e import irods.keywords\n\u003e\u003e\u003e from irods.data_object import iRODSDataObject\n\u003e\u003e\u003e from irods.models import DataObjectMeta, DataObject\n\u003e\u003e\u003e from irods.column import Like\n\u003e\u003e\u003e q = session.query(DataObject).filter( DataObjectMeta.name == 'irods::alert_time',\n                                          Like(DataObjectMeta.value, '+0%') )\n\u003e\u003e\u003e zone_hint = \"\" # --\u003e add a zone name in quotes to search another zone\n\u003e\u003e\u003e if zone_hint: q = q.add_keyword( irods.keywords.ZONE_KW, zone_hint )\n\u003e\u003e\u003e for res in q:\n...      colln_id = res [DataObject.collection_id]\n...      collObject = get_collection( colln_id, session, zone = zone_hint)\n...      dataObject = iRODSDataObject( session.data_objects, parent = collObject, results=[res])\n...      print( '{coll}/{data}'.format (coll = collObject.path, data = dataObject.name))\n```\n\nIn the above loop we have used a helper function, *get_collection*, to\nminimize the number of hits to the object catalog. Otherwise, me might\nfind within a typical application that some Collection objects are being\nqueried at a high rate of redundancy. *get_collection* can be\nimplemented thusly:\n\n```python\nimport collections  # of the Pythonic, not iRODS, kind\ndef makehash():\n    # see https://stackoverflow.com/questions/651794/whats-the-best-way-to-initialize-a-dict-of-dicts-in-python\n    return collections.defaultdict(makehash)\nfrom irods.collection import iRODSCollection\nfrom irods.models import Collection\ndef get_collection (Id, session, zone=None, memo = makehash()):\n    if not zone: zone = \"\"\n    c_obj = memo[session][zone].get(Id)\n    if c_obj is None:\n        q = session.query(Collection).filter(Collection.id==Id)\n        if zone != '': q = q.add_keyword( irods.keywords.ZONE_KW, zone )\n        c_id =  q.one()\n        c_obj = iRODSCollection(session, result = c_id)\n        memo[session][zone][Id] = c_obj\n    return c_obj\n```\n\nOnce instantiated, of course, any *iRODSDataObject*'s data to which we\nhave access permissions is available via its open() method.\n\nAs stated, this type of object discovery requires some extra study and\neffort, but the ability to search arbitrary iRODS zones (to which we are\nfederated and have the user permissions) is powerful indeed.\n\n\nGenQuery2 Queries\n-----------------\n\nGenQuery2 is a successor to the regular GenQuery interface. It is available\nby default on iRODS 4.3.2 and higher. GenQuery2 currently has an experimental status,\nand is subject to change.\n\nQueries can be executed using the `genquery2` function and passing a single querystring.  All parsing is done on the server.\n\nFor example:\n\n```\n\u003e\u003e\u003e session.genquery2(\"SELECT COLL_NAME WHERE COLL_NAME = '/tempZone/home' OR COLL_NAME LIKE '%/genquery2_dummy_doesnotexist'\")\n[['/tempZone/home']]\n```\n\nAlternatively, create a GenQuery2 object and use it to execute queries. For example:\n\n```\n\u003e\u003e\u003e q = session.genquery2_object()\n\u003e\u003e\u003e q.execute(\"SELECT COLL_NAME WHERE COLL_NAME = '/tempZone/home' OR COLL_NAME LIKE '%/genquery2_dummy_doesnotexist'\", zone=\"tempZone\")\n[['/tempZone/home']]\n```\n\nGenQuery2 objects also support retrieving only the SQL generated by a GenQuery2 query using the\n`get_sql` function and retrieving all available column mappings using the `get_column_mappings` function.\n\n\nTickets\n-------\n\nThe `irods.ticket.Ticket` class lets us issue \"tickets\"\nwhich grant limited permissions for other users to access our own data\nobjects (or collections of data objects). As with the iticket client,\nthe access may be either \"read\" or \"write\". The recipient of the\nticket could be a rodsuser, or even an anonymous user.\n\nBelow is a demonstration of how to generate a new ticket for access to a\nlogical path - in this case, say a collection containing 1 or more data\nobjects. (We assume the creation of the granting_session and\nreceiving_session for the users respectively for the users providing\nand consuming the ticket access.)\n\nThe user who wishes to provide an access may execute the following:\n\n```python\n\u003e\u003e\u003e from irods.ticket import Ticket\n\u003e\u003e\u003e new_ticket = Ticket (granting_session)\n\u003e\u003e\u003e The_Ticket_String = new_ticket.issue('read', \n...     '/zone/home/my/collection_with_data_objects_for/somebody').string\n```\n\nat which point that ticket's unique string may be given to other users,\nwho can then apply the ticket to any existing session object in order to\ngain access to the intended object(s):\n\n```python\n\u003e\u003e\u003e from irods.models import Collection, DataObject\n\u003e\u003e\u003e ses = receiving_session\n\u003e\u003e\u003e Ticket(ses, The_Ticket_String).supply()\n\u003e\u003e\u003e c_result = ses.query(Collection).one()\n\u003e\u003e\u003e c = iRODSCollection( ses.collections, c_result)\n\u003e\u003e\u003e for dobj in (c.data_objects):\n...     ses.data_objects.get( dobj.path, '/tmp/' + dobj.name ) # download objects\n```\n\nIn this case, however, modification will not be allowed because the\nticket is for read only:\n\n```python\n\u003e\u003e\u003e c.data_objects[0].open('w').write(  # raises\n...     b'new content')                 #  CAT_NO_ACCESS_PERMISSION\n```\n\nIn another example, we could generate a ticket that explicitly allows\n'write' access on a specific data object, thus granting other users\nthe permissions to modify as well as read it:\n\n```python\n\u003e\u003e\u003e ses = iRODSSession( user = 'anonymous', password = '', host = 'localhost',\n                        port = 1247, zone = 'tempZone')\n\u003e\u003e\u003e Ticket(ses, write_data_ticket_string ).supply()\n\u003e\u003e\u003e d_result = ses.query(DataObject.name,Collection.name).one()\n\u003e\u003e\u003e d_path = ( d_result[Collection.name] + '/' +\n...            d_result[DataObject.name] )\n\u003e\u003e\u003e old_content = ses.data_objects.open(d_path,'r').read()\n\u003e\u003e\u003e with tempfile.NamedTemporaryFile() as f:\n...     f.write(b'blah'); f.flush()\n...     ses.data_objects.put(f.name,d_path)\n```\n\nAs with iticket, we may set a time limit on the availability of a\nticket, either as a timestamp or in seconds since the epoch:\n\n```python\n\u003e\u003e\u003e t=Ticket(ses); s = t.string\nvIOQ6qzrWWPO9X7\n\u003e\u003e\u003e t.issue('read','/some/path')\n\u003e\u003e\u003e t.modify('expire','2021-04-01.12:34:56')  # timestamp assumed as UTC\n```\n\nTo check the results of the above, we could invoke this icommand\nelsewhere in a shell prompt:\n\n```\niticket ls vIOQ6qzrWWPO9X7\n```\n\nand the server should report back the same expiration timestamp.\n\nAnd, if we are the issuer of a ticket, we may also query, filter on, and\nextract information based on a ticket's attributes and catalog\nrelations:\n\n```python\n\u003e\u003e\u003e from irods.models import TicketQuery\n\u003e\u003e\u003e delay = lambda secs: int( time.time() + secs + 1)\n\u003e\u003e\u003e Ticket(ses).issue('read','/path/to/data_object').modify(\n                      'expire',delay(7*24*3600))             # lasts 1 week\n\u003e\u003e\u003e Q = ses.query (TicketQuery.Ticket, TicketQuery.DataObject).filter(\n...                                                            TicketQuery.DataObject.name == 'data_object')\n\u003e\u003e\u003e print ([ _[TicketQuery.Ticket.expiry_ts] for _ in Q ])\n['1636757427']\n```\n\nTracking and manipulating replicas of Data Objects\n--------------------------------------------------\n\nPutting together the techniques we've seen so far, it's not hard to write client code to accomplish\nuseful, common tasks.  Suppose, for instance, that a data object contains replicas on a given resource\nor resource hierarchy (the \"source\"), and we want those replicas \"moved\" to a second resource\n(the \"destination\").  This can be done by combining the replicate and trim operations, as in the following\ncode excerpt.\n\nWe'll assume, for our current purposes, that all pre-existing replicas are good (ie. they have a\n`status` attribute of `'1'`); and that the nodes in question are named `src` and `dest`,\nwith `src` being the root node of a resource hierarchy and `dest` just a simple storage node.\n\nThen we can accomplish the replica \"move\" thus:\n\n```python\n  path = '/path/to/data/object'\n  data = session.data_objects.get('/path/to/data/object')\n\n  # Replicate the data object to the destination.\n\n  data.replicate(**{kw.DEST_RESC_NAME_KW: 'dest'})\n\n  # Find and trim replicas on the source resource hierarchy.\n\n  replica_numbers = [r.number for r in d.replicas if r.resc_hier.startswith('src;')]\n  for number in replica_numbers:\n      session.data_objects.trim(path, **{kw.DATA_REPL_NUM:number, kw.COPIES_KW:1})\n```\n\nUsers and Groups\n----------------\n\niRODS tracks users and groups using two tables, R_USER_MAIN and\nR_USER_GROUP. Under this database schema, all groups are also users:\n\n```python\n\u003e\u003e\u003e from irods.models import User, Group\n\u003e\u003e\u003e from pprint import pprint\n\u003e\u003e\u003e pprint(list((x[User.id], x[User.name]) for x in session.query(User)))\n[(10048, 'alice'),\n (10001, 'rodsadmin'),\n (13187, 'bobby'),\n (10045, 'collab'),\n (10003, 'rods'),\n (13193, 'empty'),\n (10002, 'public')]\n```\n\nBut it's also worth noting that the User.type field will be\n'rodsgroup' for any user ID that iRODS internally recognizes as a\n\"Group\":\n\n```python\n\u003e\u003e\u003e groups = session.query(User).filter( User.type == 'rodsgroup' )\n\n\u003e\u003e\u003e [x[User.name] for x in groups]\n['collab', 'public', 'rodsadmin', 'empty']\n```\n\nSince we can instantiate iRODSGroup and iRODSUser objects directly from\nthe rows of a general query on the corresponding tables, it is also\nstraightforward to trace out the groups' memberships:\n\n```python\n\u003e\u003e\u003e from irods.user import iRODSUser, iRODSGroup\n\u003e\u003e\u003e grp_usr_mapping = [ (iRODSGroup(session.groups, result), iRODSUser(session.users, result)) \\\n...                     for result in session.query(Group,User) ]\n\u003e\u003e\u003e pprint( [ (x,y) for x,y in grp_usr_mapping if x.id != y.id ] )\n[(\u003ciRODSGroup 10045 collab\u003e, \u003ciRODSUser 10048 alice rodsuser tempZone\u003e),\n (\u003ciRODSGroup 10001 rodsadmin\u003e, \u003ciRODSUser 10003 rods rodsadmin tempZone\u003e),\n (\u003ciRODSGroup 10002 public\u003e, \u003ciRODSUser 10003 rods rodsadmin tempZone\u003e),\n (\u003ciRODSGroup 10002 public\u003e, \u003ciRODSUser 10048 alice rodsuser tempZone\u003e),\n (\u003ciRODSGroup 10045 collab\u003e, \u003ciRODSUser 13187 bobby rodsuser tempZone\u003e),\n (\u003ciRODSGroup 10002 public\u003e, \u003ciRODSUser 13187 bobby rodsuser tempZone\u003e)]\n```\n\n(Note that in general queries, fields cannot be compared to each other,\nonly to literal constants; thus the '!=' comparison in the Python list\ncomprehension.)\n\nFrom the above, we can see that the group 'collab' (with user ID\n10045) contains users 'bobby'(13187) and 'alice'(10048) but not\n'rods'(10003), as the tuple (10045,10003) is not listed. Group\n'rodsadmin'(10001) contains user 'rods'(10003) but no other users;\nand group 'public'(10002) by default contains all canonical users\n(those whose User.type is 'rodsadmin' or 'rodsuser'). The empty\ngroup ('empty') has no users as members, so it doesn't show up in our\nfinal list.\n\nGroup Administrator Capabilities\n--------------------------------\n\nSince v1.1.7, group administrator functions are available.\n\nA groupadmin may invoke methods to create groups, and may add\nusers to, or remove them from, any group to which they themselves\nbelong:\n\n```python\n\u003e\u003e\u003e session.groups.create('lab')\n\u003e\u003e\u003e session.groups.addmember('lab',session.username)  # allow self to administer group\n\u003e\u003e\u003e session.groups.addmember('lab','otheruser')\n\u003e\u003e\u003e session.groups.removemember('lab','otheruser')\n```\n\nA groupadmin may also create accounts for new users and\nenable their logins by initializing a native password for them:\n\n```python\n\u003e\u003e\u003e session.users.create_with_password('alice', 'change_me')\n```\n\niRODS Permissions (ACLs)\n------------------------\n\nThe `iRODSAccess` class offers a convenient dictionary\ninterface mapping iRODS permission strings to the corresponding integer\ncodes:\n\n```python\n\u003e\u003e\u003e from irods.access import iRODSAccess\n\u003e\u003e\u003e iRODSAccess.keys()\n['null', 'read_metadata', 'read_object', 'create_metadata', 'modify_metadata', 'delete_metadata', 'create_object', 'modify_object', 'delete_object', 'own']\n\u003e\u003e\u003e WRITE = iRODSAccess.to_int('modify_object')\n```\n\nArmed with that, we can then query on all data objects with ACLs that\nallow our user to write them:\n\n```python\n\u003e\u003e\u003e from irods.models import (DataObject, User, DataAccess)\n\u003e\u003e\u003e data_objects_writable = list(session.query(DataObject, User, DataAccess).filter(User.name == session.username,  DataAccess.type \u003e= WRITE))\n```\n\nFinally, we can also access the list of permissions available through a\ngiven session object via the `available_permissions`\nproperty. Note that (in keeping with changes in iRODS 4.3+)\nthe permissions list will be longer, as appropriate, for session objects\nconnected to the more recent servers; and also that the embedded spaces\nin some 4.2 permission strings are replaced by underscores in 4.3\nand later.\n\n```python\n\u003e\u003e\u003e session.server_version\n(4, 2, 11)\n\u003e\u003e\u003e session.available_permissions.items()\n[('null', 1000), ('read object', 1050), ('modify object', 1120), ('own', 1200)]\n```\n\nGetting and setting permissions\n-------------------------------\n\nWe can find the ID's of all the collections writable (i.e. having\na \"modify\" ACL) by, but not owned by, alice (or even alice\\#otherZone):\n\n```python\n\u003e\u003e\u003e from irods.models import Collection,CollectionAccess,CollectionUser,User\n\u003e\u003e\u003e from irods.column import Like\n\u003e\u003e\u003e q = session.query (Collection,CollectionAccess).filter(\n...                                 CollectionUser.name == 'alice',  # User.zone == 'otherZone', # zone optional\n...                                 Like(CollectionAccess.name, 'modify%') ) #defaults to current zone\n```\n\nIf we then want to downgrade those permissions to read-only, we can do\nthe following:\n\n```python\n\u003e\u003e\u003e from irods.access import iRODSAccess\n\u003e\u003e\u003e for c in q:\n...     session.acls.set( iRODSAccess('read', c[Collection.name], 'alice', # 'otherZone' # zone optional\n...     ))\n```\n\nA call to `session.acls.get(c)` -- with `c`\nbeing the result of\n`sessions.collections.get(c[Collection.name])` -- would\nthen verify the desired change had taken place (as well as list all ACLs\nstored in the catalog for that collection).\n\nThe older access manager,\n`\u003csession\u003e.permissions`, produced inconsistent results when\nthe `get()` method was invoked with the parameter\n`report_raw_acls` set (or defaulting) to\n`False`. Specifically, collections would exhibit the\n\"non-raw-ACL\" behavior of reporting individual member users'\npermissions as a by-product of group ACLs, whereas data objects would\nnot.\n\nSince v1.1.6, this inconsistency is corrected by\n`\u003csession\u003e.acls` which acts almost identically\nto `\u003csession\u003e.permissions`, except that the\n`\u003csession\u003e.acls.get(...)` method does not accept the\n`report_raw_acls` parameter. When we need to detect users'\npermissions independent of their access to an object via group\nmembership, this can be achieved with another query.\n\n`\u003csession\u003e.permissions` was therefore removed in v2.0.0\nin favor of `\u003csession\u003e.acls`.\n\nQuotas (v2.0.0)\n---------------\n\nQuotas may be set for a group:\n\n```python\nsession.groups.set_quota('my_group', 50000, resource = 'my_limited_resource')\n```\n\nor per user, prior to iRODS 4.3.0:\n\n```python\nsession.users.set_quota('alice', 100000)\n```\n\nThe default for the resource parameter is \"total\", denoting a general\nquota usage not bound to a particular resource.\n\nThe Quota model is also available for queries. So, to determine the\nspace remaining for a certain group on a given resource:\n\n```python\nfrom irods.models import Quota\nsession.groups.calculate_usage()\ngroup, resource = ['my_group', 'my_limited_resource']\nspace_left_in_bytes = list(session.query(Quota.over).filter(Quota.user_id == session.groups.get(group).id,\n                                                            Quota.resc_id == session.resources.get(resource).id))[0][Quota.over] * -1\n```\n\nAnd, to remove all quotas for a given group, one might (as a rodsadmin)\ndo the following:\n\n```python\nfrom irods.models import Resource, Quota\nresc_map = dict([(x[Resource.id],x[Resource.name]) for x in sess.query(Resource)] + [(0,'total')])\ngroup = sess.groups.get('my_group')\nfor quota in sess.query(Quota).filter(Quota.user_id == group.id):\n    sess.groups.remove_quota(group.name, resource = resc_map[quota.resc_id])\n```\n\nManaging users\n--------------\n\nYou can create a user in the current zone (with an optional auth_str):\n\n```python\n\u003e\u003e\u003e session.users.create('user', 'rodsuser', 'MyZone', auth_str)\n```\n\nIf you want to create a user from a federated zone, use:\n\n```python\n\u003e\u003e\u003e session.users.create('user', 'rodsuser', 'OtherZone', auth_str)\n```\n\nShowing client hints\n--------------------\n\nYou can get a list of available microservices, rules, etc. using the `client_hints`\nattribute of the session.\n\n```python\n\u003e\u003e\u003e session.client_hints\n```\n\nCode Samples and Tests\n----------------------\n\nAdditional code samples are available in the [test\ndirectory](https://github.com/irods/python-irodsclient/tree/main/irods/test)\n\nTesting\n=======\n\nSetting up and running tests\n----------------------------\n\nThe Python iRODS Client comes with its own suite of tests. Some amount\nof setting up may be necessary first:\n\n1.  Use `iinit` to specify the iRODS client environment.\n    For best results, point the client at a server running on the local\n    host.\n2.  Install the python-irodsclient along with the\n    `unittest unittest_xml_reporting` module or the older\n    `xmlrunner` equivalent.\n    -   `pip install ./path-to-python-irodsclient-repo[tests]`\n        (when using a local Git repo); or,\n    -   `pip install python-irodsclient[tests]'\u003e=1.1.1'`\n        (when installing directly from PyPI).\n3.  Follow further instructions in the [test\n    README file](https://github.com/irods/python-irodsclient/tree/main/irods/test/README.rst)\n\nTesting S3 parallel transfer\n----------------------------\n\nSystem requirements:\n\n    - Ubuntu 18 user with Docker installed.\n    - Local instance of iRODS server running.\n    - Logged in sudo privileges.\n\nRun a MinIO service:\n\n```\n$ docker run -d -p 9000:9000 -p 9001:9001 minio/minio server /data --console-address \":9001\"\n```\n\nSet up a bucket `s3://irods` under MinIO:\n\n```\n$ pip install awscli\n\n$ aws configure\nAWS Access Key ID [None]: minioadmin\nAWS Secret Access Key [None]: minioadmin\nDefault region name [None]:\nDefault output format [None]:\n\n$ aws --endpoint-url http://127.0.0.1:9000 s3 mb s3://irods\n```\n\nSet up s3 credentials for the iRODS s3 storage resource:\n\n```\n$ sudo su - irods -c \"/bin/echo -e 'minioadmin\\nminioadmin' \u003e/var/lib/irods/s3-credentials\"\n$ sudo chown 600 /var/lib/irods/s3-credentials\n```\n\nCreate the s3 storage resource:\n\n```\n$ sudo apt install irods-resource-plugin-s3\n```\n\nAs the 'irods' service account user:\n\n```\n$ iadmin mkresc s3resc s3 $(hostname):/irods/ \\\n  \"S3_DEFAULT_HOSTNAME=localhost:9000;\"\\\n  \"S3_AUTH_FILE=/var/lib/irods/s3-credentials;\"\\\n  \"S3_REGIONNAME=us-east-1;\"\\\n  \"S3_RETRY_COUNT=1;\"\\\n  \"S3_WAIT_TIME_SEC=3;\"\\\n  \"S3_PROTO=HTTP;\"\\\n  \"ARCHIVE_NAMING_POLICY=consistent;\"\\\n  \"HOST_MODE=cacheless_attached\"\n\n$ dd if=/dev/urandom of=largefile count=40k bs=1k # create 40-megabyte test file\n\n$ pip install 'python-irodsclient\u003e=1.1.2'\n\n$ python -c\"from irods.test.helpers import make_session\n            import irods.keywords as kw\n            with make_session() as sess:\n                sess.data_objects.put( 'largefile',\n                                       '/tempZone/home/rods/largeFile1',\n                                       **{kw.DEST_RESC_NAME_KW:'s3resc'} )\n                sess.data_objects.get( '/tempZone/home/rods/largeFile1',\n                                       '/tmp/largefile')\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Firods%2Fpython-irodsclient","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Firods%2Fpython-irodsclient","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Firods%2Fpython-irodsclient/lists"}