{"id":20358075,"url":"https://github.com/irods/irods_rule_engine_plugin_python","last_synced_at":"2026-03-03T22:02:49.144Z","repository":{"id":33431668,"uuid":"37076977","full_name":"irods/irods_rule_engine_plugin_python","owner":"irods","description":null,"archived":false,"fork":false,"pushed_at":"2025-04-01T17:32:35.000Z","size":378,"stargazers_count":10,"open_issues_count":39,"forks_count":14,"subscribers_count":14,"default_branch":"main","last_synced_at":"2025-04-12T03:15:19.093Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/irods.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-06-08T15:55:22.000Z","updated_at":"2025-04-01T17:32:39.000Z","dependencies_parsed_at":"2023-10-20T00:11:47.601Z","dependency_job_id":"1621e9c2-cf4e-4aff-bbc2-92a90790e1e3","html_url":"https://github.com/irods/irods_rule_engine_plugin_python","commit_stats":null,"previous_names":[],"tags_count":19,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/irods%2Firods_rule_engine_plugin_python","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/irods%2Firods_rule_engine_plugin_python/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/irods%2Firods_rule_engine_plugin_python/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/irods%2Firods_rule_engine_plugin_python/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/irods","download_url":"https://codeload.github.com/irods/irods_rule_engine_plugin_python/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248510000,"owners_count":21116130,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-14T23:25:16.490Z","updated_at":"2026-03-03T22:02:49.138Z","avatar_url":"https://github.com/irods.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# iRODS Rule Engine Plugin - Python\n\nThis C++ plugin provides the iRODS platform a rule engine that allows iRODS rules to be written in Python.\n\n# Build\n\nBuilding the iRODS Python Rule Engine Plugin requires version 4.2.1+ of the iRODS software from github (http://github.com/irods/irods).  The version of the `irods-dev` package will need to match the version defined in `CMakeLists.txt`.\n\n```\ncd irods_rule_engine_plugin_python\nmkdir build\ncd build\ncmake ../\nmake package\n```\n\n# Installation\n\nThe packages produced by CMake will install the Python plugin shared object file:\n\n`/usr/lib/irods/plugins/rule_engines/libirods_rule_engine_plugin-python.so`\n\n# Configuration\n\nAfter installing the plugin, `/etc/irods/server_config.json` needs to be configured to use the plugin.\n\nTo activate the plugin, add a new stanza to the \"rule_engines\" array within `server_config.json`:\n\n```json\n{\n    \"instance_name\": \"irods_rule_engine_plugin-python-instance\",\n    \"plugin_name\": \"irods_rule_engine_plugin-python\",\n    \"plugin_specific_configuration\": {}\n}\n```\n\nAdding the Python Rule Engine Plugin stanza above the default \"irods_rule_engine_plugin-irods_rule_language\" stanza will allow any defined Python rules to take precedence over any similarly named rules in the iRODS Rule Language. Placing the \"irods_rule_engine_plugin-python-instance\" stanza before the \"irods_rule_engine_plugin-cpp_default_policy-instance\" stanza will ensure that any default C++ policies will not cancel out any similarly named Python rules.\n\nAs soon as the stanza is inserted, the iRODS server expects a Python source code module to exist at the path `/etc/irods/core.py`. Once imported by the iRODS server and successfully compiled to a Python byte-code file bearing the \".pyc\" extension, any functions in the module will be callable as rules as long as they conform to a calling convention\n```\ndef a_python_rule( rule_args, callback, rei ):\n    # ... Python code to be executed in the rule context...\n```\n\nThe parameters need not be explicit or even named as they are here; they could in fact be collapsed into a parameter tuple, as in **pyfn(\\*x)**.\n\nThe first argument above, `rule_args`, is a tuple containing the optional, positional parameters fed to the rule function by its caller; these same parameters may be written to, with the effect that the written values will be returned to the caller.  The `callback` object is effectively a gateway through which other rules (even those managed by other rule engine plugins) and microservices may be called. Finally, `rei` (also known as \"rule execution instance\") is an object carrying iRODS-related context for the current rule call, including any session variables.\n\nWhen the plugin is freshly installed, a template file is located at `/etc/irods/core.py.template`. This may be copied to `/etc/irods/core.py`, or else an empty `core.py` can be created at that path if the user prefers to begin from scratch.\n\n# Default PEPs\n\nThe example `core.py.template` file in this repository contains a Python implementation of all static policy enforcement points (PEPs), equivalent to the default `core.re` rulebase. Placing the \"irods_rule_engine_plugin-python-instance\" stanza before the \"irods_rule_engine_plugin-cpp_default_policy-instance\" stanza in `/etc/irods/server_config.json` will ensure that any default C++ policies will not cancel out any similarly named Python rules copied from the example `core.py.template` file.\n\n# Python Version\n\nThis version of the Python Rule Engine Plugin uses the Python 3 interpreter.\n\n# Remote Execution\n\nThere exists a requirement for the implementation of a different `remote` microservice call for every rule language.  Given the possibility of a namespace collision with more than one rule language being configured simultaneously, the name of the microservice to use for the python language is `py_remote()`.  As with remote execution via the native rule engine, this microservice runs the given rule text on the remote host using `exec_rule_text`.   This can be done on any iRODS host (inside or outside the local zone) where the invoking user is authenticated.\n\nThe microservice's signature is: `py_remote(hostname, hints, code, recovery)`.\n\nIts four parameters are strings:\n   - `hostname`. The name of the iRODS server where the code is to be executed.\n   - `hints`.  A string containing optional XML tags. Currently scanned only for the `INST_NAME` tag, and other tags - if used - will be ignored. This\n      includes \u003cZONE\u003e, since the zone (whether remote or local) is inferred from the `hostname` parameter.\n   - `code`. The Python source code to be executed, in the form of a complete rule, i.e.: `def main(rule_args,callback,rei): ...`\n   - `recovery`.  This is currently unused.\n\nFor example:\n```\ndef main(rule_args, callback, rei):\n    rule_code = \"def main(rule_args, callback, rei):\\n    print('This is a test of the Python Remote Rule Execution')\"\n    callback.py_remote('icat.example.org', '\u003cINST_NAME\u003eirods_rule_engine_plugin-python-instance\u003c/INST_NAME\u003e', rule_code, '')\nINPUT null\nOUTPUT ruleExecOut\n```\n\n# Admin rights required for irule with Python\n\nDue to the Python language not being sandboxed (and running as the unix iRODS service account), `irule` does not allow rulefiles written in Python to be executed by non-`rodsadmin` user accounts.  If this is attempted, the following error message will be returned to the client:\n```\nInsufficient privileges to run irule in Python rule engine plugin\n```\n\nIf Python rules are needed by non-`rodsadmin` users, then iRODS administrators can install those Python rules on the server side (in `core.py`, or imported by `core.py`) where they will be made available to `irule` via the `callback` mechanism (see below).\n\n\n# Calling rules across language boundaries\n\nThe following example rulefiles (iRODS Rule Language and Python) demonstrate calling microservices in both directions when reacting to a simple `iput` (the `acPostProcForPut` PEP is fired in Python).\n\nThe Python `acPostProcForPut` rule calls `irods_rule_language_printme` with a single string parameter via the `callback` mechanism.\n\nThe iRODS Rule Language rule calls `python_printme` with a single string parameter (without using the `callback` mechanism).\n\ncore.py:\n```python\ndef acPostProcForPut(rule_args, callback, rei):\n    callback.writeLine('serverLog', 'PYTHON - acPostProcForPut() begin')\n    callback.irods_rule_language_printme('apples')\n    callback.writeLine('serverLog', 'PYTHON - acPostProcForPut() end')\n\ndef python_printme(rule_args, callback, rei):\n    callback.writeLine('serverLog', 'PYTHON - python_printme() begin')\n    callback.writeLine('serverLog', 'python_printme [' + rule_args[0] + ']')\n    callback.writeLine('serverLog', 'PYTHON - python_printme() end')\n```\n\nexample.re:\n```\nirods_rule_language_printme(*thestring){\n    writeLine(\"serverLog\",\"irods_rule_language_printme [\" ++ *thestring ++ \"]\")\n    python_printme('bananas');\n}\n```\n\nPut a file:\n```\n$ iput foo\n```\n\nThe resulting rodsLog will have the following output:\n```\nAug  8 20:57:43 pid:23802 NOTICE: writeLine: inString = PYTHON - acPostProcForPut() begin\nAug  8 20:57:43 pid:23802 NOTICE: writeLine: inString = irods_rule_language_printme [apples]\nAug  8 20:57:43 pid:23802 NOTICE: writeLine: inString = PYTHON - python_printme() begin\nAug  8 20:57:43 pid:23802 NOTICE: writeLine: inString = python_printme [bananas]\nAug  8 20:57:43 pid:23802 NOTICE: writeLine: inString = PYTHON - python_printme() end\nAug  8 20:57:43 pid:23802 NOTICE: writeLine: inString = PYTHON - acPostProcForPut() end\n```\n\n# Python globals and built-in modules\nThese built-in objects are set up at plugin initialization time and available in the python interpreter that loads `core.py` or, as the case may be, the Python rule file:\n   - `irods_rule_vars` - a dictionary for accessing variables of the form `*var` from the `INPUT` line, if present.\n   - `irods_types` - a module containing common struct types used for communicating with microservices.\n   - `irods_errors` - a module mapping well-known iRODS error names to their corresponding integer values.\n   - `global_vars` - deprecated alias for `irods_rule_vars`; only available in some contexts. Will be removed in a future release.\n   \nBy using the `irods_errors` built-in, policy may indicate how the framework is to continue through the use of a symbolic name, rather than a cryptic number reference:\n```\ndef pep_api_data_obj_put_pre(args, callback, rei):\n  # ...local processing...\n  return irods_errors.RULE_ENGINE_CONTINUE  # compare: \"return 5000000\"\n```\nFinally, the following example demonstrates the use of all three of these built-ins.\nWith the below rule contained anywhere in the Python rulebase:\n```\ndef testRule(rule_args, callback, rei):\n  from irods_errors import USER_FILE_DOES_NOT_EXIST\n  check_err = lambda e,errcode: str(e).startswith('[iRods__Error__Code:{}]'.format(errcode))\n  try:\n    ret_val = callback.msiObjStat(rule_args[0], irods_types.RodsObjStat())\n  except RuntimeError as e:\n    if check_err(e, USER_FILE_DOES_NOT_EXIST):\n      callback.writeLine('serverLog', 'ERROR in testRule: \"'+rule_args[0]+'\" not found')\n    else:\n      callback.writeLine('serverLog', 'UNKNOWN error')\n    rule_args[0] = ''\n  else:\n    objstat_return = ret_val['arguments'][1]\n    rule_args[0] = 'mtime = {!s} ; objsize = {!s} '.format(objstat_return.modifyTime, objstat_return.objSize)\n```\nand a rule script file (named `call_testRule.r`) which calls `testRule` with a data path argument:\n```\ndef main(_,callback,rei):\n  data_path = irods_rule_vars[ '*dataPath' ][1:-1]\n  retval = callback.testRule( data_path )\n  callback.writeLine( \"stdout\", retval['arguments'][0])\nINPUT *dataPath=\"\"\nOUTPUT ruleExecOut\n```\nwe can invoke the rule script and  point `*dataPath` at a trial data object path:\n```\nirule -r irods_rule_engine_plugin-python-instance -F call_testRule.r '*dataPath=\"/full/path/to/data/object\"'\n```\nand thus determine a data object's existence, as well as its mtime and size, if it does exist.\n\n# Auxiliary Python Modules\n\nIncluded with the PREP (Python Rule Engine Plugin) are some other modules that provide a solid foundation of utility for writers of Python rule code.  The plugin directly loads only the module `/etc/irods/core.py`, however any import statements in that file are honored if the modules they target are in the interpreter's import path (`sys.path`).  In addition, a `rodsadmin` irods user may use `irule` to execute Python rules within `.r` files.  By default, `/etc/irods` is included in the import path, meaning that the modules discussed in this section are accessible to all other Python modules and functions (whether or not they are \"rules\" proper) whether they be internal to `core.py`, or otherwise loaded by the PREP.\n\n## `session_vars.py`\n\nThis module contains a function `get_map` which may be used to extract session variables from the `rei` parameter passed to any rule function.\n\nAn example follows:\n\n```\nimport session_vars\ndef get_irods_username (rule_args, callback, rei):\n  username = ''\n  var_map = session_vars.get_map(rei)\n  user_type = rule_args[0] or 'client_user'\n  userrec = var_map.get(user_type,'')\n  if userrec:\n    username = userrec.get('user_name','')\n  rule_args [0] = username\n```\n\n## Special methods\n\nSome iRODS objects used with rules and microservices have been given utility methods.\n\nAn example is the map() method of a `PluginContext` PEP argument:\n```\n    def pep_resource_resolve_hierarchy_pre(rule_args, callback, rei):\n        import pprint\n        pprint.pprint(rule_args[1].map().items())\n```\n\nIn version 4.2.11.2 of the Python plugin, `BytesBuf` has methods to set and access byte content.\nThese can be used to achieve true binary reads and writes, to and from data objects.\n\nFor example:\n```\n    def my_rule(args, callback, rei):\n        buf = irods_types.BytesBuf()\n        buf.set_buffer( os.urandom(256) )\n        callback.msiDataObjectWrite( descriptor, buf, 0)  #-- Write the buffer.\n        # ...\n        buf.clear_buffer()\n```\nor:\n```\n    # ...\n    retv = callback.msiDataObjectRead( descriptor, \"256\", irods_types.BytesBuf())\n    returnbuf = retv['arguments'][2]\n    out = returnbuf.get_bytes()         #--  Returns list of integer octets.\n    start_byte = returnbuf.get_byte(0)  #--  Returns first octet\n```\n\n\n\n## `genquery.py`\n\nThis module offers writers of Python rules a convenient facility for querying and iterating over objects in the iRODS object catalog (ICAT).\nThe only declaration required within your `core.py` or other Python rule file is:\n```\nfrom genquery import *\n```\nPrevious versions of the `genquery.py` module offered the following two styles of iterator to expose iRODS' General Query capability:\n\n   * `row_iterator`, a generator function which yields results one row at a time:\n   ```\n   def my_rule(rule_args,callback,rei):\n       for result in row_iterator(\"COLL_NAME,DATA_NAME\",\n                                  \"DATA_NAME like '%.dat'\",\n                                  AS_LIST,\n                                  callback):\n           callback.writeLine( \"serverLog\", result[0] + \"/\" + result[1] )\n   ```\n   * `paged_iterator`, an iterator class that enables paging through a total of `nResults` rows from the ICAT in lists of `1 \u003c= n \u003c= N` at a time, with\n      - `N` a specified page size in the range ``1...256`` inclusive;\n      - and `n \u003c N` only at the end of the iteration when `nResults == 0` or `nResults % N != 0`:\n\n   ```\n   def get_users_in_group(args,callback,rei):\n       groupname = args[0]\n       users = []\n       for rows in paged_iterator(\"USER_NAME,USER_ZONE\",\n                                     \"USER_GROUP_NAME = '{}'\".format(groupname),\n                                     AS_DICT, callback, 32 ):\n           users += [\"{USER_NAME}#{USER_ZONE}\".format(**r) for r in rows]\n       callback.writeLine('stdout',repr(users))\n```\n\nAs of version 4.2.8 of iRODS, there is also a class-based iterator called `Query`, which offers an enhanced set of options and behaviors for use in ICAT searches.\n\nUnder the new class-based syntax, an iRODS general query can be as simple as in this example:\n\n```\nall_resource_names = [ r for r in Query(callback,'RESC_NAME') ]\n```\n\nthe result of which will be a Python `list` of `str` values, enumerating the names of all resource nodes known to the ICAT.\n\n`Query`'s constructor allows commonly defaulted attributes (otherwise tunable via a selection of keyword arguments, listed in the [reference](#queryref) at the end of this section) to take on reasonable values.  In particular, the *output* attribute (dictating the type of data structure used to return individual row results) will, unless specified, default to **AS_TUPLE** -- meaning that for single-column query results the iterator variable does not need to be indexed.  That is why, in our first `Query` example, the values yielded by the iteration were strings.\n\nThis allows for an admirable consistency of expression, as can be seen in the following two examples:\n\n```\n    for id in Query(callback, \"DATA_ID\") :\n        pass # - process data id's for iterated objects\n```  \nand\n```\n    for coll,data in Query(callback,(\"COLL_NAME\", \"DATA_NAME\")):\n        pass # - process result data objects using the logical path: (coll + \"/\" + data)\n```\nA caveat for beginners however, when expanding on such deceptively simple syntax, is that the iterations in the above two example usages yield essentially different row types.  In the first usage, the iteration variable `id` is of type `str` (string); and in the second, a `tuple` is actually returned from each iteration and then automatically unpacked as the variables `coll` and `data`.\n\n*If the uniformity of the result rows' data structure type is desirable for a given application of `Query(...)`, then `output=AS_LIST` or `output=AS_DICT` should be favored to the default `output=AS_TUPLE`. For more on this and other query attributes, see the [reference](#queryref) at the end of this section.*\n\nClones and Copies\n---\nThe `Query` class features lazy execution, meaning that only the act of iterating on a `Query` object will actually connect to the ICAT for the purpose of a general query. This has some advantages, particularly those of cheap instantiation and duplication.\n\nSpecifically, the `copy` method can be invoked on any `Query` object to instantiate a new one, and this can be done either with or without keyword arguments.  Absent any input arguments, a call to the `copy` method simply clones the original object.\n\nThe keyword arguments to Query's `copy` method, if used, are identical to those used by the constructor, and allow modification of the corresponding attributes within the new object. In this way, a `Query` object can easily serve as a bag of *attribute* variables which can be stored, transferred, modified, and re-used.\n\nBecause iteration on a `Query` object can change its state, a best practice is to favor invoking the `copy` method wherever repeated (esp. overlapping) uses are possible for the same Query instance.  This is particularly true for a `Query` object that has been stored in a Python object or variable, so as to give it a longer lifetime than a temporary instance.\n\nTo demonstrate how this could look in practice, let's incorporate a more elaborate version of the search from our first `Query` example within the following helper function:\n\n```\ndef get_resource_details(callback,*args):\n    (cond_string,) = args[:1] if args else ('',)\n    my_query = Query( callback,\n                      columns = ['RESC_NAME'],\n                      conditions = cond_string )\n    n_nodes = my_query.copy().total_rows()\n    if n_nodes == 0: return []\n\n    details = my_query.copy( columns = \\\n                               my_query.columns + ['RESC_ID','RESC_PARENT'] )\n    return [\n      \"Resource '{}': id = {}, parent-id = {}\".format(row[0],row[1],row[2])\n      for row in details\n    ]\n```\nwhich could later be called thus:\n```\ndef my_resc_summaries(_,callback,rei):\n    import socket; hostname = socket.gethostname()\n    from pprint import pformat\n    # -- all resources listed in the ICAT --\n    callback.writeLine('stdout',pformat(    get_resource_details(callback)\n                                )+\"\\n---\\n\")\n    # -- all resources on the local machine --\n    callback.writeLine('stdout',pformat(    get_resource_details(callback,\n                                            \"RESC_LOC = '{}'\".format(hostname))\n                                )+\"\\n---\\n\")\n    # -- all resources having no parent node -- (ie root resources)\n    callback.writeLine('stdout', pformat(   get_resource_details(callback,\n                                                                 \"RESC_PARENT = ''\")))\n    #...\n```\n\nAuto joins\n---\nIn iRODS's general queries, joins between related tables are automatic.  For example, queries whose column targets including both `DATA_*` and `COLL_*` fields implicitly perform \"auto-joins\" between the Data objects queried and their parent Collections. Metadata AVU's are also joined automatically with the corresponding type of iRODS object which they annotate:\n\n```\nq = Query( callback, columns = ['DATA_ID,META_DATA_ATTR_VALUE'], output = AS_DICT, conditions = \"\"\"\\\n            META_DATA_ATTR_NAME = 'modify_time' and \\\n            META_DATA_ATTR_VALUE like '0%'\n    \"\"\")\nnumber_of_data_objects = q.total_rows() # counting only, no fetch operations\nq_fetch = q.copy( q.columns + [\"COLL_NAME\", \"order(DATA_SIZE)\"] )\nfor row in q_fetch:\n    log_record = \"\"\"id = {DATA_ID} ; size = {order(DATA_SIZE)} ; path = \\\n                    {COLL_NAME}/{DATA_NAME}; mtime = {META_DATA_ATTR_VALUE}\\\n                 \"\"\".format(**row)\n    callback.writeLine(\"serverLog\",log_record)\n```\n\nQuery Options\n---\n\nThe `options` Query attribute is a mask of bit-field values which can be built up from constants in the `genquery.Options` namespace, although probably the only directly useful option is `NO_DISTINCT`.  This option might come into play in the rare case that we decide against iRODS' usual behavior of requiring all row results to be \"distinct\" (in other words, distinguishable combinations of values for the columns selected):\n\n```\nfrom genquery import * ; from genquery import Option\ny = len([ row for row in Query(callback , 'META_DATA_ATTR_NAME',\n                               \"\"\"META_DATA_ATTR_NAME = 'aa' and \\\n                                  META_DATA_ATTR_UNITS = '' \"\"\",\n                               options=Option.NO_DISTINCT) ])\n```\nThe above example enumerates metadata AVU's (both used and unused) having the attribute name of 'aa' and no units -- and, by employing the NO_DISTINCT option, suppresses the normal de-duplicating behavior of the general query.  Without the option, any subset of row results undiscernable as being different would be collapsed into a single row result.  (Another example where this might be important would be if multiple data objects had the same values in the DATA_NAME and/or DATA_SIZE columns, without a COLL_* or DATA_ID column also being selected to establish uniqueness in an alternative way.)\n\nStrict upward compatibility mode\n---\nAs of iRODS 4.2.8, the `genquery` module's `row_iterator` function returns a `Query` instance instead of the generator object of previous versions. This should not affect existing Python rule code that depends on the function unless its return value is expected to follow the generator interface -- allowing, for example, iteration via Python interpreter's `next()` built-in.  The great majority of Python statements and expressions availing themselves of this simple row-iterating facility will use the more convenient and succinct form:\n```\n  for row in row_iterator(...)\n```\n\nBut in the event of an incompatibility, users can **modify their import declarations as follows** to accommodate both new and old interfaces simultaneously:\n\n   - Use\n     ```\n     from genquery import *\n     from genquery import (row_generator as row_iterator)\n     ```\n     in place of the normal\n     ```\n     from genquery import *\n     ```\n\n   - or, in modules not using the wildcard import syntax, add\n     ```\n     from genquery import row_generator\n     ```\n     and then replace any problematic uses of `row_iterator` or `genquery.row_iterator` with `row_generator`.\n\nThis should solve the very rare issue of such dependencies.\n\n\u003ca name=\"queryref\"\u003e\u003c/a\u003e\n`Query` object creation and attributes Reference\n---\nA `Query` object's *attributes* (ie \"columns\", \"conditions\", ... ) set constraints on how an iRODS general query should be carried out and how the results should be enumerated.  These *attributes* are defined by keyword arguments of the same name, detailed below.\n\nThe `Query` object constructor requires these two arguments to be first, and in the order listed:\n\n  * *callback* (not really an attribute) can only be used in the constructor.  It is just the callback argument from the enclosing Python rule function's input argument list.\n  * *columns*  is a either a string with comma-separated column names, or a `list` or `tuple` of column name. If provided among the `copy` method's keyword arguments, it must however be of type `list`. \n\nThe following arguments are strictly optional and may be used either in `Query`'s constructor or in the `copy` method:\n\n  * *condition*  is the \"where ...\" predicate traditionally used in the general query syntax, but omitting the `where` keyword. The default (an empty string) allows for opting out of the search condition, in which case all objects of the requested type, or joined list of types,  will be returned (ie. Collections for `COLL_*`, Resources for `RESC_*`)\n  * *output* is one of: `AS_TUPLE`, `AS_DICT` or `AS_LIST`. These constants are defined in the genquery module, and they specify the Python data structure type to be used for returning individual row results.\n    - `AS_TUPLE`, the default, lets results of a single-column query be rendered simply as one string value per row returned; or, if multiple column names are specified, as a tuple of strings, indexed by a zero-based integer column offset. (Note that if a more programmatically consistent interface is desired, ie. indexing each row result by integer offset even for the case of a single column, then `AS_LIST` should be preferred.)\n    - `AS_LIST` forces a column value within each row to be indexed by its zero-based integer position among the corresponding entry in the ***columns*** attribute .\n    - `AS_DICT` lets column values be indexed directly using the column name itself (a string). The\n      seemingly higher convenience of this approach is, however, paid for by an increased **per-row** execution overhead.\n  * *offset*: 0 by default, this `Query` attribute dictates the integer position, relative to the complete set of possible rows returned, where the enumeration of query results will start.\n  * *limit*: `None` by default (ie \"no limit\"), this option can be an integer \u003e= 1 if specified, and limits the returned row enumeration to the requested number of results. Often used with *offset*, as defined above.\n  * *case-sensitive* is normally `True`. If it is set to `False`, the condition string will be uppercased, and the query will be executed without regard to case.  This allows for more permissive matching on the names of resources, collections, data objects, etc.\n  * *options* is a bitmask of extra options, and defaults to a value of 0. `genquery.Option.NO_DISTINCT` is one such extra option, as is RETURN_TOTAL_ROW_COUNT (although in the latter case, using the `Query` object's `row_count` method should be preferred.)\n  * *parser* defines which GenQuery parser to use for querying the catalog. The following values are supported:\n    - `Parser.GENQUERY1` configures the `Query` object to use GenQuery1 (i.e. the traditional parser). This is the default.\n    - `Parser.GENQUERY2` configures the `Query` object to use GenQuery2. GenQuery2 is an experimental parser with several enhancements over GenQuery1.\n      - When using the GenQuery2 parser, the following applies:\n        - The `output` constructor parameter is ignored.\n        - The `case_sensitive` constructor parameter is ignored.\n        - The `options` constructor parameter is ignored.\n        - The `total_rows` member function of the `Query` class always returns `None`.\n  * *order_by* is a string holding the sorting instructions for a GenQuery2 query. The string must be a comma-delimited list of GenQuery2 columns (or expressions). For example, `order_by='COLL_NAME, DATA_NAME'`. For more examples, see [How do I sort data using the Query class and GenQuery2?](#how-do-i-sort-data-using-the-query-class-and-genquery2). Defaults to an empty string. Only recognized by the GenQuery2 parser.\n\nWhen the processing of a GenQuery2 resultset is complete, it is best practice to call the `close()` member function. Doing this will instruct the server to immediately free any resources allocated to the `Query` object. This is extremely important when multiple `Query` objects are executed within a single rule.\n\n# Questions and Answers\n\n## What happened to my `print` output?\n\nIn iRODS server versions 4.3.0 and later, all standard streams are redirected to `/dev/null` in the server. This means that any data sent to `stdout` such as via a `print` statement in a rule run inside the server will be discarded.\n\nHere is some example code which would print a message to the `rodsLog` in servers from the 4.2.x series and earlier when run with the Python Rule Engine Plugin:\n```python\nprint('hello, server log!')\n```\n\nIn order to achieve a similar effect on server versions 4.3.0 and later, you can replace such a call like this:\n```python\ncallback.writeLine('serverLog', 'hello, server log!')\n```\n\nNote that the `writeLine` microservice can also target `stdout` in addition to `serverLog`. `callback.writeLine('stdout', ...)` will have the same effect as running `print`. The output will be discarded.\n\n## How do I pass values back to the caller, across rule engine plugin boundaries?\n\nThis is achieved by writing a value back _(as a string)_ to the appropriate index within the `rule_args` array. The caller is required to pass one or more variables as **out** parameters.\n\nHere's an example demonstrating how to pass a value, via the input arguments, from the PREP to the native rule engine plugin (NREP).\n\nThe following is a python function which overwrites the value of the first input argument.\n```python\n# file: core.py\n\ndef get_magic_number(rule_args, callback, rei):\n    # Notice the integer is represented as a string.\n    # Only strings can be passed across rule engine plugin boundaries.\n    #\n    # Take note of the index we're writing to. The index represents the\n    # position of the input argument. Because this rule expects one input\n    # argument, we use index 0. If there were more input arguments, we could\n    # write to those as well using the index which identifies them.\n    rule_args[0] = '42'\n```\n\nAnd here is the NREP rule which invokes the PREP rule.\n```python\n# file: core.re\n\nnrep_rule()\n{\n    # Due to how the NREP works, we must initialize the variable to a string.\n    # The value of the string isn't important.\n    *magic_number = '-1';\n\n    # Now, we invoke the Python rule in core.py. Remember, the rule takes one\n    # input argument. That argument maps to index 0 within the python rule.\n    get_magic_number(*magic_number);\n\n    # Finally, the value is written to the log file.\n    writeLine('serverLog', 'The magic number is [*magic_number].');\n}\n```\n\nIt's important that the value passed across rule engine plugin boundaries be of type string. Passing any other type will result in an error.\n\nUpon invoking `nrep_rule`, a message very similar to the one that follows will appear in the server log file. See the **log_message** property. It should contain `[42]` instead of `[-1]`.\n```js\n{\n  \"log_category\": \"legacy\",\n  \"log_level\": \"info\",\n  \"log_message\": \"writeLine: inString = The magic number is [42].\\n\",\n  \"request_api_name\": \"EXEC_MY_RULE_AN\",\n  \"request_api_number\": 625,\n  \"request_api_version\": \"d\",\n  \"request_client_user\": \"rods\",\n  \"request_host\": \"10.15.164.3\",\n  \"request_proxy_user\": \"rods\",\n  \"request_release_version\": \"rods4.3.1\",\n  \"server_host\": \"76f890b83c4d\",\n  \"server_pid\": 2637,\n  \"server_timestamp\": \"2024-04-10T12:43:31.381Z\",\n  \"server_type\": \"agent\",\n  \"server_zone\": \"tempZone\"\n}\n```\n\n## How do I escape embedded single quotes using the Query class and GenQuery2?\n\nGenQuery2 provides two ways to escape embedded single quotes. They are as follows:\n- Use a single quote to escape a single quote as defined by the SQL standard\n- Or, use the hexadecimal encoding\n\nBelow, we demonstrate each escape mechanism on the data object name, `'_quotes_around_me_'`.\n```python\nfrom genquery import Query, Parser\nrow = Query(callback, 'DATA_NAME', \"DATA_NAME = '''_quotes_around_me_\\\\x27'\", parser=Parser.GENQUERY2).first()\ncallback.writeLine('serverLog', f'row = {row}')\n```\n\nNotice the use of `\\\\x` rather than `\\x` for the second single quote. This is required to keep Python from interpreting the hexadecimal value.\n\n## How do I sort data using the Query class and GenQuery2?\n\nUse the `order_by` parameter. Two examples are provided below.\n\nAssume we have five collections and we want to sort them by the length of their logical path, but in descending order.\n\n```python\nfrom genquery import Query, Parser\nfor row in Query(callback, 'COLL_NAME, length(COLL_NAME)', order_by='length(COLL_NAME) desc', parser=Parser.GENQUERY2):\n    callback.writeLine('serverLog', f'row = {row}')\n```\n\nAnd here's a second example demonstrating how to sort using multiple columns.\n\n```python\nfrom genquery import Query, Parser\nfor row in Query(callback, 'COLL_NAME, DATA_NAME', order_by='COLL_NAME desc, DATA_NAME', parser=Parser.GENQUERY2):\n    callback.writeLine('serverLog', f'row = {row}')\n```\n\nThe string passed to the `order_by` parameter instructs the database to do the following:\n- First, sort by collection name in descending order\n- Second, sort the data names in each collection in ascending order\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Firods%2Firods_rule_engine_plugin_python","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Firods%2Firods_rule_engine_plugin_python","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Firods%2Firods_rule_engine_plugin_python/lists"}