{"id":17680345,"url":"https://github.com/docelic/slapd-viper","last_synced_at":"2025-03-30T18:48:44.266Z","repository":{"id":66956516,"uuid":"159969975","full_name":"docelic/slapd-viper","owner":"docelic","description":"Database backend for OpenLDAP with extensive dynamic options and behavior","archived":false,"fork":false,"pushed_at":"2018-12-01T21:36:13.000Z","size":196,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-02-05T21:33:34.716Z","etag":null,"topics":["appending","backend","dynamic","fallback","ldap","openldap","openldap-slapd","relocation","slapd","substitution"],"latest_commit_sha":null,"homepage":null,"language":"Perl","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/docelic.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-12-01T17:32:03.000Z","updated_at":"2022-09-22T07:17:07.000Z","dependencies_parsed_at":"2023-02-25T03:46:06.833Z","dependency_job_id":null,"html_url":"https://github.com/docelic/slapd-viper","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/docelic%2Fslapd-viper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/docelic%2Fslapd-viper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/docelic%2Fslapd-viper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/docelic%2Fslapd-viper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/docelic","download_url":"https://codeload.github.com/docelic/slapd-viper/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246365640,"owners_count":20765546,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["appending","backend","dynamic","fallback","ldap","openldap","openldap-slapd","relocation","slapd","substitution"],"created_at":"2024-10-24T09:06:40.028Z","updated_at":"2025-03-30T18:48:43.642Z","avatar_url":"https://github.com/docelic.png","language":"Perl","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Slapd-viper Database Backend for OpenLDAP\n\n`slapd-viper` is a flexible database backend for OpenLDAP. It has been in use since 2008.\n\nIt implements all the usual LDAP operations, as well as provides elaborate features for responding with dynamic and auto-computed data on a level not seen in traditional LDAP installations.\n\n## Quick Start\n\n1. Install system dependencies:\n\n```sh\napt install slapd libfile-find-rule-perl libnet-ldap-perl libtext-csv-xs-perl liblist-moreutils-perl\n```\n\n2. Obtain and run Viper with an example config file that provides suffix `dc=example,dc=com`:\n\n```sh\ncd /etc/ldap\ngit clone https://github.com/crystallabs/slapd-viper\n\n/usr/sbin/slapd -d 256 -f ./slapd-viper/etc/ldap/slapd.conf\n```\n\n3. While the server is running, load example data into the tree:\n\n```sh\nldapadd -x -D cn=admin,dc=example,dc=com -w nevairbe -c -f slapd-viper/ldifs/example.ldif\n```\n\n4. And query the added data:\n\n```sh\nldapsearch -x -b dc=example,dc=com\n```\n\nThat's it for the most basic example.\n\n## Overview\n\n`slapd-viper` is a custom backend for OpenLDAP built on top of OpenLDAP's `slapd-perl`.\n\nIt stores the directory tree in files and directories on disk, by default under `/var/lib/ldap/viper/`.\n\nIt can be used as a normal backend \u0026mdash; reading and writing data is just the same as for any backend.\nHowever, its primary use is for LDAP trees which contain highly dynamic data and where elaborate and configurable server-side behavior is needed.\n\nDynamic functions possible with `slapd-viper` are:\n\n1. ADD operation can overwrite existing entries without throwing an error\n1. ADD operation can ignore adds for entries which already exist without throwing an error\n1. DN of new entries can be modified (entries can be \"relocated\") before they are saved\n1. When adding Debconf templates, a prompt can be opened on a remote machine, asking the admin for Debconf answer\n1. When MODIFYing entries, no-ops can be detected and no changes made\n1. When MODIFYing (and in combination with other functionality), Copy on Write is supported\n1. DELETE can be allowed to delete whole subtrees if entries to delete are non-leaves\n1. SEARCH queries can be arbitrarily rewritten\n1. SEARCH can fallback to multiple other locations if no results are found\n1. Attribute values can expand to content of disk files\n1. Attribute values can expand to value of other entries and attributes\n1. Attribute values can expand to value of another SEARCH\n1. Attribute values can evaluate Perl code\n1. Returned entries can be appened with additional attributes or values in various ways\n\n## Configuration Reference\n\nEach database backed by Viper should be configured with something similar to the following block in `slapd.conf`:\n\n```\ndatabase           perl\nsuffix             \"dc=example,dc=com\"\nperlModulePath     \"/etc/ldap/slapd-viper\"\nperlModule         \"Viper\"\n\ndirectory          \"/var/lib/ldap/viper/\"\ntreesuffix         \"dc=example,dc=com\"\n```\n\nThe first four lines are required by `slapd` and its `back-perl` backend\nto configure the suffix and initialize Viper.\n\nThe last two lines are required by the Viper backend. The value of\n`treesufix` must be equal to `suffix`.\nThis small duplication cannot\nbe avoided because `suffix` directive is consumed by `slapd` and is not\npassed onto our backend.\n\nAfter the above standard lines, the following directives can be used:\n\n### Configuration Directives\n\nThe list is split into simple and complex directives, simple first.\n\nWithin each group, the list is sorted alphabetically, with each heading specifying configuration\ndirective name and its usage syntax.\n\nWhere applicable, the first value listed indicates the default value.\n\n#### Simple Configuration Directives\n\n##### addIgnoredups 0|1\n\nSpecify whether LDAP ADD operation should ignore adds on existing entries\nwithout throwing LDAP_ALREADY_EXISTS error. Applicable if `addoverwrites` is 0.\n\n##### addOverwrites 0|1\n\nSpecify whether LDAP ADD operation should overwrite existing entries\nwithout throwing LDAP_ALREADY_EXISTS error.\n\n##### cacheRead SPEC\n\nSpecify for how many LDAP operations to cache LDIF reads from disk.\nNo specification implies no cache.\n\nSPEC can be:\n\n```\nX    - X LDAP operations\n```\n\nOverall best value, which minimizes risk of serving stale data while reaching\nnoticeable optimization improvement (up to 25%), is 1 operation, specified as `1`.\n\n`cacheRead` and `overlayConfig` cache can be used together, amplifying the effect.\n\n##### clean\n\nInvoke removal of all saved stack files from disk.\n\n##### deleteTrees 0|1\n\nSpecify whether Viper should allow deleting non-leaf elements (deleting\nthe entry and everything under it in one go).\n\nSee notes for DELETE under \"LDAP Operations - Notes\".\n\n##### enableBind 0|1\n\nWhether to allow binding as entries under this suffix. By default this\nis disabled.\n\nSee notes for BIND under \"LDAP Operations - Notes\".\n\n##### extension .ldif\n\nSpecify file extension to use when storing server data on disk.\n\nViper's data is kept in a directory tree that corresponds to the LDAP\ntree, where DN components are directories, and leaf nodes are files.\nEach file contains one LDAP entry in LDIF format.\n\nFile extension must be specified to make directories distinguishable\nfrom files, and the default value should rarely be changed.\n\n##### load FILE [PATTERN REPLACEMENT ...]\n\nLoad and process configuration stack from FILE.\nFILE is always relative to suffix base directory.\n\nIf list of PATTERN REPLACEMENTS is specified, the s/PATTERN/REPLACEMENT/\nsubstitution is done on all loaded lines before sending them to the\nconfig processor.\n\nExample: `load default_opts`\n\n##### message TEXT\n\nPrint TEXT to the log. The log will be a console if slapd is started\nwith option -d (such as -d 256) to run in the foreground.\n\nExample: `message Test`\n\n##### modifyCopyOnWrite 1|0\n\nWhen a MODIFY request is issued for an entry that does not really exist \n(i.e. it comes from a fallback), specify whether Viper should copy the\nentry to the expected location and then modify it, or return\nLDAP_NO_SUCH_OBJECT.\n\n##### modifySmarts 1|0\n\nSpecify whether Viper should ignore MODIFY requests that do not result\nin any real change within the entry.\n\nThis is useful to enable to detect \"no-op\" modifications and\navoid writing to disk, preserving meaningful modification timestamps\non existing entries.\n\n##### parse 1|0\n\nSpecify whether in the lines that follow, variable and directive expansion\nshould be performed.\n\nThis includes expanding ${variable} to variable values and %{directive} to\nconfiguration directive values.\n\n##### reset\n\nReset current stack in memory to empty list.\n\n##### save FILE\n\nSave current stack to FILE, always relative to suffix base directory.\n\nExample: `save default_opts`\n\n##### schemaFatal 0|1\n\nSpecify whether a missing or inaccessible schemaLDIF file should trigger\na fatal error.\n\nIt is vital for Viper to be aware of server's schema (which comes from\nthe `schemaLDIF` config option). The server won't work optimally if the schema\nfile in LDIF format is missing, or is not up to date with the server's schema.\n\nHowever, we issue a warning and allow startup without it, because you are\nthen expected to use \u003ci\u003escripts/schema.pl\u003c/i\u003e to connect to the\nserver right away and obtain the schema in LDIF format, saving it to the\nexpected location. Then, restart the server to pick it up.\n\nThis must be done because the server's schema is not available to `slapd-perl`.\n\n`schemaFatal` value should be set to 1 when you are sure you do\nhave the `schema.ldif` file in the correct location.\n\n##### schemaLDIF FILE\n\nSpecify location of server's schema in a single file, in LDIF format.\n\nSee the previous option for more info.\n\nNote that the schema in LDIF format does not eliminate the need to have the\nreal schema files in `/etc/ldap/schema/*.schema`. Schema files are read by\nslapd, and schema LDIF file is read by Viper. LDIF is created on the \nbasis of real schema files, and at all times, Viper's LDIF schema should\nbe in sync with slapd.\n\nExample: `schemaLDIF /etc/ldap/schema/schema.ldif`\n\n##### treeSuffix SUFFIX\n\nThis value should always match the value of `suffix` configured for the database.\n\nThis small duplication cannot\nbe avoided because `suffix` directive is consumed by `slapd` and is not\npassed onto our backend.\n\n##### var VARIABLE \"VALUE STRING\"\n\nAssign \"VALUE STRING\" to variable VARIABLE. Variables, in this context,\nare visible only within the suffix where they are defined, and their value\nis expanded with ${variable} if option \"parse\" is enabled.\n\n#### Complex Configuration Directives\n\n##### addPrompt\n\n##### addRelocate\n\n##### entryAppend ATTRIBUTE PATTERN ... -\u0026gt; attr ATTRIBUTE [ATTRATTR [ATTR...]]\n##### entryAppend ATTRIBUTE PATTERN ... -\u0026gt; append PATTERN REPLACEMENT [ATTR...]\n\nSpecify an `entryAppend` rule, allowing adding extra attributes into an entry\nbefore returning it to the client.\n\nWhen all ATTRIBUTE-PATTERN pairs match, Viper looks to append the entry with\na set of default attributes.\n\nThe entry from which to import the attributes can be specified in two ways:\n\n1. With \"attr ATTRIBUTE\" (usually \"attr seeAlso\"). In that case,\nthe attribute `seeAlso` is looked up in the current entry. It is expected\nto contain the DN of the entry whose attributes should append the current entry.\n\nIf ATTRATTR and ATTRs are unspecified, the entry is appended with\nall allowed attributes. Otherwise, it is appended only with attributes\nlisted in the ATTRATTR attribute within the entry and/or in the literal\nlist of ATTRs.\n\n2. With \"append PATTERN REPLACEMENT\", where s/PATTERN/REPLACEMENT/ is\nperformed on the original DN, and the result is used as the entry from which\nto pull the extra attributes.\n\nWith the 'append' method, there is no ATTRATTR field, so you cannot append\nthe entry with the values of attributes listed in the entry, but you do\nhave the option of specifying ATTRs to append with.\n\nIf left unspecified, the entry is appended with all allowed attributes.\n\nExamples from production config:\n\n```\nentryAppend  objectClass \"^dhcpHost$\"                      \\\n             -\u003e                                            \\\n             append .+ cn=dhcpHost,ou=objectClasses,ou=defaults\n\nentryAppend  objectClass \"^dhcpSubnet$\"                    \\\n             -\u003e                                            \\\n             append .+ cn=dhcpSubnet,ou=objectClasses,ou=defaults\n\nentryAppend  dn          \"^cn=default,ou=networks\"         \\\n             objectClass \"^ipNetwork$\"                     \\\n             -\u003e                                            \\\n             attr seeAlso\n```\n\n##### exp MATCH_REGEX NON_MATCH_REGEX\n\nSpecify regexes that each entry DN must and must not match respectively, to have\noverlay \"exp\" run on its attributes.\n\nThe \"exp\" overlay enables expansion into values of other attributes, in the\ncurrent or other entry.\n\nExample which always matches, and so enables the `exp` overlay: `exp  .   ^$`\n\n##### file MATCH_REGEX NON_MATCH_REGEX\n\nSpecify regexes that each entry DN must and must not match respectively, to have\noverlay \"file\" run on its attributes.\n\nThe \"file\" overlay enables expansion into values of on-disk files, always\nrelative to the suffix base directory.\n\nExample: `file  .   ^$`\n\n##### find MATCH_REGEX NON_MATCH_REGEX\n\nSpecify regexes that each entry DN must and must not match respectively, to have\noverlay \"find\" run on its attributes.\n\nThe \"find\" overlay enables internal re-invocation of the search function, \nand using the values retrieved in constructing the original value.\n\nThis overlay shares many similarities with \"exp\", but contains a crucial\ndifference -- with \"exp\", you generally know where the entry and attribute\nto expand to are located. With \"find\", you generally don't, so you perform\na search to find them.\n\nExample: `find  .   ^$`\n\n##### loadDump FILE\n\nTHIS IS A DEBUG OPTION .\n\nLoad direct Perl Storable dump of configuration hash from FILE, always\nrelative to the suffix base directory.\n\nThis is an advanced option that should not be called from slapd.conf.\n\nIt is intended for scenarios where Viper is at least once initialized by slapd\n(and configured via slapd.conf), and config then dumped as Storable object\nusing saveDump.\n\nAfter that, you can run Viper \"standalone\", directly under the Perl\ninterpreter using \u003ci\u003escripts/viper.pl\u003c/i\u003e, and instead of re-parsing\nslapd.conf for configuration, simply send \"loadDump FILE\" to the config\nprocessor, to load the exact state as had by slapd/Viper.\n\nThis is almost always needed only when you want to run Viper under the Perl\ninterpreter directly, to specify Perl debug or profiling options.\n\n##### overlayConfig OVERLAY OPTION VALUE ...\n\nSpecify default overlay options.\n\nOVERLAY can be an overlay name (perl, exp, file, find) or \"default\".\n\nOPTION can be \"cache\", \"prefix\" or \"if\".\n\n\tcache SPEC - specify cache expiry time.\n\n\tCaching overlay results improves performance enormously in situations\n\twhere multiple entries are returned and all produce the same dynamic\n\tvalues for certain attributes.\n\n\tIn such cases, operations of complexity O(n) are reduced to O(1) level.\n\n\tSyntax is the same as listed under \"cacheRead\", and 1o is again the\n\toverall best setting.\n\n\tNOTE: due to deficiencies in Memoize::Expire module, time- and\n\tuses-based methods of expiry do not work correctly when caching non-scalar\n\tvalues (such as multiple values for an attribute). It is therefore suggested\n\tto always use the number-of-operations cache (like 1o).\n\n\tExample: cache 1o\n\n\tprefix PREFIX - generic prefix option, used where applicable. Currently\n\tonly the \"file\" overlay honors it, where it is a prefix to prepend on\n\tall file specifications.\n\n\tDirectory separator is not added automatically,\n\tso to prefix with a directory, include \"/\" at the end.\n\n\tExample: prefix subdir/\n\n##### perl MATCH_REGEX NON_MATCH_REGEX\n\nSpecify regexes that each entry DN must and must not match respectively, to have\noverlay \"perl\" run on its attributes.\n\nBy default, Perl overlay is disabled as it is in fact an interface for\n\"eval\", and is considered dangerous. To activate it, open Viper.pm and\nenable constant PERLEVAL.\n\nExample: `perl  .   ^$`\n\n##### saveDump FILE\n\nTHIS IS A DEBUG OPTION .\n\nSave direct Perl Storable dump of configuration hash to FILE, always\nrelative to the suffix base directory.\n\nThis is an advanced option that should usually be called as the last\nline of slapd.conf configuration for a particular suffix.\n\nThis is almost always needed only when you want to run Viper under the Perl\ninterpreter directly, to specify Perl debug or profiling options.\n\n##### searchFallback PATTERN REPLACEMENT\n\nSpecify search fallback rule, effectively implementing default entries.\n\nWhen a specific search base is requested, and it does not exist in the searched\nlocation, it is possible to fallback to a chain of default entries. The first\nentry found wins.\n\nExamples: production examples defaulting to site-wide and global defaults\n\n```\n# Fallback 1: site defaults tree.\nsearchFallback  cn=.[^,\\\\s]+,ou=hosts         ou=hosts,ou=defaults\nsearchFallback  cn=.[^,\\\\s]+,ou=templates     ou=templates,ou=defaults\n\n# Fallback 2: global defaults tree.\nsearchFallback  cn=.[^,\\\\s]+,ou=hosts,.+      ou=hosts,ou=defaults\nsearchFallback  cn=.[^,\\\\s]+,ou=templates,.+  ou=templates,ou=defaults\n```\n\n##### searchSubst KEY PATTERN ... -\u0026gt; KEY PATTERN REPLACEMENT ...\n\nSpecify searchSubst rule, allowing rewrite of any part of the search\nrequest.\n\nWhen the incoming search request matches all KEY PATTERN pairs, Viper\nperforms the specified KEY=~ s/PATTERN/REPLACEMENT/ actions to rewrite\nthe incoming search.\n\nSearch rewriting is completely free-form, and it is possible to rewrite searches to a completely different Viper suffix, as long as both are located in the same base directory.\n\nThis is a legitimate feature of the rewrite model, and is officially used to\nrewrite incoming DHCP search queries under ou=dhcp to appropriate places\nand with appropriate options under ou=clients.\n\nKEY can be one of base, scope, deref, size, time, filter, attrOnly. Rewriting\none last element of a search, the list of attributes to return, is currently\nnot possible, but the feature is on the way.\n\nExamples: production examples used in rewriting ou=dhcp to ou=clients\n\n```\nExample 1:\n\n# Solve lack of flexibility in ISC DHCP3 LDAP patch by\n# plainly specifying ldap-base-dn \"ou=dhcp\" in DHCP's\n# config, and then here, rewriting DHCP ethernet address\n# lookup to the ou=clients tree under which all clients\n# are defined.\n\nsearchSubst  base        \"^ou=dhcp$\"                       \\\n             filter      \"^\\\\(\u0026\\\\(objectClass=dhcpHost\\\\)\\\\(dhcpHWAddress=ethernet [\\\\dabcdef:]+\\\\)\\\\)$\" \\\n             -\u0026gt;                                            \\\n             base   .+   ou=clients\n\n\nExample 2:\n\n# Solve lack of flexibility in ISC DHCP3 LDAP patch by\n# rewriting a search in any shared network, tree\n# ou=dhcp, to a proper location,\n\nsearchSubst  base        \"^ou=\\\\w+,ou=dhcp$\"                \\\n             scope       \"^1$\"                             \\\n             filter      \"^\\\\(objectClass=\\\\*\\\\)$\"         \\\n             -\u0026gt;                                            \\\n             base   .+   \"ou=clients\"                      \\\n             filter .+   \"(\u0026amp;(objectClass=dhcpSubnet)(!(cn=default)))\" \\\n             scope  .+   2\n```\n\n\n## LDAP Operations - Notes\n\n1. BIND operation is supported, but only in a trivial way. Therefore, binding using a DN under this part of DIT is not encouraged nor enabled by default unless `enable_bind 1` is present in the config file. It is expected that users should authenticate using some other suffixes or via e.g. GSSAPI, so that this `bind()` is never called or needed.\n\n1. DELETE operation is supported, but does not receive an indication from OpenLDAP whether subtree delete was requested or not. Therefore, currently the way to control whether a DELETE will delete whole subtrees or refuse to work on non-leaf values is controlled using the config option `deleteTrees`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdocelic%2Fslapd-viper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdocelic%2Fslapd-viper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdocelic%2Fslapd-viper/lists"}