{"id":17680334,"url":"https://github.com/docelic/viper","last_synced_at":"2025-05-12T23:10:04.323Z","repository":{"id":631115,"uuid":"271880","full_name":"docelic/Viper","owner":"docelic","description":"Fully Automated Installation and Configuration for Debian GNU based systems","archived":false,"fork":false,"pushed_at":"2018-08-26T19:03:12.000Z","size":775,"stargazers_count":10,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-05-12T23:09:57.436Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Perl","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/docelic.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2009-08-07T17:45:21.000Z","updated_at":"2020-02-23T21:05:17.000Z","dependencies_parsed_at":"2022-08-16T10:31:23.446Z","dependency_job_id":null,"html_url":"https://github.com/docelic/Viper","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/docelic%2FViper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/docelic%2FViper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/docelic%2FViper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/docelic%2FViper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/docelic","download_url":"https://codeload.github.com/docelic/Viper/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253837452,"owners_count":21971984,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-24T09:06:33.714Z","updated_at":"2025-05-12T23:10:04.291Z","avatar_url":"https://github.com/docelic.png","language":"Perl","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Viper\n\nViper is a system for fully automated installation, configuration and monitoring of Debian GNU based systems. The development started in 2008.\n\nViper uses a modern approach (debian-installer during installation, configuration management tool later, etc.), works with existing Debian packages, and does not require any custom patching of either install media, client, or server.\n\nThe basis of the whole system is an OpenLDAP server with a custom Viper backend with radically new and dynamic functionality, and holding all configuration data.\n\n## Introduction\n\nLDAP has the potential to be enormously successful in configuration management, but its flat data structure, absence of any kind of dynamic behavior, and heaps of duplicated data typically make it an inefficent tool for smaller teams.\n\nUsing LDAP for complex configuration management in a seamless, successful, non-intrusive way has been reserved for high-end customers who have invested significant resources into making it work as desired in their environments.\n\nViper was developed to build on LDAP's great strengths while solving many of its typical deficiencies when it comes to configuration management.\n\nAs part of Viper, a custom OpenLDAP backend has been implemented. This backend should be used specifically for configuration management and not other generic data.\n\nFinally, everything has been commented, with the availability of complete, working config files and this guide.\n\nViper is released under GNU GPL v3 or later.\n\n## Features Overview\n\nHere follows a summarized list of Viper's key features, especially those that may differ from traditional config management or LDAP principles. (The explanations on how or why Viper does what it does can be found in other sections below; this section focuses only the high-level summary of user functionality.)\n\nViper uses on Debconf, OpenLDAP, ISC DHCP, and a software configuration management tool (such as Puppet) for the complete setup.\n\nOne Viper server instance can store and serve data for multiple separate customers, with multiple separate machines each.\n\nViper \"rewrites\" incoming LDAP searches in numerous ways, completely avoiding duplication of data, while at the same time transparently handling all clients who issue their standard LDAP queries and expect to find information in their standard locations.\n\nWith the help of a CGI script, Viper also handles clients who don't use LDAP searches but request data over HTTP, such as clients requesting preseed data at the beginning of machine installation.\n\nAfter the machine has been initially installed, machines' Debconf settings (as well as config management software's settings) are configured to use LDAP and query the Viper server for information.\n\nAll Viper's configuration-related key=value pairs are stored in Debconf-conformant LDAP schema (objectClass = `debconfDbEntry`). This makes it possible to treat all keys equally, regardless of whether they are part of the mandatory preseed contents, or non-essential Debconf questions, or custom additional configuration data added by site administrators.\n\nThe usual (non-Viper) way for system administrators to preseed Debconf in advance is to extract the packages' Debconf question templates, answer them, and save them to the Debconf database, so that the questions and answers are already there when the clients ask for them. However, extracting and answering Debconf questions ahead of time is quite a nuisance, especially when the packages or questions needed are not known in advance or there are many dependencies to handle as well.\n\nViper makes it possible to answer all known/existing questions automatically as usual, while all new/unseen questions are either shown interactively at the machine being installed, or they can open a Debconf popup window on the administrator's remote host of choice. This way, new questions can be answered from a central location regardless of where on the network a package is being installed, and then the answers can be saved to Debconf as expected for all future queries.\n\nFurthermore, for every Debconf question that is asked (locally or remotely), an additional follow-up question is presented to the administrator, allowing him to insert the answer in LDAP at the individual machine level (applying only to that machine), at the customer level (applying to all customer's machines), or at the global level (applying to all machines of all customers).\n\n## Server Choice Considerations\n\nViper ships with the mentioned custom backend for OpenLDAP and it must be installed for Viper to work. This backend is the primary component of Viper and it implements numerous features that make Viper an extremely powerful tool for configuration management.\n\nPlease note that Viper's OpenLDAP backend is implemented in Perl and it depends on OpenLDAP's `slapd-perl`, which is subject to a couple important notes:\n\n1. OpenLDAP's `slapd-perl` never had a particularly full-featured implementation, and it must be enabled with a config option during OpenLDAP build\n1. While Viper's backend is a full-fledged piece of software, it is bound by limitations of `slapd-perl`\n1. The two primary limitations of `slapd-perl` are 1) very modest access controls, and 2) incomplete API which makes most of slapd's data not directly accessible to Perl (thus requiring some duplication of configuration)\n\nGiven's Viper very specific and contextualized purpose, for maximum convenience the default Viper config files also reinstate the old and simple `slapd.conf` style of configuration instead of the new (and much more complex) `cn=config` style.\n\nAnd finally, necessary to mention here is that, by default, the install clients will ask for preseed data over unencrypted HTTP connections.\n\nTherefore, all things considered, we advise to only run Viper servers on dedicated machines and inside local/trusted networks, unless you first read this guide in entirety and possibly send us patches to make Viper \"thougher\" and suitable for other environments.\n\n## First Steps\n\nThe easiest way to start with Viper is to clone the files from Git and place them in `/etc/ldap/viper/`:\n\n```\nmkdir -p /etc/ldap\ncd /etc/ldap\ngit clone git://github.com/crystallabs/Viper.git viper\ncd viper\n```\n\n## Net::LDAP::FilterMatch.pm Fix\n\nIn Perl module Net::LDAP prior to version 0.4001 (2010 and earlier) there was a [bug in FilterMatch escaping](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=540938).\nIf you happen to be using one of those older versions, please upgrade or manually apply patch [support/FilterMatch.pm.patch](https://github.com/crystallabs/Viper/blob/master/support/FilterMatch.pm.patch).\n\n## Installation Procedure\n\nThere is a simple shell script [scripts/viper-setup.sh](https://github.com/crystallabs/Viper/blob/master/scripts/viper-setup.sh) delivered with Viper which contains the list of steps to be performed on a server machine to install Viper. After a mandatory manual review you could run this script to perform the installation. The script is idempotent; running it multiple times will result in no adverse effects, so in case anything fails, you can resolve the conditions and then run the script again.\n\n*Please note that the script will, by default, overwrite your existing slapd, dhcpd, and puppet config files.*\n\nIn summary, to perform the installation, you will:\n\n1. Find a suitable Devuan GNU+Linux, Debian GNU, or Ubuntu machine to use as Viper install server\n1. Configure one network interface (can be an alias) to listen on 10.0.1.1/24: `ifconfig eth0:1 inet 10.0.1.1 netmask 255.255.255.0`. This must be done to load test data and have DHCP server start properly with default config\n1. Install required packages: `slapd ldap-utils libfile-find-rule-perl libnet-ldap-perl libtext-csv-xs-perl liblist-moreutils-perl isc-dhcp-server-ldap make sudo libyaml-perl apache2`\n1. Verify and run script `scripts/viper-setup.sh`, or manually execute lines from it\n1. Make final adjustments to config files if necessary\n1. Restart OpenLDAP\n1. Review and/or modify LDIF data and load it into the server\n1. Restart ISC DHCP\n1. Run a couple tests to verify successful installation\n\nMore detailed descriptions of some of these steps follow:\n\n### Required Package Installations\n\nThe packages needed to run Viper have been listed above, repeated here for completeness.\n\n```\nslapd ldap-utils libfile-find-rule-perl libnet-ldap-perl libtext-csv-xs-perl liblist-moreutils-perl isc-dhcp-server-ldap make sudo libyaml-perl apache2\n```\n\nThe HTTP server is included in the list because Viper uses a simple CGI script to provide preseed data via HTTP for client machines during installation (queried automatically from debian-installer). Our example uses Apache, even though any web server that can execute CGI will do well.\n\nWhen installing OpenLDAP, answer \"Yes\" to the debconf question \"Omit OpenLDAP server configuration?\", because the complete config file will come supplied from Viper, and you can further tune it from there if desired.\n\nIt is known and expected that OpenLDAP and DHCP servers will not start cleanly during the `apt install` step, due to incomplete configuration. The appropriate config files for both services will be installed later from Viper's templates.\n\n### Viper's Etc Config Files\n\nViper does not intend to unconditionally or inflexibly replace your services' config files with its own. You can just as easily manually modify any of your existing services' config files to insert parts of configurations needed by Viper.\n\nHowever, it is generally assumed that you will dedicate a physical or virtual machine to the Viper server, and in that context, Viper's default config files can easily replace the services' ones. That will provide you with a known-good setup on which you can run the test suite and to which you can add your own configuration. \n\n`viper-setup.sh` will copy all the required files from `$VIPER_ROOT/etc/` (which is usually `/etc/ldap/viper/etc`) into the system's `/etc/` and overwrite any existing files.\nCurrently, this includes config files for OpenLDAP, ISC DHCP, and Puppet.\n\n### Viper LDIFs\n\n`viper-setup.sh` will also load the necessary bootstrap LDIFs into LDAP as part of the installation procedure. It will do that by invoking `make` in the directory `ldifs/`.\n\nThe LDIF contents are supposed to load into the custom Viper backend for OpenLDAP, which comes pre-configured in Viper's OpenLDAP config files.\nTherefore, be sure to restart OpenLDAP with the new configuration before loading the LDIFs.\n\nLDIFs are loaded by simply running `make` in the directory `ldifs/`. The Makefile, in turn, runs `sh add`, which finds all `*.ldif` files in the directory and loads them using `ldapadd` and the preconfigured bind credentials.\n\nOnce LDIFs are loaded, please restart the ISC DHCP server.\n\n### Preseed CGI Script\n\nViper ships with `scripts/preseed`, a CGI script used for providing preseed data to Debian-based clients. The preseed format is natively supported by Debian installer, requiring no modifications on the client. On incoming requests, the script runs and uses the client's IP address to search for matching `dhcpHWAddress` and `ipHostNumber` in the LDAP directory. Then it compiles the preseed file contents and delivers them back to the client.\n\nFor maximum convenience and manual testing, the preseed data for a client can also be obtained by accessing the CGI script with query parameter `host=HOSTNAME`, and additionally with `client=DOM.AIN` if the LDAP search for hostname alone would find more than one result.\n\nDuring our testing, it was determined that preseeding the clients during installation with all existing Debconf answers (instead of only those required by the installation) leads to unusual problems. Therefore, the preseed CGI script only returns the Debconf entries marked for inclusion in preseeds, but all keys can be requested with a query parameter `flag=` or alternatively `debug=1`.\n\n## Testing the Installation\n\nThe installation procedure hopefully finished without giving any errors, and the LDIFs were loaded.\n\nAt this point, to just do a quick and immediate test, run `ldapsearch -x`. If this prints various LDAP entries to screen, you have a working server installation.\n\nSome default data is included with the installation. That includes a \"customer\" with name \"c1.com\", and three hosts, h1, h2 and h3.\n\nAltogether, you can run the following additional ldapsearches for more testing:\n\n```\nldapsearch -x -b ou=dhcp\nldapsearch -x -b ou=defaults\nldapsearch -x -b ou=clients\n```\n\n### Testing with ldapsearch\n\nLdapsearch query for `cn=h2,ou=hosts,o=c1.com,ou=clients` is a pretty good way of determining if everything is working alright. Here's how the output from the command should look like. The exact attribute values are not important, it is just important that there are no unprocessed values in the output. That is, nothing with '$' and nothing with only half-populated information.\n\n```\n$ ldapsearch -x -b cn=h2,ou=hosts,o=c1.com,ou=clients\n\n# extended LDIF\n#\n# LDAPv3\n# base \u003ccn=h2,ou=hosts,o=c1.com,ou=clients\u003e with scope subtree\n# filter: (objectclass=*)\n# requesting: ALL\n#\n\n# h2, hosts, c1.com, clients\ndn: cn=h2,ou=hosts,o=c1.com,ou=clients\nobjectClass: top\nobjectClass: device\nobjectClass: dhcpHost\nobjectClass: ipHost\nobjectClass: ieee802Device\nobjectClass: x-puppetClient\ncn: h2\nipHostNumber: 10.0.1.8\nmacAddress: 00:11:6b:34:ae:8d\npuppetclass: test\npuppetclass: ntp::server\n\n# search result\nsearch: 2\nresult: 0 Success\n\n# numResponses: 2\n# numEntries: 1\n```\n\n### Testing with scripts/node_data\n\n```\nperl scripts/node_data h2.c1.com\n```\n\n### Testing with scripts/preseed\n\n```\nQUERY_STRING=ip=10.0.1.8 perl scripts/preseed\n```\n\n### Testing with HTTP client\n\n```\nwget http://10.0.1.1/cgi-bin/preseed.cfg?ip=10.0.1.8 -O /tmp/preseed.cfg\n```\n\n## Troubleshooting\n\nThe following two things should be done as a pre-requirement for any quick troubleshooting:\n\n1. Tune the desired internal debug options in Viper. This is done in `/etc/ldap/viper/Viper.pm` (search for \"DEBUG\")\n\n1. Run `slapd` in foreground and watch its logs. We run with slapd debug\nlevel 0 because most of the time we are interested in seeing Viper's internal\nlogs and we don't want the output to include slapd's own logs.\n\n```\nsudo /usr/sbin/slapd -h 'ldap:/// ldapi:///' -g openldap -u openldap -f /etc/ldap/slapd.conf -d 0\n```\n\n1. In addition, tail essential log files:\n\n```\nsudo tail -f /var/log/syslog /var/log/dhcp-ldap-startup.log\n```\n\n### Troubleshooting DHCP Server\n\nDHCP server will issue the equivalent of the following LDAP searches upon startup:\n\n```\nldapsearch -a never -b ou=dhcp -s sub -x \"(\u0026(objectClass=dhcpServer)(cn=HOSTNAME))\"\n```\n\nThere are many DHCP-related problems that can come up, preventing the server from starting.\n\nA good rule of thumb is to make sure that there is at least some network interface configured with the IP 10.0.1.1/24. This is one of test subnets loaded along with test data, and it should allow the DHCP server to start.\n\nAlso, when adding new clients, it is mandatory to create the network interface or alias and set it to an IP from the desired subnet before adding the new client and DHCP block to LDAP and restarting DHCP.\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdocelic%2Fviper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdocelic%2Fviper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdocelic%2Fviper/lists"}