{"id":17623411,"url":"https://github.com/voxpupuli/puppet-filemapper","last_synced_at":"2025-12-11T21:47:40.354Z","repository":{"id":3596057,"uuid":"4660135","full_name":"voxpupuli/puppet-filemapper","owner":"voxpupuli","description":"Map files to puppet resources and back","archived":false,"fork":false,"pushed_at":"2024-09-16T18:42:27.000Z","size":369,"stargazers_count":14,"open_issues_count":6,"forks_count":24,"subscribers_count":36,"default_branch":"master","last_synced_at":"2024-10-29T14:22:35.671Z","etag":null,"topics":["hacktoberfest","puppet"],"latest_commit_sha":null,"homepage":"https://forge.puppet.com/puppet/filemapper","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/voxpupuli.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":".github/CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"open_collective":"vox-pupuli","github":"voxpupuli"}},"created_at":"2012-06-14T06:39:27.000Z","updated_at":"2024-07-19T21:29:23.000Z","dependencies_parsed_at":"2024-04-15T06:32:21.547Z","dependency_job_id":"fd541af8-c797-41b2-9b35-7bac20e357ef","html_url":"https://github.com/voxpupuli/puppet-filemapper","commit_stats":{"total_commits":209,"total_committers":19,"mean_commits":11.0,"dds":0.5837320574162679,"last_synced_commit":"54c22a424b73296cd562a123380b68b740691da2"},"previous_names":["adrienthebo/puppet-filemapper"],"tags_count":19,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/voxpupuli%2Fpuppet-filemapper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/voxpupuli%2Fpuppet-filemapper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/voxpupuli%2Fpuppet-filemapper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/voxpupuli%2Fpuppet-filemapper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/voxpupuli","download_url":"https://codeload.github.com/voxpupuli/puppet-filemapper/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244722826,"owners_count":20499173,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["hacktoberfest","puppet"],"created_at":"2024-10-22T21:09:07.372Z","updated_at":"2025-12-11T21:47:35.329Z","avatar_url":"https://github.com/voxpupuli.png","language":"Ruby","funding_links":["https://opencollective.com/vox-pupuli","https://github.com/sponsors/voxpupuli"],"categories":[],"sub_categories":[],"readme":"# FileMapper module for Puppet\n\n[![Build Status](https://travis-ci.org/voxpupuli/puppet-filemapper.png?branch=master)](https://travis-ci.org/voxpupuli/puppet-filemapper)\n[![Code Coverage](https://coveralls.io/repos/github/voxpupuli/puppet-filemapper/badge.svg?branch=master)](https://coveralls.io/github/voxpupuli/puppet-filemapper)\n[![Puppet Forge](https://img.shields.io/puppetforge/v/puppet/filemapper.svg)](https://forge.puppetlabs.com/puppet/filemapper)\n[![Puppet Forge - downloads](https://img.shields.io/puppetforge/dt/puppet/filemapper.svg)](https://forge.puppetlabs.com/puppet/filemapper)\n[![Puppet Forge - endorsement](https://img.shields.io/puppetforge/e/puppet/filemapper.svg)](https://forge.puppetlabs.com/puppet/filemapper)\n[![Puppet Forge - scores](https://img.shields.io/puppetforge/f/puppet/filemapper.svg)](https://forge.puppetlabs.com/puppet/filemapper)\n\n## Synopsis\n\nMap files to resources and back with this handy dandy mixin!\n\n## Description\n\nThings that are harder than they should be:\n\n* Acquiring a pet monkey\n* Getting anywhere in Los Angeles\n* Understanding the ParsedFile provider\n* Writing Puppet providers that directly manipulate files\n\nThe solution for this is to completely bypass parsing in any sort of base\nprovider, and delegate the role of parsing and generating to including classes.\n\nYou figure out how to parse and write the file, and this will do the rest.\n\n## Synopsis of implementation requirements\n\nProviders using the Filemapper extension need to implement the following\nmethods.\n\n### `self.target_files`\n\nThis should return an array of filenames specifying which files should be\nprefetched.\n\n### `self.parse_file(filename, file_contents)`\n\nThis should take two values, a string containing the file name, and a string\ncontaining the contents of the file. It should return an array of hashes,\nwhere each hash represents {property =\u003e value} pairs.\n\n### `select_file`\n\nThis is a provider instance method. It should return a string containing the\nfilename that the provider should be flushed to.\n\n### `self.format_file(filename, providers)`\n\nThis should take two values, a string containing the file name to be flushed,\nand an array of providers that should be flushed to this file. It should return\na string containing the contents of the file to be written.\n\n## Synopsis of optional implementation hooks\n\n### `self.pre_flush_hook(filename)` and `self.post_flush_hook(filename)`\n\nThese methods can be implemented to add behavior right before and right after\nfilesystem operations. Both methods take a single argument, a string\ncontaining the name of the file to be flushed.\n\nIf `self.pre_flush_hook` raises an exception, the flush will not occur and the\nprovider will be marked as failed and will refuse to perform any more flushes.\nIf some sort of critical error occurred, this can force the provider to error\nout before it starts stomping on files.\n\n`self.post_flush_hook` is guaranteed to run after any filesystem operations\noccur. This can be used for recovery if something goes wrong during the flush.\nIf this method raises an exception, the provider will be marked as failed and\nwill refuse to perform any more flushes.\n\n## Removing empty files\n\nIf a file is empty, it's often reasonable to just delete it. The Filemapper\nmixin implements `attr_accessor :unlink_empty_files`. If that value is set to\ntrue, then if `self.format_file` returns the empty string then the file will be\ndeleted from the file system.\n\n## How it works\n\n[transaction]: http://somethingsinistral.net/blog/reading-puppet-the-transaction/\n\nThe Filemapper extension takes advantage of hooks within the\n[Transaction][transaction] to reduce the number of reads and writes needed to\nperform operations.\n\n### prefetching\n\nWhen a catalog is being applied, providers can define the `prefetch` method to\nload all resources before runtime. The Filemapper extension uses this method to\npreemptively read all files that the provider requires, and generates and stores\nthe state of the requested resources. This means that if you have a few thousand\nresources in 20 files, you only need to do 20 reads for the entire Puppet run.\n\n### post-evaluation flushing\n\nWhen resources are normally evaluated, each time a property is synchronized it's\nexpected that an action will be run right then. The Filemapper extension instead\nrecords all the requested changes and defers operating on them. When the\nresource is finished, it will be flushed, at which time all of the requested\nchanges will be applied in one pass. Given a resource with 10 properties, all of\nwhich are out of sync, the file will be written only once. If no properties are\nout of sync, the file will be untouched.\n\nTo ensure that the system state matches what Puppet thinks is going on, any file\nthat has changed resources will be re-written after each resource is flushed.\nThat means that if you have 20 resources out of sync, that file will have to be\nwritten 20 times. While it's technically possible to write the file in a single\npass, this means that some resources will be applied either early or late, which\nutterly smashes POLA.\n\n### Use on the command line\n\nThe Filemapper extension implements the `instances` method, which means that you\ncan use the `puppet resource` command to interact with the associated provider\nwithout having to perform a full blown Puppet run.\n\n### Selecting files to load\n\nIn order to provide prefetching and `puppet resource` in a clean manner, the\nFilemapper extension has to have a full list of what files to read. Implementing\nclasses need to implement the `target_files` method which returns a list of\nfiles to read. The implementation is entirely up to the implementing class; it\ncan return a single file every time, such as \"/etc/inittab\", or it can generate\nthat information on the fly, by returning `Dir[\"/etc/sysconfig/network/ifcfg-*\"]`.\nBasically, files that will be used as a source of data can be as complex or\nsimple as you need.\n\n### Writing back files\n\nIn a similar vein, resources can be written back to files in whatever method you\nneed. Implementing classes need to implement the *instance method* `#select_file`\nso that when that resource is changed, the correct file is modified.\n\n### Parsing\n\nWhen parsing a file, the implementing class needs to implement the `parse_file`\nmethod. It will get the name of the file being parsed as well as the contents.\nIt can parse this file in whatever manner needed, and should return an array of\nany provider instances generated. If the file only contains a single provider\ninstance, then just wrap that instance in an array.\n\n### Writing\n\nWhenever a file is marked as dirty, that is a resource associated with that file\nhas changed, the `format_file` method will be called. The implementing class\nneeds to implement a method that takes the filename and an array of provider\ninstances associated with that file, and return a string. The method needs to\ndetermine how that file should be written to disk and then return the contents.\nThis can be as complex as needed.\n\nUnder no conditions should implementing classes modify any files directly. No,\nseriously, don't do it. The Filemapper extension uses the built in methods for\nmodifying files, which will back up changed files to the filebucket. This is for\nyour own safety, so if you bypass this then you are on your own.\n\n### Storing state outside of resources\n\nIt's more or less expected that there will be no state outside of the provider\ninstances, but there are plenty of cases where this could be the case. For\ninstance, if one wanted to preserve the comments in a file but didn't directly\nassociate them with resource attributes, the `parse_file` method can store data\nin an instance variable, such as `@comments = my_list_of_comments`. When\nformatting the file, the implementing class can read the `@comments` variable\nand re-add that data to the content that will be written back.\n\nBasically, you can store whatever data you need in these methods and pass things\naround to maintain more complex state.\n\nUsing this sort of operation of reading outside state, you can theoretically\nhave multiple Filemapper extensions that work on shared files. By communicating\nthe state between them, you can manage multiple different resources in one file.\n**HOWEVER**, this will require careful communication, so don't take this sort of\nthing lightly. However, I don't thing that anything else in Puppet can provide\nthis sort of behavior. YMMV.\n\n### Why a mixin?\n\nWhile the ParsedFile provider is supposed to be inherited, this class is a mixin\nand needs to be included. This is done because the Filemapper extension only\n*adds* behavior, and isn't really an object or entity in its own right. This way\nyou can use the Filemapper extension while inheriting from something like the\nPuppet::Provider::Package provider.\n\n## The Backstory\n\nManaging Unix-ish systems generally means dealing with one of two things:\n\n  1. Processes - starting them, stopping them, monitoring them, etc.\n  1. Files - Creating them, editing, deleting them, specifying permissions, etc.\n\nPuppet has pretty good support in the provider layer for running commands, but\nthe file manipulation layer has been lacking. The long-standing approach for\nmanipulating files has been to select one of the following, and hope for the best.\n\n### Shipping flat files to the client\n\nUsing the `File` resource to ship flat files is a really common solution, and\nit's very easy. It also has the finesse of a brick thrown through a window.\nThere is very little customizability here, aside from the array notation for\n[specifying the `source` field](http://docs.puppetlabs.com/references/latest/type.html#file).\n\n### Using ERB templates to customize files\n\nThe File resource can also take a content field, to which you can pass the\noutput of a template. This allows more sophistication, but not much. It also\nadds more of a burden to your master; template rendering happens on the master\nand if you're doing really crazy number crunching then this pain will be\ncentralized.\n\n### Using Augeas\n\nAugeas is a very powerful tool that allows you to manipulate files, and the\n`Augeas` type allows you to harness this inside of Puppet. However, it has a\nrather byzantine syntax, and is dependent on lenses being available.\n\n### Sed\n\nI personally love sed, but sed a file configuration management tool is not.\n\n### Using the ParsedFile provider\n\n[parsedfile]: https://github.com/puppetlabs/puppet/blob/2.7.19/lib/puppet/provider/parsedfile.rb \"Puppet 2.7.19 - ParsedFile provider\"\n\nPuppet has a provider extension called the [ParsedFile provider][parsedfile]\nthat's used to manipulate text like crontabs and so forth. It also uses a number\nof advanced features in puppet, which makes it quite powerful. However, it's\nincredibly complex, tightly coupled with the FileParsing utility language, has\ntons of obscure and undocumented hooks that are the only way to do complex\noperations, and is entirely record based which makes it unsuitable for managing\nfiles that have complex structure. While it has basic support for managing\nmultiple files, *basic* is the indicative word.\n\n- - -\n\nThe Filemapper extension has been designed as a lower level alternative\nto the ParsedFile.\n\n## Examples\n\n[puppet-network]: https://github.com/voxpupuli/puppet-network\n\nThe Filemapper extension was largely extracted out of the [puppet-network][puppet-network]\nmodule. That code base should display the weird edge cases that this extension\nhandles.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvoxpupuli%2Fpuppet-filemapper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvoxpupuli%2Fpuppet-filemapper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvoxpupuli%2Fpuppet-filemapper/lists"}