{"id":20265329,"url":"https://github.com/jriguera/confinit","last_synced_at":"2025-04-11T02:32:00.075Z","repository":{"id":64305945,"uuid":"175100692","full_name":"jriguera/confinit","owner":"jriguera","description":"Linux automatic configuration at boot time, Rasberry Pi focused","archived":false,"fork":false,"pushed_at":"2021-12-05T01:00:40.000Z","size":2704,"stargazers_count":5,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-20T07:39:33.232Z","etag":null,"topics":["golang","provisioning","raspberry-pi","rpi","template-engine","yaml-configuration"],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jriguera.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-03-11T23:40:52.000Z","updated_at":"2022-08-23T22:59:10.000Z","dependencies_parsed_at":"2023-01-15T10:30:38.615Z","dependency_job_id":null,"html_url":"https://github.com/jriguera/confinit","commit_stats":null,"previous_names":[],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jriguera%2Fconfinit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jriguera%2Fconfinit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jriguera%2Fconfinit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jriguera%2Fconfinit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jriguera","download_url":"https://codeload.github.com/jriguera/confinit/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248329633,"owners_count":21085571,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["golang","provisioning","raspberry-pi","rpi","template-engine","yaml-configuration"],"created_at":"2024-11-14T11:46:48.392Z","updated_at":"2025-04-11T02:32:00.049Z","avatar_url":"https://github.com/jriguera.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"confinit\n========\n\nBoot configuration management for RaspberryPI and Linux, simple Cloud-Init idea\nbut limited to copy files, render templates and execute commands by cloning a \na source directory structure.\n\nUsage\n=====\n\nComplete example: https://github.com/jriguera/raspbian-cloud/tree/master/stageZZ/99-confinit/config\n\nGiven a source folder structure (with files and sub-folders) the primary goal of\nthis program is replicate the same structure in a destination folder. The program\nwill scan the source folder and apply a list of operations (copy, render, execute)\nin order. Each operation can define types of files/folders to process via regular\nexpressions and apply permissons on the new files.\n\nThere is also a global hooks to define startup and finish scripts.\n\nConfiguration\n-------------\n\nThe configuration is defined in `internal/config/config.go`. You which\nparameters are required and its defaults.\n\nGlobal configuration parameters\n\n```\n#### confinit configuration file ###\n\n# Log ouput can be:\n# * stdout: dump logs to stdout\n# * stderr: write logs to stderr\n# * split: errors to stderr, rest of logs to stdout\n# * - : discard all logs\n# * path/to/file.log: dump logs to file\nlogoutput: split\n\n# Log level defines the verbosity\nloglevel: debug\n\n# File (format json or yaml, by the extension) or HTTP url (GET) with additional\n# variables/structures accessible in templates\ndatafile: conf/data.yml\n\n# Global environment variables accessible to programs and templates. Also the\n# current environment variables are exported, here can be re-defined.\nenv:\n    SYSTEM: raspberry\n    LOCATION: home\n\n# Startup command, non zero exit stops the execution.\n# * timeout: defines how many seconds to wait for the execution (def)\n# * dir: folder where the program will be executed (default is current dir)\n# * env: key/value environment variables.\n# This command can perform operations on the datafile (see above), like\n# getting from a database/url and building it. Data from datafile is loaded\n# after this command runs and before the list of operations.\nstart:\n    cmd: [\"pwd\"]\n    timeout: 60\n    dir: /tmp\n    env:\n        A: a\n        B: b\n\n# Finish command, non zero exit stops the execution.\n# * timeout: defines how many seconds to wait for the execution (def)\n# * dir: folder where the program will be executed (default is current dir)\n# * env: key/value environment variables\n#\n# There are two additional environment variables defined automatically:\n# * CONFINIT_RC_START: stores the exit code of the `start` command.\n# * CONFINIT_RC_PROCESS: stores the exit code of the `process` operations.\n# * CONFINIT_RC_LOAD_DATA: stores an exit code of the result of loading the\n# datafile.\nfinish:\n    cmd: [\"env\"]\n    timeout: 600\n    dir: /tmp\n    env:\n        A: a\n        B: b\n```\n\nProcessing files\n----------------\n\nThe functionality of the program is defined in the field `process`:\n\n```\n# List of source folders. `source` is required and is the folder to clone.\n# Optional filters can be set in `match` field using shell globs format, by\n# default, it allows scanning all folders and files. The list of files is print\n# out in debug `loglevel`. if `excludedone` is true (by default) when one file\n# is processed by one operation, it will be ignored in other operations (is\n# important the order of the operations!).\nprocess:\n  - source: conf/templates\n    excludedone: true\n    match:\n        folder:\n           add: \"*\"\n           skip: \".git\"\n        file:\n           add: \"*\"\n           skip: \".backup\"\n    operations: []\n```\n\nDescriptive examples of operations:\n\n1. Copy all files to a destination (even binaries):\n```\n- destination: /\n  regex: '.*'\n  template: false\n```\n\n2. Render templates, files with extension `.template` (which will be removed at\ndestination, after rendering it because of `delextension: true`). Extra data,\nadded on top of `datafile` setting, will be used to process these templates:\n```\n- destination: /\n  regex: '.*\\.template'\n  template: true\n  delextension: true\n  data:\n    key1: value1\n    key2: []\n    key3:\n       key4: {}\n```\n\n3. Render template to determine what to do with the file (`delete` in this case)\n```\n- destination: /\n  regex: '.*\\.template'\n  template: true\n  condition: '{{if not .Data.iface}} delete {{end}}'\n  delete:\n    ifconfition: true\n```\n\nOther options to echo to `condition` field are (case insesitive, trim spaces):\n\n  * with `\"\"`, `render` or `continue`: continue processing file/template.\n  * case `skip`: stop rendering template file (do not delete destination file if exists).\n  * `delete` or `delete-file` : stop rendering and delete current file (if exists).\n  * `delete-if-empty`: delete the file only if template results in an empty file.\n  * `delete-if-fail`: delete if template calls `fail \"\u003cmsg\u003e\"` function or renderting template fails.\n  * `delete-after-exec`: delete a file (when is a command/script template, see below) after its execution.\n\nThe setting `delete.ifcondition` (default `true`) controls if rendering templates\ncan define delete actions. If is `false` it will NOT render the template if the \ncondition generates an output string, the output script will be used as informational\nmessage in the logs.\n\nApart from the conditional ouput, `delete` parameter operates by its own and has\nthese options with default values:\n\n  * `prestart` (default `false`): always delete the file before processing it.\n  * `ifempty` (default `true`): deletes if renders to an empty file.\n  * `ifconfition` (default `true`): \"do what the condition says\".\n  * `ifrenderfail` (default `true`): delete if the template does not render (or it calls `fail` function).\n  * `afterexec` (default `true`): delete after executing the file (see below).\n\n\n4. Execute shell scripts, but do not copy or render to destination, just execute it from\n   the source. Because there is no `destination`, `template` field is ignored\n   (assumes it is not a template). This is useful to run scripts in the middle\n   of processing other files. Make sure the source scripts are executable.\n   `{{.SourceAbsPath}}` is the absolute path of the source file and in this case\n   delete `afterexec` does not apply.\n```\n- regex: '.*\\.sh'\n  command:\n    cmd: [\"{{.SourceAbsPath}}\"]\n    env:\n      EXTRA_VAR: pepe2\n```\n\n1. Copy and execute shell scripts (no templates, potentially can be binary!),\n   `command.cmd` points to the destination of the file being rendered. It makes\n   sure the generated scripts have execution permissions and it will delete each\n   one after running it.\n```\n- destination: /tmp\n  regex: '.*\\.sh'\n  default:\n    mode:\n      file: \"0755\"\n  template: false\n  command:\n    cmd: [\"{{.Destination}}\"]\n  delete:\n    afterexec: true\n```\n\n1. Render shell script templates (keeping the full extension of the filename: `delextension: false`)\n   and execute each one with extra data `key: value` (to render inside the\n   temaptes) and do not delete them after execution:\n```\n- destination: /tmp\n  regex: '.*\\.sh'\n  delextension: false\n  default:\n    mode:\n      file: \"0755\"\n  template: true\n  command:\n    cmd: [\"{{.Destination}}\"]\n  data:\n    key: value\n  delete:\n    afterexec: false\n```\n\nTemplates\n---------\n\nTemplates are implemented using `golang/text.template`. There is more information\nabout how to write them in the official documentation: https://golang.org/pkg/text/template/\n\nConfinit defines template variables in the file \n[`pkg/fs/actions/templator.go:TemplateData`](https://github.com/jriguera/confinit/blob/master/pkg/fs/actions/templator.go#L105)\n```\n\tIsDir           bool\n\tMode            string\n\tSourceBaseDir   string\n\tSource          string\n\tFilename        string\n\tSourceFile      string\n\tPath            string\n\tSourceFullPath  string\n\tSourceAbsPath   string\n\tSourcePath      string\n\tExt             string\n\tDstBaseDir      string\n\tDestination     string\n\tDestinationPath string\n\tData            interface{}\n\tEnv             map[string]string\n```\n\nSo, for example, in order to get a variable defined in `datafile` you have \ndefine `{{ .Data.VARIABLE }} and to get the destination path of the current template\n`{{ .Destination }}`. Those getters can also be used in the `condition` parameter.\n\nThere are a lot of template functions defined in the file \n[`pkg/tplfunctions/tfunctions.go`](https://github.com/jriguera/confinit/blob/master/pkg/tplfunctions/tfunctions.go)\nready to be used in template files, for example:\n\n```\nUUID: {{ uuid }}\nrandomString: {{ randomString 6 }}\nrandominteger: {{ random \"0123456789\" 6 }}\nenv: {{ .Env.EEEE  }}\nenv: {{ env \"EEEE\" }}\nnow: {{ now | date \"2006-01-02\" }}\nnow: {{ now | epoch }}\nremove_spaces ({{ .Data.D }}): {{ trim .Data.D }}\n```\n\n\nDevelopment\n===========\n\nGolang 1.14 . There is a `Makefile` to manage the development actions, releases\nand binaries. `make build` generates binaries for: `linux-amd64`, `linux-arm-6`,\n`linux-arm-7` and `make deb` generates debian packages for `deb-amd64`, `deb-armhf`\n\nGolang Modules\n--------------\n\nGo 1.14 has a feature `vgo` which will replace `dep`. To use `vgo`,\nsee https://github.com/golang/go/wiki/Modules.\n\nTLDR below:\n\n```\nexport GO111MODULE=on\ngo mod init         # If you are not using git, type `go mod init $(basename `pwd`)`\ngo mod vendor       # if you have vendor/ folder, will automatically integrate\ngo build\n```\n\nThis method creates a file called `go.mod` in your projects directory. You can\nthen build your project with `go build`. If `GO111MODULE=auto` is set, then your\nproject cannot be in `$GOPATH`\n\n\nDebian package\n--------------\n\nThe usual call to build a binary package is `dpkg-buildpackage -us -uc`.\nYou might call debuild for other purposes, like `debuild clean` for instance.\n\n```\n# -us -uc skips package signing.\ndpkg-buildpackage -rfakeroot -us -uc\n```\n\nAuthor\n======\n\n(c) 2019,2020 Jose Riguera\n\nApache 2.0\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjriguera%2Fconfinit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjriguera%2Fconfinit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjriguera%2Fconfinit/lists"}