{"id":19345916,"url":"https://github.com/tolitius/cprop","last_synced_at":"2025-04-07T23:10:45.816Z","repository":{"id":29350782,"uuid":"32884994","full_name":"tolitius/cprop","owner":"tolitius","description":"likes properties, environments, configs, profiles.. ","archived":false,"fork":false,"pushed_at":"2024-03-05T23:47:27.000Z","size":246,"stargazers_count":357,"open_issues_count":15,"forks_count":25,"subscribers_count":10,"default_branch":"master","last_synced_at":"2025-03-31T22:22:28.629Z","etag":null,"topics":["clojure","configuration"],"latest_commit_sha":null,"homepage":null,"language":"Clojure","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"epl-1.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tolitius.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-03-25T19:03:32.000Z","updated_at":"2025-03-23T05:58:21.000Z","dependencies_parsed_at":"2024-02-26T01:53:29.866Z","dependency_job_id":"574a6c51-611b-4680-99b6-7c9f01bc1d49","html_url":"https://github.com/tolitius/cprop","commit_stats":{"total_commits":143,"total_committers":14,"mean_commits":"10.214285714285714","dds":"0.18881118881118886","last_synced_commit":"87ab2a071c8aac262234df92443e8260655392a7"},"previous_names":[],"tags_count":13,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tolitius%2Fcprop","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tolitius%2Fcprop/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tolitius%2Fcprop/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tolitius%2Fcprop/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tolitius","download_url":"https://codeload.github.com/tolitius/cprop/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247744335,"owners_count":20988783,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["clojure","configuration"],"created_at":"2024-11-10T04:08:21.473Z","updated_at":"2025-04-07T23:10:45.784Z","avatar_url":"https://github.com/tolitius.png","language":"Clojure","funding_links":[],"categories":[],"sub_categories":[],"readme":"# cprop\n\nwhere all configuration properties converge\n\n[![\u003c! build-status](https://travis-ci.org/tolitius/cprop.svg?branch=master)](https://travis-ci.org/tolitius/cprop)\n[![\u003c! release](https://img.shields.io/badge/dynamic/json.svg?label=release\u0026url=https%3A%2F%2Fclojars.org%2Fcprop%2Flatest-version.json\u0026query=version\u0026colorB=blue)](https://github.com/tolitius/cprop/releases)\n[![\u003c! clojars](https://img.shields.io/clojars/v/cprop.svg)](https://clojars.org/cprop)\n\n###### _any_ questions or feedback: [clojurians slack](https://clojurians.slack.com/messages/clojure/) \u003cimg src=\"doc/img/slack-icon.png\" width=\"15px\"\u003e (or just [open an issue](https://github.com/tolitius/cprop/issues))\n\n- [Why](#why)\n- [What does cprop do?](#what-does-cprop-do)\n- [Loading Config](#loading-config)\n  - [Default](#default)\n  - [Loading from \"The Source\"](#loading-from-the-source)\n- [Using properties](#using-properties)\n- [Merging Configurations](#merging-configurations)\n  - [Merging with all System and ENV](#merging-with-all-system-and-env)\n  - [Override all configs](#override-all-configs)\n- [Merging with system properties](#merging-with-system-properties)\n  - [System properties cprop syntax](#system-properties-cprop-syntax)\n- [Merging with ENV variables](#merging-with-env-variables)\n  - [Default](#default-1)\n  - [Speaking ENV variables](#speaking-env-variables)\n    - [Structure and keywords](#structure-and-keywords)\n    - [Types](#types)\n  - [Merging ENV example](#merging-env-example)\n- [Merging with .env file](#merging-with-env-file)\n- [Merging with property files](#merging-with-property-files)\n  - [Property files syntax](#property-files-syntax)\n- [Read \"as is\" (not EDN)](#read-as-is-not-edn)\n- [Customizing key path parsing](#customizing-key-path-parsing)\n- [Cursors](#cursors)\n  - [Composable Cursors](#composable-cursors)\n- [Tools](#tools)\n  - [Translating EDN](#translating-edn)\n    - [EDN to .properties](#edn-to-properties)\n    - [EDN to .env](#edn-to-env)\n- [Tips](#tips)\n  - [Setting the \"conf\" system property](#setting-the-conf-system-property)\n  - [See which files were loaded and what properties were substituted](#see-which-files-were-loaded-and-what-properties-were-substituted)\n  - [Convert properties to one level map](#convert-properties-to-a-one-level-map)\n\n## Why\n\nthere are several env/config ways, libraries.\n\n* some are _solely_ based on ENV variables exported as individual properties: 100 properties? 100 env variables exported..\n* some rely on a property file within the classpath: all good, but requires wrestling with uberjar (META-INF and friends)\n* some allow _only_ string values: no data structures, no numbers, etc.? (I love my data structures and the power of EDN)\n* some allow no structure / hierarchy, just one (top) level pile of properties\n* some keep a global internal config state, which makes it hard to have app (sub) modules with separate configs\n\n## What does cprop do?\n\n* loads an [EDN](https://github.com/edn-format/edn) config from a classpath and/or file system\n* merges it with system properties and ENV variables + the optional merge from sources (file, db, mqtt, http, etc.)\n* returns an (immutable) map\n* while keeping _no internal state_ =\u003e different configs could be used within the same app, i.e. for app sub modules\n\n## Loading Config\n\n```clojure\n(require '[cprop.core :refer [load-config]])\n\n(load-config)\n```\n\ndone.\n\n### Default\n\nBy default `cprop` would look in two places for configuration files:\n\n* classpath: for the `config.edn` resource\n* file system: for a path identified by the `conf` system property\n\nIf both are there, they will be merged. A `file system` source would override matching properties from a `classpath` source,\nand the result will be [merged with System properties](README.md#merging-with-system-properties)\nand then [merged with ENV variables](README.md#merging-with-env-variables)\nfor all the _matching_ properties.\n\ncheck out [cprop test](test/cprop/test/core.cljc) to see `(load-config)` in action.\n\n### Loading from \"The Source\"\n\n`(load-config)` optionaly takes `:resource` and `:file` paths that would override the above defaults.\n\n```clojure\n(load-config :resource \"path/within/classpath/to-some.edn\")\n```\n\n```clojure\n(load-config :file \"/path/to/another.edn\")\n```\n\nthey can be combined:\n\n```clojure\n(load-config :resource \"path/within/classpath/to-some.edn\"\n             :file \"/path/to/another.edn\")\n```\n\nas in the case with defaults, file system properties would override matching classpath resource ones.\n\n## Using properties\n\n`(load-config)` function returns a Clojure map, while you can create [cursors](README.md#cursors), working with a config is no different than just working with a map:\n\n```clojure\n{:datomic\n    {:url \"datomic:sql://?jdbc:postgresql://localhost:5432/datomic?user=datomic\u0026password=datomic\"}\n :source\n    {:account\n        {:rabbit\n           {:host \"127.0.0.1\"\n            :port 5672\n            :vhost \"/z-broker\"\n            :username \"guest\"\n            :password \"guest\"}}}\n :answer 42}\n```\n\n```clojure\n(require '[cprop.core :refer [load-config]])\n```\n```clojure\n(def conf (load-config))\n\n(conf :answer) ;; 42\n\n(get-in conf [:source :account :rabbit :vhost]) ;; \"/z-broker\"\n```\n\n## Merging Configurations\n\nBy default `cprop` will merge all configurations it can find in the following order:\n\n1. classpath resource config\n2. file on a file system (pointed by a `conf` system property or by `(load-config :file \u003cpath\u003e)`)\n3. custom configurations, maps from various sources, etc.\n4. System properties\n5. ENV variables\n\n`#1` and `#2` are going to always be merged by default.\n\nFor `#3` `(load-config)` optionally takes a sequence of maps (via `:merge`) that will be merged _after_ the defaults and in the specified sequence:\n\n```clojure\n(load-config :merge [{:datomic {:url \"foo.bar\"}}\n                     {:some {:other {:property :to-merge}}}])\n```\n\nthis will merge default configurations from a classpath and a file system with the two maps in `:merge` that would override the values that match the existing ones in the configuraion.\n\nSince `:merge` just takes maps it is quite flexible:\n\n```clojure\n(require '[cprop.source :refer [from-file\n                                from-resource]])\n```\n\n```clojure\n(load-config :merge [{:datomic {:url \"foo.bar\"}}\n                     (from-file \"/path/to/another.edn\")\n                     (from-resource \"path/within/classpath/to.edn\")\n                     {:datomic {:url \"this.will.win\"}} ])\n```\n\nin this case the `datomic url` will be overwritten with `\"this.will.win\"`, since this is the value the last map has.\nAnd notice the \"sources\", they would just return maps as well.\n\nAnd of course `:merge` well composes with `:resource` and `:file`:\n\n```clojure\n(load-config :resource \"path/within/classpath/to.edn\"\n             :file \"/path/to/some.edn\"\n             :merge [{:datomic {:url \"foo.bar\"}}\n                     (from-file \"/path/to/another.edn\")\n                     (from-resource \"path/within/classpath/to-another.edn\")\n                     (parse-runtime-args ...)])\n```\n\n### Merging with all System and ENV\n\nBy default only _matching_ configuration properties will be overridden with the ones from system or ENV.\nIn case all the system properties or ENV variables are needed (i.e. to add / override something that does not exist in the config),\nit can be done with `:merge` as well, since it does a \"deep merge\" (merges all the nested structures as well):\n\n```clojure\n(require '[cprop.source :refer [from-system-props\n                                from-env]])\n```\n\n`(from-system-props)` returns a map of ALL system properties that is ready to be merged with the config\n`(from-env)` returns a map of ALL ENV variables that is ready to be merged with the config\n\none or both can be used:\n\n```clojure\n(load-config :merge [(from-system-props)])\n```\n\n```clojure\n(load-config :merge [(from-system-props)\n                     (from-env)])\n```\n\nEverything of course composes together if needed:\n\n```clojure\n(load-config :resource \"path/within/classpath/to.edn\"\n             :file \"/path/to/some.edn\"\n             :merge [{:datomic {:url \"foo.bar\"}}\n                     (from-file \"/path/to/another.edn\")\n                     (from-resource \"path/within/classpath/to-another.edn\")\n                     (parse-runtime-args ...)\n                     (from-props-file \"/path/to/some.properties\")\n                     (from-system-props)\n                     (from-env)])\n```\n\nIt can get as creative as needed, but.. _this should cover most cases_:\n\n```clojure\n(load-config)\n```\n\n### Override all configs\n\ncprop merges _matching_ properties and ENV variables by default. In order to override that, or any other configs, properties or ENV variables `load-congfig` function takes an optional `:override-with` argument with a map that will override any matching (top level or however deeply nested) properties. If provided, this would be the last merge step applied after \"all\":\n\n```bash\n$ export DATOMIC__URL=foo\n```\n\n```clojure\n=\u003e (load-config)\n{:datomic {:url \"foo\"},\n :source ... }\n```\n\nbut could be overriden with:\n\n```clojure\n=\u003e (load-config :override-with {:datomic {:url \"bar\"}})\n{:datomic {:url \"bar\"},\n :source ... }\n```\n\n## Merging with system properties\n\nBy default cprop will merge all configurations with system properties that match the ones that are there in configs (i.e. intersection).\nIn case ALL system properties need to be merged (i.e. union), this can be done with `:merge`:\n\n\n```clojure\n(require '[cprop.source :refer [from-system-props]])\n\n(load-config :merge [(from-system-props)])\n```\n\n`(from-system-props)` returns a map of ALL system properties that is ready to be merged with the config.\n\n### System properties cprop syntax\n\nSystem properties are usually separated by `.` (periods). cprop will convert these periods to `-` (dashes).\n\nIn order to override a nested property use `_` (underscode).\n\nHere is an example. Let say we have a config:\n\n```clojure\n{:http\n {:pool\n  {:socket-timeout 600000,\n   :conn-timeout 60000,\n   :conn-req-timeout 600000,\n   :max-total 200,\n   :max-per-route 10}}}\n```\n\na system property `http_pool_socket.timeout` would point to a `{:http {:pool {:socket-timeout value}}}`. So to change a value it can be set as:\n\n```bash\n-Dhttp_pool_socket.timeout=4242\n```\n\nor\n\n```java\nSystem.setProperty(\"http_pool_socket.timeout\" \"4242\");\n```\n\n## Merging with ENV variables\n\nProduction environments are full of \"secrets\", could be passwords, URLs, ports, keys, etc.. Which are better driven by the ENV variables rather than being hardcoded in the config file.\n\n12 factor [config section](http://12factor.net/config) mentions that:\n\n\u003e The twelve-factor app stores config in environment variables\n\nWhile not _everything_ needs to live in environment variables + config files are a lot easier to visualize and develop with, this is a good point 12 factor makes:\n\n\u003e A litmus test for whether an app has all config correctly factored out of the code is whether the codebase could be made open source at any moment, without compromising any credentials.\n\nHence it makes a lot of sense for `cprop` to merge the config file with ENV variables when `(load-config)` is called.\n\n### Default\n\nBy default cprop will merge all configurations with ENV variables that match the ones that are there in configs (i.e. intersection).\nIn case ALL ENV variables need to be merged (i.e. union), this can be done with `:merge`:\n\n```clojure\n(require '[cprop.source :refer [from-env]])\n\n(load-config :merge [(from-env)])\n```\n\n`(from-env)` returns a map of ALL environment variables that is ready to be merged with the config.\n\n### Speaking ENV variables\n\n#### Structure and keywords\n\nENV variables lack structure. The only way to mimic the structure is via use of an underscore character.\nThe `_` is converted to `-` by cprop, so instead, to identify nesting, two underscores can be used.\n\nFor example to override a socket timeout in a form of:\n\n```clojure\n{:http\n {:pool\n  {:socket-timeout 600000}}}\n```\n\n```bash\nexport HTTP__POOL__SOCKET_TIMEOUT=4242\n```\n\nnotice how two underscores are used for \"getting in\" and a single underscore just gets converted to a dash to match the keyword.\n\nnamspaced and dotted keywords are also supported.\u003cbr/\u003e\nwhile `.` may be written `.` the namespace separator `/` must be written `___`\u003cbr/\u003e\nfor instance `dotted.namespaced/keyword` could be written as `DOTTED.NAMESPACED___KEYWORD`\n\nfor ENV support (i.e. ENV does not do `.`s) or more complex scenarios take a look at [customizing key path parsing](#customizing-key-path-parsing) or [this](https://github.com/tolitius/cprop/commit/64580c66ff8f394ffebcf9744095a52c634c9c07) example.\n\n#### Types\n\nENV variables, when read by [(System/getenv)](https://docs.oracle.com/javase/8/docs/api/java/lang/System.html#getenv--) are all _strings_.\n\ncprop will convert these strings to datatypes. e.g.:\n\n```bash\nexport APP_HTTP_PORT=4242                 # would be a Long\nexport APP_DB_URL=jdbc:sqlite:order.db    # would be a String\nexport APP_DB_URL='jdbc:sqlite:order.db'  # would be a String\nexport APP_DB_URL=\"jdbc:sqlite:order.db\"  # would be a String\nexport APP_NUMS='[1 2 3 4]'               # would be an EDN data structure (i.e. a vector in this example)\n```\n\nA small caveat is _purely numeric_ strings. For example:\n\n```bash\nexport BAD_PASSWORD='123456789'           # would still be a number (i.e. Long)\n```\n\nin order to make it really a String, double quotes will help:\n\n```bash\nexport BAD_PASSWORD='\"123456789\"'         # would be a String\n```\n\n### Merging ENV example\n\nLet's say we have a config file that needs values to be complete:\n\n```clojure\n{:datomic {:url \"CHANGE ME\"},\n :aws\n {:access-key \"AND ME\",\n  :secret-key \"ME TOO\",\n  :region \"FILL ME IN AS WELL\",\n  :visiblity-timeout-sec 30,\n  :max-conn 50,\n  :queue \"cprop-dev\"},\n :io\n {:http\n  {:pool\n   {:socket-timeout 600000,\n    :conn-timeout :I-SHOULD-BE-A-NUMBER,\n    :conn-req-timeout 600000,\n    :max-total 200,\n    :max-per-route :ME-ALSO}}},\n :other-things\n [\"I am a vector and also like to play the substitute game\"]}\n```\n\nIn order to fill out all the missing pieces we can export ENV variables as:\n\n```bash\nexport AWS__ACCESS_KEY=AKIAIOSFODNN7EXAMPLE\nexport AWS__SECRET_KEY=\"wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY\"\nexport AWS__REGION='us-east-1'\nexport IO__HTTP__POOL__CONN_TIMEOUT=60000\nexport IO__HTTP__POOL__MAX_PER_ROUTE=10\nexport OTHER__THINGS='[1 2 3 \"42\"]'\n```\n\n_(all the 3 versions of AWS values will be Strings, different ways are here just as an example)_\n\nNow whenever the config is loaded with `(load-config)` cprop will find these ENV variables and will merge them\nwith the original config file into one complete configuration:\n\n```clojure\nuser=\u003e (load-config)\nsubstituting [:aws :region] with a ENV/system.property specific value\nsubstituting [:aws :secret-key] with a ENV/system.property specific value\nsubstituting [:io :http :pool :conn-timeout] with a ENV/system.property specific value\nsubstituting [:io :http :pool :max-per-route] with a ENV/system.property specific value\nsubstituting [:datomic :url] with a ENV/system.property specific value\nsubstituting [:aws :access-key] with a ENV/system.property specific value\nsubstituting [:other-things] with a ENV/system.property specific value\n{:datomic\n {:url\n  \"datomic:sql://?jdbc:postgresql://localhost:5432/datomic?user=datomic\u0026password=datomic\"},\n :aws\n {:access-key \"AKIAIOSFODNN7EXAMPLE\",\n  :secret-key \"wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY\",\n  :region \"us-east-1\",\n  :visiblity-timeout-sec 30,\n  :max-conn 50,\n  :queue \"cprop-dev\"},\n :io\n {:http\n  {:pool\n   {:socket-timeout 600000,\n    :conn-timeout 60000,\n    :conn-req-timeout 600000,\n    :max-total 200,\n    :max-per-route 10}}},\n :other-things [1 2 3 \"42\"]}\n```\n\nnotice that `cprop` also tells you wnenever a property is substituted.\n\n## Merging with env file\n\n`.env` files are common in devops and other software communities: python, ruby, heroku, docker, etc.\u003cbr/\u003e\nthe content of an `.env` file is a list of environment variables\n\nthe following syntax rules apply to the .env file:\n\n- each line in an `.env` file to be in `VAR=VAL` format\n- lines beginning with `#` are processed as comments and ignored\n- blank lines are ignored\n- there is no special handling of quotation marks. this means that they are part of the `VAL`\n\ncprop follows [environment variables](#speaking-env-variables) supported syntax to read `.env` files.\n\nhere is an example of such a file that lives in [dev-resources/.env](dev-resources/.env):\n\n```bash\n# this is a comment\nSIMPLE=simple\nHYPHEN_KEY=hyphen-key\n\nEMPTY_KEY=\n\n## this is another comment\nDOTTED.KEY=dotted.key\nNAMSPACED___KEY=namespaced/key\nSUPER__NESTED__KEY=super nested key\n```\n\nin order to merge from it:\n\n```clojure\n(require '[cprop.core :as cp]\n         '[cprop.source :as cs])\n\n(cp/load-config :merge [(cs/from-env-file\n                         \"dev-resources/.env\")])\n\n;; {:empty-key \"\",\n;;  :simple \"simple\",\n;;  :super {:nested {:key \"super nested key\"}},\n;;  :namspaced/key \"namespaced/key\",\n;;  :source {:account {:rabb...}}\n;;  ...}\n```\n\n## Merging with property files\n\nIt is important to be able to integrate with existing Java applications or simply with configurations that are done as `.properties` files, i.e. not EDN.\n\n`cprop` can easily convert `.properties` files into EDN maps and merge it on top of the existing configuration by using `(from-props-file path)` function. Here is an example:\n\n```clojure\n(require '[cprop.source :refer [from-props-file]])\n\n(load-config :merge [(from-props-file \"path-to/overrides.properties\")])\n```\n\nWhich would merge:\n\n* `config.edn` as a classpath resource\n* with matching system properties\n* with matching ENV variables\n* with \"path-to/overrides.properties\" file\n\nHere is an example. Let's say we have this config:\n\n```clojure\n{:datomic {:url \"CHANGE ME\"}\n\n :aws {:access-key \"AND ME\"\n       :secret-key \"ME TOO\"\n       :region \"FILL ME IN AS WELL\"\n       :visiblity-timeout-sec 30\n       :max-conn 50\n       :queue \"cprop-dev\"}\n\n  :io {:http {:pool {:socket-timeout 600000\n                     :conn-timeout :I-SHOULD-BE-A-NUMBER\n                     :conn-req-timeout 600000\n                     :max-total 200\n                     :max-per-route :ME-ALSO}}}\n\n  :other-things [\"I am a vector and also like to place the substitute game\"]}\n```\n\nand this `overrides.properties` file:\n\n```properties\ndatomic.url=datomic:sql://?jdbc:postgresql://localhost:5432/datomic?user=datomic\u0026password=datomic\n\nsource.account.rabbit.host=localhost\n\naws.access-key=super secret key\naws.secret_key=super secret s3cr3t!!!\naws.region=us-east-2\n\nio.http.pool.conn_timeout=42\nio.http.pool.max_per_route=42\n\nother_things=1,2,3,4,5,6,7\n```\n\nWe can apply the overrides with cprop as:\n\n```clojure\n(load-config :merge [(from-props-file \"overrides.properties\")])\n```\n\nwhich will merge them and will return:\n\n```clojure\n{:datomic\n {:url \"datomic:sql://?jdbc:postgresql://localhost:5432/datomic?user=datomic\u0026password=datomic\"},\n :aws\n {:access-key \"super secret key\",\n  :secret-key \"super secret s3cr3t!!!\",\n  :region \"us-east-2\",\n  :visiblity-timeout-sec 30,\n  :max-conn 50,\n  :queue \"cprop-dev\"},\n :io\n {:http\n  {:pool\n   {:socket-timeout 600000,\n    :conn-timeout 42,\n    :conn-req-timeout 600000,\n    :max-total 200,\n    :max-per-route 42}}},\n :other-things [\"1\" \"2\" \"3\" \"4\" \"5\" \"6\" \"7\"]}\n```\n\n### Property files syntax\n\nThe traditional syntax of a `.properties` file does not change. For example:\n\n* `.` means structure\n\n`four.two=42` would be translated to `{:four {:two 42}}`\n\n* `_` would be a key separator\n\n`fourty_two=42` would be translated to `{:fourty-two 42}`\n\n* `,` in a value would be seq separator\n\n`planet.uran.moons=titania,oberon` would be translated to `{:planet {:uran {:moons [\"titania\" \"oberon\"]}}}`\n\nFor example let's take a `solar-system.properties` file:\n\n```properties\n## solar system components\ncomponents=sun,planets,dwarf planets,moons,comets,asteroids,meteoroids,dust,atomic particles,electromagnetic.radiation,magnetic field\n\nstar=sun\n\n## planets with Earth days to complete an orbit\nplanet.mercury.orbit_days=87.969\nplanet.venus.orbit_days=224.7\nplanet.earth.orbit_days=365.2564\nplanet.mars.orbit_days=686.93\nplanet.jupiter.orbit_days=4332.59\nplanet.saturn.orbit_days=10755.7\nplanet.uran.orbit_days=30688.5\nplanet.neptune.orbit_days=60148.35\n\n## planets natural satellites\nplanet.earth.moons=moon\nplanet.jupiter.moons=io,europa,ganymede,callisto\nplanet.saturn.moons=titan\nplanet.uran.moons=titania,oberon\nplanet.neptune.moons=triton\n\n# favorite dwarf planet's moons\ndwarf.pluto.moons=charon,styx,nix,kerberos,hydra\n```\n\n```clojure\n(from-props-file \"solar-system.properties\")\n```\n\nwill convert it to:\n\n```clojure\n{:components [\"sun\" \"planets\" \"dwarf planets\" \"moons\" \"comets\"\n              \"asteroids\" \"meteoroids\" \"dust\" \"atomic particles\"\n              \"electromagnetic.radiation\" \"magnetic field\"],\n :star \"sun\",\n :planet\n {:uran {:moons [\"titania\" \"oberon\"],\n         :orbit-days 30688.5},\n  :saturn {:orbit-days 10755.7,\n           :moons \"titan\"},\n  :earth {:orbit-days 365.2564,\n          :moons \"moon\"},\n  :neptune {:moons \"triton\",\n            :orbit-days 60148.35},\n  :jupiter {:moons [\"io\" \"europa\" \"ganymede\" \"callisto\"],\n            :orbit-days 4332.59},\n  :mercury {:orbit-days 87.969},\n  :mars {:orbit-days 686.93},\n  :venus {:orbit-days 224.7}},\n :dwarf {:pluto {:moons [\"charon\" \"styx\" \"nix\" \"kerberos\" \"hydra\"]}}}\n```\n\n## Read \"as is\" (not EDN)\n\nNot all configs and properties come in EDN format, and some of these not EDN properties / env variables can't be read with Clojure's EDN reader, for example:\n\n```clojure\n=\u003e (require '[clojure.edn :as edn])\n\n=\u003e (edn/read-string \"7 Nov 22:44:53 2015\")\n7\n```\nalso:\n```clojure\nboot.user=\u003e (edn/read-string \"7Nov 22:44:53 2015\")\njava.lang.NumberFormatException: Invalid number: 7Nov\n```\n\nand imagine if this `7 Nov 22:44:53 2015` is an ENV variable that you can't change, but still need to be able to use. For cases like these you can use an `:as-is` flag to communicate to cprop to treat properties/vars \"as is\", in other words take them as they come and don't try to convert them into anything.\n\n`:as-is?` optional param is available on all the source (`from-env`, `from-system-props`, `from-props-file`, etc.) functions which will read props/vars as is:\n\n```bash\n$ export FOO='\"4242\"'\n$ export BAR=4242\n$ export DATE='7 Nov 22:44:53 2015'\n$ export VEC='[1 2 3 4]'\n```\n```clojure\n=\u003e (require '[cprop.source :as s])\n\n=\u003e (:foo (s/from-env))\n\"4242\"\n=\u003e (:bar (s/from-env))\n4242\n=\u003e (:date (s/from-env))\n7                                     ;; uh.. that's bad\n=\u003e (:vec (s/from-env))\n[1 2 3 4]\n```\nbut\n```clojure\n=\u003e (:foo (s/from-env {:as-is? true}))\n\"\\\"4242\\\"\"\n=\u003e (:bar (s/from-env {:as-is? true}))\n\"4242\"\n=\u003e (:date (s/from-env {:as-is? true}))\n\"7 Nov 22:44:53 2015\"                  ;; that's good\n=\u003e (:vec (s/from-env {:as-is? true}))\n\"[1 2 3 4]\"\n```\n\nIf you need _ALL_ the properties and configs to come in \"as is\" (not as EDN) `:as-is` flag is also available at the top level:\n\n```clojure\n(load-config :as-is? true)\n```\n\nYou can also opt to keep value conversion for all your properties with the exception of a set of specified paths using `:as-is-paths`:\n\n```clojure\n(load-config :as-is-paths #{[:io :http :pool :socket-timeout]\n                            [:datomic :max-conn]\n                            [:some :other :path]})\n```\n\n## Customizing key path parsing\n\nBy default `cprop` parses keys / nestsed key paths as Clojure keywords: i.e. `{:a {:b {:c 42}}}`. There are a few corner cases where some of the keys are best parsed as different types.\n\nFor example, let's say we need to override passwords via ENV variables for this config:\n\n```clojure\n{:clusters [{:name \"first\" :url \"http://somewhere\" :password \"OVERRIDE ME\"}\n            {:name \"second\" :url \"http://elsewhere\" :password \"OVERRIDE ME\"}]}\n```\n\nThese ENV vars won't do it since the `0` and `1` are coerced to `keywords` by default:\n\n```bash\nexport CLUSTERS__0__PASSWORD=super-duper-secret\nexport CLUSTERS__1__PASSWORD=shh-don't-tell\n```\n\nIn cases like these we can provide a custom `:key-parse-fn` function, when loading config, which will be called on each part of a key path:\n\n```clojure\n=\u003e (def parse-numbers [part]\n     (if (re-matches #\"\\d+\" part)  ;; if the key is a number\n       (long part)                 ;; this could be done with (cprop.tools/str-\u003enum) to support bigint\n       (keyword part)))            ;; leave the default behavior\n```\n\nnow we can plug this function in with `:key-parse-fn`:\n\n```clojure\n=\u003e (load-config :key-parse-fn parse-numbers)\n\n{:clusters [{:name \"first\" :url \"http://somewhere\" :password \"super-duper-secret\"}\n            {:name \"second\" :url \"http://elsewhere\" :password \"shh-don't-tell\"}]}\n```\n\nand.. great success.\n\n## Cursors\n\nIt would be somewhat inconvenient to repeat `[:source :account :rabbit :prop]` over and over in different pieces of the code that need rabbit values.\n\nThat's where the cursors help a lot:\n\n```clojure\n(require '[cprop.core :refer [load-config cursor]])\n```\n```clojure\n(def conf (load-config))\n\n(def rabbit\n  (cursor conf :source :account :rabbit))\n\n(rabbit :vhost) ;; \"/z-broker\"\n```\n\nmuch better.\n\n### Composable Cursors\n\nIn case you pass a cursor somewhere, you can still build new cursors out of it by simply _composing_ them.\n\nworking with the same config as in the example above:\n\n```clojure\n{:datomic\n    {:url \"datomic:sql://?jdbc:postgresql://localhost:5432/datomic?user=datomic\u0026password=datomic\"}\n :source\n    {:account\n        {:rabbit\n           {:host \"127.0.0.1\"\n            :port 5672\n            :vhost \"/z-broker\"\n            :username \"guest\"\n            :password \"guest\"}}}\n :answer 42}\n```\n\ncreating a simple cursor to source:\n\n```clojure\nuser=\u003e (def src (cursor conf :source))\n#'user/src\nuser=\u003e (src)\n{:account {:rabbit {:host \"127.0.0.1\", :port 5672, :vhost \"/z-broker\", :username \"guest\", :password \"guest\"}}}\n\nuser=\u003e (src :account)\n{:rabbit {:host \"127.0.0.1\", :port 5672, :vhost \"/z-broker\", :username \"guest\", :password \"guest\"}}\n```\n\nnow an `account` cursor can be created out of the `src` one as:\n\n```clojure\nuser=\u003e (def account (cursor conf src :account))\n#'user/account\n\nuser=\u003e (account :rabbit)\n{:host \"127.0.0.1\", :port 5672, :vhost \"/z-broker\", :username \"guest\", :password \"guest\"}\n```\n\nor any nested cursor for that matter:\n\n```clojure\nuser=\u003e (def rabbit (cursor conf src :account :rabbit))\n#'user/rabbit\n\nuser=\u003e (rabbit :host)\n\"127.0.0.1\"\n```\n\n## Tools\n\n### Translating EDN\n\nDepending on the build infrastructure, continuous integration, deployments, some environments would require `.properties` files with overrides instead of EDN configs or ENV variable overrides.\n\nAlso it's easier to use EDN file with overrides in development before converting them to a set of ENV variables, but it can take some time, and would somewhat error prone, to convert this EDN file with overrides to a set of ENV variables.\n\nFor those cases above cprop has helper tools that can help translating from EDN.\n\nLet's use [this config file](dev-resources/config.edn) an example:\n\n```clojure\nboot.user=\u003e (pprint config)\n{:datomic\n {:url\n  \"datomic:sql://?jdbc:postgresql://localhost:5432/datomic?user=datomic\u0026password=datomic\"},\n :source\n {:account\n  {:rabbit\n   {:host \"127.0.0.1\",\n    :port 5672,\n    :vhost \"/z-broker\",\n    :username \"guest\",\n    :password \"guest\"}}},\n :answer 42}\n```\n\n#### EDN to .properties\n\nWe can pass this config to a `map-\u003eprops-file` function that will convert it to a `.properties` formatted file:\n\n```clojure\n(require '[cprop.tools :as t])\n```\n\n```clojure\n(t/map-\u003eprops-file config)\n\"/tmp/cprops-1475854845508-538388633502378948.tmp\"\n```\n\nit returns a path to a file it created, which we can look at:\n\n```clojure\n(print (slurp \"/tmp/cprops-1475854845508-538388633502378948.tmp\"))\n\nanswer=42\nsource.account.rabbit.host=127.0.0.1\nsource.account.rabbit.port=5672\nsource.account.rabbit.vhost=/z-broker\nsource.account.rabbit.username=guest\nsource.account.rabbit.password=guest\ndatomic.url=datomic:sql://?jdbc:postgresql://localhost:5432/datomic?user=datomic\u0026password=datomic\n```\n\n#### EDN to .env\n\nWe can pass this config to a `map-\u003eenv-file` function that will convert it to a `.env` formatted file:\n\n```clojure\n(require '[cprop.tools :as t])\n```\n\n```clojure\n(t/map-\u003eenv-file config)\n\"/tmp/cprops-1475854874506-8756956459082793585.tmp\"\n```\n\nit returns a path to a file it created, which we can look at:\n\n```clojure\n(print (slurp \"/tmp/cprops-1475854874506-8756956459082793585.tmp\"))\n\nexport ANSWER=42\nexport SOURCE__ACCOUNT__RABBIT__HOST=127.0.0.1\nexport SOURCE__ACCOUNT__RABBIT__PORT=5672\nexport SOURCE__ACCOUNT__RABBIT__VHOST=/z-broker\nexport SOURCE__ACCOUNT__RABBIT__USERNAME=guest\nexport SOURCE__ACCOUNT__RABBIT__PASSWORD=guest\nexport DATOMIC__URL=datomic:sql://?jdbc:postgresql://localhost:5432/datomic?user=datomic\u0026password=datomic\n```\n\n## Tips\n\n### Setting the \"conf\" system property\n\nThere are several ways the `conf` property can be set:\n\n#### command line\n\n```clojure\njava -Dconf=\"../somepath/whatsapp.conf\" -jar whatsapp.jar\n```\n\n#### boot\n\n```clojure\n(System/setProperty \"conf\" \"resources/config.edn\")\n```\n\n#### lein\n\n```clojure\n:profiles {:dev {:jvm-opts [\"-Dconf=resources/config.edn\"]}}\n```\n\n### See which files were loaded and what properties were substituted\n\nIn order to see which files were read (and merged) and which properties were substituted by the cprop merge,\nexport a DEBUG environment variable to `y` / `Y`:\n\n```bash\nexport DEBUG=y\n```\n\nif this variable is exported, cprop won't keep files and substitutions a secret:\n\n```clojure\nuser=\u003e (load-config)\nread config from stream: \"dev-resources/config.edn\"    ;;\nread config from file: \"dev-resources/config.edn\"      ;; =\u003e a sample output\nread config from resource: \"config.edn\"                ;;\n\nsubstituting [:aws :region] with a ENV/system.property specific value\nsubstituting [:aws :secret-key] with a ENV/system.property specific value\nsubstituting [:io :http :pool :conn-timeout] with a ENV/system.property specific value\nsubstituting [:io :http :pool :max-per-route] with a ENV/system.property specific value\nsubstituting [:datomic :url] with a ENV/system.property specific value\nsubstituting [:aws :access-key] with a ENV/system.property specific value\nsubstituting [:other-things] with a ENV/system.property specific value\n;; ...\n```\n\n\n\n#### Why not default?\n\nThe reason this is not on by default is merging ALL env and/or system properties with configs\nwhich is quite noisy and not very useful (i.e. can be hundreds of entries..).\n\n### Convert properties to a \"one level\" map\n\nBesides the `from-props-file` function that converts `.properties` file to a map _with hierarchy_, there is also a `slurp-props-file` function that simply converts a property file to a map without parsing values or building a hierarchy:\n\n```clojure\n(require '[cprop.source :refer [slurp-props-file]])\n\n(slurp-props-file \"solar-system.properties\")\n```\n\n```properties\n{\"star\" \"sun\",\n\n \"planet.jupiter.moons\" \"io,europa,ganymede,callisto\",\n \"planet.neptune.moons\" \"triton\",\n \"planet.jupiter.orbit_days\" \"4332.59\",\n \"planet.uran.orbit_days\" \"30688.5\",\n \"planet.venus.orbit_days\" \"224.7\",\n \"planet.earth.moons\" \"moon\",\n \"planet.saturn.orbit_days\" \"10755.7\",\n \"planet.mercury.orbit_days\" \"87.969\",\n \"planet.saturn.moons\" \"titan\",\n \"planet.earth.orbit_days\" \"365.2564\",\n \"planet.uran.moons\" \"titania,oberon\",\n \"planet.mars.orbit_days\" \"686.93\",\n \"planet.neptune.orbit_days\" \"60148.35\"\n\n \"dwarf.pluto.moons\" \"charon,styx,nix,kerberos,hydra\",\n\n \"components\" \"sun,planets,dwarf planets,moons,comets,asteroids,meteoroids,dust,atomic particles,electromagnetic.radiation,magnetic field\"}\n```\n\n## License\n\nCopyright © 2023 tolitius\n\nDistributed under the Eclipse Public License either version 1.0 or (at\nyour option) any later version.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftolitius%2Fcprop","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftolitius%2Fcprop","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftolitius%2Fcprop/lists"}