{"id":13574196,"url":"https://github.com/dacr/code-examples-manager","last_synced_at":"2025-04-13T21:22:53.058Z","repository":{"id":37486173,"uuid":"203237333","full_name":"dacr/code-examples-manager","owner":"dacr","description":"Software tool to manage your notes, scripts, code examples, configs,... to publish them as gists or snippets","archived":false,"fork":false,"pushed_at":"2025-03-31T07:32:43.000Z","size":1738,"stargazers_count":39,"open_issues_count":4,"forks_count":5,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-11T16:19:12.109Z","etag":null,"topics":["ammonite","cem","code-examples","code-examples-manager","gist","gists","github-gists","gitlab","gitlab-snippets","good-code-examples","lmdb","scala","scala-cli","scala3","snippet","snippets","snippets-manager","zio","zio-lmdb","zio2"],"latest_commit_sha":null,"homepage":"","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dacr.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-08-19T19:32:08.000Z","updated_at":"2025-04-05T22:36:29.000Z","dependencies_parsed_at":"2024-01-16T20:25:36.212Z","dependency_job_id":"d1c0acf5-e694-4cb6-b245-7d1383c348c4","html_url":"https://github.com/dacr/code-examples-manager","commit_stats":null,"previous_names":[],"tags_count":42,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dacr%2Fcode-examples-manager","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dacr%2Fcode-examples-manager/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dacr%2Fcode-examples-manager/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dacr%2Fcode-examples-manager/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dacr","download_url":"https://codeload.github.com/dacr/code-examples-manager/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248783436,"owners_count":21160921,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ammonite","cem","code-examples","code-examples-manager","gist","gists","github-gists","gitlab","gitlab-snippets","good-code-examples","lmdb","scala","scala-cli","scala3","snippet","snippets","snippets-manager","zio","zio-lmdb","zio2"],"created_at":"2024-08-01T15:00:47.967Z","updated_at":"2025-04-13T21:22:53.050Z","avatar_url":"https://github.com/dacr.png","language":"Scala","funding_links":[],"categories":["Scala"],"sub_categories":[],"readme":"# CEM - Code Examples Manager [![][CodeExamplesManagerImg]][CodeExamplesManagerLnk] ![Scala CI][scalaci-master]\n\nCode example manager (CEM) is a software managing your notes, scripts and code examples.\nIt provides publish mechanisms to [github.com][githubcom] (as [gists][gists]) or\n[gitlab.com][gitlabcom] (as [snippets][snippets]). It also automates execution for\ntestable examples, this is a quite useful to manage a high number of examples.\n\nAll my notes, scripts and code examples (my programming knowledge base) are now managed using this tool,\nyou can take a look to **[my public gists overview on github][mygists]** to illustrate the \npublishing work achieved by CEM.\n\nCurrent [Code example manager (CEM)][cem] implementation is just a command line tool\nwhich compare locally available examples with already published ones in order to find\nwhat it should do (add, update, do nothing).\n\n![](images/cloudtags.png)\n\n## Why ?\n\nCode examples are very important, each example is most of the time designed to focus\non a particular feature/characteristic of a programming language, a library or a framework.\nThey help us to quickly test, experiment and remember how bigger project or at least some\nparts of them are working.\n\n**[See the rules for good code examples][rules] for more information.**\n\nManaging hundreds of published code example files as gists (github) and/or snippets (gitlab)\nis really not easy and time-consuming, in particular if you want to keep them up to date. This\nis the main issue addressed by this software. \n\n- My open-source examples evolution trend :  \n  ![](images/created-examples-trend.png)\n- and the execution status trend for executable/testable ones :  \n  ![](images/testable-examples-status.png)\n\nAs you can see through the previous charts, once you have industrialized your notes and code\nexamples, analytics on your examples become quite easy, and a lot of advanced features become\npossible... So stay connected to this project ;)\n\n## Quick start\n\nNo particular prerequisites, just a Java \u003e=8 JVM available, and\nit will run on your Linux, Windows or MacOSX\n\nInstructions example with github.com publishing configuration :\n- Install the [coursier][csget] from @alxarchambault to automate\n  the download/update/start of code-examples-manager directly from\n  maven repositories\n- Customize your configuration (see below for token configuration)\n  ```\n  export CEM_SEARCH_ROOTS=\"/home/myuser/myexamples\"\n  export CEM_GITHUB_TOKEN=\"xxxxx\"\n  ```\n- Create an example file in `/home/myuser/myexamples` such as `hello.md`\n  ```\n  \u003c!--\n  // summary : my markdown cheat sheet\n  // keywords : markdown, hello, example \n  // publish : gist\n  // authors : someone, someonelse\n  // id : d27245fc-22fb-4c9e-9809-feb0826400e7\n  --\u003e\n  # Hello world !\n  this is just an example\n  ```\n- Run the following command from your terminal (`cs` is the [coursier][cs] CLI command):\n  ```\n  cs launch fr.janalyse:code-examples-manager_3:2.4.0\n  ```\n  - you can even use `cs launch fr.janalyse:code-examples-manager_3:latest.release` to always use the latest release\n  - current release is : [![][CodeExamplesManagerImg]][CodeExamplesManagerLnk]\n- Check the command output to get the overview URL\n\n\n## Code examples\n\nIn order to be published your code examples must come with a description header\ninserted using single line comments. You must provide a unique identifier (UUID)\nto each of your example, as well as a summary and publish keywords which define\nremote destinations.\n\nExample for languages using `//` for line comments :\n```scala\n// summary : Simplest scalatest test framework usage.\n// keywords : scala, scalatest, pi, @testable\n// publish : gist\n// authors : David Crosson\n// license : Apache\n// id : d24d8cb3-45c0-4d88-b033-7fae2325607b\n// created-on : 2020-05-31T19:54:52Z\n// run-with : scala-cli $file\n\n// ---------------------\n//\u003e using scala  \"3.1.1\"\n//\u003e using lib \"org.scalatest::scalatest:3.2.10\"\n// ---------------------\n\nimport org.scalatest._, matchers.should.Matchers._\n\nmath.Pi shouldBe 3.14d +- 0.01d\n```\n\nSupported keys in description header are the following :\n- **`summary`** : example summary in one line.\n- **`keywords`** : keywords describing your code features (comma separated). Some reserved keywords :\n  - `@testable` : allow automatic execution\n  - `@fail` : the example is expected to fail when executed\n  - `@exclusive` : all testable examples with this flag will be run sequentially (for scripts which open server sockets for example)\n- **`publish`** : publish destination keywords (comma separated)\n  - the default configuration file provide those activation keywords :\n    - `gist` : for github.com\n    - `snippet` : for gitlab.com\n- **`authors`** : code example authors list (comma separated).\n- **`license`** : the example license.\n- **`id`** : UUID for this code example. Generated using such commands :\n  - with linux command : `uuidgen`\n  - with [scala-cli][scl] : `scala-cli https://gist.github.com/dacr/87c9636a6d25787d7c274b036d2a8aad`\n  - with [ammonite][amm] : `amm -c 'println(java.util.UUID.randomUUID.toString)'`\n- **`attachments`** : List of secondary files (comma separated) which must be published with the current one\n  - must be placed in the same directory in your local storage\n  - secondary files do not require any headers\n- **`created-on`** : The ISO8601 date when this example has been created. Generated using such commands :\n  - with linux command : `date -u +\"%Y-%m-%dT%H:%M:%S.%3NZ\"`\n  - with [scala-cli][scl] : `scala-cli https://gist.github.com/dacr/4298fce08e12ba76ab91e9766be52acb`\n  - with [ammonite][amm] : `amm -c 'println(java.time.Instant.now.toString)'`\n- **`run-with`** : command used to execute this example\n  - Only examples with `@testable` keywords are eligible for automated execution\n  - on execution the exit code is used to compute execution success or failure\n  - Use `$file` (or `$scriptFile`) for example filename substitution\n- **`test-with`** : Command to test the example\n  - When `@testable` is set as keyword\n  - When your example is a \"blocking service\", you can specify an external command to test it\n    - for example : `test-with : curl http://127.0.0.1:8080/docs`\n\n## CEM operations\n\nCode examples manager operations :\n- It reads its configuration\n- It searches for code examples from the given directories roots\n  - select files matching the search pattern and not involved in the ignore mask\n  - Selects code examples if and only if they contain a unique identifier (UUID)\n- It publishes or updates remote code examples to remote destinations\n  - the code example publish scope (`publish` keyword) select target destinations\n    - comma separated publish activation keyword (`activation-keyword` parameter in configuration) \n  - It adds or updates a global overview of all published examples for a given destination\n    - this summary has its own UUID defined in the configuration file \n\n## Configuration\n\nThe configuration relies on configuration files, a default one named `reference.conf` is provided.\nThis [default configuration file][referenceconf] defines default values and default behaviors and\nallow a simple configuration way based on environment variables which override default values.\n\n### Simplified configuration\n\n| env or property name       | description                                                   | default value\n|----------------------------|---------------------------------------------------------------|---------------------------\n| CEM_CONFIG_FILE            | Your custom advanced configuration file (optional)            | *undefined*\n| CEM_SUMMARY_TITLE          | The generated summary title for all published examples        | Examples knowledge base \n| CEM_SEARCH_ROOTS           | Examples search roots (comma separated)                       | \"\"\n| CEM_SEARCH_PATTERN         | Examples files regular expression pattern                     | \".*\"\n| CEM_SEARCH_IGNORE_MASK     | Ignore file regular expression                                | \"(/[.]bsp)|(/[.]scala.*)|([.]png$)\"\n| CEM_CHAR_ENCODING          | Chararacter encoding for your examples or notes               | \"UTF-8\"\n| CEM_GITHUB_ENABLED         | To enable or disable standard GITHUB support                  | true\n| CEM_GITHUB_ACTIVATION_KEY  | Example publish keyword for github                            | \"gist\"\n| CEM_GITHUB_TOKEN           | Github authentication token for gists API access              | *more information below*\n| CEM_GITHUB_API             | Github API http end point                                     | \"https://api.github.com\"\n| CEM_GITLAB_ENABLED         | To enable or disable standard GITLAB support                  | true\n| CEM_GITLAB_ACTIVATION_KEY  | Example publish keyword for the gitlab                        | \"snippet\"\n| CEM_GITLAB_TOKEN           | gitlab authentication token for snippets API access           | *more information below*\n| CEM_GITLAB_API             | Gitlab API http end point                                     | \"https://gitlab.com/api/v4\"\n| CEM_GITLAB_VISIBILITY      | Gitlab published examples chosen visibility                   | \"public\"\n\nConfiguration examples :\n```shell\nexport CEM_SEARCH_ROOTS=\"/tmp/someplace,/tmp/someotherplace\"\nexport CEM_SEARCH_PATTERN=\"[.](sc)|(sh)|(md)|(jsh)$\"\nexport CEM_GITHUB_TOKEN=\"fada-fada-fada-fada\"\n```\n\n### Advanced configuration\n\nTake a look to the [configuration file][referenceconf] for more information about advanced configuration.\n\nOnce CEM installed you can modify the provided `conf/application.conf` file (whose content is by default\nthe same as the default [reference.conf][referenceconf] file), remember that any unset parameter in `application.conf`\nwill default to the value defined in `reference.conf`.\n\nNote : It is also possible to provide a custom configuration file through the `config.file` java property or the\n`CEM_CONFIG_FILE` environment variable.\n\n## Authentication tokens\n\n### Gitlab authentication token configuration\n\nGet an access token from gitlab :\n- Go to your user **settings**\n  - Select **Access tokens**\n    - Add a **Personal access token**\n      - Enable scopes : `api` and `read_user`\n- setup your `CEM_GITLAB_TOKEN` environment variable or `token` parameter in your configuration file\n  with the generated token\n- **Keep it carefully as it is not possible to retrieve it later.**\n- **And of course KEEP IT SECURE**\n\n### Github authentication token configuration\n\nGet an access token from github.com :\n- Got to your user **settings**\n  - Select **Developer settings**\n    - Select **Personal access tokens**\n      - Then **generate new token**\n        - Enable scopes : `gist` and `read:user`\n- setup your `CEM_GITHUB_TOKEN` environment variable or `token` parameter in your configuration file\n  with the generated token, the value shown within curl json response\n- **Keep it carefully as it is not possible to retrieve it later.**\n- **And of course KEEP IT SECURE**\n\n## Project history\n\n- 2019-06 - PoC#1 example proof of concept\n- 2019-07 - PoC#2 example proof of concept (in prod)\n- 2019-08 - Switch to a real world project\n- 2019-09 - In prod for my own usage\n- 2020-07 - First public release\n- 2021-05 - Full refactoring to use [ZIO][zio] - pure functional\n- 2021-06 - Migration to Scala3\n- 2021-12 - PoC#3 Search \u0026 Execution engines\n- 2022-01 - Migrate to ZIO2 and add support for attachments\n- 2023-04 - Use ZIO standard configuration\n- 2023-05 - Add ZIO LMDB for local storage cache AND data sharing with external applications \n\n[CodeExamplesManager]:    https://github.com/dacr/code-examples-manager\n[CodeExamplesManagerImg]: https://img.shields.io/maven-central/v/fr.janalyse/code-examples-manager_3.svg\n[CodeExamplesManagerLnk]: https://mvnrepository.com/artifact/fr.janalyse/code-examples-manager\n[scalaci-master]: https://github.com/dacr/code-examples-manager/workflows/Scala%20CI/badge.svg\n[mygists]: https://gist.github.com/c071a7b7d3de633281cbe84a34be47f1\n[cem]: https://github.com/dacr/code-examples-manager\n[amm]: https://ammonite.io/\n[scl]: https://scala-cli.virtuslab.org/\n[githubcom]: https://github.com/\n[gitlabcom]: https://gitlab.com/\n[snippets]: https://docs.gitlab.com/ce/user/snippets.html\n[gists]: https://docs.github.com/en/github/writing-on-github/creating-gists\n[uuid-sc]: https://gist.github.com/dacr/87c9636a6d25787d7c274b036d2a8aad\n[scala]: https://www.scala-lang.org/\n[lihaoyi]: https://github.com/lihaoyi\n[ac2019]: https://www.alpescraft.fr/edition_2019/\n[ac2019talk]: https://www.youtube.com/watch?v=61AGIBdG7YE\n[referenceconf]: https://github.com/dacr/code-examples-manager/blob/master/src/main/resources/reference.conf\n[latest]: https://github.com/dacr/code-examples-manager/releases/latest\n[rules]: https://github.com/dacr/the-rules-for-good-code-examples\n[cs]: https://get-coursier.io/\n[csget]: https://get-coursier.io/docs/cli-installation\n[zio]: https://zio.dev/","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdacr%2Fcode-examples-manager","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdacr%2Fcode-examples-manager","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdacr%2Fcode-examples-manager/lists"}