{"id":18099371,"url":"https://github.com/kevinlang/catalog","last_synced_at":"2025-04-13T15:55:26.514Z","repository":{"id":57482374,"uuid":"397399257","full_name":"kevinlang/catalog","owner":"kevinlang","description":"Compile time content and data processing engine for markdown, json, yaml, and more.","archived":false,"fork":false,"pushed_at":"2021-08-22T16:24:42.000Z","size":53,"stargazers_count":6,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-03T20:38:21.762Z","etag":null,"topics":["blogging","elixir","markdown"],"latest_commit_sha":null,"homepage":"https://hexdocs.pm/catalog","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kevinlang.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-08-17T21:48:43.000Z","updated_at":"2024-08-31T07:27:52.000Z","dependencies_parsed_at":"2022-09-02T04:20:18.440Z","dependency_job_id":null,"html_url":"https://github.com/kevinlang/catalog","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kevinlang%2Fcatalog","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kevinlang%2Fcatalog/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kevinlang%2Fcatalog/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kevinlang%2Fcatalog/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kevinlang","download_url":"https://codeload.github.com/kevinlang/catalog/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248741144,"owners_count":21154250,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["blogging","elixir","markdown"],"created_at":"2024-10-31T21:08:10.282Z","updated_at":"2025-04-13T15:55:26.495Z","avatar_url":"https://github.com/kevinlang.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Catalog\n\n[Online Documentation](https://hexdocs.pm/catalog).\n\n\u003c!-- MDOC !--\u003e\n\n`Catalog` does compile-time transformation and processing of data and content files\nin your codebase and embeds the result into the module in which it was used.\n\nIt intends to make integrating non-code files within your Elixir projects \nas pleasant as a developer experience as possible, taking inspiration from\nthe seamless integration static site generators provide for editing in-repo \ndata and content files.\n\nIt supports the following content types:\n\n* `markdown/3`\n* `json/3`\n* `csv/3`\n* `file/3`\n* `yaml/3`\n* `toml/3`\n\n## Comparison to other approaches\n\nBecause the content is preprocessed at compile time, any expensive transformations\nare no longer done at runtime, such as transforming markdown to HTML in response to\na web request. Likewise, since everything is stored in memory, no disk lookups need\nto be made to access the processed entries. This compile-time approach means any processing\nare easily detected during development or in a basic CI system, instead of encountering issues\nin response to a request or at boot time. \n\n## Example\n\nWe can install Catalog into our application by updating our `deps()` in our `mix.exs`\nto include `:catalog`.\nSince we will be processing markdown files here with Toml frontmatter, we will also need\nto install `:earmark` and `:toml`, respectively.\n\n```elixir\ndef deps() do\n  [\n    {:catalog, \"~\u003e 0.1.1\"},\n    {:earmark, \"~\u003e 1.4\"},\n    {:toml, \"~\u003e 0.6.2\"}\n  ]\nend\n```\n\nWith that done, we can define a new module in our application:\n\n```elixir\ndefmodule MyApp.Catalog do\n  use Catalog\n\n  markdown(:posts, \"posts/**.md\")\n\n  def all_posts(), do: @posts\nend\n```\n\nIn the example above, we defined a new module for our Elixir application, `MyApp.Catalog`,\nthat will serve as the API for our processed assets. We then use the `markdown/2` macro,\nspecifying first the name of the module attribute, `:posts`, we want to stored the processed markdown file,\nthen specifying the wildcard path for where those files are stored, `\"posts/**.md\"`.\n\nIf our specified directory has only the following markdown file at `posts/hello.md`:\n\n```markdown\n+++\nauthor = \"Kevin Lang\"\ntitle = \"Hello World\"\ndate: 2021-08-19\n+++\nThis is a markdown *document*.\n```\n\nThen the `@posts` attribute above will look like the following:\n\n```elixir\n[%{\n  content: \"\u003cp\u003e\\nThis is a markdown \u003cem\u003edocument\u003c/em\u003e.\u003c/p\u003e\\n\"\n  frontmatter: %{\n    author: \"Kevin Lang\",\n    title: \"Hello World\",\n    date: ~D[2021-08-19]\n  },\n  path: \"posts/hello.md\"\n}]\n```\n\nWe can customize how we build each entry by specifying our own `:build` option.\n\n```elixir \ndefmodule MyApp.Catalog.Post\n  @enforce_keys [:id, :author, :title, :date, :content]\n  defstruct [:id, :author, :title, :date, :content]\n\n  def build(path, frontmatter, content) do\n    [id] = path |\u003e Path.rootname() |\u003e Path.split() |\u003e Enum.take(-1)\n    struct!(__MODULE__, [id: id, content: content] ++ Map.to_list(frontmatter))\n  end\nend\n```\n\nThen our `@posts` attribute will look like:\n\n```elixir\n[%MyApp.Catalog.Post{\n  id: \"hello\",\n  content: \"\u003cp\u003e\\nThis is a markdown \u003cem\u003edocument\u003c/em\u003e.\u003c/p\u003e\\n\",\n  date: ~D[2021-08-19],\n  title: \"Hello World\",\n  author: \"Kevin Lang\"\n}]\n```\n\nAdditionally, we can add syntax highlighting and customize our markdown to HTML\ntransformation. See `markdown/2` for more info.\n\n### Using and modifying the module attribute\n\nAfter the module attribute is defined, as shown in the example above, you may want to\nmodify it further. For example, you may want to sort all of the `@posts` according to\ntheir date. This can be done like so:\n\n```elixir\ndefmodule MyApp.Catalog do\n  use Catalog\n\n  markdown(:posts, \"posts/**.md\")\n\n  # The @posts variable is first defined by the markdown macro above.\n  # Let's further modify it by sorting all posts by descending date.\n  @posts Enum.sort_by(@posts, \u0026 \u00261.date, {:desc, Date})\n\n  def all_posts(), do: @posts\nend\n```\n\n**Important**: Avoid injecting the `@posts` attribute into multiple functions,\nas each call will make a complete copy of all posts. For example, if you want\nto show define `recent_posts()` as well as `all_posts()`, DO NOT do this:\n\n```elixir\ndef all_posts, do: @posts\ndef recent_posts, do: Enum.take(@posts, 3)\n```\n\nInstead do this:\n\n```elixir\ndef all_posts, do: @posts\ndef recent_posts, do: Enum.take(all_posts(), 3)\n```\n\n## Frontmatter\n\nAll of our content types support frontmatter. Frontmatter is a block at the top of your\ncontent file that contains additional data about the file. They are commonly used for markdown\nfiles, but are supported for all of our macros.\n\n### TOML frontmatter\n\nTOML frontmatter can be used by specifying a TOML block fenced in by the `+++` seperator.\n\n```markdown\n+++\nhello = \"toml\"\n+++\nThis markdown *document* has TOML frontmatter!\n```\n\nTOML frontmatter is processed by the `Toml` libary.\nTo use TOML frontmatter, `Toml` must be added as a dependency:\n\n```\n{:toml, \"~\u003e 0.6.2\"}\n```\n\nYou can customize the TOML processing by providing \n`:toml_options` in your macro call, which will be passed along to `Toml.decode!/1`. \n\n### Elixir frontmatter\n\nElixir frontmatter can be used by including Elixir code fenced in by the `===` seperator.\n\n```markdown\n===\n%{\n  hello: \"elixir\"\n}\n===\nThis markdown *document* has Elxiir frontmatter!\n```\n\nThe code in the block is passed to `Code.eval_string/1`. It must return a `Map`.\n\n### YAML frontmatter (not recommended)\n\nYou can use YAML frontmatter by specifying a YAML block fenced in by the `---` seperator.\n\n```markdown\n---\nhello: yaml\n---\nThis markdown *document* has YAML frontmatter!\n```\n\nYAML frontmatter is processed by the `YamlElixir` libary.\nTo use YAML frontmatter, `YamlElixir` must be added as a dependency:\n\n```\n{:yaml_elixir, \"~\u003e 2.8\"}\n```\n\nYou can customize the YAML processing by providing\n`:yaml_options` in your macro call, which will be passed along to `YamlElixir.read_from_string!/2`.\n\nUnfortunately, this library does not allow us to have the keys returned as atoms instead of strings, which\nmakes it slightly more cumbersome to use. Because of this, we do not recommend using YAML frontmatter.\n\n## Live reloading\n\nIf you are using Phoenix, you can enable live reloading by simply telling Phoenix to watch the “posts” directory. Open up \"config/dev.exs\", search for `live_reload:` and add this to the list of patterns:\n\n```elixir\nlive_reload: [\n  patterns: [\n    ...,\n    ~r\"posts/*/.*(md)$\"\n  ]\n]\n```\n\n## Credits\n\nThis work draws heavily on the [NimblePublisher](https://github.com/dashbitco/nimble_publisher) library by Dashbit.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkevinlang%2Fcatalog","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkevinlang%2Fcatalog","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkevinlang%2Fcatalog/lists"}