{"id":19769557,"url":"https://github.com/tjmahr/notestar","last_synced_at":"2025-10-23T17:39:10.738Z","repository":{"id":43455168,"uuid":"329370954","full_name":"tjmahr/notestar","owner":"tjmahr","description":"notestar is a notebook system built on the targets package: notes with targets.","archived":false,"fork":false,"pushed_at":"2022-06-15T21:07:40.000Z","size":768,"stargazers_count":29,"open_issues_count":7,"forks_count":2,"subscribers_count":3,"default_branch":"main","last_synced_at":"2023-06-28T12:20:40.461Z","etag":null,"topics":["bookdown","knitr","pandoc","r","rmarkdown","targets"],"latest_commit_sha":null,"homepage":"","language":"R","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tjmahr.png","metadata":{"files":{"readme":"README.Rmd","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-01-13T16:43:47.000Z","updated_at":"2023-06-28T12:20:40.463Z","dependencies_parsed_at":"2022-08-23T22:51:44.318Z","dependency_job_id":null,"html_url":"https://github.com/tjmahr/notestar","commit_stats":null,"previous_names":[],"tags_count":null,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tjmahr%2Fnotestar","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tjmahr%2Fnotestar/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tjmahr%2Fnotestar/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tjmahr%2Fnotestar/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tjmahr","download_url":"https://codeload.github.com/tjmahr/notestar/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224219737,"owners_count":17275477,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bookdown","knitr","pandoc","r","rmarkdown","targets"],"created_at":"2024-11-12T04:43:49.394Z","updated_at":"2025-10-23T17:39:05.688Z","avatar_url":"https://github.com/tjmahr.png","language":"R","funding_links":[],"categories":["R"],"sub_categories":[],"readme":"---\noutput: github_document\n---\n\n\u003c!-- README.md is generated from README.Rmd. Please edit that file --\u003e\n\n\u003c!-- say directory, not folder --\u003e\n\n```{r, include = FALSE}\nknitr::opts_chunk$set(\n  collapse = TRUE,\n  comment = \"#\u003e\",\n  fig.path = \"man/figures/README-\",\n  out.width = \"100%\"\n)\nknitr::opts_template$set(\n  file_content = list(\n    comment = \"\",\n    collapse = FALSE\n  )\n)\n```\n\n# notestar 📓⭐\n\n\u003c!-- badges: start --\u003e\n[![R-CMD-check](https://github.com/tjmahr/notestar/workflows/R-CMD-check/badge.svg)](https://github.com/tjmahr/notestar/actions)\n\u003c!-- badges: end --\u003e\n\nnotestar is a notebook system built on the\n[targets](https://docs.ropensci.org/targets/) package: *notes* with\n*tar*gets.\n\n## Installation\n\nYou can install notestar from [GitHub](https://github.com/) with:\n\n``` r\n# install.packages(\"devtools\")\ndevtools::install_github(\"tjmahr/notestar\")\n```\n\n## A demo notebook\n\nHere is an example project/notebook showing how notestar combines\nvarious .Rmd files into a single HTML file:\n\u003chttps://github.com/tjmahr/notestar-demo\u003e.\n\n\n## Big picture idea of how notestar works\n\nFirst, we follow a [targets](https://docs.ropensci.org/targets/)-based\nworkflow. We develop datasets and models and so on, writing functions in\nR/functions.R and describing a build pipeline in `_targets.R`. Some\nfamiliarity with the big ideas of the targets packages is required.\n\nWe then work with our data-analysis products in RMarkdown .Rmd files in\na notebook directory. We read in these targets using `targets::tar_read()`,\nand we might develop several entries notebook as we tackle different\nparts of our analysis problem.\n\nIn the `_targets.R` file, there are special notebook-related targets.\nWhen we run `targets::tar_make()`, notestar does the following:\n\n  - For each .Rmd file, it knits (`knitr::knit()`) the corresponding .md\n    output file: computing running the code, printing the results,\n    saving and inserting figures.\n    \n  - These .md files are collated and assembled into a single-page\n    bookdown document. It looks kind of a data-analysis blog (a page\n    with a sequence of entries in reverse-chronological order).\n    \nImportantly, **notestar only does these jobs when needed**. For example,\na notebook entry's .md file will only be created if it is *outdated*.\nThat is,\n\n  - if the .md has not been created yet.\n  - if the .Rmd source file has changed at all.\n  - if a dependency loaded via targets (i.e., `data \u003c-\n    targets::tar_read(data)`) in the .Rmd source file has changed.\n  - if any other dependencies for the .md file have changed. (For\n    example, there is an R script that run before knitting each entry\n    that sets knitr options. If that changes, then the notebook entry is\n    outdated.)\n\nnotestar's role in all of this is to link the data analysis targets to\nthe .Rmd files and then orchestrate the assembly of a notebook from\nthese entries. \n\n### Packages that help make this happen\n\nLet's highlight some packages that are indispensable for this scheme to work.\n\n  - [targets][targets] (backend)\n  - [knitr][knitr] (running R code in .Rmd files to .md files)\n  - [bookdown][bookdown] (assembling .md files into a .md file)\n  - [rmarkdown][rmarkdown] / [pandoc][pandoc] (for converting\n    markdown into other formats)\n  - [cleanrmd][cleanrmd] (theme/template for final html document)\n  - [usethis][usethis] (functions for setting up R-based data\n    analysis projects)\n\n[targets]: https://docs.ropensci.org/targets/ \"targets site\"\n[knitr]: https://yihui.org/knitr/ \"knitr site\"\n[bookdown]: https://bookdown.org/yihui/bookdown/ \"bookdown site\"\n[rmarkdown]: https://rmarkdown.rstudio.com/ \"rmarkdown site\"\n[pandoc]: https://pandoc.org/ \"pandoc site\"\n[cleanrmd]: https://github.com/gadenbuie/cleanrmd \"cleanrmd: Clean Class-Less R Markdown HTML Documents\"\n[usethis]: https://usethis.r-lib.org/ \"usethis site\"\n\n\n## Most important functions for users\n\nBelow I show a worked example and describe things in great detail. But\nbefore that I want to note that as a user, I only really use 3--4\nfunctions from this package.\n\n  - `use_notestar()` to set up a notestar project\n  - (optional) `use_notestar_makefile()` to set up a Makefile that runs\n    `targets::tar_make()`. I then use the RStudio's Build commands to\n    build projects.\n  - (optional) `use_notestar_references()` to set up a .bib and .csl file \n    for the notebook.\n  - `notebook_create_page()` to create a new notebook entry\n  - `notebook_browse()` to open the final notebook file in a browser.\n\n\n## A worked example\n\nnotestar works best inside of a data analysis project and specifically,\nas a part of an [RStudio\nproject](https://support.rstudio.com/hc/en-us/articles/200526207-Using-RStudio-Projects).\nThat is, we have some directory for our project. Everything we do or\ncreate will live in that directory, and that directory is the default\nworking directory for all of our R code.\n\nFor demonstration, let's create a new directory inside of a temporary\ndirectory and make that the home base for our project.\n\n```{r}\nproject_dir \u003c- file.path(tempdir(), pattern = \"my-project\")\ndir.create(project_dir)\nsetwd(project_dir)\n```\n\n```{r, echo = FALSE}\nknitr::opts_knit$set(root.dir = project_dir)\n```\n\nNothing here!\n\n```{r}\nfs::dir_tree(all = TRUE)\n```\n\n### Initial file skeleton\n\n`use_notestar()` will populate the project directory with the basic\nskeleton for the project. We set the theme to `\"water-dark\"` so that the\nscreenshots below stick out better from the white background on GitHub.\n\n```{r}\nlibrary(notestar)\nuse_notestar(cleanrmd_theme = \"water-dark\")\n\nfs::dir_tree(all = TRUE)\n```\n\nThe file **`config.yml`** is a\n[config-package](https://rstudio.github.io/config/) configuration file.\nThese configuration options were set when we called `use_notestar()`, so\nthese are all the default configuration options (except for\n`cleanrmd_theme`). Each of these is described by a `comment` field.\n\n```{r, opts.label = \"file_content\"}\nwriteLines(readLines(\"config.yml\"))\n```\n\nTwo .Rmd files are automatically included: `index.Rmd` and\n`0000-00-00.Rmd`. These are the first and last entries (top and bottom\nparts) of the notebook. **`index.Rmd`** houses metadata for the\nnotebook:\n\n```{r, opts.label = \"file_content\"}\nwriteLines(readLines(\"notebook/index.Rmd\"))\n```\n\nThe yaml metadata in `index.Rmd` is created automatically inside the\n`_targets.R` file. More on that later.\n\n**`0000-00-00.Rmd`** is not meant to be edited. As it tells us, it\nprovides a \"References\" heading. When the bibliography is appended to\nthe end of the notebook, it will be printed under this heading.\n\n```{r, opts.label = \"file_content\"}\nwriteLines(readLines(\"notebook/0000-00-00-references.Rmd\"))\n```\n\nThe file **`_targets.R`** orchestrates the compilation of the notebook using\nthe targets package. `targets::tar_make()` compiles the notebook by:\n\n  - knitting each .Rmd file in `notebook` *if* necessary to produce a\n    corresponding .md file `notebook/book/`.\n  - collating the .md files in `notebook/book/` into a single-document\n    bookdown book with bookdown/RMarkdown/pandoc (*if* necessary).\n\nI say \"*if* necessary\" because targets only builds the targets in\nworkflow if the target has not been built yet or if the target is out of\ndate. Thus, notestar doesn't waste time regenerating earlier entries if\nthey or their dependencies have not changed.\n\nFinally, **`.here`** is a sentinel file for the\n[here](https://here.r-lib.org/) package. It indicates where the project\nroot is located. **`R/functions.R`** is an (as-yet empty) R script that\nis `source()`-ed at the start of `_targets.R`.\n\n### Building, rebuilding\n\nHere we build the notebook and see targets build each target.\n\n```{r}\ntargets::tar_make()\n```\n\nIf we ask it to build the book again, it skips everything---none of the\ndependencies have changed---but a special spell-checking target set to\nalways run.\n\n```{r}\ntargets::tar_make()\n```\n\nRight now, our compiled notebook (`\"notebook/book/docs/notebook.html\"`)\nis just the title page:\n\n```{r shot1, out.width = \"35%\", echo = FALSE}\nwebshot::webshot(\n  \"notebook/book/docs/notebook.html\",  \n  file = \"shot1.png\",\n  vwidth = 400, \n  vheight = 400,\n  zoom = 2\n)\n```\n\nIf we look at the project tree, we see some additions.\n\n```{r}\nfs::dir_tree(all = TRUE)\n```\n\n`_targets/` is a new directory. It is the object and metadata storage\nfor targets. We don't worry about it.\n\nThere are some md files in `notebook/book/` as well as some\nbookdown-related files (`_bookdown.yml`, `_output.yml` and\n`notebook.rds` file). There is also the output of bookdown in\n`notebook/book/docs`. (`notebook/book/docs/notebook.html` is the file we\nscreenshotted earlier.) \n\n`knitr-helpers.R` was also copied to the `notebook/book/` directory. This\ncopying reflects design decision by the package. **Namely, the contents\nof the `notebook/book` directory should not be edited by hand.** Its\ncontents should be reproducible whether by regenerating files (like the\n.md files) or by copying files (like `knitr-helpers.R`. The user should\nonly have to worry about editing files in the `notebook/` directory or in\n`_targets.R` (or perhaps `config.yml`). \n\nWe can create a new entry from a template using `notebook_create_page()`\nand regenerate the notebook. (A slug is some words we include in the\nfilename to help remember what the entry is about.)\n\n```{r}\nnotebook_create_page(date = \"2022-02-22\", slug = \"hello-world\")\n```\n\nNow targets has to rebuild the notebook because there is a new entry\nthat needs to be folded in. The network diagram shows that\n`entry_2022_02_hello_world_rmd` is outdated (blue) so everything\ndownstream from it is also outdated.\n\n```{r network1, dpi = 144}\ntargets::tar_visnetwork(targets_only = TRUE)\n```\n\nWhen we rebuild the notebook, that entry now appears in \nthe HTML file.\n\n```{r shot2, out.width = \"35%\", echo = 1:2, results = \"hide\"}\ntargets::tar_make()\n#\u003e [output omitted]\n\nwebshot::webshot(\n  \"notebook/book/docs/notebook.html\",\n  file = \"shot2.png\",\n  vwidth = 400, \n  vheight = 500, \n  zoom = 2\n)\n```\n\n### Next steps\n\nFrom here, we go with the flow. We use targets as we normally would,\nmodifying `R/functions.R` and `targets.R` to set up our data-processing\npipeline. We can now use our notebook to do reporting and exploration as\npart of our data-processing pipeline. Things we make with targets can be\n`tar_read()` into our notebook entries and tracked as dependencies.\n\n\n### Behind-the-scenes details\n\nIn this section, we will describe some behind-the-scenes details about\nnotestar using the worked example.\n\n#### 1. .Rmd/.md files have hidden timestamps\n\nHere is what a minimal Rmd file entry looks like:\n\n```{r, opts.label = \"file_content\"}\nwriteLines(readLines(\"notebook/2022-02-22-hello-world.Rmd\"))\n```\n\nThat first `\u003c!--- comment ---\u003e` line on top is an HTML comment. It will\nnot be displayed when we view the final html file, but when the .Rmd\nfile is knitted to produce the corresponding .md, the timestamp will be\nupdated. Here is the first line of that .md file:\n\n```{r, opts.label = \"file_content\"}\nwriteLines(readLines(\"notebook/book/2022-02-22-hello-world.md\")[1])\n```\n\nThis timestamp allows us to mark a notebook entry as outdated even if\nnone of the text in the .md file has changed. Here is a motivating\nexample. Let's append a code chunk to the bottom of the notebook entry.\nIt will plot a histogram.\n\n```{r}\nentry_v0 \u003c- readLines(\"notebook/2022-02-22-hello-world.Rmd\")[1:13]\nwriteLines(\n  c(\n    entry_v0,\n    \"Would you look at all these 4's?\",\n    \"```{r old-faithful, fig.width = 4, fig.height = 4}\",\n    \"hist(faithful$eruptions)\",\n    \"```\"\n  ),\n  \"notebook/2022-02-22-hello-world.Rmd\"\n)\n```\n\nAnd then we regenerate the notebook.\n\n```{r shot3, out.width = \"35%\", echo = 1:2, results = \"hide\"}\ntargets::tar_make()\n#\u003e [output omitted]\n\nwebshot::webshot(\n  \"notebook/book/docs/notebook.html\",  \n  file = \"shot3.png\",\n  vwidth = 400, \n  vheight = 400,\n  zoom = 2\n)\n```\n\nLet's store the current .md file lines so we can compare it to a later\nversion.\n\n```{r}\nentry_v1 \u003c- readLines(\"notebook/book/2022-02-22-hello-world.md\")\n```\n\nNow, suppose we wanted to change size or resolution of the plot. In this case, \nwe will change the `fig.width` and `fig.height` values to 6 here and regenerate\nthe notebook\n\n```{r shot4, out.width = \"35%\", echo = 1:5, results = \"hide\"}\nwriteLines(\n  c(\n    entry_v0,\n    \"Would you look at all these 4's?\",\n    \"```{r old-faithful, fig.width = 6, fig.height = 6}\",\n    \"hist(faithful$eruptions)\",\n    \"```\"\n  ),\n  \"notebook/2022-02-22-hello-world.Rmd\"\n)\n\ntargets::tar_make()\n#\u003e [output omitted]\n\nwebshot::webshot(\n  \"notebook/book/docs/notebook.html\",  \n  file = \"shot4.png\",\n  vwidth = 400, \n  vheight = 400,\n  zoom = 2\n)\n```\n\n\nThe figures image files have definitely changed: they are different\nsizes! The text in the plots in the two screenshots are different\nsizes. But the text of the .md files is the same---except for the\ntimestamp.\n\n```{r}\nentry_v2 \u003c- readLines(\"notebook/book/2022-02-22-hello-world.md\")\nentry_v1 == entry_v2\nentry_v1[1] \nentry_v2[1]\n```\n\nThis phenomenon, where a change to an .Rmd file would not cause a change\nin the text of a .md file, is the reason for the timestamp at the top of\nthe .Rmd file.\n\n#### 2. Despite the timestamp, image dependencies are tracked\n\nOur targets graph has a node called `entry_2022_02_22_hello_world_md`. \n\n```{r network2, dpi = 144}\ntargets::tar_visnetwork(\n  targets_only = TRUE, \n  names = \"entry_2022_02_22_hello_world_md\"\n)\n```\n\nThat node does not represent just the file\n`notebook/book/2022-02-22-hello-world.md`. Its plot is also tracked as\nbyproduct of the entry:\n\n```{r}\ntargets::tar_read(\"entry_2022_02_22_hello_world_md\")\n```\n\nThus, if I removed the image, that notebook entry becomes outdated and needs\nto be reprocessed.\n\n```{r network-3, dpi = 144}\nfile.remove(targets::tar_read(\"entry_2022_02_22_hello_world_md\")[2])\n\ntargets::tar_visnetwork(\n  targets_only = TRUE, \n  names = \"entry_2022_02_22_hello_world_md\"\n)\n```\n\n(I forget the problem that motivated me to add this layer of tracking on\ntop of the timestamping, but it's there.) \n\n```{r, results = \"hide\"}\n# undo the deletion before moving on\ntargets::tar_make()\n#\u003e [output omitted]\n```\n\n\n#### 3. Unused figures are automatically deleted\n\nThink about any other time you've used knitr or RMarkdown. When you\nremove the code to produce a figure in an .Rmd file, what happens to the\nplot's image file? Normally, it sticks around, and you eventually find\nyourself with all kinds of old, no-longer used figures. When knitting a\n.Rmd file, notestar removes all existing figures associated with an\nentry beforehand. As a result, only figures that were created during the\nmost recent knitting are retained. This move is what allows image\ndependencies (see last point) to be inferred: If an image file is\ncreated as a result of knitting an .Rmd file, we can associate it with\n.md file.\n\nLet's demonstrate this feature. Here are the current notebook assets:\n\n```{r}\nfs::dir_tree(\"./notebook/book/assets\")\n```\n\nWe will restore the original version of the entry so that the plot is no\nlonger created.\n\n```{r, results = \"hide\"}\nwriteLines(entry_v0, \"notebook/2022-02-22-hello-world.Rmd\")\n\ntargets::tar_make()\n#\u003e [output omitted]\n```\n\nWhat we have now is an empty directory.\n\n```{r}\nfs::dir_tree(\"./notebook/book/assets\")\n```\n\nThis behavior is controlled in the `knitr-helpers.R` file, specifically\nthe last line:\n\n```{r, opts.label = \"file_content\"}\nwriteLines(readLines(\"notebook/knitr-helpers.R\"))\n```\n\n\n```{r, echo = FALSE}\nknitr::opts_knit$set(root.dir = NULL)\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftjmahr%2Fnotestar","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftjmahr%2Fnotestar","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftjmahr%2Fnotestar/lists"}