{"id":18999088,"url":"https://github.com/crossr/rlog","last_synced_at":"2026-04-18T16:30:21.562Z","repository":{"id":53964915,"uuid":"178088532","full_name":"CrossR/rLog","owner":"CrossR","description":"A CLI tool for logging scientific workflows.","archived":false,"fork":false,"pushed_at":"2021-03-12T11:39:03.000Z","size":211,"stargazers_count":1,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-01-01T17:42:32.267Z","etag":null,"topics":["reasonml"],"latest_commit_sha":null,"homepage":"","language":"OCaml","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CrossR.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-03-27T23:13:59.000Z","updated_at":"2019-11-10T00:19:12.000Z","dependencies_parsed_at":"2022-08-13T05:20:46.866Z","dependency_job_id":null,"html_url":"https://github.com/CrossR/rLog","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CrossR%2FrLog","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CrossR%2FrLog/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CrossR%2FrLog/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CrossR%2FrLog/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CrossR","download_url":"https://codeload.github.com/CrossR/rLog/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240018705,"owners_count":19734873,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["reasonml"],"created_at":"2024-11-08T17:49:16.897Z","updated_at":"2025-02-21T13:14:27.013Z","avatar_url":"https://github.com/CrossR.png","language":"OCaml","readme":"# rLog\n\n\u003cp align=\"center\"\u003e\n\u003ca href=\"https://dev.azure.com/CrossR-1/rLog/_apis/build/status/CrossR.rLog?branchName=master\"\u003e\u003cimg alt=\"Build Status\" src=\"https://dev.azure.com/CrossR-1/rLog/_apis/build/status/CrossR.rLog?branchName=master\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\nrLog is a tool for logging the output of commands for scientific\nworkflows.\n\nI find myself very often running some long-running command to produce\nsimulation output or similar, where the output depends very heavily on the\ncurrent state of my environment.\n\nHaving the automated ability to run a set of commands, and store their output\nmeans that I can log out not only the result of a command, but the state of\nthe environment at the time of the simulation.\n\nFor example:\n\n - `echo` out environment variables to check library versions.\n - `git` commands to get the current commit and `diff` of the repository.\n - Dumping out the current contents of a configuration file.\n - Status of input files.\n\n## Configuration\n\nConfiguration is achieved with a `config.json` in `~/.config/rLog/`:\n\n```json\n{\n    \"outputPath\": \"~/rLogOut\",\n    \"logCommands\": [],\n    \"linkCommands\": [],\n    \"loadLocalCommands\": false,\n    \"valuesToLog\": [\"@LOG@: \"]\n}\n```\n\nWhere:\n\n - `outputPath` is the folder log files will be stored in.\n - `logCommands` is the additional logging commands to run.\n - `linkCommands` is the commands to run when calling `rLog link`.\n - `localLocalCommands` enables the ability to use a per project config file.\n    This will cause `rLog` to load a local `config.json`, if there is\n    one in the current directory. It will also attempt to get one from the `git`\n    root, if one exists. Any values in the `logCommands` and `valuesToLog`\n    will be combined, but the `outputPath` will be overridden. The override\n    order is Current directory \u003e Project \u003e Global, so the current folder config\n    has the highest priority.\n - `valuesToLog` is a list of strings that appear at the start of output\n    that is especially important to log. For example `echo \"@LOG@: Cosmics\n    disabled\"` will cause `Cosmics disabled` to appear in the metadata log\n    file separately, so it is more easily found. These values can be printed\n    in anyway, as long as they appear in the terminal `stdout`/`stderr`. This\n    must be explicitly enabled by passing `-p` or `--parse` to the CLI, since\n    it can slow down commands considerably.\n\n### Command variables\n\nBoth `logCommands` and `linkCommands` can contain variables which will\ndynamically be replaced.\n\n| Variable | Replacement | Command |\n|----------|-------------|---------|\n| `${CWD}`   | The directory `rLog` was opened in. | Both |\n| `${GITROOT}` | The git root of the `${CWD}` if applicable. If not, this command will not run. | Both |\n| `${OUTPUTFILE}` | The passed output file(s). If multiple files are given, the command is invoked multiple times. | Link |\n| `${OUTPUTFILES}` | The full list of passed files. | Link |\n| `${COMMAND}` | The full passed command. | Run |\n\n### Example Config\n\n```json\n{\n    \"outputPath\": \"~/rLogOut\",\n    \"logCommands\": [\n       \"git -C ~/git/simulator status\",\n       \"git -C ~/git/simulator diff\",\n       \"git -C ${GITROOT} status\",\n       \"git -C ${GITROOT} diff\",\n       \"env\",\n       \"bash ~/scripts/parseConfigFiles.sh ${COMMAND}\"\n    ],\n    \"linkCommands\": [\n       \"cksum ${OUTPUTFILE}\",\n       \"bash ~/scripts/parseOutputFile.sh ${OUTPUTFILE}\",\n       \"echo ${OUTPUTFILES} \u003e\u003e ~/logs/totalOutputFiles.txt\"\n    ],\n    \"loadLocalCommands\": false,\n    \"valuesToLog\": [\"@LOG@: \"]\n}\n```\n\nIn this example, when I use `rLog` I'll get the current status of my\nsimulator, as well as the state of the `git` repository, both of which can\nplay a big part in why an output looks the way it does.\n\nThe environment is fully logged out, to pick up the library versions and such\nthat are setup with environment variables, and a script is ran on the full\ncommand that will parse the config file out and store the status of the\nconfig file that was used.\n\nAfter running, the files are passed through a quick checksum, parsed further\nand also stored in a secondary log file.\n\n## Usage\n\nUsage should just be `rLog -- commandToRun`, to run `commandToRun` and have\nit logged to the location outlined in the config file, with the `logCommands`\nran to produce metadata for that command.\n\nRunning `rLog --help` will show the command line flags and their usage.\n\n`rLog link -- output1.txt` can be used to link to the most recent run. Link\nin this context will create a symlink to a new `output` file, which will run\nany of the `linkCommands`, as well as giving an easy way of accessing the\ncommand output and metadata output.\n\n`rLog genconfig` will generate a default config file in\n`~/.config/rLog` (or in a custom location if `--config-path` is\ngiven.) This can be useful for making multiple configs, so that aliases can\nbe setup for certain workflows. That is, using something like\n`alias pLog=rLog --config-path ~/configs/pConfig.json` means `pLog` can now\nbe used to automatically use a project specific config, if a project spans\nmultiple repositories, so using a project local config is not an option.\n\n## ToDo\n\nStuff to do:\n\n - [X] Make a release, so I have a binary.\n - [X] Return the ran commands status code.\n - [X] Implement the project specific config.\n - [X] Add command to get a link to the most recent config (so then I can do\n    `do_simulation.sh` and then `rLog link` and have a link to the\n    log file in the data location too).\n - [X] Add a variable syntax to the commands (so I can define dynamic\n    commands, that depend on the env/command ran, not just static ones).\n - [ ] Add a search/list command for listing the ran commands.\n - [ ] Check if I can get this running on Windows (not a priority since none\n    of my workflow that benefits from this is on Windows...).\n\n## Thanks\n\nThanks to both\n\n - @Schniz : https://github.com/Schniz/fnm\n - @ulrikstrid : https://github.com/ulrikstrid/reenv\n\nFor their existing CLI apps that I used heavily to help fix things I couldn't\nget working! Especially around making a static binary.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcrossr%2Frlog","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcrossr%2Frlog","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcrossr%2Frlog/lists"}