{"id":14066792,"url":"https://github.com/brshallo/macro-shell-example","last_synced_at":"2025-10-29T23:12:28.063Z","repository":{"id":139240741,"uuid":"366213011","full_name":"brshallo/macro-shell-example","owner":"brshallo","description":null,"archived":false,"fork":false,"pushed_at":"2021-05-24T20:39:08.000Z","size":123,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-12-04T06:36:48.769Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"R","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/brshallo.png","metadata":{"files":{"readme":"README.Rmd","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-05-11T00:41:39.000Z","updated_at":"2024-09-25T20:41:55.000Z","dependencies_parsed_at":"2023-05-17T17:15:26.489Z","dependency_job_id":null,"html_url":"https://github.com/brshallo/macro-shell-example","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/brshallo/macro-shell-example","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brshallo%2Fmacro-shell-example","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brshallo%2Fmacro-shell-example/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brshallo%2Fmacro-shell-example/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brshallo%2Fmacro-shell-example/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/brshallo","download_url":"https://codeload.github.com/brshallo/macro-shell-example/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brshallo%2Fmacro-shell-example/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260621233,"owners_count":23037669,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-13T07:05:15.838Z","updated_at":"2025-10-29T23:12:27.959Z","avatar_url":"https://github.com/brshallo.png","language":"R","funding_links":[],"categories":["R"],"sub_categories":[],"readme":"---\noutput: github_document\n---\n\n\u003c!-- README.md is generated from README.Rmd. Please edit that file --\u003e\n\n```{r, include = FALSE}\nknitr::opts_chunk$set(\n  collapse = TRUE,\n  comment = \"#\u003e\",\n  warning = FALSE,\n  message = FALSE\n)\n```\n\n# Macros in the Shell... Example\n\nThis repo provides the example walk-through for my blog post on [Macros in the Shell: Integrating That Spreadsheet From Finance Into A Data Pipeline](https://www.bryanshalloway.com/2021/05/10/macros-in-the-shell-integrating-that-spreadsheet-from-finance-into-a-local-data-pipeline/). \n\nClick \"Code\" to download or clone this repository so that you can go through it on your own machine.\n\n![](images/clone-repo.PNG)\n\n## Example\n\nLet's break down the kinds of pipelines we are talking about in [the post](https://www.bryanshalloway.com/2021/05/10/macros-in-the-shell-integrating-that-spreadsheet-from-finance-into-a-local-data-pipeline/) into steps:\n\n1. [Pipeline Computes Things]\n2. [Pipeline Passes Things Through Spreadsheet / Macros]\n3. [Pipeline Collects Outputs from Spreadsheet / Macro]\n\n### Pipeline Computes Things\n\nFor our simple example, pretend you are evaluating deals for investments. Deals come in batches of projects. Projects can have varying...\n\n* costs (paid in lump sums)\n* revenues (received in lump sums)\n* years from costs (paid) until revenue (received)\n\nAn example deal may look like this:\n\n```{r}\nlibrary(tidyverse)\n\nsource(here::here(\"scripts\", \"funs.R\"))\n\nset.seed(1234)\ncreate_deal(12) %\u003e% \n  knitr::kable()\n```\n\nSay you fetched and/or predicted these values through a pipeline you or your team set-up. \n\nFor our example, we are just pretending like this happened, the results of which are simulated by the `create_deal()` function whose logic is in \"scripts/funs.R\" which is called as the first non set-up step in \"run-all.R\".\n\n### Pipeline Passes Things Through Spreadsheet / Macros\n\nThis is where most of the things relevant to the post happen. The attributes for your deal need to be passed as inputs through an excel workbook + VBA macro (maintained by finance or someone other than yourself). The key workbook / macros for this example live in \"data-spreadsheet/deal_calculator.xlsm\". \n\nBelow, I walk through the steps of the operations, indicating at the start of each line with either {DS} or {FINANCE} for whether setting-up the step would likely be the responsibility of the data scientist or finance:\n\n1. {DS} Deal information is written to a location where the spreadsheet is expecting it \n\n\n* This could be a separate .csv, a location within the .xlsm, or as parameters through the VBScript. \n    * In our case the deal \"data-spreadsheet/example_deal.csv\" is expected to be written to the same folder where \"data/deal_calculator.xlsm\" lives; \n    * we also pass in a relative file path via the shell script in *step 2* below.\n2. {DS/FINANCE} Now that the relevant information from upstream is set: the relevant macros are triggered by a VBScript (\"scripts/run_vbs.vbs\") --\u003e which is triggered by a shell function --\u003e which is run via an R script (\"scripts/run-all.R\")\n3. {FINANCE} The spreadsheet in our case does the following:\n\n* Updates all connections in the workbook (i.e. whatever is in \"data/example_deal.csv\") -- via the macro `refresh_data`\n* calculate the present value associated with each project on the deal as well as the total value for the deal -- via excel table calculations and excel formulas\n* copies total deal value to a new cell (this macro is essentially meaningless, but just an extra macro thrown in for example / testing) -- via `copy_deal_value` macro\n\n*...you could imagine there being some more complicated calculations...*\n\n### Pipeline Collects Outputs from Spreadsheet / Macro\n\nThe `readxl::read_excel()` line in \"run-all.R\" then reads the *total deal value* back into R (owner of the spreadsheet would need to ensure that location this is written to will be consistent). And the data pipeline continues on its way...\n\n## Appendix\n\n### Threads helpful for setting-up VBScript\n\nSetting-up the VB wrapper script was a bit annoying (as was setting-up relative paths and passing in parameters to the VB script). Below are a few threads I found helpful:\n\n* https://techcommunity.microsoft.com/t5/excel/power-query-source-from-relative-paths/m-p/206150\n* https://stackoverflow.com/a/15621773/9059865 \n* https://stackoverflow.com/questions/20979154/how-to-close-excel-file-from-vbscript-without-being-prompted\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbrshallo%2Fmacro-shell-example","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbrshallo%2Fmacro-shell-example","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbrshallo%2Fmacro-shell-example/lists"}