{"id":34108213,"url":"https://github.com/avikhagol/alfrd","last_synced_at":"2026-03-09T08:32:09.179Z","repository":{"id":239009991,"uuid":"796241117","full_name":"avikhagol/alfrd","owner":"avikhagol","description":"ALFRD : Automated Logical FRamework for executing scripts Dynamically  This program is written for the SMILE project supported by the ERC starting grant.","archived":false,"fork":false,"pushed_at":"2026-02-13T15:54:59.000Z","size":98,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-02-14T00:11:44.196Z","etag":null,"topics":["alfrd","automation-framework","big-data","google-sheets","gspread","logframe","smilescience"],"latest_commit_sha":null,"homepage":"https://www.smilescience.info/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/avikhagol.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-05-05T11:20:49.000Z","updated_at":"2026-02-13T15:46:26.000Z","dependencies_parsed_at":"2024-05-09T13:34:49.977Z","dependency_job_id":"d69bb2e9-6f4b-45da-86e8-7befcfa48a23","html_url":"https://github.com/avikhagol/alfrd","commit_stats":null,"previous_names":["avialxee/alfrd","avikhagol/alfrd"],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/avikhagol/alfrd","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/avikhagol%2Falfrd","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/avikhagol%2Falfrd/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/avikhagol%2Falfrd/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/avikhagol%2Falfrd/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/avikhagol","download_url":"https://codeload.github.com/avikhagol/alfrd/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/avikhagol%2Falfrd/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30287834,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-09T02:57:19.223Z","status":"ssl_error","status_checked_at":"2026-03-09T02:56:26.373Z","response_time":61,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["alfrd","automation-framework","big-data","google-sheets","gspread","logframe","smilescience"],"created_at":"2025-12-14T18:14:55.823Z","updated_at":"2026-03-09T08:32:09.170Z","avatar_url":"https://github.com/avikhagol.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ALFRD : Automated Logical FRamework for Dynamic script execution(ALFRD)\n\nThis program is written for the [SMILE project](https://smilescience.info) supported by the ERC starting grant, particularly with following purposes in mind:\n\n- Communicate with google spreadsheets(or local csv file) and update progress periodically when required.\n- Execute a pipeline workflow that dynamically operates based on the progress and requirements recorded in the spreadsheet or CSV file.\n\n# Contents:\n- [ALFRD : Automated Logical FRamework for Dynamic script execution(ALFRD)](#alfrd--automated-logical-framework-for-dynamic-script-executionalfrd)\n- [Contents:](#contents)\n  - [1. Create API credentials on Google Cloud](#1-create-api-credentials-on-google-cloud)\n  - [2. Installing](#2-installing)\n  - [3. Using ALFRD](#3-using-alfrd)\n    - [3.1 - Basic Usage](#31---basic-usage)\n      - [3.1.1 Example: Google Spreadsheet - Initializing and creating instance](#311-example-google-spreadsheet---initializing-and-creating-instance)\n      - [3.1.2 Example: Google Spreadsheet - Update the data](#312-example-google-spreadsheet---update-the-data)\n      - [3.1.3 Example: CSV - Update the data (more soon)](#313-example-csv---update-the-data-more-soon)\n    - [3.2 - Advance](#32---advance)\n      - [3.2.1 Example : Execute functions using alfrd](#321-example--execute-functions-using-alfrd)\n      - [3.2.2 Example : Pipeline step execution/update using the Spreadsheet/CSV](#322-example--pipeline-step-executionupdate-using-the-spreadsheetcsv)\n  - [4. Attribution](#4-attribution)\n  - [5. Acknowledgement](#5-acknowledgement)\n\n\n-------\n\n\n## 1. Create API credentials on Google Cloud\n\n\nA similar guide is available [here](https://developers.google.com/workspace/guides/create-credentials) or [https://developers.google.com/workspace/guides/create-credentials](https://developers.google.com/workspace/guides/create-credentials)\n\nNOTE: In order to successfully create a google console project a billing detail is usually required. But the sheets API service is available for free, refer [here](https://developers.google.com/sheets/api/limits)\n\n- step 1:\n  Go to https://console.cloud.google.com/\n\n- step 2: \n\tclick on the drop-down to create a new project \n\t\t- can choose organization or leave on default\n\n- step 3\n  Search in the top bar (or press / ) and type : \"Google sheets api\"\n  \n- step 4\n  In the search results - under Marketplace select the first result which should be the same : Google sheets api\n  \n- step 5\n  Enable the service In the product details page \n  \n- step 6\n  Now select create credentials \u003e Application Data\n  \n- step 7\n  Create an account name\n  Select create and continue\n  \n- step 8\n  Search and select \"Editor\" in role \u003e Continue\n  \n- step 9\n  Skip next optional step\n  Select Done\n  \n- step 10\n  The Credentials are successfully created\n  Select \"Credentials\" on the left menu\n  \n- step 11\n  select/edit account that was just created\n  also copy the email address that is shown\n  \n- step 12\n  Select keys tab \u003e Add keys \u003e Create New Keys \u003e JSON \u003e save\n  \n- step 13\n  Go to the Google spreadsheet and \"share\" the sheet to the email address that was copied, as Editor.\n\n## 2. Installing\n\n- Install using the pip package manager:\n  ```bash\n  pip install alfrd\n  ```\n- Alternatively Download [ALFRD](https://github.com/avialxee/alfrd) and unzip / Or \n    ```bash\n   git clone https://github.com/avialxee/alfrd\n   cd alfrd/\n   pip install .\n    ```\nthis should install alfrd and all the dependencies automatically.\n\n\n## 3. Using ALFRD\n\nALFRD can be used for structuring the pipeline/workflow steps, such that each step (e.g., Step A, Step B, Step C) is represented as a column (A, B, C) in a table, with the workflow executing these steps sequentially according to their order in the table.\n\n### 3.1 - Basic Usage\n\nALFRD relies on pandas dataframe to read/write tabular data.\n\n####  3.1.1 Example: Google Spreadsheet - Initializing and creating instance\n\n```python\nfrom alfrd.lib import GSC, LogFrame\nfrom alfrd.util import timeinmin, read_inputfile\n\nurl='https://spreadsheet/link'\nworksheet='worksheet-name'\n\ngsc = GSC(url=url, wname=worksheet, key='path/to/json/file')      # default path for key = home/usr/.alfred\ndf_sheet = gsc.open()\n```\n\n\n####  3.1.2 Example: Google Spreadsheet - Update the data\n\nThe instance of LogFrame can be used to manipulate the dataframe and update the Google Sheet.\n\n```python\n\nlf = LogFrame(gsc=gsc)\nlf.df_sheet.loc[0, 'TSYS'] = True\n\nlf.update_sheet(count=1, failed=0,csvfile='df_sheet.csv')                     # if updating the sheet fails, a copy of the dataframe is saved locally at the csvfile path.\n\n```\n\n####  3.1.3 Example: CSV - Update the data (more soon)\n\nIt is also possible to use just the CSV file as an alternative to the Google Sheet.\n\n```python\n\nlf = LogFrame(csv='in.csv')\nlf.df_sheet.loc[0, 'TSYS'] = True\nlf.df_sheet.to_csv('out.csv')\n\n```\n\n### 3.2 - Advance\n\n#### 3.2.1 Example : Execute functions using alfrd\n\nCreate `pipe.py` and use the register decorator for creating a pipeline step.\n\n```python \n# pipe.py\nfrom alfrd.plugins import register\n\n\n@register(\"A hello world function\")\ndef step_hello_world(name):\n  print(f\"hello, {name}\")\n\n```\n\nNow we should create a pipeline project and add the script to the project, this can be done as follows:\n\n```bash\nalfrd init PROJECT_NAME # change the PROJECT_NAME to something desirable\nalfrd add /path/to/pipe.py PROJECT_NAME\n```\n\nthis will create a symlink to the project directory found at  `~/.alfrd/projects/PROJECT_NAME/pipe.py`\nThat's it! You have created a pipeline flow, now execute the created script by running the following:\n\n```bash\nalfrd run step_hello_world PROJECT_NAME name=World\n```\n\u003e NOTE: you can create a config file e.g config.txt and modify the above as follows: \\\n\u003e   `alfrd run step_hello_world PROJECT_NAME config.txt` \\\n\u003e and in the config.txt write something like:\n```\n# config.txt\nname = World\n```\n\n#### 3.2.2 Example : Pipeline step execution/update using the Spreadsheet/CSV\n\nNow modify the `pipe.py` and use the validator decorator for defining functions which can be executed just before and after the main pipeline step. \nThe parameters accessed between the pipeline steps and validators can be defined by the `Pipeline.params`. \nAlso one can use the config file to initialize the parameter values in the execution time.\n\n```python \n# pipe.py\n\nfrom alfrd.lib import GSC, LogFrame\n\nfrom alfrd import Pipeline\nfrom alfrd.plugins import validator, validate, register\n\n@validator(desc=\"Update values on the google sheet\", run_once=False, after=True)\ndef update_sheet(lf, success_count, failed_count):\n    lf.update_sheet(success_count, failed_count)\n  \n@validator(desc=\"Connect with the google sheet and return an instance for the runtime parameter\", run_once=True)\ndef connect_sheet(sheet_url, worksheet):\n    gsc = GSC(url=sheet_url, wname=worksheet, key='/path/to/credentials.json')\n    gsc.open()\n    Pipeline.params['lf'] = LogFrame(gsc=gsc)\n\n@validate(by=[connect_sheet, update_sheet])\n@register(\"A hello world function\")\ndef modify_tsys(lf):\n  lf.df_sheet.loc[0, 'TSYS'] = True\n  Pipeline.params['success_count'] = 1\n  Pipeline.params['failed_count'] = 0\n\n```\n\nthe above can be executed as follows:\n```bash\n alfrd run modify_tsys PROJECT_NAME sheet_url=/path/to/sheet worksheet=main\n```\n\n## 4. Attribution\n\nWhen using ALFRD, please add a link to this repository in a footnote.\n\n## 5. Acknowledgement\n\nALFRD was developed within the \"Search for Milli-Lenses\" (SMILE) project. SMILE has received funding from the European Research Council (ERC) under the HORIZON ERC Grants 2021 programme (grant agreement No. 101040021).","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Favikhagol%2Falfrd","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Favikhagol%2Falfrd","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Favikhagol%2Falfrd/lists"}