{"id":19184620,"url":"https://github.com/gatoreducator/gatortracer","last_synced_at":"2026-06-13T01:32:49.830Z","repository":{"id":180783702,"uuid":"656321181","full_name":"GatorEducator/GatorTracer","owner":"GatorEducator","description":"Fetching JSON assignments","archived":false,"fork":false,"pushed_at":"2023-09-04T19:23:38.000Z","size":193,"stargazers_count":0,"open_issues_count":4,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-23T02:31:52.640Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/GatorEducator.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-06-20T17:55:22.000Z","updated_at":"2024-06-18T17:08:01.000Z","dependencies_parsed_at":null,"dependency_job_id":"f3e15b65-a0f8-4e54-a496-82e9958ed4fb","html_url":"https://github.com/GatorEducator/GatorTracer","commit_stats":null,"previous_names":["gatoreducator/gatortracer"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/GatorEducator/GatorTracer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GatorEducator%2FGatorTracer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GatorEducator%2FGatorTracer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GatorEducator%2FGatorTracer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GatorEducator%2FGatorTracer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/GatorEducator","download_url":"https://codeload.github.com/GatorEducator/GatorTracer/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GatorEducator%2FGatorTracer/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34269363,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-12T02:00:06.859Z","response_time":109,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T11:07:41.148Z","updated_at":"2026-06-13T01:32:49.810Z","avatar_url":"https://github.com/GatorEducator.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# GatorTracer\n\nFetching Json from GitHub with the rule you make to get a dataset the way you want\n\n## Setup\n\n- GitHub: To access to one person's repository in a GitHub organization, that person has to be at least a member in that organization. Outside collaborator permission won't allow our tool to fetch the target repositories. This tool also expects to have a personal access token(PAT) as input. To setup a PAT, please check this [GitHub tutorial](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens#). Or it's more recommended to use GitHub CLI to authenticate in shell and run `gh auth status --show-token` to display the PAT automatically generated by `GitHub CLI`. For the steps of setting up `GitHub CLI`, please check [GitHub CLI QuickStart](https://docs.github.com/en/enterprise-cloud@latest/github-cli/github-cli/quickstart)\n- GatorTracer: our tool currently hasn't been shipped on the `pypi`. The only approach to use this tool is by cloning this repository link and run `poetry install` in the root directory of this tool to set up an poetry virtual environment.\n\n## Features\n\nBefore everything, please run `poetry install` to install all the required dependencies.\n\n### Token Manipulation\n\nAs mentioned above, this tool needs GitHub Authentication like PAT to communicate with GitHub to access to the those target Json files. It's reasonable to assume passing tokens every time running command is annoying and inhuman. So this tool gives users choice of using a temporary token or a saved token set up by users themselves.\n\nTo manipulate or control saved token, please use the official CLI related to saved-token:\n\n- `poetry run gatortracer saved-token --verify`\n- `poetry run gatortracer saved-token --save CERTAIN_TOKEN`\n- `poetry run gatortracer saved-token --remove`\n  \nto check existence, save a new token and remove saved token.\n\nHere is an example:\n\n```bash\n\u003e\u003e ~/cs/yes/garage/GatorTracer: poetry run gatortracer saved-token --save fake_token                                       \nToken has been saved\n```\n\nThe list of flags associated with `poetry run gatortracer saved-token`:\n\n```md\n\n│ --verify  -v            verify if there is stored gh token                                                                                         │\n│ --save    -s      TEXT  save a new gh token                                                                                                        │\n│ --remove  -r            remove the currente stored gh token                                                                                        │\n│ --help                  Show this message and exit.                                                                                                │\n```\n\n### Configuration Manipulation\n\nHere are currently two configuration files: `include.json` and `exclude.json` in charge of included organizations plus repositories as well as excluded ones. `json_fetch` related commands consults those two Json files to find wanted organizations and repositories. **It's required to set up those two files before running any other commands**. P.S.: while both `include.json` and `exclude.json` being specified, only `include.json` will work.\n\nTo write one configuration json file, please have a Json file following the template below and using regular expression to specify the content you want to include/exclude.**Here the content of organization should be the url name like `my-org-1` other than display name `My Organization Name`**. You can find url name from url and fetch only the organization part. e.g.: fetching `GatorEducator` in `https://github.com/GatorEducator`\n\n```json\n\n{\n    \"organization\": [],\n    \"repository\": []\n}\n\n```\n\nHere is an example of `include.json` that includes all the organizations whose name starts with `allegheny-college-sand` or starts with `allegheny-haunted` and include all the repositories whose name ends with `Yanqiao4396` or starts with `hey` within those selected organizations\n\n```json\n{\n    \"organization\": [\n        \"^allegheny-college-sand+.\",\n        \"^allegheny-haunted+.\"\n    ],\n    \"repository\": [\n        \".+Yanqiao4396$\",\n        \"hey.+\"\n    ]\n}\n\n```\n\nTo interact with configuration files, use the commands like\n\n- `poetry run gatortracer config --display-in`\n- `poetry run gatortracer config --clear-in`\n- `poetry run gatortracer config --in-from-file`\n\nAn example of usig `poetry run gatortracer config --in-from-file`\n\n```bash\n\u003e\u003e ~/cs/yes/garage/GatorTracer: poetry run gatortracer config --in-from-file example_in.json\n\u003e\u003e ~/cs/yes/garage/GatorTracer: poetry run gatortracer config --display-in\n\ninclude.json\n\n{\n    \"organization\": [\n        \"^yes+.\",\n        \"^org+\"\n    ],\n    \"repository\": [\n        \"^sample-repo+.\"\n    ]\n}\n\n\n```\n\nHere is a list of available flags associated with `poetry run gatortracer config`\n\n```md\n│ --display-all               display both exclude and include config json file                                                                      │\n│ --display-in                display the include config json file                                                                                   │\n│ --display-ex                display the exclude config json file                                                                                   │\n│ --clear-all                 clear both exclude and include config json file                                                                        │\n│ --clear-in                  clear include config json file                                                                                         │\n│ --clear-ex                  clear exclude config json file                                                                                         │\n│ --in-from-file        TEXT  Write configuration include.json from another json file                                                                │\n│ --ex-from-file        TEXT  Write configuration exclude.json from another json file                                                                │\n│ --help                      Show this message and exit.                                                                                            \n```\n\n### Main Feature: Fetch Json\n\n`poetry run gatortracer js-fetch` commands fetch Json files from certain path in certain branch in certain repositories in certain organizations.\n\nThe output of `js-fetch` will be a series of tables that could fall in two categories: `MainTable` and `CheckTables`. `MainTable` includes the report level information generated by report of `GatorGrade`. And the specific checks of `GatorGrade` will be shipped to sub-tables naming after the name of check. the insight report in `MainTable` and its subordinate-checks are linked by `uid`, a unique identifier generated based on report level information. `uid` exists in all the tables. **If two rows share the same `uid` in or across tables, then it literately means these two checks belong to a same insight report.**\n\nIf you try to store output of `js-fetch` to a path where `MainTable` and other tables have already existed, **then the new output will be appended to the old tables other than overwriting old ones.**\n\nHere is a list of flags associated with it.\n\n```md\n│ *  --token          -t      TEXT  Choose the kind of token to use: either the saved token or the temporary token [default: None] [required]        │\n│    --branch         -b      TEXT  The branch where json(s) reside [default: insight]                                                               │\n│ *  --dir            -d      TEXT  The directory where json(s) reside [default: None] [required]                                                    │\n│    --file           -f      TEXT  The file names in the regex format [default: .]                                                              │\n│    --parse-insight  -p            parsing insight ```checks to output matrix [default: True]                                                          │\n│    --store-path     -s      TEXT  The path where the output files will inhabit. [default: .]                                                       │\n│    --help                         Show this message and exit. \n```\n\nHere is an example of command using js-fetch:\n`poetry run gatortracer js-fetch -t s -b insight -d insight -f \"(^insight+.)|(^hello-world+.)\" -s tables`\nwhat this command does is: with saved token, fetch all the Json files in path `insight` of branch `insight`. Json file names should start with `insight` or `hello-world`. Finally save all the output tables under a directory called `tables`.\n\n### Check Selection\n\n`poetry run gatortracer select-check` selects all the qualified checks with `attribute` and `attribute-value` and qualified check DataSet as a csv file.\n\nHere is a list of flags associated with it:\n\n```md\n│ *  --main-path    -p      TEXT  The directory where main table inhabit [default: None] [required]                                                  │\n│ *  --attribute    -a      TEXT  the attribute check selection is subject to [default: None] [required]                                             │\n│ *  --value        -v      TEXT  the value associate with the attribute [default: None] [required]                                                  │\n│    --save-file    -s      TEXT  if specified, then save output as csv in the path you choose                                                       │\n│    --table        -t      TEXT  the table where you want to select checks from, all the available tables will be selected by default. [default: .] │\n│    --with-report  -r            combine checks with report file information [default: True]                                                    │\n│    --help                       Show this message and exit.                                                        \n```\n\nHere is an example of it: `poetry run gatortracer select-checks --main-path examples/tables --attribute status --value False --save-file examples/status_false.csv`\n\nHere by using the tables in `example/tables`, I want to fetch all the checks whose status is False and save those checks into a csv file named `examples/status_false.csv`\n\n## Using BranchWrite\n\nUsing `BranchWrite` to automatically generate Json files in a certain branch within workflow, here it's recommended to use [BranchWrite](https://github.com/GatorEducator/BranchWrite) dynamically write Json files. For [GatorGrade](https://github.com/GatorEducator/gatorgrade), BranchWrite is extremely helpful to store students GatorGrade reports for future data analysis.\n\n## License\n\nThis project is licensed under the terms of the MIT license.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgatoreducator%2Fgatortracer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgatoreducator%2Fgatortracer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgatoreducator%2Fgatortracer/lists"}