{"id":26385168,"url":"https://github.com/danwallach/github-classroom-utils","last_synced_at":"2025-03-17T07:40:11.712Z","repository":{"id":39990720,"uuid":"187918017","full_name":"danwallach/github-classroom-utils","owner":"danwallach","description":"Python tools for instructors working with GitHub Classroom","archived":false,"fork":false,"pushed_at":"2023-04-22T19:50:23.000Z","size":104,"stargazers_count":87,"open_issues_count":0,"forks_count":20,"subscribers_count":8,"default_branch":"master","last_synced_at":"2024-04-18T14:12:49.854Z","etag":null,"topics":["classroom","github-api","github-classroom","python","teachers"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/danwallach.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-05-21T21:46:36.000Z","updated_at":"2023-12-19T17:24:48.000Z","dependencies_parsed_at":"2022-09-01T10:31:58.934Z","dependency_job_id":null,"html_url":"https://github.com/danwallach/github-classroom-utils","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danwallach%2Fgithub-classroom-utils","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danwallach%2Fgithub-classroom-utils/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danwallach%2Fgithub-classroom-utils/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danwallach%2Fgithub-classroom-utils/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/danwallach","download_url":"https://codeload.github.com/danwallach/github-classroom-utils/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243995278,"owners_count":20380778,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["classroom","github-api","github-classroom","python","teachers"],"created_at":"2025-03-17T07:40:10.873Z","updated_at":"2025-03-17T07:40:11.698Z","avatar_url":"https://github.com/danwallach.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# GitHub Classroom Utils\n\nThis repository contains a number of utilities that I've written (and rewritten)\nover the years to help with [GitHub Classroom](https://classroom.github.com). \nMost recently (Oct 2019), I revamped the code to launch multiple, concurrent requests\nto the GitHub servers whenever possible. These tools now run *significantly* faster.\n\n## Setup\n\nI typically have an administrative repository, private to just the instructors,\nwhere I keep grades, slides, and other materials related to running my classroom. \nI'll copy these utilities to that directory and configure them to operate on the\nstudent repositories for just that class.\n\n## Installation \u0026 Configuration\n\n\n1) If you haven't already done this, you'll first need to `pip3 install\niso8601 pandas matplotlib requests aiohttp` for necessary libraries.\n(Everything here requires Python3 and is tested with Python 3.7.2.\n Earlier versions of Python3 might or might not work.)\n \n2) Copy all the `py` files here into your class administrative repository.\n\n3) Get a GitHub token with all the \"Repo\" privileges. You do\nthis on the GitHub website\n[(instructions)](https://github.com/blog/1509-personal-api-tokens). \n\n4) Edit the `github_config.py` file. In this file\n   you can save values that every tool here will use.\n   These parameters can be specified on the command-line\n      for every tool here, but it's nice to save them so you're\n      not typing them over and over again.\n    \n   - `default_github_organization`: Your organization's name\n   (e.g., for `https://github.com/RiceComp215-Fall2018`, the\n   organization name is `RiceComp215-Fall2018`). \n   \n   - `default_github_token`: Your API token goes here.\n\n   - `default_prefix`: When you're cloning and otherwise working\n     with a specific assignment for your students, you can specify\n     this here. \n     \n   - `default_grader_list`: [Used by github_graders, see below](#github_graders)\n   \n   - `default_grader_ignore_list`: [Used by github_graders, see below](#github_graders)\n\n   - `default_timezone`: [used by github_completion_times, see below](#github_completion_times)\n     \n      \n## Tools\n\nAll of these tools use a common library to interact with GitHub that tries\nto avoid rescanning student repositories unless something has changed.\nThese scans can take a while to run and also burn through your available\nGitHub API request limit, so it's important to cache the results. (You'll\nsee a multi-megabyte JSON file written out as a dot-file in the current directory.)\n\nThe cache uses the \n[ETag headers](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/ETag)\ngenerated by GitHub to try to avoid repeated downloads of identical lists\nof all the student repositories. \n[GitHub's implementation of ETag](https://developer.github.com/v3/#conditional-requests) \nseems to be unreliable, so you'll be wanting to manually delete the cache\nif you know that your students have created new repositories. You\ndo this by removing the cache file, which has a name like\n`.github-classroom-utils.RiceComp427-Spring2019.json`).\nThis forces a rescan of the students' repositories the next time\nyou run one of the tools here.\n\n**Tool usage.** Each tool below let's you run it with a `--help` argument which will summarize\nthe command-line arguments. \n\n### github_clone_all\n\nYou often want to get a local copy of every repo beginning with a common prefix,\ne.g., `comp215-week06` for the week6 projects.\nRun `python3 github_clone_all.py --prefix comp215-week06 --out codedump-week06`\nand it will create the directory `codedump-week06` and will check out all\nof the matching repos into the desired directory.\n\nAn optional flag, `--safe`, creates repositories that do not have your\nAPI token embedded in them. This means that remote Git actions that\nrequire the token might not work, but the repos are safer to share.\nBy default, the API token is embedded in the cloned Git repos.\n\n### github_rate_limit\n\nIf you keep running these tools, you'll eventually hit the wall with\nGitHub's rate limits. This tool tells you how many requests you have\nleft and when the timer will reset.\n\n### github_private_all\n\nIf you checked the wrong box when setting up GitHub Classroom, and all\nyour students' repositories are public, when you meant them to be private,\nyou can go back into GitHub Classroom's settings and make sure that *future*\ncloned repositories will be private, but what about the existing ones? This\ntool will tell GitHub to make private all the matching repositories. (I've\nneeded this twice in so many years, so I figure others might need this as well.)\n\n### github_no_partners\n\nYou tell the students that they're required to have a partner, or to have\na minimum group size. How do you detect the missing students? This script\ndoes all that. It scans all the projects, extracts the teams, and lets\nyou know about any project with fewer than the minimum number of members.\nLikewise, if there are students in your database who aren't attached to\nany GitHub projects, you get their information as well.\n\n### github_graders\n\nIf you're using GitHub Classroom, one of the things you may need to do is assign\nstudent submissions to graders. This project does this as a random mapping,\nprinting a document that you might share with your graders on Piazza or whatever\nforum, with grader names and student project hyperlinks.\n\nFirst, create a list of GitHub IDs that correspond to your graders and\nplace that in the `default_grader_list` in `github_config.py`.\nThis tells the tool who your graders are, and also any repos that they\nmight have cloned for their own benefit will be ignored. If you want to\nignore any other names, such as the professors, you can add them to\n`default_grader_ignore_list`.\n\nOur graders need to know how to go from GitHub identifiers to our\ninternal NetIDs, emails, and so forth. The tool will read in a CSV\nfile with all this specified (by default, `student-data.csv`). To\nthe extent anything is standard in the CSV universe, the first row\nshould be a list of strings giving the names of each column. We\nuse a `GitHubID` column for GitHub user ids, and then `Name` for\ntheir printable name, `Email` for their full university email address,\nand `NetID` for their university unique \"network\" identifier (which\nis often, but not always, their email address).\n\nTypical usage: `python3 github_graders.py --prefix comp215-week06` will \nprint out everything you need, assuming your assignment repos are named `comp215-week06`\nwith the students' names afterward. \n\nA new feature, `--teams` is useful when students are working as teams\nwith GitHub Classroom. This will use GitHub's APIs to identify the\nnames of each student associated with each repo and will adjust its\nprinted output appropriately. *We also have our students write their\nteam information into their `README.md` files, which is a helpful\nbackstop in case the GitHub metadata is incorrect.*\n\nAnother new feature, `--ignore` lets you specify a substring of a repo\nname to ignore when assigning grading. We tell our graders, when they\nwant to check out a repo to play with it, to add the word `STAFF` in\nthe name. This helps us skip those so they don't get assigned to be\ngraded.\n\nThe output of this tool is in Markdown format, which Piazza has recently added.\nSelect the Markdown button before cutting-and-pasting. We post this on Piazza,\nvisible only to the graders, and we ask the graders to edit the post to mark the\nstudents as \"done\" when they're done with their grading session. (This helps us see\nwhat graders haven't finished their work and, if necessary, assign other\ngraders to pick up the slack.)\n\n### github_event_times\n\nThis program uses the GitHub \"Events\" API to print all of the *push* times\nfor each commit. This might be\nuseful if you have a student who you suspect of falsifying commit times\naround a deadline and you need to document what happened. By default,\nyou get a LaTeX \"table\" with a \"tabular\" inside. If the table is\ntoo large, you've got two useful flags, `--longtable` and `--tiny`.\nThe former uses the LaTeX longtable package for multi-page tables.\nThe latter uses a smaller font.\n\nLets say you want to get the commit times for a series of repos\nwith\nnames like `assignment3-student1` and `assignment3-student2`, \nyou run `python3 github_event_times.py assignment3-student1\nassignment3-student2`\nand it will print a table with the commit IDs (7 digit prefix, same\nas reported on GitHub's list of commits), the commit string, and the\ntime at which that commit was pushed to GitHub, converted to your\nlocal timezone (from the UTC times reported by GitHub).\n\nNote that GitHub only retains the underlying event data for a small\namount of time, maybe three months. If you see something unusual, \ncapture this output while it's still available.\n\n### github_completion_times\n\nThis reads all the available\nCI data for every commit in every repo\nand produces a plot over time of how many students have passed all\nthe tests and gotten a green checkmark. Here's an example from my\nown students, showing work in progress toward a deadline on 2019-09-01;\nyou can see roughly 100 of 170 students have completed the work on\nthe evening of 2019-08-30. \n\n![Example completion graph](example_completion.png)\n\nThe timezone used to render the chart is set from the\n`default_timezone` setting in `github_config.py`.\n\n### github_invite_to_org\nThis will read a specified JSON file (`--file` parameter) to a GitHub organization (either the default in `github_config.py` or through the `--org` parameter) and invite the users specified in the file to the specified organization under the role associated with the user in the file.  \n\nThe JSON file should define a dictionary (mapping) of `{role: [username,...]}` where `role` is designated role of the usernames in the associated list.   `role` is restricted to `member` or `admin` (owner).  Multiple roles can be specified in the dictionary but do not specify the same username for multiple roles as which role they will actually be assigned is ill-defined.\n\nThe role to which each username is being invited will be printed as well as the response from GitHub showing the status of that request.\n\nThe resultant printout is in the format of a series of JSON dictionaries.\n\n\n### github_org_teams\nThis will print out information about the teams in a GitHub organization (either the default in `github_config.py` or through the `--org` parameter). \n\nNote: The team name \"slug\" is included in the printout.\n\nThe resultant printout is in a JSON dictionary format.\n\n\n### github_team_members\nThis will print out the members of a specified team (`--team` parameter) a GitHub organization (either the default in `github_config.py` or through the `--org` parameter). \n\nNote: The team name must be the team name `slug`.   This is the part of the GitHub URL that specifies team when browsing to the team on the web.   Typically, the slug is the team name with spaces replaced with dashes and no capital letters.\n\nThe resultant printout is in a JSON dictionary format.\n\n\n### github_invite_to_team\nThis will read a specified JSON file (`--file` parameter) to a specified team (`--team` parameter) a GitHub organization (either the default in `github_config.py` or through the `--org` parameter) and invite the users specified in the file to the specified organization under the role associated with the user in the file.  \n\nNote: The team name must be the team name `slug`.   This is the part of the GitHub URL that specifies team when browsing to the team on the web.   Typically, the slug is the team name with spaces replaced with dashes and no capital letters.\n\nThe JSON file should define a dictionary (mapping) of `{team: [username,...]}` where `team` is team to invite the associate list of usernames.   Multiple teams can be specified in the dictionary and any username can be invited to multiple teams.\n\nThe team to which each username is being invited will be printed as well as the response from GitHub showing the status of that request.\n\nThe resultant printout is in the format of a series of JSON dictionaries.\n\n\n\n## See also\n\n- I did a talk at SIGCSE 2019 about an earlier version of these tools.\n  - [Slides](https://www.cs.rice.edu/~dwallach/sigcse2019.pdf)\n  - [YouTube video](https://youtu.be/xFPskG4ctsI?t=318)\n\n- For dealing with Travis-CI, check out [travis-activate](https://github.com/danwallach/travis-activate).\n  (Travis-CI normally \"activates\" immediately when a new repository is created,\n   but at least once I've seen this fail. This script was something I originally\n   ran from a `cron` job to force all repos to \"activate\" with Travis, years ago,\n   when Travis didn't know how to automatically activate new repos.)\n   \n- https://classroom.github.com/assistant - GitHub's web-based tool for bulk repo downloads ([open source](https://github.com/education/classroom-assistant/))\n\n- https://github.com/dwalkes/github-classroom-scripts - knows how to set up pull requests\n\n- https://github.com/ccannon94/github-classroom-utilties - knows how to clone assignments, add files, and to\n  set things up for a run of [MOSS](https://theory.stanford.edu/~aiken/moss/)\n\n- https://github.com/osteele/multiclone - another repo cloning tool, written in Golang\n\n- https://github.com/konzy/mass_clone - there are many forks of this, some with additional features\n\n\n## Recent Updates\n* (8/27/2021 by S. Wong) Added new functions to invite users to organizations and teams as well as to print out the teams in an organization and the members of a given team.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdanwallach%2Fgithub-classroom-utils","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdanwallach%2Fgithub-classroom-utils","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdanwallach%2Fgithub-classroom-utils/lists"}