Automate all the things

Who am I

Assoc. Professor of the Practice, Dept. of Statistical Science, Duke University

All three courses run on GitHub - one (private) organization per course, students added as members, one repository per assignment per student or team.

Click or script?

Myself and others have written a lot about why you should be using Git & GitHub for your courses.

Having adopted GitHub, you still have to decide how to manage things.

• Point-and-click is fine for small courses and tasks

• GitHub Classroom reduces the number of clicks (but doesn’t eliminate them) and constrains workflows

• Using the API is what makes things scale

  • one function call can iterate over an entire roster

ghclass

An R package that wraps the GitHub REST and GraphQL APIs in class focused operations.

Course as organization

One org per course per semester. Students added as members (to use teams). The default repository permission is set to none, so students only see their assigned repos.

org_repos("ghclass-paper")

 [1] "ghclass-paper/.github"                   
 [2] "ghclass-paper/hw1"                       
 [3] "ghclass-paper/hw1-key"                   
 [4] "ghclass-paper/hw1_lab01_team01"          
 [5] "ghclass-paper/hw1_lab01_team02"          
 [6] "ghclass-paper/instructor"                
 [7] "ghclass-paper/jsm26_lab01_team1"         
 [8] "ghclass-paper/jsm26_lab01_team2"         
 [9] "ghclass-paper/midterm1"                  
[10] "ghclass-paper/midterm1_ghclass-alice"    
[11] "ghclass-paper/midterm1_ghclass-bob"      
[12] "ghclass-paper/midterm1_ghclass-carol"    
[13] "ghclass-paper/midterm1_ghclass-dave"     
[14] "ghclass-paper/midterm1_ghclass-elizabeth"
[15] "ghclass-paper/midterm1_ghclass-fred"     

1. Setting up the course

Configure the organization before anyone is invited:

org_set_repo_permission("ghclass-paper", "none")

✔ Set org "ghclass-paper"'s repo permissions to "none".

org_set_workflow_permissions("ghclass-paper", "read")

✔ Set org "ghclass-paper"'s default workflow permissions to "read".

org_sitrep("ghclass-paper")

── ghclass-paper sitrep: ───────────────────────────────────────────────────────

• Admins: "rundel"

• Members: 0

• Public repos: 13

• Private repos: 2

• Default repository permission: "none"

• Default workflow permissions: "read" <- Warning: this may prevent some GitHub
actions from working correctly.

• Members can create public repos: TRUE

• Members can create private repos: TRUE

• Members can fork private repos: FALSE

Adding students

Collect student usernames via a web form / survey and validate before inviting:

user_exists(c("rundel", "mine-cetinkaya-rundel", "ghclass-alice"))

[1]  TRUE  TRUE FALSE

Use ghclass to manage invites:

need = setdiff(
  responded$github,
  c(org_members("ghclass-paper"), org_pending("ghclass-paper"))
)

org_invite("ghclass-paper", need)

2.1 Assignments as repositories

repo_tree("ghclass-paper/hw1")

ghclass-paper/hw1
├─.github
│ └─workflows
│   └─check_assignment.yml
├─.gitignore
├─README.md
├─hw1.Rproj
└─hw1.qmd

repo_tree("ghclass-paper/hw1-key")

ghclass-paper/hw1-key
├─.gitignore
├─README.md
└─hw1.qmd

2.2 Assigning teams

Per assignment we build a roster that tracks student details and random team assignments,

roster = readr::read_csv("github_roster.csv", show_col_types = FALSE) |>
  team_roster(
    size = 3, 
    by   = "section",
    name = "jsm26_lab{section}_team{team_id}",
    pad = 2,
    seed = 20250901
  )

roster |> dplyr::select(name, email, section, github, team)

# A tibble: 6 × 5
  name      email                 section github            team              
  <chr>     <chr>                 <chr>   <chr>             <chr>             
1 Alice     alice@example.edu     01      ghclass-alice     jsm26_lab01_team02
2 Bob       bob@example.edu       01      ghclass-bob       jsm26_lab01_team02
3 Carol     carol@example.edu     01      ghclass-carol     jsm26_lab01_team01
4 Dave      dave@example.edu      01      ghclass-dave      jsm26_lab01_team01
5 Elizabeth elizabeth@example.edu 01      ghclass-elizabeth jsm26_lab01_team02
6 Fred      fred@example.edu      01      ghclass-fred      jsm26_lab01_team01

2.3 Distributing an assignment

Make sure the source repo is a template,

repo_set_template("ghclass-paper/hw1")

✔ Changed the template status of repo "ghclass-paper/hw1" to TRUE.

then create student repos from it,

org_create_assignment(
  org         = "ghclass-paper",
  repo        = roster$team,
  user        = rep("rundel", nrow(roster)),
  team        = roster$team,
  source_repo = "ghclass-paper/hw1",
  add_badges  = TRUE
)

✔ Mirrored repo "ghclass-paper/hw1" to repo "ghclass-paper/jsm26_lab01_team02".

✔ Mirrored repo "ghclass-paper/hw1" to repo "ghclass-paper/jsm26_lab01_team01".

✔ Created team "jsm26_lab01_team02" in org "ghclass-paper".

✔ Created team "jsm26_lab01_team01" in org "ghclass-paper".

✔ Added user "rundel" to team "jsm26_lab01_team02".

✔ Added user "rundel" to team "jsm26_lab01_team01".

✔ Team "jsm26_lab01_team02" given "push" access to repo "ghclass-paper/jsm26_lab01_team02"

✔ Team "jsm26_lab01_team01" given "push" access to repo "ghclass-paper/jsm26_lab01_team01"

✔ Modified file "ghclass-paper/jsm26_lab01_team01/README.md".

✔ Modified file "ghclass-paper/jsm26_lab01_team02/README.md".

3.1 Live assignments

While an assignment is open, student work is queryable and assignments can be edited / corrected,

repos = org_repos("sta523-fa25", "hw01_")

repo_n_commits(repos)

# A tibble: 12 × 3
   repo                          branch     n
   <chr>                         <chr>  <int>
 1 sta523-fa25/hw01_lab01_team01 main      41
 2 sta523-fa25/hw01_lab01_team02 main      14
 3 sta523-fa25/hw01_lab01_team03 main      75
 4 sta523-fa25/hw01_lab01_team04 main      49
 5 sta523-fa25/hw01_lab01_team05 main      30
 6 sta523-fa25/hw01_lab01_team06 main      20
 7 sta523-fa25/hw01_lab01_team07 main      35
 8 sta523-fa25/hw01_lab01_team08 main      50
 9 sta523-fa25/hw01_lab01_team09 main      27
10 sta523-fa25/hw01_lab02_team01 main      23
11 sta523-fa25/hw01_lab02_team02 main      11
12 sta523-fa25/hw01_lab02_team03 main      13

repo_add_file(
  repos, 
  file = "late_addition.csv", 
  repo_folder = "data"
)

repo_modify_file(
  repos, path = "README.md",
  pattern = "Due: .*", 
  content = "Due: 2026-09-22", 
  method = "replace"
)

3.2 Automated feedback

GitHub Actions provide a powerful tool for automating a varierty of things,

• Our feedback usually focuses process not correctness (reproducibility)

• We include workflows in student repos (.github/workflows/check_assignment.yml) as part of the source repo

• The checks are mostly R functions from checklist - required files present, document renders, etc.

• Uses a prebuilt docker image - r_gh_action

• add_badges = TRUE at distribution put a pass / fail badge at the top of each README

What students see

4.1 Collecting assignments

After the due date, clone every repo, downloads CI artifacts, and scaffolds comment markdown files.

org_grade_assignment(
  "hw1", "ghclass-paper", "jsm26_",
  key_repo = "ghclass-paper/hw1-key",
  comment_template = "## hw1 feedback\n\n"
)

ℹ Cloning 2 student repos matching "jsm26_".

✔ Cloned "ghclass-paper/jsm26_lab01_team1".

✔ Cloned "ghclass-paper/jsm26_lab01_team2".

ℹ Creating comment files.

✔ Created 2 comment files in 'hw1/comments'.

ℹ Cloning key repo "ghclass-paper/hw1-key".

✔ Cloned "ghclass-paper/hw1-key".

fs::dir_tree("hw1")

hw1
├── comments
│   ├── jsm26_lab01_team1.md
│   └── jsm26_lab01_team2.md
├── hw1-key
│   ├── README.md
│   └── hw1.qmd
└── repos
    ├── jsm26_lab01_team1
    │   ├── README.md
    │   ├── hw1.Rproj
    │   └── hw1.qmd
    └── jsm26_lab01_team2
        ├── README.md
        ├── hw1.Rproj
        └── hw1.qmd

4.2 Scores & feedback

Numeric scores go to the gradebook,

scores = readr::read_csv("hw1/scores.csv")
upload = dplyr::left_join(roster, scores, by = "team")

stopifnot(!any(is.na(upload$score)))

readr::write_csv(upload, "hw1/hw1_upload.csv")

comments = fs::dir_ls("hw1/comments", glob = "*.md")
repos    = paste0("ghclass-paper/", fs::path_ext_remove(fs::path_file(comments)))

issue_create(repos, title = "hw1 feedback", body = purrr::map_chr(comments, readr::read_file))

✔ Created issue "hw1 feedback" for repo "ghclass-paper/jsm26_lab01_team1".

✔ Created issue "hw1 feedback" for repo "ghclass-paper/jsm26_lab01_team2".

markermd

An interactive (Shiny) grading interface for Quarto and R Markdown assignments that works with org_grade_assignment().

Using markermd

A grading project (folder) is initialized with init_project() which creates .markermd/.

Once initialized you can use:

• template() to build an assignment template - document subsections and validation rules

• mark() to validate submissions, build a rubric, and apply it across submissions

• export_marks() to collects results and populates scores.csv and comments/

Some advice

Reach out