Quarto and automation

JSM 2023 - Toronto

Colin Rundel

Duke University

Context

  • Sta 323 / Sta 523 - R based statistical computing (UG / MS)

  • Sta 663 - Python based statistical computing (MS)

  • All courses run using GitHub

    • 1 organization / course

    • Students added as org members

    • 1 repo / assignment (team or individual)

    • Management via ghclass

Sta 523 - HW 1

.github/workflows/check_assignment.yml

on: push
name: Check Assignment
jobs:
  check-allowed-files:
    runs-on: ubuntu-latest
    container:
      image: ghcr.io/sta523-fa22/sta523-base:latest
    steps:
    - name: Checkout
      uses: actions/checkout@master
    - name: Check Files
      run: |
        checklist::quit_on_failure({
          checklist::check_allowed_files(
            c("hw1.qmd", "hw1.Rproj", "README.md", "fizzbuzz.png")
          )
        })
      shell: Rscript {0}
  
  check-renders:
    runs-on: ubuntu-latest
    container:
      image: ghcr.io/sta523-fa22/sta523-base:latest
    steps:
    - name: Checkout
      uses: actions/checkout@master
    - name: Install Missing Packages
      run: |
        missing = checklist::install_missing_pkgs(dir = "./", glob = "hw1.qmd")
      shell: Rscript {0}
    - name: Check Renders
      run: |
        checklist::check_qmd_renders("hw1.qmd")
      shell: Rscript {0}
    - name: Create artifacts
      uses: actions/upload-artifact@v3
      with:
        name: hw1-html
        path: hw1.html

.github/workflows/check_assignment.yml

on: push
name: Check Assignment
jobs:
  check-allowed-files:
    runs-on: ubuntu-latest
    container:
      image: ghcr.io/sta523-fa22/sta523-base:latest
    steps:
    - name: Checkout
      uses: actions/checkout@master
    - name: Check Files
      run: |
        checklist::quit_on_failure({
          checklist::check_allowed_files(
            c("hw1.qmd", "hw1.Rproj", "README.md", "fizzbuzz.png")
          )
        })
      shell: Rscript {0}
  
  check-renders:
    runs-on: ubuntu-latest
    container:
      image: ghcr.io/sta523-fa22/sta523-base:latest
    steps:
    - name: Checkout
      uses: actions/checkout@master
    - name: Install Missing Packages
      run: |
        missing = checklist::install_missing_pkgs(dir = "./", glob = "hw1.qmd")
      shell: Rscript {0}
    - name: Check Renders
      run: |
        checklist::check_qmd_renders("hw1.qmd")
      shell: Rscript {0}
    - name: Create artifacts
      uses: actions/upload-artifact@v3
      with:
        name: hw1-html
        path: hw1.html

ghclass + artifacts

repos = ghclass::org_repos("sta523-fa22", "hw01_")
ghclass::action_artifact_download(repos, "html/")
✔ Downloaded artifact 846527824 from repo "sta523-fa22/hw01_lab01_team01" to "html/hw01_lab01_team01.html".
✔ Downloaded artifact 846527748 from repo "sta523-fa22/hw01_lab01_team02" to "html/hw01_lab01_team02.html".
✔ Downloaded artifact 846527719 from repo "sta523-fa22/hw01_lab01_team03" to "html/hw01_lab01_team03.html".
✔ Downloaded artifact 846527656 from repo "sta523-fa22/hw01_lab01_team04" to "html/hw01_lab01_team04.html".
✔ Downloaded artifact 846527730 from repo "sta523-fa22/hw01_lab01_team05" to "html/hw01_lab01_team05.html".
✔ Downloaded artifact 846527655 from repo "sta523-fa22/hw01_lab01_team06" to "html/hw01_lab01_team06.html".
✔ Downloaded artifact 846527725 from repo "sta523-fa22/hw01_lab01_team07" to "html/hw01_lab01_team07.html".
✔ Downloaded artifact 846527650 from repo "sta523-fa22/hw01_lab02_team01" to "html/hw01_lab02_team01.html".
✔ Downloaded artifact 846527717 from repo "sta523-fa22/hw01_lab02_team02" to "html/hw01_lab02_team02.html".
✔ Downloaded artifact 846527682 from repo "sta523-fa22/hw01_lab02_team03" to "html/hw01_lab02_team03.html".
✔ Downloaded artifact 846527759 from repo "sta523-fa22/hw01_lab02_team04" to "html/hw01_lab02_team04.html".
✔ Downloaded artifact 846527728 from repo "sta523-fa22/hw01_lab02_team05" to "html/hw01_lab02_team05.html".
✔ Downloaded artifact 846527712 from repo "sta523-fa22/hw01_lab02_team06" to "html/hw01_lab02_team06.html".
✔ Downloaded artifact 846527703 from repo "sta523-fa22/hw01_lab02_team07" to "html/hw01_lab02_team07.html".

Sta 663 - HW 1 (Python)

.github/workflows/repo_checks.yml

on: push
name: Repo Checks
jobs:
  check_allowed_files:
    runs-on: ubuntu-latest
    container:
      image: ghcr.io/sta663-sp23/sta663-base:latest
    timeout-minutes: 3
    steps:
    - name: Checkout
      uses: actions/checkout@master
    - name: Check Files
      run: |
        checklist::quit_on_failure({
          checklist::check_allowed_files(
            c("hw1.ipynb", "advent.py", "wordle.py", "README.*", "data/*")
          )
        })
      shell: Rscript {0}
  check_renders:
    needs: check_allowed_files
    runs-on: ubuntu-latest
    container:
      image: ghcr.io/sta663-sp23/sta663-base:latest
    timeout-minutes: 5
    steps:
    - name: Checkout
      uses: actions/checkout@master
    - name: Check Renders
      run: |
        quarto render hw1.ipynb --execute --to html --embed-resources
    - name: Archive html
      uses: actions/upload-artifact@master
      with:
        name: html
        path: hw1.html

Advice

  • Each organization gets 2000 minutes of action runner time per month for free (3000 with Team upgrade)

    • Easy to exhaust for even a medium sized course

    • Additional minutes can be paid for (~$0.008 / min)

    • or you can use a self hosted runner(s) (our setup guide + helper scripts)

  • Artifacts similarly count towards organization storage limits (500 MB Free or 2 GB Team). Old artifacts can be removed using,
  remove_old_artifacts:
    runs-on: self-hosted
    timeout-minutes: 3
    steps:
    - name: Remove old artifacts
      uses: c-hive/gha-remove-artifacts@v1
      with:
        age: '0 seconds'
        skip-tags: false
        skip-recent: 0

Advice

  • If you are not interested in creating your own Docker images the rocker is an excellent option.

    • Specifically, rocker/verse has quarto bundled along with most everything else you would need
    container:
      image: rocker/verse:latest

Questions?



  rundel.github.io
  rundel
  rundel@gmail.com
colin.rundel@duke.edu
  rundel/ghclass
rundel/checklist

bit.ly/teach-quarto