Lab: Agentic Workflows Repo Analyzer
Duration: ~45 minutes | Level: Intermediate | Prerequisites: GitHub CLI (
gh) authenticated, a GitHub repository with some PR/issue history
Objectiveβ
In this lab you will create a GitHub Agentic Workflow that automatically analyzes a repository's health and produces a comprehensive report as a GitHub issue. By the end, you'll have a workflow that runs weekly and generates insights about AI-agent readiness, CI/CD health, DORA metrics, review efficiency, Copilot adoption, and more.
GitHub Agentic Workflows (gh-aw) let you write GitHub Actions workflows in Markdown with natural language instructions. The gh aw CLI compiles your Markdown into a secure GitHub Actions .lock.yml file that runs an AI agent with carefully scoped permissions, tools, and guardrails. Think of it as "CI/CD for AI agents."
What You'll Buildβ
A Repository Health Analyzer workflow that:
- Runs on a weekly schedule and on-demand via
workflow_dispatch - Reads repository data through the GitHub MCP server (issues, PRs, commits, workflow runs)
- Generates Python charts (matplotlib/seaborn) for trend visualization
- Produces a GitHub issue with 8 report sections:
| Section | What It Covers |
|---|---|
| π€ AI Agent Readiness | Scorecard assessing if the repo is ready for AI agents |
| π§ CI/CD Health | Workflow pass rates, durations, flaky test detection |
| π DORA Metrics | Deployment frequency, lead time, change failure rate, MTTR |
| π§ SPACE Metrics | Activity, performance, efficiency, collaboration proxies |
| π Review Efficiency | Time to first review, review-to-merge, reviewer throughput |
| β±οΈ PR Lead Times | Openβreviewβmerge lifecycle by PR size |
| π€ Copilot Adoption | Copilot-assisted PRs, commits, reviews, and adoption trends |
| π Trends & Recommendations | Week-over-week comparison and prioritized action items |
Exercise 1 β Install gh-awβ
The gh aw CLI is a GitHub CLI extension that compiles Markdown workflows into GitHub Actions.
1.1 Verify GitHub CLIβ
Make sure gh is installed and authenticated:
gh auth status
You should see your GitHub username and authentication method. If not, run gh auth login.
1.2 Install the Extensionβ
The extension requires Linux, macOS, or Windows with WSL. Install as follows:
curl -sL https://raw.githubusercontent.com/github/gh-aw/main/install-gh-aw.sh | bash
If you already have it installed, upgrade to the latest version:
gh extension upgrade aw
1.3 Verify Installationβ
gh aw version
You should see a version number (e.g., v0.64.2 or newer).
β Checkpointβ
You have gh authenticated and gh aw installed. Run gh aw version to confirm.
Exercise 2 β Understand the Anatomyβ
Before writing your own workflow, let's understand the structure of an agentic workflow.
2.1 The Two-Part Fileβ
Every agentic workflow is a single .md file in .github/workflows/ with two parts:
βββββββββββββββββββββββββββββββββββββββββββ
β YAML Frontmatter (between --- markers) β β Configuration
β - Triggers (on:) β Requires recompilation
β - Permissions β when changed
β - Tools & MCP servers β
β - Safe outputs β
β - Setup steps β
βββββββββββββββββββββββββββββββββββββββββββ€
β Markdown Body β β Agent Instructions
β - What the agent should do β Can be edited without
β - How to process data β recompilation!
β - Output formatting β
β - Step-by-step guidance β
βββββββββββββββββββββββββββββββββββββββββββ
The Markdown body is loaded at runtime. You can edit agent instructions directly on GitHub.com and changes take effect on the next run. No recompilation needed!
2.2 Security Modelβ
Agentic workflows follow a read-only agent model:
- The agent job has read-only permissions for all scopes
- All write operations (creating issues, adding comments) go through the
safe-outputssystem - Safe outputs enforce validation, rate limiting, and audit trails
- This prevents a runaway or compromised AI agent from causing damage
# β
Correct: Read-only agent, writes via safe-outputs
permissions:
contents: read
issues: read
safe-outputs:
create-issue:
max: 3
# β Wrong: Never give write permissions directly
permissions:
issues: write # This bypasses safety controls!
2.3 The Compilation Stepβ
After writing your .md file, you compile it:
gh aw compile <workflow-name>
This generates a .lock.yml file β a standard GitHub Actions workflow with all the security guardrails baked in. The .lock.yml is what GitHub Actions actually runs.
β Checkpointβ
You understand the two-part structure (frontmatter + body), the read-only security model, and the compile step.
Exercise 3 β Write the Frontmatterβ
Now let's build the workflow. Create the file:
mkdir -p .github/workflows
touch .github/workflows/repo-health-analyzer.md
Open it in your editor and add the YAML frontmatter.
3.1 Description and Triggersβ
Start with the description and triggers:
---
description: |
Weekly repository health analyzer that assesses AI-agent readiness,
CI/CD health, DORA/SPACE metrics, review efficiency, PR lead times,
and GitHub Copilot adoption. Produces a comprehensive report as a GitHub issue
with trend charts and actionable recommendations.
on:
schedule:
- cron: '0 6 * * 0' # Every Sunday at 6:00 AM UTC
workflow_dispatch:
timeout-minutes: 30
Key points:
cron: '0 6 * * 0'runs every Sunday at 6:00 AM UTC. Adjust the hour to match your timezone (e.g.,0 12 * * 0for 7 AM ET / 4 AM PT during daylight saving time).workflow_dispatchallows manual triggering at any time.timeout-minutes: 30gives the agent enough time to collect data, generate charts, and create the report.
3.2 Permissions and Environmentβ
Add read-only permissions and a configurable target repository:
permissions:
contents: read
issues: read
pull-requests: read
actions: read
env:
TARGET_REPOSITORY: ${{ vars.TARGET_REPOSITORY || github.repository }}
By setting the TARGET_REPOSITORY repository variable in your repo's Settings β Variables, you can point the analyzer at any repository you have access to. Note that the built-in GitHub Actions token will only have permissions for the current repository, so cross-repo analysis would require additional setup (e.g., a PAT with broader access).
3.3 Network and Toolsβ
Configure network access and tools:
network:
allowed:
- defaults
- python
- node
tools:
edit:
bash:
- "*"
github:
lockdown: false
min-integrity: none
toolsets: [default]
allowed-repos: all
What each tool does:
editβ Allows the agent to create/edit files (for Python scripts)bashβ Allows running commands (for executing Python scripts)githubβ The GitHub MCP server, providing access to issues, PRs, commits, workflow runs, etc.network: pythonβ Allows pip to download packages from PyPI
3.4 Safe Outputs and Setup Stepsβ
Add the output configuration and Python setup:
safe-outputs:
mentions: false
allowed-github-references: []
upload-asset:
create-issue:
title-prefix: "[Repo Health] "
labels: [report, repo-health, weekly]
close-older-issues: true
steps:
- name: Setup Python environment
run: |
mkdir -p /tmp/charts /tmp/data
pip install --user --quiet numpy pandas matplotlib seaborn scipy
python3 -c "import pandas, matplotlib, seaborn; print('Python environment ready')"
---
What this configures:
upload-assetβ Lets the agent upload chart imagescreate-issueβ Lets the agent create the report issue with[Repo Health]prefix and auto-close older reportsmentions: falseβ Prevents the agent from @-mentioning peoplestepsβ Pre-installs Python data science libraries before the agent runs
β Checkpointβ
You should have a complete frontmatter block between --- markers. The closing --- is important because it separates the frontmatter from the Markdown body.
Exercise 4 β Write the Agent Instructionsβ
Below the frontmatter, add the Markdown body. This is where you tell the agent what to do in natural language. The agent will follow these instructions step by step.
Add the sections to your markdown body.
4.1 Introduction and Data Collectionβ
# Repository Health Analyzer
Analyze repository `${{ env.TARGET_REPOSITORY }}` and produce a comprehensive health report as a GitHub issue. The report should help teams understand their repository's health, AI-agent readiness, and improvement trends over time.
## Data Collection
Use GitHub tools to gather data for the **past 30 days** from `${{ env.TARGET_REPOSITORY }}`:
1. **Repository metadata** β README contents, file tree (top 2 levels), key config files
2. **Pull requests** β All PRs (opened, merged, closed) with authors, reviewers, timestamps, and commit trailers
3. **Issues** β All issues opened and closed with labels and timestamps
4. **Commits** β Recent commits on the default branch with author info and co-author trailers
5. **Workflow runs** β CI/CD workflow run results (success, failure, duration)
6. **Releases/tags** β Recent releases or deployment-tagged events
Collect enough data to compute weekly trends for the past 4 weeks.
---
4.2 AI Agent Readiness Sectionβ
This section is based on the criteria from Preparing a Repository for AI Agents. Add:
## Section 1: AI Agent Readiness Assessment
Evaluate the repository against the criteria below. For each criterion, assign a status:
- β
**Present & Good** β criterion is fully met
- β οΈ **Partial** β exists but incomplete or could be improved
- β **Missing** β not present
### Criteria Checklist
| # | Criterion | What to Check |
|---|-----------|---------------|
| 1 | **Clear README** | README.md exists and includes: project purpose, architecture summary, build steps, test steps, repository layout |
| 2 | **Architecture Documentation** | Presence of `/docs/architecture.md`, `/docs/components.md`, system diagrams, or similar |
| 3 | **Predictable Repo Structure** | Uses clear folder names (`/src`, `/tests`, `/docs`, `/scripts`); avoids ambiguous names (`/misc`, `/stuff`, `/temp`) |
| 4 | **In-Repo Documentation** | Build instructions, test instructions, and contribution guidelines are in markdown files in the repo |
| 5 | **Architecture Decision Records** | Presence of `/docs/adr/` or similar ADR directory |
| 6 | **Scriptable Build/Test** | Makefile, npm scripts, or similar with simple commands (`make build`, `npm test`) |
| 7 | **Test Suite** | Test directory exists, test framework configured, evidence of meaningful test coverage |
| 8 | **Automated CI** | GitHub Actions workflows that run tests, linting, or security scans on PRs |
| 9 | **Linting & Formatting** | Config files present (`.eslintrc*`, `.prettierrc*`, `ruff.toml`, `.editorconfig`, `pyproject.toml` with tool config, etc.) |
| 10 | **Copilot Auto-Review** | Check for Copilot review configuration or evidence of Copilot as a reviewer on PRs |
| 11 | **Custom Instructions** | Presence of `.github/copilot-instructions.md` |
| 12 | **CONTRIBUTING.md** | File exists with meaningful content (coding standards, branch strategy, PR expectations) |
### Output Format
Present the results as a scorecard table with the status emoji, criterion name, and a brief finding. Then compute an overall **AI Agent Readiness Grade**:
- **A (Agent-Ready)**: 10+ criteria met (β
)
- **B (Nearly Ready)**: 7-9 criteria met
- **C (Needs Work)**: 4-6 criteria met
- **D (Not Ready)**: 0-3 criteria met
Follow the scorecard with a **Recommendations** section listing the top 3-5 most impactful improvements to make the repo AI-agent ready, in priority order. Each recommendation should be specific and actionable (e.g., "Create a `.github/copilot-instructions.md` file with project overview, tech stack, and build instructions" rather than "Add custom instructions").
---
Repositories structured for machine consumption get dramatically better results from AI agents. This assessment gives teams a concrete checklist to improve their agent outcomes.
4.3 CI/CD Health Sectionβ
This section analyzes the health of the repository's CI/CD pipelines based on GitHub Actions workflow runs. Note if you have another CI/CD system you would need to adjust the data collection and analysis steps accordingly.
## Section 2: CI/CD Health
Analyze GitHub Actions workflow runs from the past 30 days:
- **Workflow inventory** β List all workflows with their trigger types
- **Pass/fail rates** β Success rate per workflow (runs that succeeded vs. failed)
- **Average duration** β Mean and median run duration per workflow
- **Failure frequency** β Number of failures per week, trending up or down
- **Flaky detection** β Identify workflows/jobs that alternate between pass and fail on the same branch (potential flakiness)
Present as a summary table:
| Workflow | Runs | Pass Rate | Avg Duration | Trend |
|----------|------|-----------|--------------|-------|
---
4.4 DORA Metrics Sectionsβ
Continue adding instructions for each metrics section. Here's the DORA metrics section as an example:
## Section 3: DORA Metrics
Approximate DORA metrics from GitHub data for the past 30 days:
### Deployment Frequency
- Count releases, tags, or deployments per week
- If no formal releases, count merges to default branch as a proxy
- Classify: **Elite** (multiple/day), **High** (weekly), **Medium** (monthly), **Low** (less)
### Lead Time for Changes
- Median time from **first commit on a branch** to **PR merge** into default branch
- Classify: **Elite** (<1 day), **High** (<1 week), **Medium** (<1 month), **Low** (>1 month)
### Change Failure Rate
- Identify reverted PRs (commits with "revert" in message), hotfix branches, or issues labeled as bug/regression opened shortly after a merge
- Calculate as percentage of total merged PRs
- Classify: **Elite** (0-5%), **High** (5-10%), **Medium** (10-15%), **Low** (>15%)
### Mean Time to Recovery (MTTR)
- For issues labeled `bug`, `incident`, or `regression` β time from opened to closed
- Classify: **Elite** (<1 hour), **High** (<1 day), **Medium** (<1 week), **Low** (>1 week)
Present a summary table with metric, value, classification, and week-over-week trend arrow.
---
4.5 SPACE Metrics Sectionβ
Add instructions for the SPACE metrics section:
## Section 4: SPACE Metrics
Compute proxy indicators for SPACE framework dimensions:
### Satisfaction & Well-being
- **Contributor retention**: How many unique contributors from 30-60 days ago are still active in the last 30 days?
- **New contributors**: Count of first-time contributors in the past 30 days
### Performance
- **PR merge rate**: Percentage of opened PRs that got merged (vs. closed without merge)
- **CI pass rate**: Overall CI success rate across all workflows
### Activity
- **Commits per week** on default branch
- **PRs opened/merged per week**
- **Issues opened/closed per week**
- Present as a 4-week activity trend table
### Communication & Collaboration
- **Review comments per PR** (average)
- **Discussion activity**: Count of discussion posts if discussions are enabled
- **PR review participation**: Average number of reviewers per PR
### Efficiency
- **PR cycle time**: Median time from PR open to merge
- **Review turnaround**: Median time from PR open to first review
- **Code review load**: Average open review requests per reviewer
---
4.6 Review Efficiency Sectionβ
Add instructions for the review efficiency section:
## Section 5: Review Efficiency
Deep-dive into the code review process:
- **Time to first review** β Median time from PR creation to first review (comment, approval, or request changes)
- **Review to merge** β Median time from first review to merge
- **Total review cycle** β Median time from PR open to merge
- **Review throughput** β Number of reviews completed per reviewer per week
- **Review backlog** β Count of PRs currently open and awaiting review (no reviews yet)
- **Stale PRs** β PRs open for more than 7 days with no activity
Present a summary table and highlight any bottlenecks.
---
4.7 PR Lead Times Sectionβ
Add instructions for the PR lead times section:
## Section 6: PR Lead Times
Detailed PR lifecycle analysis:
- **Open β First Review** β time distribution
- **First Review β Merge** β time distribution
- **Open β Merge** β total lead time distribution
- **By PR size**: Categorize PRs as Small (<50 lines), Medium (50-250 lines), Large (>250 lines) and show median lead times per size category
### Chart: PR Lead Time Distribution
Write a Python script to create a chart:
- Horizontal grouped bar chart showing median lead time for each phase (openβreview, reviewβmerge) by PR size category
- Save as `/tmp/charts/pr_lead_times.png` at 300 DPI, 12Γ7 inches
- Use seaborn style with a clear, professional palette
- Use `matplotlib.use('Agg')` for headless rendering
Run the script via bash and verify the file exists.
---
4.8 Copilot Adoption Sectionβ
Add instructions for the GitHub Copilot adoption section:
## Section 7: GitHub Copilot Adoption
Measure GitHub Copilot's contribution to the repository:
### Detection Method
- Search commit messages and PR descriptions for `Co-authored-by:` trailers containing "Copilot" or "copilot"
- Check for PRs authored by GitHub Copilot bots
- Look for Copilot code review activity on PRs
### Metrics
- **Copilot-assisted commits**: Count of commits with Copilot co-author trailers per week
- **Copilot-assisted PRs**: Count of PRs with Copilot co-author trailers or authored by Copilot
- **Copilot reviews**: Count of PR reviews by Copilot
- **Adoption rate**: Percentage of total commits/PRs that involve Copilot
- **Trend**: Week-over-week change in Copilot adoption
### Chart: Copilot Adoption Trend
Write a Python script to create a chart:
- Dual-axis line chart: Copilot-assisted commits (left axis), adoption rate % (right axis) over 4 weeks
- Save as `/tmp/charts/copilot_adoption.png` at 300 DPI, 12Γ7 inches
- Use seaborn style
- Use `matplotlib.use('Agg')` for headless rendering
Run the script via bash and verify the file exists.
---