Skip to main content

Project: Tutorial Coach Agent (Harness Engineering Team)

Purpose​

Build a Tutorial Coach Agent workflow that:

  • guides learners through tutorial projects
  • tracks progress with a simple state machine
  • verifies work using evidence + checkpoint questions
  • produces summaries for the engineering team

This project belongs under the Harness Engineering (Team Project Tracking) umbrella, but is implemented as its own project so it can evolve independently.

Scope​

In scope​

  • Learner onboarding + project assignment
  • Progress tracking model (states + evidence requirements)
  • Evaluation workflow (question bank + rubric)
  • Storage of learner logs (GitHub as system-of-record)
  • “Coach agent” operating rules (how it prompts, how often, what it records)

Out of scope (initial)​

  • Full LMS platform
  • Automated grading from device telemetry (future)

Roles​

  • Learner: completes tutorial + submits evidence
  • Coach Agent: asks questions, checks evidence, updates status
  • Admin/Reviewer (Arif): approves completions, resolves disputes

Store tracking logs in the KB repo (private):

  • sessions/learners/<learner>/<project>/YYYY-MM-DD.md

Store reusable templates:

  • docs/templates/LEARNER_TRACKER_TEMPLATE.md
  • docs/templates/TUTORIAL_EVAL_PACK_TEMPLATE.md

Progress Model (state machine)​

Per learner per project:

  • Not Started
  • Environment Ready
  • Project Created
  • Build OK
  • Flash OK
  • Monitor OK
  • Menuconfig Used
  • Verified (Q&A passed)
  • Complete

Evidence Pack (minimum)​

To advance checkpoints, learner submits:

  1. repo link (or zip)
  2. proof of build/flash/monitor (screenshot/photo or pasted terminal output)
  3. config proof (sdkconfig or key values)
  4. short reflection (5–10 lines): what broke + fix

Verification (Q&A) workflow​

  • Each checkpoint has 2–5 questions.
  • Coach agent asks questions after evidence is submitted.
  • If failed: agent gives remediation steps and re-asks a smaller question set.

Rubric (10 points)​

  • Environment setup correctness (2)
  • Build/flash/monitor success (3)
  • menuconfig understanding + change applied (2)
  • Troubleshooting ability (2)
  • Reproducibility / cleanliness (1)

Pass criteria:

  • ≥ 7/10 AND minimum evidence pack complete.

Cadence (anti-spam)​

  • Daily check-in only for learners marked “Active”.
  • Weekly: agent generates progress summaries and flags blockers.

Deliverables​

  • Templates: learner tracker + eval pack
  • Eval pack #1: ESP32 ESP-IDF VS Code Hello World
  • Operating SOP: coach agent prompt rules
  • Pilot with 1 learner

Coaching Surface​

  • Coaching happens in Mattermost (confirmed).

Recommended structure:

  • One team/channel: #tutorial-coaching
  • One thread per learner per project (or a channel per learner if you prefer)
  • Learners post evidence in-thread; coach agent replies with checkpoint questions and next tasks

Next steps​

  1. Add templates to KB. ✅
  2. Create first eval pack for /tutorials/esp32-esp-idf-vscode-hello-world. ✅
  3. Create Mattermost channel(s) and decide threading convention.
  4. Pilot with one learner.

Changelog​

  • 2026-02-14: Created project plan.