AI Assessment
Short-answer grading, automated.
An AI assistant evaluates open responses across eight certification exams.
Challenge
The Certified Admin exams included over 80 unique short-answer questions across eight assessments, each requiring manual grading. The process was time-consuming, inconsistent across graders, and difficult to scale.
Delivering contextual, personalized feedback was also hard given vague, overly detailed, or partially correct responses from learners of varying backgrounds.
Solution
An engine trained on course materials, answer keys, and company context handles vague, nuanced, and over-detailed answers, while excluding biased or incorrect historical data.
Generates precise, supportive feedback (e.g. "missing concept," "good structure, incomplete logic") without pass/fail bias.
Rubric-based scoring, aligned with SME validation and refined through blind comparison against human graders.
My Role
- Led project design and development from ideation to pilot testing.
- Authored the custom prompt architecture and validation scripts.
- Ran blind testing with SMEs and iterated based on feedback.
- Designed the rollout and data-governance framework.
Impact
50%
Reduction in manual grading time
95–99%
Agreement with human graders
70+
Questions across 7 certification exams
Consistent
High-quality feedback on every response
Bring this to your team
Want a result like this for your organization? Tell me what you're trying to accomplish and I'll show you how I can help.
Start a project