AI Evals
Design rigorous evaluation systems that turn expert judgment into measurable, repeatable AI quality signals.
Outcomes
- Translate product quality into clear evaluation dimensions and measurable rubrics
- Build representative test sets, golden datasets, and human-review workflows
- Use LLM-as-judge responsibly with calibration, agreement checks, and failure analysis
- Create an eval operating cadence that improves models, prompts, agents, and product decisions
Who it's for
AI product managers, engineers, researchers, and quality leaders who need a practical system for measuring whether generative-AI experiences are reliable, useful, and improving.
Syllabus
An end-to-end evaluation system for a real AI feature: rubric, dataset, judge prompts, baseline results, and an improvement plan.
Questions
Is everything taught live?
The Fellowship is built around live, cohort-based learning. You learn directly from the named practitioners, ask questions, complete applied work, and receive the latest insights from people working at the frontier.
Advanced Product Management is included as an on-demand program for strengthening timeless product fundamentals.
How much time will I need each week?
Most cohorts are designed for busy working professionals and require only a few hours per week. The exact commitment varies by program. You'll get the most value by attending live, completing the exercises, and making time for your capstone.
What if I miss a live session?
Attend live whenever possible because the discussions, feedback, and accountability are important parts of the experience.
However, missing an occasional session won't ruin your progress. You can also retake eligible cohorts during your active membership.
Can I retake a cohort?
Yes. You can retake eligible cohorts at no additional cost while your membership remains active. This allows you to experience the updated curriculum, refresh your knowledge, and apply what you learn to a new project as AI evolves.
Can my employer pay for it?
Yes. Many professionals use their learning and development budgets to cover the Fellowship.
We provide a manager-approval email template explaining the practical value to your company. Corporate cohort options are also available for teams. If your company needs an invoice, email support@productfaculty.com and we'll sort it.
Every other certification, included.
AI Product Management
Ship enterprise-ready AI products: PRDs, RAG, evals, agents.
AI Product Strategy for Leaders
Defensible AI strategy: moats, AI P&L, pricing, GTM.
AI-Native PM with Claude
Turn Claude and Claude Code into your daily PM team.
AI Product Manager Interviews
Ace the AI product interview, from product sense to strategy.
Pick your first cohort. Take the rest as they run.
See all 8 courses