Strategy to sunset as one sequence: who hands what to whom, in order, and the gate that closes each phase. Built to be product- and company-agnostic: consumer, B2B SaaS, marketplace or developer platform. Indigo marks the work an AI product adds on top of a classic PDLC. Every phase has a template, and a worked example fills them all in.
Swipe sideways to see all eight roles →
| # | Phase | AI PM's job | Artifacts | Key metrics | Pitfall to avoid | Template |
|---|---|---|---|---|---|---|
| 01 | Strategy | Pick where AI creates a durable advantage, tied to company OKRs | Strategy memo, market sizing, AI bets, data-moat map | TAM / SAM, proprietary data coverage, build vs. buy TCO | Starting from 'we need AI' instead of a customer or business problem | AI strategy memo |
| 02 | Vision | Paint the end state and how much autonomy the product earns over time | Vision narrative, concept prototype, North Star Metric + metric tree | NSM and its input metrics; target autonomy per workflow | A vision with no measurable North Star, or a vanity metric as the North Star | North Star + metric tree |
| 03 | Discovery | Prove the problem is real and that AI beats a rules-based fix | Research synthesis, personas, JTBD map, funnel analysis, risk tier | Baseline quality on real samples; user error tolerance | Demo-driven conviction: testing on 5 cherry-picked examples instead of real data | Discovery kit |
| 04 | Roadmap | Rank bets by value, effort, model cost and risk | Now / Next / Later roadmap, sizing, funding ask | Cost per task and latency in the prioritization score | Ignoring inference cost until after launch, when margins break | Roadmap prioritization Unit economics |
| 05 | Define | Write the launch bar before anyone builds | PRD, KPI targets, instrumentation plan, eval spec + golden set, model card draft | Quality bar, hallucination ceiling, p95 latency, cost per task | Starting build with no eval set, so 'good enough' is argued, not measured | Agent PRD Eval plan GenAI + agentic metrics |
| 06 | Build | Run the eval loop and triage every failure to its root cause | Eval dashboard, failure log, instrumented prototype | Offline eval score trend across iterations | Tuning prompts by vibes; fixing one failure and silently breaking three others | Experimentation guide |
| 07 | Trust review | Make sure the system fails safely | Red-team report, mitigations, sign-off | Jailbreak / injection pass rate, bias checks, PII leakage | Treating red-teaming as a launch checkbox instead of a test suite you keep | Responsible AI review |
| 08 | Launch | Ramp in steps and let guardrail metrics decide each step | GTM brief, rollout plan, KPI dashboard, A/B readout, AI disclosure | Activation, retention, acceptance and override rate vs. control | Going 0 → 100% in one step with no guardrail metrics or kill switch | Launch checklist |
| 09 | Operate + iterate | Turn production feedback into eval cases and roadmap changes | Metrics review, NPS / CSAT, QBR deck, updated golden set | NSM trend, drift, cost per task, incidents, ROI | Shipping and moving on while the model drifts and costs creep | Post-launch runbook |
Every AI product needs all five layers. The North Star says if you are winning; guardrails say if you are winning the wrong way.
| Layer | What it answers | Consumer app | B2B SaaS | Marketplace | Developer platform |
|---|---|---|---|---|---|
| North Star Metric | Is the product delivering its core value? | Weekly tasks completed with AI help | Weekly accounts where AI completes a workflow | Successful matches per week | Weekly active builders shipping with the API |
| Input metrics | Which levers move the North Star? | New users, % using AI weekly, tasks per user | Seats activated, workflows automated per account | Supply listed, search-to-match rate | Time to first call, calls per active key |
| Business KPIs | Is it working as a business? | D30 retention, paid conversion, ARPU | NRR, expansion, churn, CAC payback | GMV, take rate, repeat rate | Paid conversion, usage revenue, churn |
| AI quality metrics | Is the model good enough to trust? | Answer acceptance, edit rate, hallucination rate | Task success, accuracy vs. golden set, human-override rate | Ranking relevance (NDCG), match quality | Eval pass rate, tool-call success, p95 latency |
| Guardrail metrics | Are we winning the wrong way? | Unsafe output rate, complaints, cost per task | Escalations, data-leak incidents, cost per task | Fraud, bias across segments, cancellations | Error rate, abuse, cost per 1K calls |
The metrics stack above says which layers you need. These are the GenAI- and agent-specific metrics inside them. Pick one North Star, three to five inputs, a guardrail for each, and the layers your product type needs. Picks for eight product types, formulas, eval statistics, an event schema and a weekly review table: GenAI and agentic AI product metrics template.
| Layer | Core metrics | Trap to watch |
|---|---|---|
| Business value | Incremental revenue vs. holdout, cost-to-serve reduction, measured time saved, payback | Comparing adopters to non-adopters instead of a holdout |
| North Star + adoption | Tasks completed with output users kept, activation (first success), repeat usage, retention curve | Activity metrics (prompts, tokens, chat time) rise when users are confused |
| User-perceived quality | Acceptance rate, edit distance after accept, regenerate rate, override rate, CSAT | Acceptance is not accuracy |
| Reliance + teaming | Over-reliance (wrong outputs accepted), under-reliance, verification time, human + AI vs. human alone, sycophancy | Rising trust with flat accuracy means rubber-stamping |
| Model quality (offline evals) | Task success, pairwise win rate, faithfulness, factuality, top-harm error, calibration, pass^k, regression rate, judge agreement, golden label quality | Thresholds set after results; gating on pass@k instead of pass^k |
| Retrieval + context | Hit rate@k, recall@k, precision@k, MRR, NDCG, context position sensitivity, truncation rate, citation accuracy, answerable-but-refused, unanswerable-but-answered, index lag | Low hallucination rate built on refusals |
| Task outcomes | Containment with resolution and false containment, automation and straight-through rate, revert rate (coding), keep rate (generation), source coverage (research) | Containment reported alone counts giving up as a win |
| Agent behavior | Verified task completion, claimed vs. verified gap (silent failure), wrong-tool rate, steps per task, loop rate, correct + unnecessary handoff, irreversible-action errors, undo rate | Trusting the agent's own “done” |
| Agent permissions + security | Unauthorized actions, least-privilege coverage, destructive actions blocked, indirect prompt injection, tool-description poisoning, permission-boundary leakage, credential exposure | Testing only direct attacks, not injection through documents and tool results |
| Multi-agent | Token multiplier and quality gain vs. single agent, handoff failures, duplicated work, orchestrator overhead | No single-agent baseline |
| Memory + personalization | Wrong or stale memory, memory helpfulness, cross-user leakage, deletion compliance | Deleting the memory store but not derived indexes |
| Trust + safety | Unsafe output, jailbreak success, data leakage, over-refusal, fairness gap, disclosure + content marking (EU AI Act Art. 50) | A UI notice without machine-readable marking |
| Performance | TTFT, output tokens/sec, p95 end-to-end latency, time to task completion, fallback rate | Reporting only p50 |
| Voice + multimodal | Word error rate, voice-to-voice latency, barge-in success, MOS, image/document accuracy, prompt adherence | WER measured on clean audio only |
| Cost + unit economics | Blended cost per successful task, quality per dollar, tokens per task, cache hit rate, model mix, review cost, gross margin | Dividing cost by all requests makes failures look cheap |
| LLMOps health | Production quality score, offline–online agreement, quality and input drift, golden-set freshness, feedback-to-eval time, MTTD / MTTR | A frozen golden set while user behavior moves on |
| Phase | Question | Methods | Output |
|---|---|---|---|
| Strategy | Which market and segment are worth it? | Market sizing, analyst + competitor teardown, win/loss review | Target segments, TAM / SAM |
| Vision | What would ideal look like for the user? | Co-design sessions, concept tests, diary studies | Vision prototype users react to |
| Discovery | Is the problem real and frequent? | Interviews, JTBD, surveys, contextual inquiry, funnel + cohort data, support-ticket mining | Personas, opportunity tree, problem ranking |
| Roadmap | Which problems matter most? | Kano / MaxDiff surveys, sales + support input, usage data | Ranked opportunities |
| Define | Does our solution concept fit? | Wizard-of-Oz, fake-door tests, prototype walkthroughs | Validated scope and AI UX |
| Build | Can people use it and trust it? | Moderated usability, think-aloud, beta cohort | Fixes to UX, trust cues, hand-off |
| Launch | Does it change behavior at scale? | A/B tests, in-product surveys, session replays | KPI deltas, adoption blockers |
| Operate | What keeps users, what loses them? | NPS / CSAT, churn interviews, feedback tagging, logs of overrides | Roadmap inputs and new eval cases |