What you will be doing:

• Drive high-impact AI initiatives within the Caseware platform transformation — shape priorities and sequencing for your workstream, grounded in user research, domain expertise, and a clear view of technical feasibility.

• Design and run rigorous eval frameworks: build golden datasets, define task taxonomies, write rubrics that score AI outputs the way a senior practitioner would, and run structured eval cycles before and after every major release.

• Work daily with engineering to translate precise problem statements into well-scoped product requirements — including clear acceptance criteria and eval bars before development begins.

• Embed with domain experts and partner accounting and audit firms to validate AI outputs against professional standards — not just user preferences, but what a qualified practitioner would actually sign off on.

• Drive continuous discovery with practitioners: shadow workflows, test prototypes in context, and turn 'this doesn't feel right' into specific, actionable failure modes.

• Stay ahead of the AI landscape — monitor model developments, new architectures, emerging agent patterns, and competitor moves, and bring a clear point of view on what matters for Caseware.

• Define and own product health metrics for AI features: task completion, correction rates, override frequency, trust signals, and time-to-completion — and use them to drive product decisions.

• Synthesize signals across engineering, partners, domain experts, customer success, and leadership into a coherent product direction — and communicate it with clarity at every level of the organization.

What you will bring:

• 4+ years of experience in technology-focused product management or product development, including 1–2 years working deeply with AI or machine learning–driven products.

• Direct experience building and running eval frameworks for AI products — golden datasets, rubrics, human-in-the-loop validation, and regression tracking across releases.

• Technical fluency in LLMs, RAG, prompt engineering, embeddings, and agentic patterns — enough to have substantive conversations with engineers and know when a product problem is a model problem.

• High agency: you identify what needs to happen, move without waiting to be told, and take ownership of outcomes rather than just activities.

• Genuine curiosity about AI — you follow model releases, read research, understand what frontier labs are shipping, and form your own views on what matters.

• Experience working with domain experts or subject matter specialists to define quality bars in high-stakes, regulated, or professional-grade contexts.

• Comfortable with ambiguity. Many of the problems this role works on do not have clear playbooks — you are energized by that, not slowed down by it.