About Onit
We're redefining the future of legal operations through the power of AI. Our cutting-edge platform streamlines enterprise legal management, matter management, spend management and contract lifecycle processes, transforming manual workflows into intelligent, automated solutions.
We’re a team of innovators using AI at the core to help legal departments become faster, smarter, and more strategic. As we continue to grow and expand the capabilities of our new AI-centric platform, we’re looking for bold thinkers and builders who are excited to shape the next chapter of legal tech.
If you're energized by meaningful work, love solving complex problems, and want to help modernize how legal teams operate, we’d love to meet you.
Position Summary
• The Director of AI Ethics and Integrity leads Onit's AI red team function, responsible for rigorous validation of all AI-powered products, both legal AI features and general LegalOps automation, before release to customers.
• This role combines AI evaluation methodology with domain expertise to ensure our products meet the highest standards of accuracy, safety, and reliability across all use cases.
• Critically, this position is independent from the model development and product engineering teams, providing objective, adversarial testing that identifies risks before they reach production.
• The successful candidate will lead a team of AI Researchers who stress-test products for hallucinations, reasoning failures, security vulnerabilities, and domain-specific risks including legal accuracy where applicable.
Key Responsibilities
- Lead the AI red team function: design and execute adversarial testing protocols for all AI-powered products across the Onit portfolio
- Build comprehensive evaluation frameworks covering general AI failure modes (hallucination, reasoning errors, instruction following) and domain-specific risks
- Identify and document failure modes across product categories: contract automation, document processing, workflow AI, and legal research features
- Establish quality gates and release criteria that all AI products must pass before customer deployment
- Manage relationships with external red team vendors and academic partners for independent validation
- Develop and maintain AI ethics guidelines applicable across Onit's product suite
- Create feedback loops between red team findings and model development to drive continuous improvement
- Report directly to leadership on product readiness, risk assessments, and recommended mitigations
- Represent Onit in industry discussions on AI safety, responsible deployment, and enterprise AI governance
- Collaborate with Legal/Compliance on regulatory requirements and professional responsibility considerations where applicable
Required Qualifications
- 7+ years of experience in AI/ML evaluation, quality assurance, or product validation
- Experience with AI/ML red teaming, adversarial testing, or systematic evaluation methodology
- Strong understanding of LLM failure modes: hallucination, prompt injection, reasoning errors, and edge case behavior
- Proven ability to design systematic testing protocols and maintain rigorous documentation
- Strong analytical skills with ability to identify edge cases and failure modes across diverse product types
- Excellent communication skills for translating technical risks to business stakeholders
- Experience leading teams in a quality, validation, or research capacity
Preferred Qualifications
- JD or legal background (valuable for legal AI features but not required)
- Background in AI safety, responsible AI, or ML evaluation at a technology company or research organization
- Experience with enterprise software validation in regulated industries
- Familiarity with LLM behavior: prompt engineering, fine-tuning effects, RLHF impacts
- Published research or thought leadership in AI safety, evaluation methodology, or responsible AI
Red Team Focus Areas
- Factual Accuracy: Verify AI outputs are grounded in source documents and don't fabricate information
- Reasoning Validation: Test for logical consistency, correct inference, and sound conclusions
- Instruction Following: Ensure AI respects user instructions, constraints, and formatting requirements
- Security and Robustness: Stress-test for prompt injection, jailbreaking, data leakage, and manipulation attempts
- Bias and Fairness: Evaluate for demographic bias, outcome disparities, or unfair treatment in AI recommendations
- Edge Case Handling: Test behavior with ambiguous inputs, conflicting instructions, and unusual data
- Legal-Specific (where applicable): Citation accuracy, jurisdictional compliance, unauthorized practice prevention
Benefits & Perks That Support You:
Onit offers a comprehensive total rewards package designed to support the whole employee at work and beyond:
Health Coverage: Employee and immediate family members.
Time Away: Flexible paid time off and 10 company paid holidays annually.
Family Support: Exceptional paid leave for birth parents, non-birth parents, and caregivers. Onit also offers surrogacy and adoption reimbursement.
Income Protection: 100% employer-paid life and disability insurance.
Additional Coverage Options: Voluntary benefits including hospital indemnity, critical illness, accident, and even pet insurance.
Tax-Advantaged Accounts: Flexi, NPS.
Community Engagement: One paid volunteer day each year to give back to the community.
Our Commitment to Applicants
We know that not everyone will check every box in a job description. At Onit, we value diversity, inclusion, and authenticity. If you’re excited about this role but your experience doesn’t align perfectly with every qualification, we encourage you to apply. You may be exactly who we’re looking for.
Onit Values
Customer First - Customer success is our success. We deliver value, listen, and act on customer needs.
Purposeful Innovation - Innovation fuels our growth. We harness creativity to solve problems and lead with the intentions and expertise.
Win as One - Teamwork is how we win. We are accountable, act with integrity, and communicate openly.
Intentional Growth - Our people are the difference. We create an environment with compelling work, impactful contributions, and career growth.
