Caseware is one of Canada's original Fintech companies, having led the global audit and accounting software industry for over 30 years, with more than 500,000 users across 130 countries and available in 16 different languages. While you might not have heard of us (yet) over 36,000 accounting and audit professionals list Caseware as a skill on their LinkedIn profiles!
We are hiring AI Quality Engineers to join our Central Quality Engineering function to drive the future of agentic testing across our AI first SaaS platform. You will work within our Platform Engineering organization, shaping the quality strategy for foundational infrastructure and new intelligent systems. This includes both modern SaaS components and advanced AI-driven, agentic platform capabilities—areas that require new thinking in test strategy, automation, and verification.
📍 Location: This is a fully remote position located in Colombia.
You will be reporting to:
Contact:
What You’ll Be Doing
-
1.Quality Engineering Leadership Across the Platform
- Represent and elevate the Central QE community, partnering closely with product and engineering teams to embed quality at every stage of development.
- Contribute to a modern Quality strategy for a fast-scaling AI First SaaS architecture.
- Improve CI/CD workflows with better test isolation, data setup, and execution reliability.
2. AI-Focused Test Strategy & Execution
- Develop primarily deterministic and contribute into non-deterministic testing strategies for LLM-based agentic systems.
- Build automation to validate the offline and online evaluation pipelines for correctness, safety, drift, and reliability.
- Design and implement test harnesses for agent endpoints, tool-calling flows, and planning sequences.
- Build automation to validate AI artifacts including prompt libraries, golden datasets, rubric scoring, eval outputs and model versions.
- Validate RAG pipelines end-to-end and integrate AI-specific metrics into CI/CD.
- Build safety, compliance, and guardrail validation mechanisms.
3. Functional & Cross-Team Testing
- Partner across teams to ensure timely, high-quality delivery of roadmap commitments.
4. Quality Metrics, Telemetry & Observability
- Reporting discipline for DORA metrics, test coverage, test effectiveness, and health execution.
- Extend metrics with AI-specific quality indicators.
- Expand observability for AI systems including degradation patterns and anomalous outputs.
5. Culture, Leadership & Collaboration
- Promote a quality-first mindset across Engineering.
- Lead cross-functional initiatives to ensure quality influences design, development, deployment, and incident reviews.
What You Will Bring
-
- 5+ years experience in Quality Engineering within cloud-native SaaS organizations.
- Expertise in automated testing infrastructure and CI/CD pipelines.
- Strong understanding of testing strategies across the pyramid.
- Solid frontend and full-stack testing collaboration skills.
- Familiarity with observability tooling.
- Experience testing LLMs, AI agents, RAG pipelines.
- Knowledge of AI safety risks including hallucinations and prompt injections.
- Strong communication, leadership, and coaching skills. C1 english.
- Bachelor's/Master's in Computer Science, AI, or related; certifications
Perks & Benefits
- Contrato a termino Indefinido with all the legal benefits
- Prepaid Medicine
- Life insurance and funeral assistance
- Internet allowance
- Home office stipend
- Competitive compensation — above the market average
- 100% remote work environment and an excellent work-life balance
- Opportunity to work for a growing global SaaS leader company
- A culture that promotes independence, innovation, trust, and accountability
- Open space to be creative, innovative, and strategize for the future
- Mentorship by a highly experienced professional
- Budget for training, we want you to grow
- 5 Personal Time Off days per year
- Sick Leave Top up to total 100% of salary paid by the employer from Day 3 to 90.
- Recognition Award, additional paid time off in recognition of the corresponding year of service
- Upgrade vacation starting at 5 years of service
- AI-first environment: Be part of an AI-first engineering organization that embraces modern tools, automation, and AI-driven ways of working.
What's in it for you:
▪️Innovation is at our core. We work with cutting-edge technology in accounting and financial reporting, constantly pushing the boundaries to create impactful software solutions.
▪️We are committed to a collaborative culture, where your ideas are valued, and knowledge sharing is encouraged within a supportive, inclusive team.
▪️Work-life balance is important to us. We offer flexible work options, remote opportunities, and generous time-off policies to ensure a healthy work-life balance.
▪️We offer competitive compensation, including a competitive salary and comprehensive benefits such as health insurance and retirement plans.
▪️We are driven by impactful work. Your contributions directly affect how our clients manage financial processes and drive their success.
▪️Recognition and rewards matter to us. We celebrate hard work through recognition programs, performance bonuses, and opportunities for career growth.
▪️We embrace global opportunities. Work on international projects and collaborate with a diverse, global team.
About Caseware:
Caseware's cutting-edge software products are meticulously designed for accounting firms, corporations, and governments. Our teams are continually collaborating, innovating, and building upon our existing suite of products. With a customer-focused mindset, we are building technology that is shaping what the future of audits, financial reporting, and financial data analytics will look like.
With a recent strategic investment from Hg Capital in 2020, Caseware is now in its next major growth phase as we double down on the people and products that have made Caseware so successful to date.
One of Caseware's core values is Many Voices, One Team and with that in mind, we're dedicated to building teams as diverse as our customers in an equitable and inclusive way. We welcome and encourage candidates of all backgrounds to apply. Should you require accommodations or have any questions at any point during the application or interview process, please e-mail our People Operations team at [email protected].
AI Usage:
The recruitment process may use AI assisted tools but not for candidate screening or assessment. All final hiring decisions are made by humans to ensure fairness, transparency, and oversight.
Background Check:
Any candidates successful in obtaining an offer for a position will need to successfully complete a background check through Certn.co which typically includes an Identity Verification and Criminal Record Check. Executives and Senior Managers will undergo a Soft Credit Check as well. Candidates residing in the Netherlands and Germany are excluded from undergoing background checks via Certn.co
Security and Fraud:
Caseware takes the security of candidates seriously. All legitimate communication from us will come from email addresses ending in @caseware.com and our open positions are always listed on reputable job boards and on our website https://jobs.lever.co/caseware. We will NEVER ask for payment or financial information from you. If you receive an unsolicited job offer, proceed with extreme caution.
