Senior Fullstack Engineer
•SerpApi· over 2y
💻 Engineering⏰ Full-time⭐ Senior💰 $150k-180k🌍 Worldwide
over 2y ago
3Pillarglobal·about 3 hours ago
Senior QA Automation Engineer
You'll own test architecture and semantic validation for a mission-critical AI-powered system where consistency guarantees are non-negotiable. This isn't traditional QA; you're designing test harnesses that prove AI outputs are correct, not just crash-free.
Your core responsibility: build a golden-set validation harness that measures whether the system's LLM stage produces the right decisions at the right confidence levels. You'll design automated tests across unit, integration, E2E, and access-control layers. You'll also establish CI/CD quality gates that block bad builds—schema validation failures, cross-tenant isolation defects, missing floor controls, semantic inconsistencies—before they reach production.
This is front-loaded work (weeks 1–5), then you support the team through delivery. You'll mentor a junior test engineer and set the standard for test coverage in a compressed timeline.
Leadership skills
Mentoring and coaching abilities
Strategic thinking and decision-making skills
Project management skills
Stakeholder management and collaboration skills
Ability to motivate and inspire team members
Continuous improvement mindset
Familiarity with DevOps processes and tools.
Proficiency in programming languages such as Java, Python, or Ruby.
Design comprehensive test architecture – plan unit, integration, E2E, and semantic validation test suites that cover the full pipeline from profile intake through roadmap generation; define coverage targets and acceptance criteria
Build semantic validation harnesses for AI outputs – design golden-set test cases (5–8 CISO-approved reference profiles) that measure AI stage precision/recall, rigor-tier correctness, floor-control invariants, traceability completeness, and run-to-run stability; a semantic regression blocks merge
Implement access-control testing – design automated probes at API and database layers that verify cross-tenant isolation; simulate compromised sessions and verify RLS policies enforce tenant boundaries
Design and maintain CI/CD quality gates – build automated test runners that execute on every commit: unit tests, integration tests, schema validation, migration guards, image scanning, access-control probes; red builds block merge
Lead test automation practices – expand automation frameworks and practices; identify opportunities for improving test processes; recommend tools and technologies for testing efficiency
Collaborate across the team – serve as the QA voice in technical design reviews; flag testability issues early; work with the AI/LLM Lead to define what "correct" means for each pipeline stage
Mentor junior test engineers – guide the team on test design, semantic validation, and CI/CD best practices; share knowledge on testing AI systems where reproducibility is critical
Own test documentation and reporting – maintain clear test plans, results, and traceability matrices; report test coverage and quality metrics to leadership; communicate blockers and risks transparently
5+ years of hands-on software QA with a track record of:
Test strategy, planning, execution, and maintenance
Comprehensive test documentation and traceability
Advanced test automation (custom frameworks, advanced scripting)
3+ years with automated testing tools – Selenium, Pytest, or equivalent; ability to write custom testing frameworks
Advanced proficiency in a programming language relevant to test automation:
Python (primary for this engagement; Pytest, async testing)
Or Java, C#, Go
Deep knowledge of databases and SQL – execute complex data validation, verify multi-tenant isolation at the DB layer, test RLS policies
CI/CD pipeline expertise – understand how tests integrate into automated deployment workflows; experience optimizing test execution for speed and reliability
Understanding of Agile/Scrum methodologies – ability to drive QA strategies in rapid, iterative development cycles
Proficiency in defect tracking (Jira or equivalent)
Proven leadership – mentor junior engineers, advocate for quality standards, drive process improvements
Exceptional communication skills – articulate complex technical issues to engineers, product, and stakeholders
Self-motivated and detail-oriented – work independently under time pressure, manage multiple priorities
Exceptional English proficiency – work effectively with distributed, global teams
ISTQB Advanced Level, CSTE, or equivalent QA certification
Performance and load testing (JMeter, LoadRunner) – design and execute tests under various conditions
Security testing practices – knowledge of OWASP vulnerabilities, experience with security testing tools (OWASP ZAP, Fortify)
Test-driven development (TDD) and behavior-driven development (BDD) – familiarity with Gherkin, Cucumber, or equivalent
Experience testing multi-tenant systems – understanding of isolation, RLS, and cross-tenant security validation
Experience with AI/ML systems – familiarity with testing LLM outputs, semantic validation, and consistency guarantees
API testing – REST/GraphQL endpoint validation, contract testing
Experience in regulated environments – audit logging, reproducibility, compliance testing
Demonstrated track record of process improvement – initiated and led projects that enhanced quality, reduced costs, or improved efficiency
Medical Insurance benefits as per company policy.
Life Insurance as per company policy
15 days of paid vacation, sick leave, and paid holidays as per local law
Paternity and maternity leave as per as per local law
Marriage, bereavement and graduation leaves as per company policy
Sick leave and paid holidays as per local law
Christmas and mid-year bonuses as per local law
Monthly productivity bonus as per local law
Discounts in local shops
Direct deposit of payroll.
Paid professional certifications
At 3Pillar, we offer a world of opportunity:
Imagine a flexible work environment – whether it's the office, your home, or a blend of both. From interviews to onboarding, we embody a remote-first approach.
You will be part of a global team, learning from top talent around the world and across cultures, speaking English everyday. Our global workforce enables our team to leverage global resources to accomplish our work in efficient and effective teams.
We’re big on your well-being – as a company, we spend a whole trimester in our annual cycle focused on wellbeing. Whether it is taking advantage of fitness offerings, mental health plans (country-dependent), or simply leveraging generous time off, we want all of our team members operating at their best.
Our professional services model enables us to accelerate career growth and development opportunities - across projects, offerings, and industries.
We are an equal opportunity employer. It goes without saying that we live by values like Intrinsic Dignity and Open Collaboration to create cutting-edge technology AND reinforce our commitment to diversity - globally and locally.
Join us and be a part of a global tech community! 🌍💼 Check out our Linkedin site and Careers page to learn more about what it’s like to be part of our #oneteam!