Software Engineer, Discovery Eval

Within Perception, the Discovery Eval team delivers early, actionable, and strategic insights into driver performance. We design automated analytic workflows using Large Language Models to cluster, summarize, and surface critical issues from simulation and real-world data. Our work orchestrates the full lifecycle of these discoveries to directly feed the ML training flywheel and ensure perception models operate with high precision and reliability.

In this hybrid role, you will report to an engineering manager.

You will:

• Design, develop, and implement automated eval analytic workflows using large language models and other AI techniques to analyze and categorize events from perception evaluation systems.
• Build and maintain scalable data pipelines to automatically cluster, summarize, and surface critical issue categories discovered through large-scale simulation and real-world data.
• Develop an orchestration system to manage the lifecycle of discovered issues, including data sampling, annotation, and integration into the ML training and evaluation flywheel.
• Enhance and support core discovery tools and frameworks, focusing on improving the precision, recall, and reliability of automated issue detection.
• Expand the capabilities of discovery evaluation systems to cover a broader range of perception signals and other advanced outputs.
• Collaborate with other engineers, ML infrastructure teams, and researchers to integrate cutting-edge AI techniques and build robust, end-to-end evaluation solutions.

You have:

• BS or MS in Computer Science or a related field.
• 2+ years of software development experience (3+ years for L5).
• Strong programming skills in Python and/or C++.
• Experience with data analysis, data pipelines, and working with large datasets.
• Solid understanding of machine learning concepts and evaluation methodologies.
• Excellent problem-solving skills and ability to work independently and collaboratively.

We prefer:

• Experience in developing or utilizing AI-powered agentic workflows.
• Familiarity with evaluation framework in autonomous vehicle systems or robotics.
• Hands-on experience with ML infrastructure and tools.
• Experience with large-scale data processing systems.
• Proven ability to design and implement novel evaluation metrics and techniques.
• Experience with data visualization tools and techniques.