Senior Staff Threat Hunter & Intelligence Engineer - Databricks

Senior Staff Threat Hunter & Intelligence Engineer - Databricks

About the Role

What if you could hunt threats across one of the world's largest data platforms—using that same platform as your weapon? At Databricks, our threat hunters don't just protect a data company; they leverage petabyte-scale analytics, real-time streaming, and ML infrastructure that most security teams can only dream of. This is threat hunting without constraints.

We're looking for a Senior Staff Threat Hunter & Intelligence Engineer to define and lead our threat hunting and intelligence capabilities across AWS, Azure, and GCP. You'll set the strategic direction for how we detect and pursue adversaries, build the tooling and infrastructure to hunt at scale, and serve as a technical authority across our security organization.

Key Responsibilities

Advanced Threat Hunting Operations

• Define the strategic vision and roadmap for a structured, repeatable threat hunting program using hypothesis-driven methodologies aligned with industry frameworks.
• Develop Databricks-based hunting capabilities and logic to analyse security telemetry at a massive scale across our multi-cloud environment.
• Build reusable hunting notebooks and automated intelligence pipelines using Databricks workflows.
• Serve as the technical authority for threat hunting across Security, influencing detection strategy and incident response capabilities.
• Mentor and develop threat hunting capabilities across the security organization.

Strategic Threat Intelligence Leadership

• Operationalize threat intelligence from multiple sources (commercial feeds, OSINT, industry sharing groups) into actionable hunting hypotheses.
• Work with internal partners to develop and maintain Priority Intelligence Requirements (PIRs).
• Build automated enrichment pipelines using Databricks to correlate intelligence with internal telemetry.
• Produce intelligence assessments on threats relevant to our business.
• Represent Databricks in external security communities, industry working groups, and with strategic customers on advanced threat topics.

Platform Innovation & Tooling

• Architect scalable hunting infrastructure using Databricks notebooks, Delta Lake, and Unity Catalog.
• Develop libraries of reusable detection logic and hunting queries optimized for distributed computing.
• Build automated workflows for threat intelligence ingestion, enrichment, and correlation.
• Create dashboards and visualizations for threat exposure and hunt findings.
• Integrate security tools with Databricks platform.

Required Qualifications

• 12+ years in cybersecurity with 6+ years focused on threat hunting, threat intelligence, or detection engineering.
• Deep expertise with nation-state and e-crime threat actors’ TTPs, trends, and historical targets.
• Experience working with large-scale security datasets and big data platforms.
• Strong Python programming experience with a background in PySpark, distributed computing frameworks, or Databricks’ platform.
• Deep understanding of cloud security across AWS, Azure, and GCP—including cloud-native logging, security controls, and container/Kubernetes security.
• Strong knowledge of OS internals across macOS, Linux, and containerized environments.
• Experience with enterprise-scale software development practices including infrastructure-as-code, code review, and large codebase management.

• Demonstrated experience conducting hypothesis-driven threat hunts with measurable outcomes.
• Experience defining and driving multi-year security program strategy.
• Thought leadership around the application of cybersecurity frameworks, such as MITRE ATT&CK and D3FEND.
• Applied CTI skills including consuming and operationalizing IOCs/TTPs, tracking campaigns, and conducting research.

• Experience influencing technical decisions beyond your immediate team.
• A track record of mentoring Staff+ engineers.

Preferred Qualifications

• Experience with Databricks platform or similar (Spark, Delta Lake, MLflow).
• Experience protecting multi-tenant SaaS/PaaS environments.
• Experience using AI, Large Language Models or machine learning to automate cybersecurity operations.
• Experience with purple team operations and adversary emulation.
• Published research at major cybersecurity conferences or in academic journals.
• Contributions to impactful open-source security projects or software patents in the cybersecurity domain.

What You'll Bring

• The ability to think like an adversary while understanding defender constraints.
• Comfort with ambiguity and ability to define structure where none exists.
• A track record of building consensus across competing priorities.
• The ability to translate technical capabilities into business risk reduction.
• The desire to automate and scale hunting operations using our modern data platform.
• The drive to stay current with rapidly evolving cloud threats and technologies.
• The ability to work effectively across security, engineering, and product teams.