DataOps Engineer - Lakehouse Operations

DeepLight AI is a specialist AI and data consultancy with extensive experience implementing intelligent enterprise systems across multiple industries, with particular depth in financial services and banking. Our team combines deep expertise in data science, statistical modeling, AI/ML technologies, workflow automation, and systems integration with a practical understanding of complex business operations.

As a DataOps Engineer, you will be the guardian of our modern Lakehouse environment, ensuring that our complex data ecosystem remains reliable, transparent, and high-performing. Positioned at the vital intersection of Data Engineering and Operations, you will lead the charge in proactive monitoring and rapid incident response for mission-critical pipelines, including AWS Glue, dbt, and Kafka. 

This is a high-visibility role where you won't just react to issues—you will own the end-to-end incident lifecycle, drive deep root-cause analysis (RCA), and collaborate with development teams to build permanent automated preventions. Whether you are managing real-time operational dashboards or leading daily reviews of platform health, your mission is to guarantee data freshness and platform integrity, turning operational stability into a competitive advantage for our data-driven decision-making.

Your responsibilities as a DataOps Engineer will include, but not be limited to;

• Monitoring & Alerting
• Monitor and act on incidents related to:
• AWS Glue job executions
• Current DMS and dbt pipelines
• Kafka lag and streaming health
• Data freshness SLA breaches
• Data quality issues
• Platform health alerts
• Perform L1 triage or distribute incidents to L2/L3 teams as needed.
• Incident Management
• Own the incident response process:
• Initial triage and severity assessment.
• Coordinate with development teams for resolution.
• Create and assign JIRA tickets with full context.
• Track incident resolution and closure.
• Escalate high-priority or long-running incidents to management.
• Root Cause Analysis & Prevention
• Conduct post-incident root cause analysis.
• Maintain incident logs and post-mortem documentation.
• Implement preventive measures for recurring issues in collaboration with dev teams.
• Operational Reporting
• Manage the TV dashboard providing real-time status of critical flows (with color-coded indicators).
• Deliver a daily 10–15 min operational review of previous day’s executions, open incidents, and follow-ups.
• Share daily summary emails with the team.
• Continuous Awareness
• Stay up-to-date on the status of all critical flows and remediation efforts.
• Ensure proactive communication on risks and delays.

As an AI consultancy, our greatest asset is the expertise of our people.

While technical mastery is the foundation of what we do, the ability to bridge the gap between complex data science and actionable business value is what defines your success with Deeplight.

We're looking for individuals who are not only world-class in their fields of specialism, but also compelling communicators and persuasive advocates for their own skills.

You will be the face of our firm, tasked with building trust, articulating the "why" behind your technical decisions, and effectively "selling" your vision to high-level stakeholders.

If you thrive on the challenge of presenting cutting-edge solutions as much as you do on building them, you will fit right in.