Production Engineer

Mysten Labs believes that decentralized and open protocols are the bedrock of the internet of value. This is why at Mysten Labs, we are creating foundational infrastructure to accelerate the adoption of decentralized protocols based on blockchain technologies.

Team Description

Production Engineers at Mysten Labs are the custodians of our decentralized infrastructure—keeping the Sui blockchain, Walrus storage network, and related protocols humming under massive traffic, adversarial attacks, or global-scale events. With expertise in automation, observability, and resilient systems, PEs collaborate with distributed systems experts to deploy, monitor, and harden software that powers Web3's most demanding workloads. This is hands-on work in a fast-scaling decentralized environment: hybrid cloud/bare-metal infrastructure, Grafana/Mimir/Loki observability stack, and other cutting-edge integrations.

Role Description

You'll own the four PE pillars—Infrastructure, Observability, Release Engineering, and Reliability—while embedding operational rigor into internal services as well as the decentralized Sui Stack (including protocols like Walrus and SEAL). Expect autonomy to hunt down bottlenecks, automate toil out of existence, and ship fixes that prevent outages at decentralized infrastructure scale. Partner with core engineers on productionizing Rust-based systems, support internal teams building new products, and tackle Web3-unique challenges. If you thrive on turning chaos into uptime and love automating your way out of incidents, this is your shot to shape the infrastructure for the next billion users on Sui.

What You’ll Do:

• Scale hybrid cloud + bare-metal infrastructure for Sui validators and Walrus storage nodes, optimizing for cost, latency, and resilience against attacks.

• Supercharge observability with our Grafana stack (Mimir, Loki)—building dashboards, alerts, and tracing to pinpoint issues in real-time during peak loads.

• Evolve deployment pipelines beyond GitHub Actions: design next-gen CI/CD for high-velocity releases, incorporating IaC (e.g., Pulumi) and Kubernetes orchestration.

• Drive reliability wins—measure/optimize performance, author runbooks, automate incident response, and harden systems for decentralized threats like DDoS or chain halts.

• Support Engineering partners (Greece/Europe focus) collaborating cross-functionally to unblock product velocity.

What You’ll Have:

• 5+ years hands-on Production Engineering/SRE experience, owning infrastructure as code, containerized workloads and orchestration (Kubernetes), observability (metrics/logs/traces), deployments, and reliability at scale.

• Built/maintained infrastructure code and tooling with Pulumi or equivalents; deployed on GCP/AWS; automated releases/monitoring pipelines.

• Systems programming fluency (Rust preferred—Mysten's stack; or Go) + scripting (Python/Bash/Shell) for debugging distributed systems.

• Proven runbook discipline, incident leadership, and a bias for automation in high-availability environments.

• Thrives independently spotting issues, driving fixes autonomously, and partnering with eng/product teams—no handholding needed.

• (We encourage strong candidates with adjacent scalable infrastructure experience to apply—even if you don't check every box.)

If you have it… Nice!:

• Blockchain/crypto production experience (Validators, RPC nodes, L1s, DeFi infra).

• Grafana/Mimir/Loki expertise.

• Experience as a DBA, Network Engineer, and/or Linux Systems Administrator (e.g., managing production databases, networks, or bare-metal environments).

• Rust depth for contributing to Sui/Walrus.

Employment is contingent upon the successful completion of a background check, which may include verification of employment history, education credentials, criminal history, and other relevant information.

Regarding the recent rash of technology job scams: Be aware that emails from genuine Mysten Labs group recruiters will always come from the @mystenlabs.com domain or related subdomains (e.g., mystenlabs.com/careers). Remember: you can always verify positions on our job boards at www.mystenlabs.com/careers.

To support an efficient and fair hiring process, we may use technology-assisted tools, including artificial intelligence (AI), to help identify and evaluate candidates. All hiring decisions are ultimately made by human reviewers.

Our team is remote first and we are hiring across the world. Here at Mysten Labs, you’ll be joining a world-class team with tremendous growth potential as we bring the next billion users to web3. We raised a $300M Series B round from top Silicon Valley led venture funds like Jump Crypto, Andreessen Horowitz (a16z), Binance Labs, Redpoint, Lightspeed, Coinbase Ventures, Electric Capital, Standard Crypto, NFX, Slow Ventures, Scribble Ventures, Samsung Next, Lux Capital, among other investment firms and strategic partners. Come join us and build the future of web3!