Staff Software Engineer, Replication Foundations

Temporal Technologies

About The Position

Temporal is an open source programming model that can simplify code, make applications more reliable, and help developers focus on the important things like delivering features faster. We are on a mission to be the reliable foundation of every developer’s toolbox, and are building the team that will make that happen. Our values guide us —they are present in how we show up, make decisions, and work together to make an impact. We’re curious, driven, collaborative, genuine and humble. Temporal is growing and we are looking for those who share our values, challenge 'standard' thinking, and want to influence our future. If you have a passion for improving the developer experience, building world-class open-source software and communities, and want to be a part of our amazing team, we'd love to hear from you! Summary We’re hiring a Staff Software Engineer to join the Replication Foundations team, part of Temporal’s Cloud Global Services (CGS) organization. Replication Foundations owns and evolves Temporal’s core replication stack in Temporal OSS—the distributed systems backbone behind key Temporal Cloud capabilities like High Availability namespaces, cross-cluster and cross-region failover, and migration products that let customers move workloads between self-hosted Temporal and Temporal Cloud. The team also drives fundamental scalability and reliability mechanisms that define how Temporal operates at large scale. At the Staff level, you’ll operate as a technical leader in Temporal’s distributed core: shaping protocols and correctness guarantees, leading complex efforts end-to-end, raising engineering rigor across replication foundations, and partnering cross-functionally to ensure OSS primitives unlock current and future cloud products. This role is deeply technical and requires comfort reasoning about consistency, concurrency, failure modes, performance, and operational safety in correctness-critical systems. Key outcomes you’ll drive: Evolve replication protocols and correctness mechanisms that make failures (and cloud outages) a non-event for customers. Deliver scalability primitives (e.g., multi-cell namespaces) that unlock the next phase of Temporal Cloud growth. Raise the bar on safety, observability, and operability of Temporal’s replication layer across OSS and cloud.

Requirements

10+ years building production systems, including significant experience with distributed systems and correctness-critical infrastructure.
Strong experience with replication, consistency, fault tolerance, and failure recovery in distributed environments.
Demonstrated ability to design and implement concurrent, correctness-critical systems with clear invariants and safety guarantees.
Proven track record of leading complex technical projects across teams—setting direction, driving execution, and landing changes safely in production.
Hands-on experience debugging complex production issues involving race conditions, data consistency, partial failures, and performance degradation.
Proficiency writing production-quality concurrent code, preferably in Go (Java/C++ or similar systems languages also welcome).
Solid understanding of distributed systems fundamentals such as replication, sharding/partitioning, backpressure, failure detection, and durability mechanisms.
Ability to operate with high ownership and minimal oversight, balancing deep technical rigor with pragmatic delivery.
Curiosity and rigor in understanding how systems behave under stress, failure, and scale.

Nice To Haves

Experience designing or maintaining replication protocols or data-plane infrastructure.
Experience with multi-cluster or multi-region architectures (active-active / active-passive).
Familiarity with database internals, log-based replication, or event-sourced systems.
Prior contributions to large OSS projects or distributed systems infrastructure.

Responsibilities

Lead the design and implementation of core components of Temporal’s OSS replication stack, from initial design through rollout and long-term operational ownership.
Design and evolve replication protocols that power: High Availability namespaces Cross-cluster and cross-region replication Migration between Temporal clusters (cloud ↔ self-hosted, cloud ↔ cloud)
Build scalability and reliability capabilities such as: Multi-cell namespaces Protocols enabling a single namespace to span multiple clusters Dynamic split/merge strategies based on usage, hot spots, and capacity needs
Reason deeply about correctness: consistency models, ordering guarantees, idempotency, failure recovery, and safe rollouts of protocol changes.
Drive cross-team alignment with Cloud Enablement and other CGS teams to ensure OSS foundations support current and future cloud products.
Author high-quality design docs that clarify invariants, trade-offs, failure modes, and operational playbooks for complex changes.
Raise engineering standards through reviews, mentorship, and technical leadership—improving correctness testing, fault injection, and incident readiness.
Participate in on-call/incident response related to replication and core system behavior, helping build durable fixes and prevention mechanisms.

Benefits

Unlimited PTO, 12 Holidays + 2 Floating Holidays
100% Premiums Coverage for Medical, Dental, and Vision
AD&D, LT & ST Disability, and Life Insurance (Standard & Supplemental Available)
Empower 401K Plan
Additional Perks for Learning & Development, Lifestyle Spending, In-Home Office Setup, Professional Memberships, WFH Meals, Internet Stipend and more!
Paid Time Off (PTO) and Benefits outside the United States vary by country, and are issued in partnership with Remote.com. Additionally, Temporal offers perks to all international employees for learning & career development, a lifestyle spending account, in-home office setup (in addition to company-issued hardware), professional memberships, work-from-home meals, and access to the Calm app for mental wellness.
$3,600 / Year Work from Home Meals
$1,800 / Year Professional Enrichment (Career Development & Professional Memberships)
$1,200 / Year Lifestyle Spending Account
$1,000 / Year In-Home Office Setup (In addition to Temporal issued equipment - laptop, monitor, keyboard, mouse, trackpad, and extension power cable at no cost to you)
$74 / Month Reimbursement for Internet
Calm App Subscription for Mental Health & Wellness