About The Position

We are hiring a Senior Software Engineer to join the Replication Foundations team, part of Temporal’s Cloud Global Services (CGS) organization. The Replication Foundations team owns and evolves Temporal’s core replication stack in Temporal OSS. This stack underpins key Temporal Cloud offerings, including High Availability namespaces, cross-cluster and cross-region failover, and migration products that allows customers to move workloads freely between self-hosted Temporal and Cloud. Beyond cloud features, the team works on fundamental scalability and reliability mechanisms that define how Temporal operates at large scale. As a Senior Engineer on this team, you will work deep in Temporal’s distributed systems core—designing and implementing replication protocols, correctness mechanisms, and scalability features such as multi-cell namespaces and protocols that allow a single namespace to span multiple clusters and split or merge dynamically based on workload demands. This role is highly technical and requires comfort reasoning about consistency, concurrency, failure modes, and performance in complex distributed systems.

Requirements

  • Strong experience building distributed systems with a focus on replication, consistency, and fault tolerance
  • Demonstrated ability to design and implement concurrent, correctness-critical systems
  • Hands-on experience debugging complex production issues involving race conditions, data consistency, or performance degradation
  • Proficiency writing production-quality concurrent code, preferably in Go (Java or similar systems languages also welcome)
  • Solid understanding of distributed systems fundamentals (consensus, replication, sharding, backpressure, failure detection, etc.)
  • Ability to take ownership of technically deep areas and drive improvements with minimal oversight
  • Curiosity and rigor in understanding how systems behave under stress, failure, and scale
  • As a Senior Engineer on this team, you will work deep in Temporal’s distributed systems core—designing and implementing replication protocols, correctness mechanisms, and scalability features such as multi-cell namespaces and protocols that allow a single namespace to span multiple clusters and split or merge dynamically based on workload demands.
  • This role is highly technical and requires comfort reasoning about consistency, concurrency, failure modes, and performance in complex distributed systems.

Nice To Haves

  • Experience designing or maintaining replication protocols or data-plane infrastructure
  • Experience with multi-cluster or multi-region architectures
  • Familiarity with database internals, log-based replication, or event-sourced systems
  • Prior contributions to large OSS projects or distributed systems infrastructure

Responsibilities

  • Design and implement core components of Temporal’s OSS replication stack
  • Build and evolve replication protocols that power: High Availability namespaces Cross-cluster and cross-region replication Migration between Temporal clusters
  • Develop scalability and reliability features such as: Multi-cell namespaces Protocols enabling a single namespace to span multiple clusters Dynamic split and merge of namespaces based on usage and capacity
  • Reason deeply about consistency models, ordering guarantees, and failure recovery in replicated systems
  • Own complex changes end-to-end, from design through implementation, testing, and rollout in OSS and cloud environments
  • Write clear design documents and code comments that explain protocol behavior, invariants, and tradeoffs
  • Collaborate closely with Cloud Enablement and other CGS teams to ensure OSS foundations support current and future cloud products
  • Participate in on-call and incident response related to replication and core system behavior

Benefits

  • Unlimited PTO, 12 Holidays + 2 Floating Holidays
  • 100% Premiums Coverage for Medical, Dental, and Vision
  • AD&D, LT & ST Disability, and Life Insurance (Standard & Supplemental Available)
  • Empower 401K Plan
  • Additional Perks for Learning & Development, Lifestyle Spending, In-Home Office Setup, Professional Memberships, WFH Meals, Internet Stipend and more!
  • Paid Time Off (PTO) and Benefits outside the United States vary by country, and are issued in partnership with Remote.com. Additionally, Temporal offers perks to all international employees for learning & career development, a lifestyle spending account, in-home office setup (in addition to company-issued hardware), professional memberships, work-from-home meals, and access to the Calm app for mental wellness.
  • $3,600 / Year Work from Home Meals
  • $1,800 / Year Professional Enrichment (Career Development & Professional Memberships)
  • $1,200 / Year Lifestyle Spending Account
  • $1,000 / Year In-Home Office Setup (In addition to Temporal issued equipment - laptop, monitor, keyboard, mouse, trackpad, and extension power cable at no cost to you)
  • $74 / Month Reimbursement for Internet
  • Calm App Subscription for Mental Health & Wellness
© 2024 Teal Labs, Inc
Privacy PolicyTerms of Service