Senior Software Engineer(Distributed Systems)

Your work days are brighter here.

We’re obsessed with making hard work pay off, for our people, our customers, and the world around us. As a Fortune 500 company and a leading AI platform for managing people, money, and agents, we’re shaping the future of work so teams can reach their potential and focus on what matters most. The minute you join, you’ll feel it. Not just in the products we build, but in how we show up for each other. Our culture is rooted in integrity, empathy, and shared enthusiasm. We’re in this together, tackling big challenges with bold ideas and genuine care. We look for curious minds and courageous collaborators who bring sun-drenched optimism and drive. Whether you're building smarter solutions, supporting customers, or creating a space where everyone belongs, you’ll do meaningful work with Workmates who’ve got your back. In return, we’ll give you the trust to take risks, the tools to grow, the skills to develop and the support of a company invested in you for the long haul. So, if you want to inspire a brighter work day for everyone, including yourself, you’ve found a match in Workday, and we hope to be a match for you too.

About the Team

The Data Platform and Observability Engineering (DPOE) team is building Workday’s next-generation, multi-petabyte-scale Observability Platform. We are a distributed team responsible for the libraries, distributed services, and infrastructure that power observability across Workday. Our stack includes Iceberg, ClickHouse, Tempo, Grafana, S3, Kafka, Elasticsearch, and LangSmith for LLM/agentic tracing and evaluation. Together, we support traces, metrics, logs, and LLM/agent observability across Workday workloads at scale.

About the Role

As a Senior Software Development Engineer, you will own the technical design and execution of core components of Workday’s distributed tracing platform, built on ClickHouse and/or Grafana Tempo and backed by a big-data pipeline using Kafka, Spark/Flink, Iceberg, and S3. This is a hands-on role where you will solve multi-petabyte-scale challenges, optimize low-latency infrastructure, and shape the future of Observability AI by extending distributed tracing to LLM and agentic workflows with LangSmith and LangChain, enabling visibility into agent execution, tool calls, and prompt/response chains.

Basic Qualification

8+ years experience in software development engineering.

4+ years experience specifically focused on designing, building, and operating complex distributed system architectures, evidenced by successful deployment of systems with high availability (e.g., 99.9% uptime) and fault tolerance.

5+ years experience with at least two of the following programming languages Java, Go, Scala, Python, including experience in writing production-level code for distributed systems.

Bachelor’s degree in a relevant field such as Computer Science, Engineering, or a related discipline; a Master's degree (e.g., MS in Computer Science, Distributed Systems, or related field) is strongly preferred or equivalent practical experience.

Other Qualification

Strong ability in Algorithmic Thinking, including to build highly efficient and scalable solutions for complex high-throughput data ingestion and sub-second query performance challenges.

Solid experience in API Development, including an understanding of gRPC, REST, and OpenTelemetry (OTLP), with practical experience designing and building scalable distributed APIs for observability data.

Strong understanding of Code Testing methodologies, such as distributed load testing and integration testing, and experience contributing to end-to-end telemetry pipeline testing and CI/CD automation.