Principal Data Scientist

About the team

At Zillow Group, our mission is to help more people get home. The CoreTech Org delivers on this mission by building the foundational platforms, data, and developer capabilities that power every consumer, partner, and employee experience at Zillow.

You will work most closely with the Connections Platform (CP) — the engine that turns consumer interest into a connection with the right partner, owning the conversational experience, the lead funnel and follow-up workflows, and the shared agent services that sit underneath both. Increasingly, that experience is powered by LLM-based and agentic systems: agents that qualify leads, summarize and extract intelligence from conversations for partner teams, and drive proactive outreach. Beyond CP, you will be a measurement partner for the broader CoreTech Org — the foundational platforms every consumer, partner, and employee experience at Zillow is built on — wherever a hard data science problem needs an owner.

These are platforms with hundreds of touchpoints and dozens of dependent teams — and they have historically been among the hardest places at Zillow to measure well. CP itself is mid-transformation, shifting from NLP to LLM and agent-based systems, with agents increasingly built and relied on in production but no rigorous way to evaluate them yet. That combination — high dependency, hard-to-measure, and changing fast — is exactly where rigorous measurement creates the most leverage.

About the role

We are seeking a Principal Data Scientist to be a measurement and impact partner in CoreTech. This is a senior individual-contributor role embedded with the Data & Infrastructure Platform team and who will partner tightly with Product leadership.

You will own four outcomes: (1) make the impact of platform work legible — quantifying what shipped, what moved, and what didn’t, in terms leaders can act on; (2) define the metrics that matter for CP and the other platforms you take on — what belongs on a progress dashboard, what is a leading vs. lagging indicator, and what is vanity; (3) build the evaluation layer CP is missing — standing up rigorous, repeatable evals for its LLM and agentic systems (including LLM-as-a-judge) so the team can tell whether an agent actually works, not just that it shipped; and (4) operationalize and instrument those metrics and evals in partnership with engineering — getting the events emitted, the pipelines stable, the dashboards live, and the data trusted.

The defining challenge of this role is twofold. The first is measuring causal impact: CP has live, expensive, executive-facing questions that hinge on making a credible impact claim when a clean A/B test isn’t available — conversational and agent-driven flows that span a lead’s journey across sessions and channels, and changes that roll out to everyone at once. The second is establishing an evaluation feedback loop: much of CP now runs on LLM-based and agentic systems that were built by engineers with no rigorous way to measure whether they actually work. You will own how we evaluate these systems — defining the signals that matter, standing up eval and experimentation frameworks (including LLM-as-a-judge), and separating activity and adoption from genuine quality and outcomes. The same question recurs across CoreTech in many different areas, and you will need to help prove whether new features or infrastructure actually pay off, rather than trusting activity or adoption as proof. Owning these questions is the heart of the job.