Backend Software Engineer (Evals)

This role is based in San Francisco and Seattle.

Compensation:

$255K – $405K/year + Offers Equity

Responsibilities:

• Design eval pipelines that are reliable, reproducible, and extendable.
• Build infrastructure for continuous eval monitoring frameworks.
• Design, build, and maintain backend services and APIs.
• Integrate and structure data across internal platforms.
• Collaborate with data, research, and engineering teams.
• Own the full development lifecycle of new backend systems.

Requirements:

• 4+ years of backend engineering experience.
• Proficiency in Python, FastAPI, and Postgres.
• Experience designing and scaling distributed systems.
• Familiarity with evaluation methods for LLMs.
• Experience creating production evals and measuring performance of ML/LLM models.