Sr. Big Data Engineer, Energy Service Engineering

This role is based in Palo Alto, California.

Compensation:

$100,000 - $216,000/year + cash and stock awards + benefits

Responsibilities:

• Design and implement robust, scalable solutions for processing terabyte-scale data.
• Develop and maintain high-performance data pipelines using Apache Spark, Kafka, and other modern data technologies.
• Build and optimize batch and streaming data processing systems.
• Implement and maintain CI/CD pipelines for data applications.

Requirements:

• Strong data engineering background with expertise in at least two programming languages (Python, Scala, Java, or Rust).
• Expert-level experience with Apache Spark (PySpark, Spark SQL, Spark Streaming).
• Experience with containerization and orchestration (Docker, Kubernetes).
• Demonstrated experience working with terabyte-scale data processing.