Staff Software Engineer - StreamingActive$182K

Databricks
Apply now

The opportunity

We're seeking an experienced Staff Engineer to drive the technical direction of Spark Structured Streaming, spanning both open source and Databricks-specific components. Your mission is to make Spark Structured Streaming the state-of-the-art stream processing engine — adding…

What you'll do

  • Set and drive the technical vision for Spark Structured Streaming across OSS: and the Databricks Data Intelligence Platform

  • Design and build core engine capabilities: state management, new operators, and architectural improvements to latency and throughput

  • Raise the bar for engineering quality and operability, building software that: is not just high quality but easy to run in production

  • Make company-wide impact by driving stream processing adoption across the Databricks product portfolio

  • Guide long-term architecture and technical-debt decisions, balancing them against the product roadmap

  • Mentor and technically lead engineers on the team, and partner with the: Engineering Manager to attract and grow top-tier talent

What they're looking for

  • BS (or higher) in Computer Science or a related technical field, or equivalent practical experience
  • + years building related systems: big-data ecosystems, Apache Spark, or database internals
  • A passion for database systems, storage systems, distributed systems,: language design, or performance optimization
  • Comfortable working toward a multi-year vision with incremental deliverables