Senior Data EngineerActive$220K
The opportunity
As a Data Engineer on the Analytics Team, you will be collaborating with stakeholders across the company to design, build and implement data pipelines and models that enable our next generation of technology to be deployed around the world. You will also shape the current and future vision of the entire data platform.
What you'll do
Lead the design and roadmap for our data platform
Partner with operations, product, and engineering to advocate best practices: and build supporting systems and infrastructure for the various data needs
Own the ingest and egress frameworks for data pipelines that stitch together: various data sources in order to produce valuable data products that drive the business
Palantir Foundry is our central hub for data driven applications,: visualizations and large scale data analysis across the Anduril org. We manage a large user base and provide true data self service at scale.
We use SQLMesh for data transformations, giving our engineers the ability to: work with SQL while enjoying software engineering best practices like version control, testing, and CI/CD for data pipelines.
For querying data, we utilize Athena, an aws serverless solution that allows: our team to run analytics directly on our data lake without managing infrastructure.
What they're looking for
- We've implemented Apache Iceberg as our table format, providing advanced data: management capabilities like schema evolution and time travel queries.
- We use Flyte for orchestration. It is a highly scalable, kubernetes native: modern orchestration platform similar to Airflow or Dagster.
- Operate in a "player-coach" capacity, leading members of the team, while remaining hands on as a contributor
- + years of experience in a data engineering role building products, ideally in a fast-paced environment