The opportunity
Elastic, the Search AI Company, enables everyone to find the answers they need in real time, using all their data, at scale — unleashing the potential of businesses and people. The Elastic Search AI Platform, used by more than 50% of the Fortune 500, brings together the…
What you'll do
Own prioritized competitive benchmarking initiatives across search, vector: and hybrid retrieval, and analytical and observability workloads.
Build and maintain automated benchmark harnesses, datasets, query sets, and: load profiles that reflect real-world usage, including concurrent ingestion and querying.
Design fair comparisons with documented system versions, configurations,: tuning decisions, resource budgets, and functional differences.
Evaluate latency distributions, throughput, resource consumption, and cost: efficiency. For retrieval workloads, measure performance at comparable recall or relevance targets.
Adapt and extend workload-appropriate tools such as Elasticsearch Rally,: ClickBench, and vector search benchmark frameworks.
Investigate performance differences using profiling, system metrics, and: controlled experiments; collaborate with engineering teams to identify causes and validate improvements.
What they're looking for
- Evaluate relevant self-managed and managed cloud deployments, including: serverless offerings, accounting for scaling behavior and service constraints.
- Work with engineering and technical content partners to publish reproducible: benchmark artifacts and clear technical reports that explain methodology, findings, limitations, and tradeoffs.
- Partner with engineering and product management to prioritize benchmark: coverage and translate findings into actionable product improvements.
- Solid software engineering skills, with experience building maintainable: automation and analysis tools in Python or another relevant programming language.
