The opportunity
Ready to do the most impactful work of your career? At Coinbase , we are uncompromising on our mission to increase economic freedom.
What you'll do
Design and execute quantitative testing for AI and ML models before and after: launch, including input data quality testing, variance and stability testing, and performance testing for agentic and LLM-based systems, producing documentation that holds up to independent model validation and regulatory examination.
Build and maintain monitoring pipelines and dashboards that track model: inputs, outputs, and downstream outcomes (e.g., SAR performance metrics), applying statistical tests to detect drift and degradation and delivering clear health readouts to model owners and Compliance leadership.
Own data lineage for compliance-critical data elements, designing data: quality checks that feed Compliance's data quality control framework and partnering with Engineering to trace data from source systems to control consumption.
Write production-quality SQL and Python pipelines, diagnosing and fixing: pipeline bugs and leading data migrations that support external regulatory reporting.
Build detective monitoring scenarios that test whether compliance controls: are working, writing and validating the underlying SQL logic.
Prepare testing evidence, model documentation, and walkthroughs for Internal: Audit, model validators, and regulators, responding to findings and tracking remediation to closure.
What they're looking for
- + years of experience in data science, model risk, quantitative analytics, or: data engineering, ideally in financial services, fintech, or another regulated setting.
- Strong grounding in applied statistics (hypothesis testing, drift and: stability measures, sampling, confidence intervals) with hands-on experience evaluating ML models, including testing LLM-based or agentic systems (e.g., accuracy against human-reviewed samples, variance across runs, input and payload monitoring).
- Advanced SQL and Python skills with experience building monitoring dashboards: and working in cloud data environments (e.g., Snowflake or Databricks), including data lineage tracing and data quality rule design.
- Demonstrated ability to write testing and model documentation that withstands: independent review, and to communicate technical results clearly to non-technical compliance stakeholders and auditors.
