The opportunity
At FourKites we have the opportunity to tackle complex challenges with real-world impacts. Whether it's medical supplies from Cardinal Health or groceries for Walmart, the FourKites platform helps customers operate global supply chains that are efficient, agile and sustainable.
What you'll do
Define the reference architecture for FourKites agents: the agent loop, tool execution, state and checkpointing, retries and recovery, guardrails, permissions, and human-in-the-loop handoffs
Design for long-running, multi-step work that spans many context windows and: recovers cleanly when a tool, carrier API or model call fails
Decide where each problem is best solved (frontier LLM, open-weight model,: fine-tuned encoder, or plain code) and own model routing, cost-per-task and latency budgets across the fleet
Integrate agents with FourKites' event-driven platform (Kafka, microservices,: Postgres) and with customer systems through APIs and MCP servers
Set the standard for what goes into an agent's context window (system: prompts, tool definitions, retrieved documents, memory, prior turns) within an explicit token budget
Build shared components for retrieval, compaction and summarization,: sub-agent isolation, prompt caching and persistent memory so teams stop reinventing them
What they're looking for
- Diagnose context failures (lost instructions, tool confusion, quality: degradation as the window fills) and turn fixes into reusable harness patterns
- Build the offline eval framework: golden datasets from production traces, code-based checks, LLM-as-judge graders calibrated against human labels, and CI regression suites that gate every prompt, model or tool change
- Build online evaluation: scoring sampled production traffic, trace- and task-level quality metrics, drift detection, shadow and A/B rollouts, user feedback signals, and quality alerts
- Connect eval metrics to the business outcomes customers pay for, such as: tracking accuracy, exceptions resolved without a human, and appointment fill rate
