AI Benchmark Engineer (Ernakulam)

About Turing: Turing is one of the world’s fastest-growing AI companies, accelerating the advancement and deployment of powerful AI systems. Turing helps customers in two ways: working with the world’s leading AI labs to advance frontier model capabilities in thinking, reasoning, coding, agentic behavior, multimodality, multilinguality, STEM, and frontier knowledge; and leveraging that work to build real-world AI systems that solve mission-critical priorities for companies.

Role Overview: We are seeking experienced AI Benchmark Engineers — Data Analysis to design and develop high-quality multi-agent benchmark tasks that evaluate the analytical reasoning, coordination, and execution capabilities of advanced AI systems.

In this role, you will build realistic benchmark tasks that require AI agents to analyze large, messy, multi-source datasets, decompose work across specialist sub-agents, and arrive at specific, verifiable conclusions. These tasks may involve structured and semi-structured data such as CSVs, JSON files, logs, reports, survey results, vendor assessments, or financial and operational documents.

Your work will help measure how effectively AI systems perform complex analytical workflows involving cross-referencing, contradiction detection, anomaly identification, and statistical reasoning across multiple data sources.

What does day-to-day look like:
- Design and author multi-agent benchmark tasks centered on complex data analysis workflows
- Create realistic synthetic datasets or curate real-world style datasets across domains such as finance, operations, security, or market analysis
- Build tasks that require agents to perform cross-referencing, anomaly detection, contradiction identification, and statistical computation across multiple sources
- Develop decomposition guides that split analytical work across specialist sub-agents such as financial, technical, security, or operations analysts
- Write accurate oracle logic or verification scripts that va Apply on Kit Job: kitjob.in/job/4nbxwu