Product Aligned Agentic Quality Engineering Lead Vapi

We are hiring a senior Quality Engineering leader who owns end-to-end quality, release sign-off, and automation strategy across our platform — including AI agents, web applications, APIs, and databases.

This role blends:

Quality Ownership

Automation & Agentic Testing Leadership

Release & Go-Live Authority

Product-aligned execution (PO-style ownership)

You will start by establishing robust manual and exploratory testing foundations, then systematically drive automation-first quality, including AI agent validation, critic agents, eval frameworks, and safety guardrails.

This is not a passive QA role.

You are accountable for production quality.

Key Responsibilities

End-to-End Quality Ownership & Release Sign-Off

Own quality outcomes across the full delivery lifecycle

Act as release authority, driving go/no-go decisions

Define and enforce quality gates across features, agents, and releases

Partner with Product and Engineering to balance speed and quality

Reduce production defects through proactive automation and validation

Automation-First Quality Strategy

Define and own the automation roadmap (UI, API, DB, agentic systems)

Build and maintain hybrid automation frameworks using Python

Drive CI/CD-integrated automated testing pipelines

Ensure automation coverage continuously replaces repetitive manual testing

Guide aligned Quality Engineers in automation implementation

Agentic AI & LLM Quality Engineering

Own quality for AI agents, tool calls, and multi-step workflows

Validate:

Agent behavior & reasoning paths

Tool invocation correctness

Hallucinations & unsafe outputs

RAG accuracy and grounding

Define and execute agent evaluation (evals) frameworks

Collaborate with Critic Engineers to design:

Critic agents

Automated agent validation

Benchmark datasets & scoring rubrics

Lead innovation in agent testing methodologies

Manual, Exploratory & Scenario-Based Testing

Drive high-quality manual and exploratory testing wher Apply on Kit Job: kitjob.in/job/4n4l6y