Artificial Intelligence LEAD (Vapi)

About your role: We are seeking a highly skilled Technical Lead for AI Development to drive the architecture, design, and execution of advanced AI systems using LLM frameworks, multi-agent architectures, RAG pipelines, and Model Context Protocol (MCP) integrations. The ideal candidate has strong hands-on experience building production-grade AI features, orchestrating agent ecosystems, evaluating model performance, and iterating through continual refinements. You will lead a team of engineers, collaborate with product and research teams, and play a key role in shaping our AI strategy and platform capabilities. Key Responsibilities: AI Architecture & Development
- Design and implement multi-agent systems, including agent orchestration, delegation, and tool interaction patterns.
- Build scalable RAG (Retrieval-Augmented Generation) architectures using vector databases, embedding pipelines, and data chunking strategies.
- Integrate and extend MCP (Model Context Protocol) tools for robust model-tool communication and workflow automation.
- Lead development of AI-based features, prototypes, and production solutions using LLM APIs or self-hosted models.
- Architect and optimize prompt engineering, prompt chains, agent loops, and refinement pipelines. Model Evaluation & Continuous Improvement
- Implement and maintain agent evaluation frameworks (agent evals, scenario tests, regression testing).
- Design automated evaluation harnesses for LLM quality, reliability, hallucination control, and performance metrics.
- Drive iterative improvements through A/B testing, reward models, and feedback loops.
- Monitor system performance, latency, cost, and reliability — and implement optimization strategies. Technical Leadership
- Lead and mentor engineers working on AI, data, and backend components.
- Collaborate with product managers, researchers, and cross-functional teams to align tech strategy with business outcomes.
- Conduct code reviews, enforce best practices, and maintain architectural standards.
- Own technical roadmaps, sprint planning, and engineering execution. Systems & Infrastructure
- Work with cloud platforms (AWS/GCP/Azure) to deploy scalable AI services.
- Integrate vector databases (Pinecone, Weaviate, Elasticsearch, etc.).
- Build APIs and microservices to expose AI capabilities to internal and external stakeholders.
- Maintain secure, compliant, and effective data pipelines for ingestion and retrieval. Qualifications
- Bachelor’s/Master’s degree in Computer Science, Engineering, AI, or related field.
- 8+ years of software engineering experience with strong backend architecture skills.
- 3+ years deep experience with LLMs, GPT models, agents, or advanced ML systems.
- Strong hands-on experience with:
- MCP tools and LLM tool integration
- Agent frameworks (e.g., OpenAI Agents, LangChain, LlamaIndex, custom agents)
- RAG pipelines, embedding models, vector stores
- Agent evaluation, reliability testing, and model refinements
- Proficiency in Python, TypeScript/Node.js, or similar languages.
- Experience deploying LLM apps and APIs in production environments.
- Deep understanding of AI limitations, hallucination control, and safety measures. Preferred / Nice to Have Experience with:
- Fine-tuning LLMs
- OpenAI API, Claude, or Azure OpenAI
- Distributed embeddings and high-throughput retrieval systems
- MLOps frameworks
- Knowledge of DevOps, CI/CD, containerization (Docker/Kubernetes).
- Prior leadership experience managing small to mid-size engineering teams. Apply on Kit Job: kitjob.in/job/4mxc8g