Terminal Bench Expert (Vapi)
Terminal Bench Expert (Vapi)
-
Vapi, India -
Posted: less than a week ago
-
Save -
Share
Description
Company Description MillionLogics, a trusted Oracle Partner, is a global IT solutions leader with a presence in London, UK, and a development hub in Hyderabad, India. Specializing in transformative technologies, the company empowers organizations through Data & AI services, Cloud migrations, and enterprise application optimization, with a solid focus on Oracle Cloud and database technologies. With a dedicated team of over 55+ AI experts, MillionLogics tailors cutting-edge IT solutions to drive tangible outcomes for clients. Guided by a commitment to innovation and excellence, MillionLogics delivers strategic IT consulting, custom application development, and security architecture solutions, among other offerings, to help businesses unlock their full potential. Discover more about their team and services at: millionlogics.com.
Role Description This is a contract-based remote position for a Terminal Bench Expert. We are looking for highly analytical engineers, researchers, and domain specialists to contribute benchmark tasks for AI agent evaluation systems (e.g., Terminal-Bench). Design realistic, technically deep tasks simulating real-world scenarios such as debugging, data corruption, infrastructure failures, and complex workflows.
Offer Details:
- Mode of work: Fully Remote
- Pay: INR 1.25 to INR 2 lakhs per month (net/take-home)
- Duration: 12 months (likely extended)
- Experience: 3-10 years
- Number of positions: 28
- Evaluations: 1 round of technical interview
What does day-to-day look like:
- Design high-quality Terminal-Bench task ideas and specifications.
- Develop complex tasks requiring reasoning, investigation, and debugging.
- Write clear task descriptions, solution approaches, and verification logic.
- Define deterministic, outcome-based evaluation criteria.
- Identify realistic failure modes, edge cases, and operational constraints.
- Create tasks that challenge AI systems while remaining solvable by experts.
- Collaborate with reviewers to refine Apply on Kit Job: kitjob.in/job/4ms8xn
Role Description This is a contract-based remote position for a Terminal Bench Expert. We are looking for highly analytical engineers, researchers, and domain specialists to contribute benchmark tasks for AI agent evaluation systems (e.g., Terminal-Bench). Design realistic, technically deep tasks simulating real-world scenarios such as debugging, data corruption, infrastructure failures, and complex workflows.
Offer Details:
- Mode of work: Fully Remote
- Pay: INR 1.25 to INR 2 lakhs per month (net/take-home)
- Duration: 12 months (likely extended)
- Experience: 3-10 years
- Number of positions: 28
- Evaluations: 1 round of technical interview
What does day-to-day look like:
- Design high-quality Terminal-Bench task ideas and specifications.
- Develop complex tasks requiring reasoning, investigation, and debugging.
- Write clear task descriptions, solution approaches, and verification logic.
- Define deterministic, outcome-based evaluation criteria.
- Identify realistic failure modes, edge cases, and operational constraints.
- Create tasks that challenge AI systems while remaining solvable by experts.
- Collaborate with reviewers to refine Apply on Kit Job: kitjob.in/job/4ms8xn
Highlights
-
Company nameMillionlogics -
Job positionTerminal Bench Expert (Vapi)
-
Flag -
Block ad
Safety Tips
Do not pay a ’prospective employer’ anything in order to secure a job.
Related searches
More info about this ad
Terminal Bench Expert (Vapi) has been posted in the Vapi Other Jobs category on Locanto.
For Vapi, there are no other ads posted in this category.
There are more ads within a 15 km radius for this category. If you want to view those ads, click here.
