AI Quality Analyst | $17/hr | Remote

Position: Agent Function Calling Specialist

Type: Contractor Assignment (6 weeks)

Compensation: $16-$17/hr

Location: Remote

Commitment: At least 20 hrs/week (options: 20, 30, or 40 hrs/week) with minimum 4 hours overlap with PST

Role Responsibilities

* Design multiturn conversations that simulate real interactions between users and AI assistants using apps like calendaremail, maps, and drive.

* Emulate both the user and the assistant in conversations, including assistant tool calls when necessary.

* Carefully determine when and how the assistant should use available tools and function calls.

* Create dialogues that demonstrate natural language understanding and intelligent task completion.

* Generate examples where the assistant completes feasible tasks, identifies infeasible ones, or maintains natural conversation when tools are not required.

* Ensure all conversations follow defined formatting and quality guidelines.

* Iterate on conversation examples based on feedback to improve realism, clarity, and usefulness for training datasets.

* Collaborate with reviewers and team members to maintain consistency and high-quality outputs.

Requirements

* Strong technical reasoning and logical problem-solving skills.

* Ability to simulate realistic user–assistant conversations and workflows.

* Experience with any programming language or technical stack.

* Familiarity with APIs, JSON, or structured data formats.

* Excellent written English communication skills.

* Strong attention to detail and ability to follow structured guidelines.

* Creativity in designing realistic scenarios and conversations.

* Ability to work independently in a remote environment.

Preferred Qualifications

* Experience working with Large Language Models, AI assistants, or function-calling frameworks.

* Background in software development, data analysis, or technical research.

* Experience with conversational AI, prompt design, or AI evaluation projects.

Application Process

* Application Form

* ICF + Profile Review

* Assessment

* Submit