We Love AI JobsAI Quality Analyst | $17/hr | Remote
Crossing HurdlesJoin Crossing Hurdles as an AI Quality Analyst in a remote, contract role focused on designing realistic user-assistant interactions. Collaborate with a dedicated team to enhance AI functionality through engaging conversation simulations that help shape the future of AI assistants.
Skills & Expertise
Key Responsibilities
Design multiturn conversations for AI interactions using various applications.
Emulate user and assistant roles in conversations with tool calls when necessary.
Iterate on dialogue examples to enhance realism and usefulness for training.
Full Description
Position: Agent Function Calling Specialist
Type: Contractor Assignment (6 weeks)
Compensation: $16-$17/hr
Location: Remote
Commitment: At least 20 hrs/week (options: 20, 30, or 40 hrs/week) with minimum 4 hours overlap with PST
Role Responsibilities
* Design multiturn conversations that simulate real interactions between users and AI assistants using apps like calendaremail, maps, and drive.
* Emulate both the user and the assistant in conversations, including assistant tool calls when necessary.
* Carefully determine when and how the assistant should use available tools and function calls.
* Create dialogues that demonstrate natural language understanding and intelligent task completion.
* Generate examples where the assistant completes feasible tasks, identifies infeasible ones, or maintains natural conversation when tools are not required.
* Ensure all conversations follow defined formatting and quality guidelines.
* Iterate on conversation examples based on feedback to improve realism, clarity, and usefulness for training datasets.
* Collaborate with reviewers and team members to maintain consistency and high-quality outputs.
Requirements
* Strong technical reasoning and logical problem-solving skills.
* Ability to simulate realistic user–assistant conversations and workflows.
* Experience with any programming language or technical stack.
* Familiarity with APIs, JSON, or structured data formats.
* Excellent written English communication skills.
* Strong attention to detail and ability to follow structured guidelines.
* Creativity in designing realistic scenarios and conversations.
* Ability to work independently in a remote environment.
Preferred Qualifications
* Experience working with Large Language Models, AI assistants, or function-calling frameworks.
* Background in software development, data analysis, or technical research.
* Experience with conversational AI, prompt design, or AI evaluation projects.
Application Process
* Application Form
* ICF + Profile Review
* Assessment
* Submit