We Love AI JobsAI Quality Analyst | $15/hr | Remote
Crossing HurdlesAs an AI Quality Analyst at Crossing Hurdles, you'll work remotely in a contract role focused on evaluating personalized AI responses and ensuring the quality of conversational models. With a commitment of just 20 hours per week, you'll have the opportunity to influence AI interactions while enjoying a flexible work environment.
Skills & Expertise
Key Responsibilities
Design and execute multi-turn conversational prompts based on personal context.
Evaluate personalized AI responses for relevance, grounding, integration, and helpfulness.
Assess correct and incorrect use of personal data in model outputs.
Full Description
Position: LLM – AI Quality Analyst (Personalization) – German
Type: Short-Term Contract
Location: Remote
Commitment: 20 hours/week with minimum 2 hours overlap with PST
Engagement Length: 1 month
Start Date: Immediate
Role Responsibilities
* Design and execute multi-turn conversational prompts (typically 1–5 turns) based on personal context
* Evaluate personalized AI responses for relevance, grounding, integration, and overall helpfulness
* Assess correct and incorrect use of personal data in model outputs
* Perform side-by-side (SxS) evaluation and ranking of AI responses
* Identify grounding errors, poor inferences, hallucinations, and forced personalization
* Write clear, structured, and defensible rationales referencing specific conversation turns
* Extract and verify model debug information and data source usage
* Maintain strict data hygiene by deleting evaluation conversations
Requirements
* German fluency (reading and writing) with high proficiency
* Experience in data annotation, AI quality evaluation, content moderation, or related roles
* Strong analytical thinking and attention to detail
* Ability to evaluate nuanced and ambiguous AI responses
* Experience in prompt design and understanding of personalization concepts
* Comfortable using a primary personal Google account with enabled data sources
* BS/BA degree or equivalent experience in a relevant analytical field
* Strong written communication and structured feedback skills
* Self-motivated and able to work independently in a remote setting
* Reliable desktop/laptop with stable internet connection
Application Process
* Fill out the application form
* Complete the ICF
* Assessment (within 24 hours of sharing)