Position: LLM – AI Quality Analyst (Personalization) – Finnish

Type: Short-Term Contract (1 month)

Compensation: $11–$15 per hour

Location: Remote

Commitment: Part-time availability required (30-40 hours/week with 4 hours overlap with PST)

Role Responsibilities

* Evaluate a personalization feature for Gemini

* Design and execute multi-turn conversational prompts that require the AI to utilize personal information and experiences

* Assess how effectively the model uses past conversations, Gmail, Google Search, and YouTube activity to generate relevant and helpful responses

* Evaluate model responses based on intent and appropriate personalization

* Analyze responses for grounding issues, including flawed inferences or hallucinations

* Assess integration quality to ensure personal data is incorporated naturally into responses without overnarration

* Perform side-by-side evaluations and stack-rank model responses based on helpfulness, naturalness, and usability

* Write clear rationales referencing specific conversation turns

* Extract and verify debug information to confirm correct use of summaries and data sources

* Maintain data hygiene by deleting evaluation conversations after completion

Requirements

* Experience in data annotation, AI quality evaluation, content moderation, or related roles

* Strong Finnish proficiency (reading and writing)

* BS/BA degree or equivalent experience in a relevant field (Policy, Law, Ethics, Linguistics, Journalism, Computer Science, or similar)

* Willingness to use a primary personal Google account and enable personal data sources for assessment

* Strong analytical thinking and attention to detail

* Experience with creative prompt engineering and personalization concepts

* Ability to evaluate nuanced and ambiguous AI responses

* Ability to provide structured feedback and clear written explanations

* Excellent written communication and collaboration skills

* Ability to work independently in a remote environment

* Desktop or laptop with a stable internet connection

Application Process

* Apply/Easy Apply & check email for application form

* Fill Google form

* Fill out the application form

* Complete the ICF

* Complete the assessment

Generalist (Finnish) | $15/hr Remote