Work Snapshot

Type: Hourly contract

Location: Remote

Commitment: Full-time or Part-time

What You’ll Be Doing

* Evaluate AI-generated responses to coding and software engineering queries for accuracy, reasoning, clarity, and completeness

* Conduct fact-checking using trusted sources and validate outputs by executing code

* Annotate model responses by identifying strengths, gaps, and inaccuracies

* Assess code quality, readability, and algorithmic soundness

* Ensure responses align with expected conversational behavior and engineering best practices

* Apply consistent evaluation standards using defined taxonomies, benchmarks, and guidelines

What We’re Looking For

* Strong experience in software engineering, data science, or systems design

* Strong academic background in computer science or a related technical field

* Strong experience in at least two programming languages such as Python, Java, C++, JavaScript, Go, Rust, or similar

* Strong problem-solving skills with the ability to handle complex coding challenges

* Experience contributing to open-source projects is a plus

* Familiarity with using AI tools for coding and understanding their strengths and limitations

* High attention to detail with the ability to identify logical flaws and inefficiencies

* Strong experience in model evaluation, data annotation, or RLHF is a plus

* Ability to explain complex technical concepts clearly

How To Apply

* Upload resume

* Interview

* Submit form

Data Scientist | Remote