🚀 We’re Hiring: AI Systems & Models Testing Engineer

📍 Montreal, QC, Canada / Hybrid (3 days)

💼 Contract Opportunity

We are looking for an experienced AI Testing Engineer with strong expertise in Generative AI, LLM evaluation, and RAG-based systems testing.

🔹 Key Responsibilities:

• Design and execute test strategies for AI systems and LLM-based applications

• Perform prompt engineering, hallucination detection, bias/safety testing, and output validation

• Evaluate model quality across accuracy, tone, coherence, and reliability dimensions

• Test AI applications integrated with RAG pipelines, vector databases, and knowledge bases

• Validate retrieval quality, embedding behavior, similarity search thresholds, and edge cases

• Work with LangChain / LangGraph frameworks to identify failure points and build test harnesses

• Test MCP integrations, tool availability, fallback handling, and error scenarios

• Analyze LLM behaviors including tokenization, embeddings, attention mechanisms, and inference patterns

🔹 Required Skills:

✅ Experience testing LLM / Generative AI systems

✅ Strong understanding of RAG architecture and vector databases

✅ Hands-on experience with LangChain and/or LangGraph

✅ Knowledge of embeddings, similarity search, hallucinations, and AI evaluation metrics

✅ Experience validating AI outputs for safety, bias, and reliability

✅ Strong Python and automation testing skills preferred

AI/LLM Testing Engineer