Accountabilities:

In this role, you will support AI model improvement by performing detailed evaluations of generated responses and applying structured quality standards. You will analyze content across multiple formats, identify differences in usefulness and accuracy, and provide clear reasoning behind your decisions.

• Compare AI-generated responses side-by-side and determine which response better addresses user needs.
• Evaluate responses based on factual accuracy, relevance, completeness, reasoning, clarity, tone, safety, and instruction-following.
• Apply Castilian Spanish expertise to assess language quality, regional accuracy, cultural context, terminology, and natural usage in Spain.
• Review content in English, Spanish, or mixed-language scenarios across various AI tasks, including conversations, search results, file-based tasks, image-related responses, and content generation.
• Identify subtle quality differences, including unsupported claims, missing information, unclear reasoning, unnatural phrasing, and cultural inconsistencies.
• Follow detailed annotation guidelines and maintain consistent evaluation standards across a high volume of assignments.
• Document evaluation decisions with concise, evidence-based explanations when required.
• Participate in training sessions, calibration activities, qualification reviews, and ongoing quality improvement processes.
• Incorporate feedback to continuously improve evaluation accuracy and alignment with quality expectations.

Requirements:

The ideal candidate combines advanced Castilian Spanish knowledge with strong analytical abilities and an interest in artificial intelligence. Success in this role requires careful judgment, strong written communication, and the ability to evaluate complex information objectively.

• Native-level or professional fluency in Castilian Spanish, with deep understanding of Spain-specific linguistic and cultural conventions.
• Strong English fluency and reading comprehension, including the ability to understand detailed guidelines and AI-generated content.
• Strong analytical and critical-thinking skills beyond language evaluation alone.
• Ability to assess content quality across different topics, formats, and user scenarios.
• Ability to evaluate factual accuracy, relevance, reasoning, clarity, instruction adherence, and overall usefulness.
• Excellent attention to detail and ability to maintain accuracy while completing repetitive evaluation tasks.
• Strong written communication skills with the ability to explain decisions clearly and concisely.
• Ability to independently apply structured criteria in ambiguous situations.
• Comfort working with detailed frameworks, rubrics, and quality standards.
• Ability to manage a high volume of evaluations while maintaining consistent judgment.
• Ability to receive feedback, adapt to evolving guidelines, and continuously improve evaluation quality.
• Preferred experience includes AI response evaluation, data annotation, content quality assessment, search relevance evaluation, or structured labeling projects.
• Experience working with AI-generated content or model-quality assessment is a plus.
• Familiarity with Python, SQL, or technical workflows is not required but may be beneficial.

Benefits:

• Competitive compensation ranging from $38.46 to $40.87 USD per hour, with a midpoint of $39.66 USD per hour.
• Fully remote work arrangement within the United States.
• Flexible work schedule after completion of the training and qualification period.
• Comprehensive onboarding program including training, guided practice, calibration, and quality support.
• Medical, dental, and vision insurance options.
• Flexible Spending Account (FSA).
• 401(k) retirement plan.
• Paid time off and parental leave benefits.
• Professional growth and development opportunities.
• Opportunity to contribute to the advancement of AI technologies through meaningful evaluation work.