Annotation Specialist

$30,000 USD/year ^{Pay is set based on global value, not the local market. Most roles = hourly rate x 40 hrs x 50 weeks}  ($15 USD/hour)

• Label student session recordings by detecting, categorizing, and timestamping behavioral events according to a comprehensive rubric
• Audit and refine LLM pre-annotations by eliminating false positives, inserting overlooked events, and sharpening timestamp accuracy
• Document clear rationale for ambiguous decisions, citing rubric sections and the logic applied
• Record edge cases and open questions for unclear situations, and maintain an annotation log with session details
• Engage in calibration tasks, incorporate QA feedback, and adapt to rubric revisions to enhance precision continuously

• Construct AI models, conduct experiments, or perform research into student behavior patterns
• Author the annotation rubric or modify category definitions based on subjective judgment
• Prioritize throughput over accuracy, consistency, or timestamp exactness
• Handle sporadic, disconnected tasks spanning unrelated fields without background or feedback mechanisms

This role ensures that student session recordings are transformed into ≥95%-accurate, temporally precise labeled datasets that dependably indicate when model performance advances or declines.

• At least 1 year of experience in data annotation, content moderation, QA evaluation, or similar rubric-driven review work
• Strong English reading comprehension and the ability to follow complex written instructions without drifting from the rules
• Capacity to maintain focus and precision during 4–6 hours of video-based tasks each day
• Ability to detect nuanced visual and on-screen behavioral signals and apply consistent classification across multiple sessions
• Strong written documentation skills for explaining edge cases, assumptions, and clarification questions
• Reliable internet connection capable of streaming video
• Comfort reviewing, correcting, and supplementing AI/LLM-generated annotations