Content Operations Analyst

Overview

Key Responsibilities:

• Linguistic Excellence: Apply your native-level fluency and strong written communication skills in US English to ensure the accuracy and cultural appropriateness of our content throughout the software development cycle.

• Cross-Functional Collaboration: Work closely with Design, Product, Editorial, Safety, and Engineering teams to define evaluation requirements for machine learning and large language models (LLMs) and deliver high-quality results on time.

• Quality Assurance: Maintain and enhance data quality through rigorous evaluation, research, and verification of metadata across various media types. Ensure all content meets our high standards through quality assurance reviews, editing, scoring, transcribing, and copywriting.

• Process Improvement: Utilize both quantitative and qualitative feedback to continuously refine and improve processes. Provide clear, actionable feedback to leads and communicate effectively with editorial and engineering teams.

• Workflow Management: Leverage established workflows and tools for content tasks. Adhere to guidance from Project Leads, Style Guides, and Terminology Databases to align with Apple’s style and project requirements.

• Research and Benchmarking: Conduct research to validate and create automated benchmarks for LLM evaluations. Gather and analyze data from human participants, ensuring data quality and validity.

Qualifications:

• Education: Advanced degree in Linguistics, Computer Science, Machine Learning, Cognitive Science, Psychology, Economics, or a related field is preferred.

• Experience: Minimum of 3 years of relevant experience, including working with large language models and evaluating their performance using both human and automated benchmarks.

• Skills: Expertise in grammar, spelling, and cultural nuances in US English. Proficiency in evaluating content across various media and content types. Strong problem-solving abilities and effective communication skills.

Desirable Experience:

• Familiarity with LLM evaluation techniques and methodologies.
• Experience conducting research with human participants for data collection and validation.

The target hiring compensation range for this role is $30 to $33. Compensation is based on several factors including, but not limited to education, relevant work experience, relevant certifications, and location.

About Aquent Talent:

Aquent Talent connects the best talent in marketing, creative, and design with the world’s biggest brands.

Our eligible talent get access to amazing benefits like subsidized health, vision, and dental plans, paid sick leave, and retirement plans with a match. We also offer free online training through Aquent Gymnasium. More information on our awesome benefits! 

Aquent is an equal-opportunity employer. We evaluate qualified applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, veteran status, and other legally protected characteristics. We’re about creating an inclusive environment—one where different backgrounds, experiences, and perspectives are valued, and everyone can contribute, grow their careers, and thrive.