Audio Annotation Analyst (Preference Data & Long-Form Captioning)
We need an analyst who can execute both comparative listening evaluations and detailed
musicological captioning at scale. You will run structured preference tasks, craft grounded
long-form descriptions of tracks, and keep throughput, quality, and metadata spotless for
model-training partners.
Key Responsibilities: Preference Data Collection
- Follow structured rubrics covering overall preference, holistic ratings, appeal axes, adherence checks, and engagement scoring for A/B and A/B/X tasks.
- Document concise rationales for every overall preference choice, calling out musical, stylistic, or technical drivers.
- Ensure smooth playback verification (scrubbing, repeated listens) and confirm clips align with task prompts before logging results.
- Tag each submission with the correct prompt, model version, rater ID, and timestamp to maintain metadata fidelity.
- Surface anomalies - audio defects, unclear prompts, UI bugs - to the lead so tasks can be recycled or corrected quickly.
Key Responsibilities: Long-Form Music Captioning
- Produce 300- to 500-word captions per track that describe structure, timbre, instrumentation, stylistic influences, and emotional arcs in present tense.
- Create timeline callouts (start time, section type, narrative notes, instrument/vibe tags) plus summary tables for genre, instrumentation, mix, vibe, quality, and wow factor.
- Apply the Foreground Rule and grounded subjectivity guidelines to stay concise, accurate, and conversational.
- Maintain throughput targets (20 - 30 minutes per track) while protecting quality
- Contribute calibration feedback by highlighting edge cases, ambiguous cues, or
- instructions that need refinement.
Quality, Tooling, and Reporting
- Participate in pilot runs (50 - 500 tasks), logging timing, UI, and instruction issues for
- iteration before scale-up.
- Complete gold-standard qualification sets and ongoing drift checks; incorporate
- reviewer notes immediately into future work.
- Keep dashboards or trackers updated with daily task counts, rejection reasons, and
- rework so leads can forecast burn-down.
- Collaborate with engineers or ops partners to request UI tweaks, metadata fixes, or
- schema updates without slowing delivery.
Must-Have Qualifications
- 2+ years in human annotation, music journalism, audio QA, or related analyst roles.
- Demonstrated ear for musical structure, instrumentation, and production details across
- multiple genres.
- Comfortable working inside configurable annotation tools, spreadsheets, or content management systems.
- Strong written communication with the ability to summarize subjective listening impressions using precise, grounded language.
- High attention to detail for metadata tagging, file naming, and prompt/model crosschecks.
- Reliable remote setup with headphones suited for repeat listening sessions and the focus to handle long-form writing blocks.
Nice-to-Have Skills
- Background in music theory, ethnomusicology, or audio engineering.
- Experience supporting generative audio, recommender systems, or creator tools.
- Working knowledge of SQL, Python, or automation scripts to accelerate QC or reporting.
- Familiarity with data privacy or content sensitivity guidelines for global annotation programs.
Success Measures
- Completes assigned preference tasks with <1% metadata errors and on-time delivery.
- Maintains caption acceptance rate above internal quality threshold with minimal rewrite requests.
- Keeps daily throughput at or above plan while respecting 20 - 30 minute per-track guidance.
- Logs 100% of anomalies or blockers within the team tracker within 24 hours.
