AI Evaluator Expert | $105/hour Remote | Mercor

At Crossing Hurdles, we work as a referral partner. We refer candidates to Mercor that collaborates with the world's leading AI research labs to build and train cutting-edge AI models.

Organization: Mercor

Position: STEM PhD (AI Subject Matter Experts)

Referral Partner: Crossing Hurdles

Type: Hourly contract

Compensation: $105/hour

Location: Remote

Commitment: 10–40 hours/week

Role Responsibilities:

• Critically evaluate machine learning and artificial intelligence research papers for methodological rigor, novelty, and clarity.
• Inspect Python source code, experimental pipelines, and experiment logs to validate reported results and metrics.
• Conduct structured literature reviews using tools such as Google Scholar or Semantic Scholar to assess originality and identify prior work.
• Provide detailed, structured feedback on research quality, including potential ethical, safety, or credibility concerns.
• Assess "human-quality" research outputs and document findings in a clear, standardized format.
• Submit non-identifying professional background details (e.g., affiliation, highest degree, publication links) to establish evaluator credibility for research studies.

Requirements:

• Strong experience in artificial intelligence, machine learning, or closely related research domains.
• Demonstrated ability to read, interpret, and evaluate ML research papers and technical methodologies.
• Comfort reviewing Python code, experimental workflows, and quantitative logs.
• Experience conducting academic literature reviews and assessing research originality.
• High attention to detail with the ability to analyze complex technical arguments.
• Ability to work independently in a remote, asynchronous environment.

Application Process (Takes 20 min):

• Upload resume or CV
• AI interview based on your background (15 min)
• Submit form