We are sharing a specialised full-time opportunity for experienced, board-certified physicians with substantial post-residency clinical practice and deep expertise in a recognised medical specialty.

This role supports advanced research focused on improving how AI systems reason through real-world clinical work. Selected physicians will review complex medical tasks and outputs, develop authoritative reference solutions and instructions, and translate expert clinical judgement into rigorous standards for medically accurate and professionally credible reasoning.

Key Responsibilities

Clinical Quality Review

• Evaluate medical tasks and outputs for clinical accuracy, completeness, and professional realism
• Identify missing considerations, weak reasoning, unsafe recommendations, and departures from accepted clinical standards
• Assess whether diagnostic and management conclusions are appropriately supported
• Distinguish clinically sound reasoning from responses that are plausible but medically flawed
• Apply practising-physician judgement across complex patient scenarios

Diagnostic & Treatment Reasoning

• Review cases involving diagnosis, differential diagnosis, investigation, treatment, and follow-up
• Evaluate whether clinical information is interpreted appropriately
• Assess management decisions against relevant evidence and professional standards
• Identify overlooked contraindications, risk factors, or alternative explanations
• Develop reference approaches reflecting real-world clinical decision-making

Clinical Task Development

• Create challenging tasks that reflect authentic medical practice
• Develop scenarios requiring nuanced clinical reasoning and specialist expertise
• Write clear task instructions and define appropriate expected outcomes
• Ensure problems capture the complexity and uncertainty of real clinical work
• Develop cases that distinguish genuine expertise from superficial medical knowledge

Reference Solution Development

• Produce authoritative solutions to complex clinical problems
• Define what constitutes a complete, safe, and well-supported medical response
• Explain important diagnostic and management considerations clearly
• Translate tacit clinical judgement into explicit evaluation criteria
• Ensure reference outputs reflect current professional standards

Evaluation Standards & Calibration

• Develop structured criteria for evaluating clinical reasoning
• Identify recurring weaknesses and failure patterns in model-generated medical outputs
• Work with research teams and specialists from adjacent disciplines to maintain consistent quality standards
• Refine tasks and evaluation criteria based on observed performance
• Help establish reliable standards for assessing complex medical knowledge work

Specialty Expertise

• Contribute deep professional expertise in a defined clinical specialty such as:
  • Internal Medicine
  • Oncology
  • Radiology
  • Emergency Medicine
  • Surgery
  • Psychiatry
  • Medical subspecialties
• Apply specialist knowledge to realistic clinical scenarios
• Evaluate cases requiring advanced judgement within the relevant field
• Experience in utilisation management, clinical informatics, or medical affairs is advantageous

Ideal Profile

• MD or DO from an accredited medical school
• Completed residency in a recognised medical specialty
• 4+ years of post-residency clinical practice
• Active, unrestricted licence to practise medicine in at least one U.S. state
• Current board certification in the relevant specialty
• Demonstrated specialist expertise within a defined area of clinical medicine
• Clear progression to a senior clinical level such as Attending Physician, Medical Director, Division Chief, Associate Professor, Professor, Chief Medical Officer, or comparable role
• Demonstrated ownership of significant clinical decisions and patient-care responsibilities
• Hands-on experience using large language models in professional workflows
• Strong ability to distinguish rigorous clinical reasoning from superficially plausible conclusions
• Excellent written communication and ability to provide precise, structured medical feedback

Engagement Details

• Full-time position
• Hybrid — Bay Area, California
• Expected commitment of 40 hours per week
• Initial engagement of approximately 6 months
• Compensation: $60–$100/hour
• Candidates must live in the Bay Area and be available to work on-site with the assigned team multiple days per week when required
• Candidates not currently based in the Bay Area must be willing to relocate there at their own expense before the engagement begins
• Relocation assistance is not provided
• Client-issued accounts and equipment may be provided for work within designated systems and workflows
• Projects may be extended, shortened, or concluded based on organisational needs and performance

About the Platform

This opportunity is available through 24-MAG LLC. We connect experienced professionals with remote consulting opportunities across technical, evaluation, and project-based workstreams.

By submitting this application, you acknowledge that your information may be processed by 24-MAG LLC for recruitment and opportunity matching in accordance with our Privacy Policy: https://www.24-mag.com/privacy-policy.