Applied Research Engineer at Sully.ai (S21)
$180K - $220K
AutonomousOS for healthcare organizations
Mountain View, CA, US / Remote (Mountain View, CA, US; San Francisco, CA, US; Berkeley, CA, US; Stanford, CA, US; Walnut Creek, CA, US; Pleasanton, CA, US; Danville, CA, US; CA, US; San Jose, CA, US; Santa Rosa, CA, US; Oakland, CA, US; Menlo Park, CA, US; San Mateo, CA, US; San Ramon, CA, US; Dublin, CA, US; Richmond, CA, US; Concord, CA, US; Fremont, CA, US; Milpitas, CA, US; Livermore, CA, US; Sunnyvale, CA, US; Palo Alto, CA, US)
Full-time
US citizen/visa only
3+ years
About Sully.ai

Our mission is "One Human, One Doctor". We are creating superhuman doctors, because access to doctors is a basic human right.

Start with making doctors superhuman. Our vision is to eliminate doctor distractions and help them navigate the best treatments for their patients.

About the role

About Us

At Sully.ai, We’re Building the Most Impactful Healthcare Company on Earth

We believe that access to a great doctor is a basic human right. Today, that’s not a reality. Delays, misdiagnoses, administrative chaos, and burnout plague the system.

Our Mission:

One Human, One Doctor. We build AI teammates that augment clinicians — scribes, nurses, receptionists, translators — all powered by our own world-class models and deployed in real-world care.

Our Traction

  • 450+ organizations signed 16 months
  • AI agents cut admin by ~2.8 hours daily and reduce onboarding 85%.
  • 5M+ Clinical Tasks completed to date, serving 36+ specialties.
  • Raised $25M from YC, Eric Yuan, Amity, Semper Virens
  • Patented AI architecture (MedCon-1) outperforms GPT-4.5, Gemini, Claude on clinical reasoning tasks

Sully requires A-players capable of 4 months = 1 year output.

What You'll Do

Build and scale automated evaluation pipelines (LLM-as-judge + human review) with clinical-grade benchmarks.

What you Must Bring

  • Proven experience designing agentic processes and LLM evaluation/benchmarking frameworks.
  • Strong Python and ML background (PyTorch/TensorFlow, Hugging Face, LangChain/LlamaIndex).
  • Demonstrated ability to design rigorous experiments and translate findings into production.
  • Track record of published research or deep applied work in LLMs and agent evaluation.
  • Strong communication and technical writing skills to articulate complex findings clearly.

First-Month Focus

  • Audit existing evaluation approaches for clinical and agentic tasks.
  • Define initial benchmarks and build early automated pipelines.
  • Partner with engineering to land first set of CI gates for accuracy, factuality, and safety.

90 Days

  • Deliver a repeatable evaluation framework with automated pipelines in production.
  • Demonstrate measurable improvements in robustness, hallucination reduction, or safety.
  • Publish or present internal research findings that directly shape product reliability.

If you’ve ever said, “I want to do work that actually matters”, this is it. Let’s build something life-changing, together.

Who Thrives Here

  • Entrepreneurial to your core: You think in outcomes, thrive in chaos, and take ownership without limits
  • Mission-obsessed: You’re here to save lives, not just ship features — patients and doctors are your why.
  • Impact-driven & fast-moving: You sprint toward hard problems and ship with sharp judgment.
  • Elite teammate: You raise the bar through high standards, direct feedback, and craft excellence.

Why Join Sully.ai? 🔥 Revolutionizing the antiquated $800B+ Healthcare market

🧠 50%+ of us are ex-founders. We hire A-players, not passengers

⚡️ Speed matters - we operate with urgency, autonomy, and ownership

🧪 You’ll work on real, first-of-their-kind problems at the edge of AI and medicine

❤️ Your work helps doctors reclaim their time - and patients get better, faster care

Other jobs at Sully.ai

fulltimeMountain View, CA, US$180K - $200K6+ years

fulltimeUS / Remote (US)Backend$230K - $260K6+ years

fulltimeMountain View, CA, US / Remote (Mountain View, CA, US; San Francisco, CA, US; Berkeley, CA, US; Stanford, CA, US; Walnut Creek, CA, US; Pleasanton, CA, US; Danville, CA, US; CA, US; San Jose, CA, US; Santa Rosa, CA, US; Oakland, CA, US; Menlo Park, CA, US; San Mateo, CA, US; San Ramon, CA, US; Dublin, CA, US; Richmond, CA, US; Concord, CA, US; Fremont, CA, US; Milpitas, CA, US; Livermore, CA, US; Sunnyvale, CA, US; Palo Alto, CA, US)Machine learning$180K - $220K3+ years

fulltimeUS / Remote (US)Devops$175K - $200K6+ years

fulltimeMountain View, CA, US / Remote (US)$200K - $300K6+ years

contractDubai, Dubai, AE / AE / Remote (Dubai, Dubai, AE; AE)Backend$5.45K - $10.9K / monthly6+ years

fulltimeUS / Remote (US)$250K - $300K6+ years

fulltimeMountain View, CA, US$150K - $250K6+ years

fulltimeMountain View, CA, US / Remote (Mountain View, CA, US; San Jose, CA, US; San Francisco, CA, US; Berkeley, CA, US; Stanford, CA, US; Pleasanton, CA, US; Livermore, CA, US; Santa Rosa, CA, US; Fremont, CA, US; San Mateo, CA, US; CA, US)Engineering manager$210K - $260K6+ years

fulltimeUS / Remote (US)Full stack$175K - $250K6+ years

fulltimeMountain View, CA, USDevops$165K - $200K6+ years

fulltimeMountain View, CA, US / Remote (San Francisco, CA, US; Mountain View, CA, US; Stanford, CA, US; Berkeley, CA, US; San Mateo, CA, US; Livermore, CA, US; Pleasanton, CA, US; San Jose, CA, US; Menlo Park, CA, US; Santa Rosa, CA, US; Dublin, CA, US; Danville, CA, US; Walnut Creek, CA, US; Fremont, CA, US)Full stack$175K - $220K6+ years

fulltimeMountain View, CA, USBackend$180K - $220K6+ years

contractUS / Remote (US)Sourcer$80K - $110K3+ years

fulltimeUS / Remote (US)QA engineer$165K - $180K6+ years

fulltimeSanta Clara, CA, US / Remote (San Francisco, CA, US; US)$120K - $150K6+ years

Hundreds of YC startups are hiring on Work at a Startup.

Sign up to see more ›