Scientific Evals
About The Position
Edison Scientific builds and commercializes AI agents for science. Scientific discovery moves too slowly, and autonomous AI agents are how we intend to fix that. We're assembling a team of top researchers and engineers across AI and biology to build an AI scientist. Role We are seeking an ambitious, scientifically grounded person to join our team focused on developing rigorous benchmarks and training datasets that advance AI capabilities in biology. This role sits at the intersection of biology, data curation, and machine learning, and is ideal for someone with deep scientific training who is excited to shape how frontier AI systems learn to do science. This role is on-site at our San Francisco office in the Dogpatch neighborhood. Our office is a converted warehouse with high ceilings, open space, and a team that genuinely believes in what they're building.
Requirements
- Graduate-level training in biology, biochemistry, computational biology, or a related field, with hands-on research experience.
- Working knowledge of machine learning concepts, particularly deep learning and large language models.
- Comfortable with Python and building workflows for data processing, analysis, and experimentation.
- Possess strong scientific taste and able to identify what distinguishes expert-level reasoning from surface-level pattern matching.
- Detail-oriented and willing to take on high-value but occasionally tedious work.
- Energized by ambiguous, open-ended problems that require creativity, collaboration, and first-principles thinking to solve.
- Organized and communicative, able to manage multiple workstreams and coordinate across teams.
Nice To Haves
- Prior experience creating evaluation datasets, annotation guidelines, or working on human-in-the-loop data pipelines.
- Experience with bioinformatics pipelines, biological databases, or sequence analysis tools.
- Hands-on experience fine-tuning or evaluating large language models, or familiarity with RLHF and preference-based training.
- Publications or research experience in areas relevant to AI for science.
Responsibilities
- Design benchmarks that capture the complexity of real biological research, drawing on your domain expertise to identify what makes scientific reasoning hard. This will include open-ended scientific benchmarks and building on prior work like LAB-Bench and BixBench.
- Curate and vet biological datasets to ensure scientific rigor.
- Analyze model outputs, identify failure modes, and contribute to iterative improvements in both datasets and evaluation criteria.
- Collaborate with AI/ML researchers to translate scientific intuition into training signal, helping AI systems learn not just facts but how scientists think.
- Coordinate operations and manage workflows, including working with domain experts, tracking task progress, and maintaining documentation.
Benefits
- Competitive salary and equity
- Full healthcare coverage — we pay 100% of premiums for you and your dependents
- Support for growing families, including a yearly new parent stipend and fertility coverage through Carrot
- 401(k) company matching
- $300 health and wellness benefit
- Lunch is on us every day you're in the office, and dinner is on us when you're working late
- Regular team offsites and company events
- A fast-moving, mission-driven culture where smart people do their best work and actually enjoy doing it
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Mid Level
