Research Aide– LCF – Rauniyar, Krishna – 2.23.26
About The Position
AskALCF is an AI-powered assistant designed to help users interact with ALCF systems by answering questions related to documentation, system usage, job scheduling, storage, and performance troubleshooting. AskALCF leverages large language models (LLMs) from multiple sources, including commercial models accessed through Argo and AskSage, as well as open-source models deployed on ALCF inference endpoints. These models have different performance characteristics in terms of answer quality, latency, and reliability, and their behavior can vary significantly depending on model configurations and retrieval-augmented generation (RAG) settings. In this intern project, the student will develop a systematic evaluation framework to benchmark and compare the performance of various LLMs and configurations used by AskALCF. The student will construct a representative set of AskALCF queries based on real user workflows, run controlled experiments across different models and inference platforms, and collect metrics such as answer quality, grounding to documentation, response latency, and robustness.
Requirements
- The entirety of the appointment must be conducted within the United States.
- Currently enrolled in undergraduate or graduate studies at an accredited institution.
- Graduated from an accredited institution within the past 3 months; or
- Actively enrolled in a graduate program at an accredited institution.
- Must be 18 years or older at the time the appointment begins.
- Must possess a cumulative GPA of 3.0 on a 4.0 scale.
- If accepting an offer, must pass a screening drug test
- Must complete a satisfactory background check.
Responsibilities
- Develop a systematic evaluation framework to benchmark and compare the performance of various LLMs and configurations used by AskALCF.
- Construct a representative set of AskALCF queries based on real user workflows.
- Run controlled experiments across different models and inference platforms.
- Collect metrics such as answer quality, grounding to documentation, response latency, and robustness.
Benefits
- comprehensive benefits are part of the total rewards package.
- Click here to view Argonne employee benefits!
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Intern
Education Level
No Education Listed
