AI Member Experience Quality Analyst

BlueCross BlueShield of Tennessee•Chattanooga, TN

5d•Remote

About The Position

We’re hiring an AI Member Experience Quality Analyst who is passionate about quality and eager to influence how AI‑powered member experiences are delivered across daily operations. In this role, success is defined by the impact of your quality insights and the improvements they drive across AI performance, compliance, and member trust. You’ll conduct rigorous quality evaluations grounded in real operational workflows, identify high‑risk issues, and translate findings into clear, actionable recommendations that strengthen AI responses at scale. You’ll partner closely with training, operations, product, and business stakeholders to ensure quality learnings directly influence how our AI learns, evolves, and performs in live member environments. By connecting quality data, trends, and operational realities, you’ll tell a compelling quality story that reduces risk, elevates the member experience, and helps set the standard for AI quality excellence. Our ideal candidate brings a strong foundation in quality, operations, or analytics and thrives in environments where precision, judgment, and collaboration matter. You’re comfortable working with data, spotting patterns and risks, and clearly communicating insights to diverse stakeholders. You understand how quality shows up in day‑to‑day operations and are motivated by the opportunity to influence AI‑driven experiences at scale. If you’re excited to apply your quality and operational expertise to the future of AI‑driven member experiences, apply today and help shape how trust, accuracy, and innovation come together at scale.

Requirements

Associate degree or equivalent work experience required. Equivalent experience is defined as 2 years of professional work experience in healthcare or related field.
3+ years of experience in quality assurance, quality monitoring, or auditing with a track record of driving improvement
2+ years of experience in health insurance or healthcare operations with strong product and process knowledge
Expertise in quality evaluation methodologies, scoring rubrics, and calibration processes
Deep knowledge of health insurance products, terminology, and member service standards
Eagle-eye attention to detail with ability to identify errors and inconsistencies others might miss
Strong written communication skills; you can document findings clearly and deliver constructive feedback

Responsibilities

Be the quality champion: conduct thorough reviews of AI conversations, starting with 100% coverage at launch and evolving to strategic sampling as we prove excellence
Apply your expertise: evaluate AI interactions against approved quality and experience standards, spanning accuracy, completeness, compliance, and member impact
Document with precision: capture quality findings, categorize issues by impact (critical, major, minor), and track trends that tell the quality story
Drive AI improvement: identify training gaps and knowledge base opportunities, providing specific, actionable recommendations to appropriate stakeholders
Bridge quality and training: collaborate with Business Requirements Analysts to transform your findings into AI enhancements
Create quality narratives: prepare compelling reports and scorecards that highlight trends, celebrate wins, and focus attention on opportunities
Ensure consistency: participate in calibration sessions that align quality evaluation across the team
Protect compliance: review AI responses for regulatory adherence, ensuring required disclosures and catching prohibited content
Sound the alarm: escalate critical quality issues immediately, protecting members and the organization from potential harm
Investigate root causes: support deep-dive analysis when quality incidents or member complaints require thorough understanding
Shape quality standards: contribute to developing and refining the evaluation criteria and scoring approaches that define AI excellence
Build quality knowledge: maintain evaluation guides, job aids, and training materials that elevate quality practice across the team