AI has changed how we understand member calls – and how we respond to them
By Ethan Cohen and Alexis Ditomassi
At Oscar, we use AI to help make our members’ experience faster, simpler, and more personalized. At the same time, privacy remains our first priority, and we protect our members’ data with strong safeguards and security practices. Read our full healthcare AI commitments here.
Every day, members call and chat with Oscar looking for answers. For our team, this often leads to a big question: What are common themes behind member calls, and how can we prevent the member confusion and friction that causes them to call us in the first place?
Answering that question is sometimes hard because of the complexities of US healthcare, and yet it’s critical to our mission of building a better, simpler health insurance experience. This post looks at how we developed a structured call intent taxonomy and AI model to help us do so.
Building an AI-powered intent taxonomy
Oscar regularly reviews member interactions to understand pain points and improve the member experience.
Historically, the best signal we had about why members called us came from our care guides tagging and taking notes on calls. That data isn’t sufficiently nuanced. As a result, we lacked accurate, structured data on why members were reaching out, which made it harder to design better experiences, reduce call volume, and anticipate emerging issues.
WeTo build our model, we first manually categorized call transcripts to understand how to best identify the many different ways members call and chat with us. Through this process we developed a granular view of member pain points and informed our taxonomy. After rounds of LLM-assisted iteration, we landed on a refined set of 66 AI intents. These intents, which were centered around the seven stages of the member journey, included:
Inquires About Plan Selection → Ex: 1.1 Plan Enrollment
Onboarding→ Ex: 2.1 Online Account Setup and Access
Pre-Care→ Ex: 3.1 Finding Care
Care→ Ex: 4.1 Prior Authorizations
After Care→ Ex: 5.1 Claim Status and Issues
Issue Resolution→ Ex: 6.1 Billing and payments
Miscellaneous→ Ex: 7.1 Tax Forms and Documentation
The LLM we’re using follows this approach:
Use a ranking system for likely intents, with multiple re-ranking iterations to mitigate hallucinations and allow for multiple intents when calls had multiple different objectives.
Utilize AI-compiled intent descriptions to clearly identify categories.
Integrate internal error-checking mechanisms to rectify mislabeling, such as mismatched codes and labels.
Conduct iterative prompt and model enhancements based on accuracy assessments across thousands of real-world transcripts.
This combination of prompt engineering and structured taxonomy gave us a system that could analyze a full year’s worth of call transcripts in hours, and do so with meaningful accuracy.
From call insight to member impact
With the AI intent model, we can identify what’s driving member calls with far greater specificity. Here are a few examples of recent discoveries:
Assigned primary care provider (PCP) confusion: One of the top call drivers turned out to be members wanting to switch from a PCP that was auto-assigned during enrollment. This insight has led to improvements in PCP assignment and the process of switching PCPs without agent intervention.
Referral requirements in “gatekeeping” plans: When we launched our HMO product, the AI tool revealed an early spike in calls about referrals. We used that signal to target communications to both members and providers, resulting in faster resolution and reduced denials stemming from new referral procedures.
Auto-pay misunderstanding: The model uncovered persistent confusion about how monthly auto-pay works, which triggered a redesign of our payment communications and led to a broader affordability project.
We’ve also used the tool to explore ways to better support our member education efforts. Ahead of a recent member event in El Paso, the team used common themes in member questions to proactively prepare answers and improve the in-room experience. For example, because an analysis of Oscar member calls taught us that many members struggled to understand the difference between urgent care and emergency care, we were able to ensure we focused on that topic during the event.
This capability also covers secure messaging and chats, where members often reach out with slightly different kinds of questions.
What’s next: messages, providers, and urgency
We’re also expanding into provider interactions. Our team is developing a provider-specific taxonomy to better understand why providers call us and how we can improve those and other touchpoints, including our website.
Finally, we’ve launched an AI-powered urgency flag to identify calls where members express high levels of frustration or concern.
What began as an effort to improve data quality has evolved into a broader capability, one that informs product decisions, supports member events, and flags issues before they become larger problems. We’re looking forward to continuing to develop this tool using it to improve more parts of the member and provider experience.
At Oscar, we’re building smarter ways to serve members, powered by AI and driven by empathy. Want to help us shape what’s next? Check out our open roles.
