What does it take for insurers and MGAs to trust AI in production?

Transcript

Keith Raymond: I’ll use, I’ll use one word: trust, right?

Keith Raymond: Last year, 88, almost 90% of organizations said everything they did with the new AI technologies, you know, the generative AI technologies was internally facing. And what is that? And why is that? And 83% this year. So it only went down about 5-6% from last year.

Keith Raymond: And why is it? Because organizations are learning. Organizations are trying to figure out from the standpoint of, you know, probabilistic artificial intelligence, how to get it to a point where they trust it to use it externally facing, right?

Keith Raymond: And so that’s a lot of what’s going on is the building of trust. So a lot of organizations that they’re just getting their feet wet with the latest iterations of artificial intelligence, they’re understanding it, they’re learning it, you know, all that kind of stuff. And they’re building that trust and capability. And that in order to, you know, move in the direction of using it externally facing.

Keith Raymond: And in the governance side, you know, we are seeing a big lift. It was a key area of focus from one year to the next. Last year, it was less than 30% of the organizations indicated they had a governance model in place. This year, it was almost 70%.

Keith Raymond: So a lot of attention is now being paid to the governance and framework and guardrails you put around artificial intelligence. The question mark really is, are they consistent, right?

Keith Raymond: Because there’s all sorts of different areas you can look at, whether it be the NIST AI model framework, whether it be NAIC’s model bulletin, you know, you name it. They’re the starting points for where you build from, but you got to go deeper, too.

Lawrence Ngo: I think one big aspect is really trying to figure out how you can, whether it’s proving internally, it’s whether proving to your insurers or to your customers that the system is doing what you intend it to do and is doing it at a level of reliability that you want it to do.

Lawrence Ngo: So I’ll just take one example out of many. But as you deploy a certain product, you want to make sure that, again, one of other themes that’s related to governance was the accuracy, reliability, etc.

Lawrence Ngo: But not only is it accurate in the general case, the mean across the entire selection of data points, but that within various subgroups that you have this reliable amount of accuracy across subgroups.

Cameron Scott: The right first use case or determining which one. You know, I think we’ve seen a shift in talking with carriers and MGAs where, you know, there was a thought of we just need to be in AI. Whatever, just we need something with AI to prove AI, which can be a backwards way of looking at it.

Cameron Scott: You know, I think we need to look at what problem can we solve and what’s the best use of technology to solve that problem? Is it AI deployment? Is it more deterministic workflow?

Cameron Scott: Determining the right problem at the source and then seeing if AI is the solution is the right way to go from our perspective.

Cameron Scott: And talking with some carriers and we’ve seen what a few years ago was AI. Well, and what at conferences, AI is the silver bullet. It’ll solve all your problems.

Cameron Scott: But that’s, you know, we need to dial that right back to what’s this very specific problem that we can solve, that will prove trust, that we can wrap in proper governance to get people excited about AI, to get people excited about removing those mundane tasks from their desk.

Cameron Scott: Whether it’s, you know, automatic cancellations based on unstructured email content or even prompting those for review by a user, preparing those declinations, you know, a very specific business case at first, as opposed to thinking that AI is just going to broadly replace an underwriter.