Why does someone use a digital assistant (aka AI, ChatGPT, chatbots, et al)?
Before you glide past that question, take a moment and really think about it: Why would your users/customers/team members use a chatbot?
The seemingly obvious answer: To get a quick and accurate answer to a question they have.
Would you be surprised to learn that many chatbot products aren’t focused on providing relevant, accurate answers to users’ questions?
According to IntraSee founder Andrew Bediz, most organizations introduce an artificial intelligence/chatbot platform expecting it to help users quickly find answers to their questions, only to experience poor usage rates and negative user feedback. “We often see organizations with a chatbot strategy of focusing on a single purpose (i.e., recruiting, financial aid, etc.) and they expect people will only ask questions relating to those pre-built topics,” Andrew shared. “But this approach isn’t accurate. Research data that IntraSee has collected finds that the most popular search topics on a chatbot max out around 20% of the total volume and the average topic sees only single digit percentages. Translation: If you implement a bot that has really narrowed its focus to the top question, you are only helping 20% of those asking questions, leaving 8 out of 10 frustrated. That’s a sure way to never get 80% of your audience back.”
Andrew often cautions prospective customers that “cheap is expensive” when it comes to implementing a digital assistant. “Without some effort and research, you’re not going to have a positive user experience,” Andrew said. “And if you don’t get good results, people won’t use your chatbot. If your organization isn’t committed to really investing in the project, you’re setting yourselves up for failure. Next thing you know, your three-year contract is up, and you discover that people don’t like your chatbot and that they’re not using it. They don’t think it’s accurate. They don’t think it’s relevant. They don’t think it covers enough of the questions they might be asking, or they might not think it’s personalized to them or have anything to do with their needs. That’s when cheap becomes very expensive.”
Another way chatbot projects become expensive is when a customer buys a solution and deploy it only with built-in questions and answers. At first this can seem like a great way to keep costs down if, for example, a customer buys a chatbot pre-loaded only with student financial aid Q & A. But what if a new student asks for help finding the library? In response to this concern, the vendor might counter with “We have a web crawler that will crawl your website and pull that data into your chatbot.” Crawling a website falls far short of delivering a personalized experience. Keep in mind that the user expects a different experience interacting with a chatbot than they do with classic web search. When that experience is disappointing, they stop using the offering. We’ve seen organizations spending $200,000+ over a three-year contract with very little to show for their investment and now they are considering starting from scratch. Cheap is expensive.
“If you don’t get good results, people won’t use your chatbot. If your organization isn’t committed to really investing in the project, you’re setting yourselves up for failure.”
Start Right, Start Now
Gideon Taylor CEO and Founder Paul Taylor often talks about how hard it can be to “catch up” on innovation. In Paul’s words, “It’s not as simple as thinking ‘We can start whenever we want and quickly arrive at the same place as Organization X who started innovating months or years before us. Enterprise-grade chatbot technology is constantly learning. It is building a database of understanding about your organization and what your users are asking. You can’t overcome that head-start without a lot of cost and sustained effort.”
Andrew adds that asking, “What does ‘cost’ mean to me?” can make a material difference in the outcome of your digital assistant investment. “What’s important is net cost. If I spend more on the bottom line but make more on the top line, then it’s not necessarily more expensive. Gartner metrics show that when you use a bot, at minimum, every interaction could represent a savings opportunity of $18 (live support is $19 per interaction and a chatbot is $1 per interaction.) Let’s imagine the difference between a good chatbot and a bad one. A good one has more adoptions, with more people using it to get answers instead of through an email or a phone call. Let’s say this difference leads to 1,000 more interactions per month. By that measurement, you’re saving almost $20K per month vs. the bad bot. Let’s presume the bad bot costs $70K per year and the better solution costs twice that much. Based on this comparison, you are making up the cost difference in just over three months. The remaining 8.5 months represents pure savings. It becomes a wiser economic decision to choose the better chatbot that is meeting more users’ needs and is being used much more often.”
“Chatbot technology is constantly learning. It is building a database of understanding about your organization and what your users are asking. You can’t overcome that head-start without a lot of cost and sustained effort.”
Here are two important questions that can help you to avoid having a negative and expensive chatbot project:
Will my chatbot be accurate?
Will my chatbot be personalized?
“All chatbot vendors will tell you they provide these two experiences but have a hard time backing these up with objective measures and analytics,” Andrew says. “Customers don’t always understand that there is a difference in chatbots because that difference can be hard to see at first. On RFPs they’ll include features they want or that would be nice to have, such as ‘Does the bot have the ability to go to live chat?’ or ‘Can it be placed on any website?’ or ‘Can it use SMS?’ They then confirm if the vendor checks these and other boxes but miss the most important question – ‘Can your bot quickly and accurately answer user questions?’ If it can’t do this, all the other features don’t matter. It’s like you bought a fancy shiny car with nice rims and a great sound system but you can’t get it to drive faster than 25 miles per hour.”
Accuracy is Job #1
Achieving accuracy is a matter of algorithms and training models. It is not something that is easy to see in a controlled environment like a sales demo. This is something that a website or salesperson may skim over in pitching the benefits of their chatbot platform.
At IntraSee, we use Oracle’s Digital Assistant (ODA) PaaS service for machine learning driven natural language processing (NLP). The language model is seeded by Oracle who used over forty years of enterprise data. They are also one of the largest enterprise data companies in the past forty years, which gives them access to massive amounts of enterprise-level data. Oracle has used this data to inform their language models so that things like misspellings, alternative words, and grammar mistakes are more easily tolerated. They also employ something called Machine-Learning Artificial Intelligence. What that means is the algorithms aren’t looking for specific words or strings or characters in a certain sequence. Instead, this type of artificial intelligence is designed to learn over time and can handle inputs it has never seen before, leading to remarkable results. You can ask ODA questions it has never seen in the past and has never been programmed to handle and still receive accurate responses. The AI tech that Oracle uses is like what Tesla uses to determine the difference between a stoplight, a go light, yield sign, or a line in the road.
“Can your bot quickly and accurately answer user questions? If it can’t do this, all the other features don’t matter. It’s like you bought a fancy shiny car with nice rims and a great sound system but you can’t get it to drive faster than 25 miles per hour.”
How’s Your Training Model?
Another frequent problem Andrew mentioned is the chatbot training model that some vendors employ. “Not all these products use true machine learning AI and those that do save by training one model to generically serve all customers. The downside to this approach is that the model knows nothing about your unique environment, not to mention any bias a vendor might be introducing. It doesn’t know your slang or acronyms. Up front it sounds like a good idea. ‘Oh cool! We don’t have to do any of this work.’ But the result is that the bot has become less personalized to you. It doesn’t understand you or your organization. Personalization in a chatbot is so important. When users encounter what they think is overly generic engagement, they just see it as spam. They don’t see it as a valuable interaction and won’t repeat it. Think of the value lost in that moment and then multiply that many times over each day. Once again, cheap is expensive.”
So, how is IntraSee’s Ida enterprise chatbot providing a measurably different outcome to customers?
- We use the top-tier backend technology: Oracle Digital Assistant or ODA.
- Security is always at the top of our list of priorities.
- Each customer has a unique training model for their language. Not very many, if any, vendors are going to give you that. This means the bot gets to know you and your organization and that learning can make all the difference in user outcomes.
- We know most customers don’t have an AI architect, a data scientist, and a computational linguist on their team. That’s not their day job or focus, nor should it be. AI Architects can be $200K or more in today’s market. They are also hard to find because they are in high demand. We couple our tech with a managed service team that’s always monitoring a customer’s data, accuracy, etc. If we see a dip in accuracy because people are asking new questions about parking, etc. and we haven’t trained the bot to handle those, we don’t just say “good luck with that.” Our work doesn’t end when someone buys a license. We have a team of people constantly engaged, always making sure an organization is successful, always making sure we’re calling attention to pitfalls and roadblocks.
“Personalization in a chatbot is so important. When users encounter what they think is overly generic engagement, they just see it as spam. They don’t see it as a valuable interaction and won’t repeat it.”
Right Model, Right Results
Add all these elements and more and you have an offering that is much deeper, much broader, and always learning. Accuracy is top-level – better than any competitor. Although the initial cost may be at a premium, the value you get in return is orders of magnitude better. When you net it all out, you may even call it cheap! We think it’s a good idea to pay a little more in exchange for 10x more in value. If you invest in the right investment, the return should be far greater than what you put into it. The stock price doesn’t matter if you get a great return on it. It matters a lot if you lose what you invested.
Ready to learn more? Schedule a personalized solution presentation with one of our team members today and discover what Ida can do for you.