Does your AI BOT hallucinate?

Hi Mandy,

are you using Advance AI or an external bot? In fact the Generative AI will forget details after usage and a hallucination is possible if you are using more than valid articles of your HC and due to the fact that the performance of neural networks will shrink after months.

I currently do not use a Generative AI Bot in our instance, but we are working with the tools in other processes. 

Please keep us informed.

Thanks, Stephan

Thanks Stephan for the comment. We are planning on using the Generative AI bot in our help centre and in-app.  What do you mean when you say, using more than valid articles?  Do you mean "answers" configured in flows?

In fact there are several options if you are integrating other bots. For example: Some companies are offering products which can handle 5 data bases (help center, sharepoint or whatever). Therefore the articles of the HC and the website etc. can be classified as trustful. The internet data (Wikipedia etc.) will be able to give you generated or less trustful knowledge.

I think you are using the zendesk solution, so your articles of HC will be your knowledge for your chatbot. Are you using the flow builder? I changed to answer triggered use-cases during summer. :-)

Yes we are only using the help centre articles for now anyway.  We didn't want to have to build any bot flows which is why we went with generative AI thinking that it would hunt and peck thru articles and deliver the answer to the customer without having to build flows.  I do see the benefits of using the answer buttons though, they can be quite handy.

Hi Mandy Labriola, 

Currently I'm not using Generative AI Bot but I want to give some feedback on this topic, if AI bot doesn't understand your input or doesn't have sufficient information, it'll rely on the limited dataset and it will give inaccurate information. If your input is idioms or any expression which is not matched to your data then it will give inaccurate results so you have to train your AI for the vast dataset.

Thanks

Hey @mandy12 - 

Generative AI always comes with risk - which is why Zendesk has very cautiously rolled it into the product set.  That said, there is an EAP (early access program) going currently that allows your bot to 'conversationalize' your existing Help Center content.  You can learn more about that here.

Brandon

We've developed an AI agent (bot) for Zendesk that's available on the marketplace: https://www.zendesk.co.uk/marketplace/apps/messaging/139/my-askai/

One of our core goals is to bring hallucinations down to virtually zero. You train your AI agent on your help docs, and it will only answer questions from content within those help docs. The AI will say when it can't answer a question or if something is off topic.

Over the last month, we've handled 30,000+ conversations, with over 75% of them being resolved by our AI agent.

Hope this helps! I know first hand how a few hallucinations can make a business really weary about implementing AI in customer support.

We are determined to achieve zero AI hallucinations.

We've created an AI-powered solution for customer service. It automates 60% of cases, including complex scenarios beyond FAQs, by integrating with back-office systems and ticketing platforms. Most importantly, customers are happier, receiving more precise responses in under 30 seconds, and CSAT increases by over 20%! 

Check it out at Rauda.ai :)

Yeah, I’ve been using the AI bot in our help center setup for a bit. It does throw some weird article suggestions now and then, especially if the question is a bit vague. But honestly, it's been decent overall.

When it gets stuff wrong, most users just end up clicking “contact us” or rewording their question, so it's not a disaster. Still feels like it needs some fine-tuning though. Curious to hear what others are seeing too.