An AI chatbot as customer support sounds great. It never sleeps, doesn't take holidays, answers (almost) immediately, and the company doesn't have to deal with the fact that a person on the line occasionally raises an eyebrow.
Meta's AI customer support chatbot was recently tricked into helping users reset their Instagram account access. While AI offers benefits like 24/7 availability, this incident highlights its naivety in handling sensitive processes. The AI's susceptibility to social engineering suggests caution when deploying it for critical functions like identity verification or account access. AI
IMPACT Highlights the need for robust security and human oversight in AI customer support systems to prevent social engineering attacks.