Today, many websites and applications are using chatbots.
For common user queries, chatbots are programmed to reply with scripted messages. Chatbots have a very simple, intuitive user interface – spoken or written human language. But with a variety of user behaviors and no human involvement from website’s end, dealing with chatbots can be tricky.
Speaking of user interface, we’re testing the UI/UX using relevant software testing tools. In this context, a test case can be described as a conversation a chatbot should be able to follow.
UI/UX testing, in the test level hierarchy, is on the highest level, as the whole system is being tested including all dependent systems. Usually, for a chatbot testing strategy you start with lower levels.
A test suite for a chatbot consists of a library of conversation transcripts. Head for the most important use cases for your chatbot, and the most important edge cases. Identify the “Happy Paths”, the use cases where user behaves as expected, and continue with other conversation flows then. So, whenever one of those conversations with the Chatbot fail, you know there is some trouble and you have to investigate.
Running the test suite manually is not feasible for medium/large-scale projects. With test automation, you can run your chatbot conversations without your intervention. This means that a chatbot will be interacting with your chatbot, saving you the hard work, and upon completion, you will receive test reports.
It is to be noted here that “conversation ↔ test case” approach does not address whether the chatbot actually meets the requirements. However, this approach simplifies the process of getting started with test automation. “Verification” and “Validation” are two often terms often misunderstood by people from non-testing backgrounds. So, let’s put it this way:
Verification: “Are you building it right?”
Validation: “Are you building the right thing?”
In this case, we’re checking the verification since we assume that chatbots are doing the right thing. But are they working as designed?
This approach is usually easier to follow when getting started with test automation.
Conclusion
Testing chatbots is imperative. Since they’re interacting with users who are your clients or potential clients, they are the front-line representatives of your company. Their performance can affect website’s reputation. Wrong replies can misguide users and, in most cases, annoy them for being too wrong. Stated in this article is a basic approach to strategizing chatbot testing to get you started. Relevant other testing tools can help ensure error-free performance of these bots.
Bots serve all kinds of purposes. This article focuses on chatbots that serve a good purpose of catering to user queries. But not all bots are designed with a good motive. In my article, Bots and Cyber Security: The Good, The Bad, and The Ugly, I discuss how bots are used around the world to attack organizations and even the governments.
Read Also: How AI Chatbots Benefit Customer Service