Wizard of Oz Experiment

-> to BOTwiki - The Chatbot Wiki

In the field of human-computer interaction, the Wizard of Oz method refers to a research experiment in which test persons interact with a computer system which is considered autonomous by the test persons but which is actually operated or partially operated by an invisible human being. [1]

Chatbots and Wizard of Oz

Chatbots are extremely well suited for the Wizard of Oz experiment. This allows a use case to be examined for its "chatbot suitability" before implementation. The findings can then be used to iteratively expand existing flows and define new lines of communication. In addition, the collected data, such as Utterances, can be used directly for the Chatbot Training can be used. A sophisticated human handover tool is even able to automatically convert the tested data into a chatbot.

Recommended procedure

  1. Define different chatbot flows
  2. Integration of a live chat or an empty chatbot that only triggers a human handover.
  3. Manual response to user queries
  4. Derivation of Chatbot Flows
  5. Answering the questions with the Chatbot Flows
  6. Iterative revision and extension of the Chatbot Flows
  7. Manual or automated transfer of flows into a chatbot builder

> Back to the BOTwiki - The Chatbot Wiki


[1] Kelley, J. F., "An empirical methodology for writing user-friendly natural language computer applications". Proceedings of ACM SIG-CHI '83 Human Factors in Computing systems (Boston, 12-15 December 1983), New York, ACM, pp. 193-196