The 3 AIs needed to Create truly Intelligent Assistants and Chatbots (Part II)
[Versión en Español disponible aquí]
The Story so far
In Part I, we talked about the state of Artificial Intelligence (AI) and how it is not enough to guarantee success. There are three other types of AIs needed for applications to be truly intelligent. We already covered the first — Aided Introductions (aka Onboarding). Now, let’s look at the second one:
2. Active state Indicators
At its core, a “conversational experience” is based on the premise of good communication between the system and the user. Since users should always have the freedom to decide their preferred modality — tapping, typing or speaking — it is the system’s responsibility to always make its current state visible. It helps answer questions such as “What is going on?” and “What should I do next?”
To accomplish that, your user interface should leverage existing mobile design patterns and rely on human behavior so the experience always feels natural and familiar. For example, if your system supports speech recognition, then it should indicate when the system is ready to accept input (listening mode), when it is actively listening (indicating what has been recognized as well as the volume level being registered), when it is processing the user’s input (processing, transcribing, understanding, etc.), when it is providing a response or asking a follow up question (which in some cases may not be interrupted), when it is retrieving data from other systems, and even when the system considers the interaction to be complete (dormant state).
Furthermore, in addition to text and visual indicators, the state can be reinforced via non-verbal sounds and visual animations and transitions (for example, a color palette can be reversed when a chat interaction switches from a virtual assistant to a live chat conversation within the same interface).
This constant communication of a system’s state helps compensate for user distraction (particularly on mobile), allows you to alert users ‘at a glance’ to emphasize important items, all while reinforcing the brand experience you want to deliver. The easier and clearer you make the experience, the better your engagement rate will be.
“Good UI design gives users a comprehensible sense of power that consistently helps them feel in control.” — Jim Nielsen
For example, Amazon recently added Alexa capabilities to their mobile shopping app. As you can see from the sequence below, the app indicates speech capabilities via the microphone button. Then, when activated, the screen visually changes, disables the rest of the items and provides a just-in-time hint to educate users.
After providing an input (“Oakley glasses”), the bar at the bottom of the screen animates and indicates an active recognition state while also providing feedback on the volume level being recognized. Upon a successful recognition and data retrieval, the system replies with “<bing> Showing items matching Oakley glasses” hence providing implicit confirmation of what it recognized while reinforcing the visual change in the search box.
Finally, the results screen also does a really good job at presenting a set of relevant follow-up choices such as immediately allowing a follow-up search, allowing users to filter results in various ways, and presenting a “tip” at the bottom about how to save items for later. Also, in terms of error recovery, users can tap the screen at the listening state, which results in a different 2-tone sound and keeps the user in the same screen as before. Or if the user stays quiet and times out, the system plays a different tone and stops recognition, keeping the user in that same screen.
Another example of a speech-enabled shopping assistant comes from Humansfirst’s HER app whose assistant offers help with online shopping, travel and fashion (version 2.0 seems will also add tickets). In this case, we see a lot more opportunities to improve communication about the state of the system.
As you can see below, once the user activates recognition, the system plays a <beep> sound and transitions into a “spinning tag” screen, but it seems to be missing a hint on what could be said or feedback on whether the system is hearing something and at what volume level. Then, upon a successful recognition, the system reuses the same <beep> as before (Is it listening again?) and then updates the screen with the recognized text but only for a couple of seconds.
Afterwards, the system immediately moves on to the results page and you hear “Here’s what I have found for you”. At this point, adding an audible and visual indication of what was recognized and providing filtering capabilities would be of huge help. Furthermore, users are likely to be looking for ways to refine their search or to perform a new search immediately (which right now requires the extra step of tapping “BACK”).
Also, in terms of error recovery, there doesn’t seem to be a way to stop the recognition, and a time out results in the same <beep> as before (Is it listening again? Was it successful?) and the alert screen depicted below which upon dismissal doesn’t have a way to go back or retry a search without restaring. Improvements like these would make an enormous difference in the user experience and increase the perceived “intelligence” of any shopping assistant.
In light of that, how would your users score your system intelligence at conveying state and next steps? Are you closer to an A+ or a D-? I look forward to your feedback
