Voice User Interface Design Guidelines¶

In a nutshell

Designing for voice is different from designing a screen: people speak to your skill in many different ways, and you can't show them buttons or menus to guide them. This page shares simple methods to plan, sketch, and test a skill idea before you start coding, so the finished result feels natural to talk to instead of something you have to learn. Think of it as drawing a conversation on paper first. New terms are explained in the Glossary.

Scope: voice interaction design

Applying principles of Voice User Interface Design helps you define and validate a skill's features before you start development. The methods below help plan, prototype, and test your skill during the early design stages.

This page is about designing the conversation: when to confirm, how much to say, how to phrase prompts. It is independent of any programming language. For the code-level practices that implement these ideas (which method to override, which API to call), see Skill Design Best Practices.

Credits: Voice User Interface Design Guidelines based on the original work of Derick Schweppe.

Interactions¶

Intents¶

Let's start with an example. A user in Melbourne, Australia might want to know about the weather. To ask for this information, they might say:

"Hey Mycroft, what's today's weather like?"

"Hey Mycroft, what's the weather like in Melbourne?"

"Hey Mycroft, weather"

Even though these are three different expressions, for most of us they probably have roughly the same meaning. In each case we would assume the user expects OVOS to respond with today's weather for their current location.

It is up us as Skill creators to teach OVOS the variety of ways that a user might express the same intent. This is a key part of the design process. It is the key difference between a Skill that kind of works if you know what to say, and a Skill that feels intuitive and natural to talk to.

This is handled by a pipeline plugin whose job it is to learn from your Skill what intents it can handle, and extract from the user's speech and key information that might be useful for your Skill. In this case it might include the specified date and location.

Statements and Questions¶

A Question asks the user for a response, while a Statement provides information to the user that does not need a follow-up response. For example a weather forecast like this would be considered a statement:

Today's forecast is sunny with a high of 60 and a low of 45.

Statements¶

For a lot of skills the conversation might end with a simple statement from OVOS, and no further action is necessary. Try to imagine what the user is trying to accomplish, if a simple statement gets the job done there is no reason to keep the conversation rolling, and in fact a follow-up might annoy the user with unnecessary interaction.

Questions¶

It may be tempting to always give users specific instructions like traditional automated phones systems \(Interactive Voice Response\). Many phone systems are notorious for being too verbose and difficult to follow.

With OVOS we're trying to break that mold and make the interaction natural. If you follow the phone system method you may be giving the user the exact phrasing to say, but you're also taking up valuable time and training them to think the system is very narrow in capability. If the user gives a response that your skill cannot handle, create follow-up questions that steer the user back on track.

Remember, there are no errors in a cooperative conversation.

Avoid

Speaker
Mycroft	How many people are playing? For example, you can say 2 players.

Better

Speaker
Mycroft	How many players?
User	My Brother and Myself
Mycroft	I'm sorry, what was the number of players?
User	Two

In the first example the user is told explicitly what they can say, but the question is unnatural, we don't typically suggest responses to a conversation partner in real life. These long-winded questions can become tiresome if they are used repeatedly throughout the skill. Remember the phone system example, typically poorly designed automated phone systems inundate the user with many options and additional instructions at every step of the interaction.

In the second example we see a better question, although the user gives a response that is easy for a human to understand it is more difficult to design a skill to understand. Instead, the skill designer can ask the question again in a different way to steer the user back on track and doesn't require them to start the interaction over. In this case the follow-up question changes the wording slightly to make it clear that the user needs to say a number. The next time the user interacts with this Skill, they will likely say a number in their first interaction.

Determining whether to respond to the user with a statement or a question can be a bit tricky. When the user is somewhat vague it is tempting to assume the user's intent to speed along the interaction. Sometimes this is the right approach and sometimes it is not. If you are very confident in the user's input then it is alright to assume, and respond with a statement. For example in the Wikipedia Skill the Wikipedia API is used to confidently select the best response.

Wikipedia Skill

Speaker
User	Tell me about Abraham Lincoln
Mycroft	Abraham Lincoln was an American statesman and lawyer who served as the 16^th president of the UnitedStates from 1861 to 1865. Lincoln led the nation through its greatest moral, constitutional, and political crisis in the American Civil War.

In contrast, let's look at an example of where a follow-up question is a better approach. You're working on a skill that gives important information about birds, such as wingspan, laden and unladen airspeed, etc..

Avoid

Speaker
User	what is the airspeed velocity of an unladen swallow?
Mycroft	The unladen airspeed of the european swallow is 20.1 miles per hour.

Better

Speaker
User	What is the airspeed velocity of an unladen Swallow?
Mycroft	What do you mean, African or European Swallow?
User	European Swallow.
Mycroft	The unladen airspeed of the European Swallow is 20.1 miles per hour.

In the first example Mycroft assumes the user is referring to the European Swallow, however there is nothing to give confidence to that assumption. In this case it's dangerous to assume the User meant the European Swallow since it is just as likely they meant the African Swallow. When there is no confidence in one answer versus another, OVOS should follow up with a question as demonstrated in the second example.

Another approach to solving this problem is to offer the most likely answer and inform the user that there are other answers available with a statement.

Confirmations¶

Confirmation approaches can also be defined by Statements or Questions, but when we talk about them in the context of confirmations we call them Implicit and Explicit.

Implicit Confirmation¶

This type of confirmation is also a statement. The idea is to parrot the information back to the user to confirm that it was correct, but not require additional input from the user. The implicit confirmation can be used in a majority of situations. For example Mycroft's Timer Skill uses implicit confirmation when creating new timers.

Timer Skill

Speaker
User	Hey Mycroft set a timer for 5 minutes
Mycroft	Alright, I'm starting a 5 minute timer

Avoid

Speaker
User	Hey Mycroft, add bananas to my shopping list
Mycroft	done

Better

Speaker
User	Hey Mycroft, add bananas to my shopping list
Mycroft	Adding bananas

Explicit Confirmation¶

This type of confirmation requires an input from the user to verify everything is correct. Only use explicit confirmation when needed as it can be tedious for the user to always confirm their input. This strategy is best when there are multiple steps, and it would be difficult to fix a misstep without starting over, or when deleting or modifying an existing multistep input.

Avoid

Speaker
User	Hey Mycroft, set an alarm for 9:00 am tomorrow
Mycroft	Ok, I've set an alarm for 9:00 am tomorrow, is that correct?
User	Yes
Mycroft	9:00 am alarm set

Better

Speaker
User	Hey Mycroft, set an alarm for 9:00 am tomorrow
Mycroft	I've set an alarm for tomorrow at 9:00 am, 16 hours from now

The first example requires the user to confirm the alarm. Since the interaction is quite short it isn't necessary to use explicit confirmation.

Avoid

Speaker
User	Hey Mycroft, set a calendar event for a meeting with Chris Stevens
Mycroft	When is the event?
User	Tomorrow at 11:00 am
Mycroft	should I send an invite to Chris Stevens?
User	Yes
Mycroft	Alright, I've created a meeting for Tomorrow
Mycroft	Alright, I've created a meeting for Tomorrow at 11:00 am with Chris Stevens
User	Cancel my 11:00 am meeting tomorrow
Mycroft	Done

Better

Speaker
User	Hey Mycroft, set a calendar event for a meeting with Chris Stevens
Mycroft	When is the event?
User	Tomorrow at 11:00 am
Mycroft	should I send an invite to Chris Stevens?
User	Yes
Mycroft	Alright, I've created a meeting for Tomorrow at 11:00 am with Chris Stevens
User	Cancel my 11:00 am meeting tomorrow
Mycroft	Are you sure you want to delete your meeting tomorrow at 11:00 am?
User	Yes

Both examples use explicit and implicit confirmation correctly until the end of the interaction when the user wants to cancel the event. Unlike setting a simple timer or an alarm creating this calendar event requires multiple steps and would require reasonable effort to recreate. There may also be significant consequences if the request was misinterpreted and the meeting was canceled unintentionally.

Conversations¶

Any time the user needs to input a lot of information or the user needs to sort through a variety of options a conversation will be needed. Users may be used to systems that require them to separate input into different chunks. For example, many personal navigation devices \(GPS units\) require users to input the street number, street name, city, and state separately, like this:

Speaker
Navigation Device	What is the street number
User	123
Navigation Device	What is the street name
User	Main Street
Navigation Device	What is the city
User	Springfield
Navigation Device	Oregon

This was a limitation of the technology, both the STT and intent-matching pipeline plugin were not capable of handling an entire address. With OVOS you do not need to separate the address into multiple inputs. It's also easy to think like a traditional GUI and translate what are typically separate fields into questions for the user, but that will only slow down the interaction.

The best solution is to allow the user to input information both ways, all at once, or in pieces. For example the Alarm skill allows the user to set a complicated alarm all in one go, or in steps.

In Steps:

Speaker
User	Hey Mycroft, set a recurring alarm
Mycroft	For what day?
User	weekdays
Mycroft	For what time?
User	7 am
Mycroft	I've set a recurring Alarm for 7 am on weekdays

All at once:

Speaker
User	Hey Mycroft, set a recurring alarm for weekdays at 7 am
Mycroft	I've set a recurring Alarm for 7 am on weekdays

Context¶

Conversational context is a given when speaking to another person, but not always when interacting with a computer-based system. Many skills are designed for only a quick back-and-forth interaction and can operate well without conversational context. Context can still add much to a skill and make the interaction feel more natural. Here are some examples of how to use context in conversation.

Speaker
User	Who is Shaquille O'Neal
Mycroft	Shaquille O'Neal is an American former professional basketball player who is a sports analyst on the television program Inside the NBA on TNT. He is considered one of the greatest players in National Basketball Association history
User	How tall is he?
Mycroft	seven foot 1 inches tall
User	when did he retire
Mycroft	2011

Error Handling¶

Inevitably, the user will say something that your skill can't handle. It's best not to think of these as errors on the part of the user, remember there aren't really errors in conversations. Conversations are naturally cooperative with both parties seeking to continue to a successful outcome. Keeping that in mind you should do your best to avoid Error Handling that is a dead-end.

Avoid

Speaker
User	Timer
Mycroft	I'm sorry I can't help you with that

Better

Speaker
User	Timer
Mycroft	A timer for how long?
User	5 minutes

In the first example OVOS does not give the user any options to finish the job they set out to do. Try to avoid situations where the user has to start the conversation over.

Source code: OpenVoiceOS/ovos-workshop.