Our Approach to Voice User Testing
As the role of voice continues to expand and evolve at a rapid pace, the product team at Arrive embarked on an extensive research and testing track to discover how to best utilize Alexa in what we call the last mile of mobility.
To successfully design an easy-to-use voice parking product, it’s imperative to stay at the forefront of technical capabilities while fully grasping user comprehension. With testing, the Arrive product team hoped to refine our Alexa Skill to create a thoughtful and beneficial experience for our customers.
Before getting started, we identified a series of goals:
- Gauge current user comprehension of Alexa capabilities
- Gain an understanding of how users expect to use Alexa for parking
- Identify areas of frustration
- Identify and prioritize enhancements based on issue frequency and severity level
- Discover when and where the user expects information
Currently, most user experience testing practices are meant for visual interfaces, such as phones, touch screens, etc. And since voice products are relatively new, there are few defined formal recommendations for user testing. The product team bridged this gap by developing a non-biased research approach for our Skill.
User Testing
We began each testing session by interviewing the participants about how they used Alexa. Turns out most participants used Alexa for weather forecasts and driving directions, but not much else.
After gathering background information, we then asked users to book parking uninterrupted without providing instructions or context, in order to mimic real-world interactions.
When the user finished, we asked high-level questions, such as “What was your general impression of the Skill?” and “Did you feel frustrated with Alexa at any point?” These questions were designed to garner expansive and thoughtful responses about the Skill without leading them down a specific path.
Solutions
As testing went on, we realized how important it was to provide detailed instructions and phrases to help the user navigate the parking experience. Some users struggled when they used language that the Skill couldn’t recognize. After we outlined common phrases and actions within the Skill, confusion was greatly reduced for inexperienced Alexa users.
We also identified key transitions along the journey where the user expected specific parking information. By focusing on these areas and highlighting essential information, we minimized deadends, reduced frustrations, and helped the user understand the Skill’s capabilities.
Also, we realized It’s important to create an experience that’s customized specifically for Alexa users. Google, Siri, and other voice users have different expectations and preferences.
Results
Once we implemented these findings, the product team was able to reduce the number of turns within the user flow from nine or more to as little as five. This reduction lowered the average completion time from 98 seconds to 69 seconds, which was a huge accomplishment!
Through research and testing we were able to identify areas of improvement and enhance the overall user experience. This will ultimately make it easier for customers to find and book parking with Alexa.
Once our updates and improvements were made, we were able to create an easy path for booking seen below.
If you have any thoughts or voice research you’d like to share, we would love to hear from you!
Contact us to get the conversation started.