User Testing
The Task
We asked the users to imagine themselves as a computer science student and search for “how to implement Dijkstra’s algorithm” by following some task instructions which we have given. We asked them to verbalize their thoughts throughout the testing process. We informed them that they would not be using an actual website but instead testing an interactive mockup.
Testing Instructions
- Without leaving the homepage, what are your initial impressions of the design and the functionalities available? Be specific.
- Start your search by choosing Java as your intended language. Tell us how you selected it. Comment on the intuitivity of this step.
- Search “How to implement Dijkstra’s algorithm” by clicking the correct suggestion in the dropdown menu.
- Scroll through the pages. Comment on the dual layout of the search results. Describe in your own words how you might benefit from using this page.
- Navigate to “Web Answers.” Comment on the design of this page.
- Change your display to “Hello Answers.” Describe what you expect to find on this page. Comment both on the design and benefits of this view.
- Go to the first referenced source link.
After the users completed all of the tasks, we asked them to answer some post-test questions. The questions were the following:
- What frustrated you most about this site?
- If you had a magic wand, how would you improve this site?
- What did you like about the site?
- How likely are you to recommend this site to a friend or colleague (0=Not at all likely, and 10=Very likely)? Why or why not?
Analysis of Results
Overall, our UserTesting results were in line with our expectations. The users didn't have difficulty in completing the tasks and found our design intuitive.
Task 1 indicated that users understood the homepage layout and purpose, though minor feedback surfaced about adding a profile button. Task 2 feedback suggested adding more languages to the mockup and displaying language change options after selection. Task 3 and Task 4 were both deemed intuitive, with users finding the split screen effective, but there were differing opinions on the placement of web answers. Task 5 revealed users' ease in locating web answers, while differing preferences emerged concerning similarity to Google. Task 6 highlighted a preference for 'Hello Answers' over 'Web Answers', and users appreciated the algorithm's responses, suggesting a 'see all answers' button. Lastly, Task 7 showed unanimous agreement on the ease of accessing the first source link.