User Testing
2 sages of the testing plan were conducted (In person user testing and A/B testing) for the mobile app prototype.
​
In Person User Testing
​
First thing I did was to develop a testing protocol, which would allow others to conduct the same test in a consistent manner.
The protocol would include
- Preparation and setup
- The steps & roles for executing the test
- Instruction Script
- How will the observations be recorded
- Debriefing the participant
- Interview questions after user complete tasks
​
I invited 2 of my colleagues who I expected would use my app (representative testers) to join the user testing. During the testing I’d focus on capturing the breakdown and design opportunities from participants’ feedback, and again made a list of changes for my next design iteration.
Here’s one of the pain points I captured from the testers. She though that the she can add more time for current event by clicking either the “+” button or the text “Need more time”(She thought that was also a button). So she kept clicking the text part several time but found nothing happened. But actually she can only click the “+” button to activate “add more time” function.
Key Insights
After the in-person testing I found some interesting general patterns in people’s behavior that could be applied to improve my prototype:
​
-
People are expecting that all buttons in the same pages should be directly related to the purpose of the page.
-
People tend to take both the icon and the text that describe the icon as bounded together and clickable.
-
People are expecting that “next step related” buttons should be at right sides, and the buttons at the left side of screen are implying related to “previous step”.
-
People don’t like too many buttons in the same page. If it is inevitable to have these amount of buttons, then grouping is important to categorize these buttons to make users much easily to navigate around.
-
The layout of the buttons and text should not be like the ads layout, or it’ll let user to less likely click that button.
A/B Testing
​
The following are the plan for my A/B testing.
​
- Experiment -
The button for “Discover” recommended activities should be appeared
before or during user adding events?
From the in-person testing, I found that since the “Discover” button was originally designed for recommending activities to users when users have no idea what to do for their night, so the “Discover” button should be put just right up the “add event” icon allowing users to first get some events inspiration before adding their events.
Originally the “Discover” button will appear after users click the “add events” button, which I hypothesized it might make users less likely to click on it since user should already decide what to do. Put the “Discover” button before user adding events could let users more likely to click on it.
​
So for the A/B test, the A version would be the original design- the “Discover” button would appear during users are adding events, and the B version the users will find the “Discover” button before they are going to create new events.
​
Based on my hypothesis, the “Discover” button would be clicked more times in the B version than in the A version.
Here I used UserTesting.com to conduct A/B test.
I assigned 2 of participants for version A and the other 2 for version B.
Conclusion for the Test Result
​
Based on the A/B test conducted, “Discover” button in screen B tends to have more user click than the button in screen A for the following reason:
In Screen A, whenever users want to click on the Discover button, they need to first click the blue “+” button then they can click on the Discover button. However in screen B user would be more easily to click on the Discover button because it just located in the first layer of the add event page. It would be good if more testing samples are involved and analyzed by Chi-square methods to strengthen this conclusion.
However I would apply screen B to redesign my prototype since it match my point of view more. Since my point of view that finally reach this prototype should be designed for both people who are busy at night and who needs inspiration to fulfill their night, separating the 2 buttons in the following manner would fit my point of view more adequately: One of them (the blue “+” button) is to allow user to add events they’ve planned and the other one (the Discover button) is to inspire users what activities they could do.