Skip to main content

5 Tips of User Test

I go to Pragmatic Product Meetup in Hong Kong, the host Leon is a very inspiring product manager working for a toy company; we sometimes discuss how to do user testing. We find many things in common. And I guess the common points we discovered can be shared across different industries since we are in totally different industries: Leon, babies, physical, Hong Kong market; mine, adults, smartphone users, mobile application and website, South East Asia, China, United States and United Kingdom.

What we found in common can be concluded as the following 5 points:

What do you want to test determines how you want to test it: there is not “the” right way to conduct user test, you can do a questionnaire, you can do a drawing on a piece of paper or you can let them test out your prototype or beta version product; 

In the Hierarchy of Actions, there are 4 types of products: desirable, usable, persuasive and emotional; Paul Doncaster, Senior UX Designer at Thomson Reuters, in his … explain that desirability test is a quick and cost-efficient way to seperate what users actually desire versus what they say they desire. 

If users have a positive impression, they are more likely to overlook or forgive poor usability or limited functionality, while with a negative impression, users are more likely to find fault with an interaction, even if a product’s overall usability is good and the product offers real value.

I used Concept Testing a lot with new products and features, as it is closely associated with the concept of MVP (minimum viable product). After all, what can be more bare-bone than only testing for the viability of an idea. It provides the feedback to turn a “deliberately sketchy idea for a product or service” into something that users might actually want.
I usually use surveys and interviews. A concept test can focus on New Product Concept 
Tests identify the benefits that resonate most with customers and the features to create these benefits which help you to prioritize your design elements and better schedule the development process, plus allow you to plan ahead for after the release. Usability Test help us understand what direction might make the most sense for updates to existing products (whether it’s ease of use, simpler navigation, etc). or Price Test which obviously test how much people are willing to pay for it. 

While going beyond into the actual design and interaction, I found InVision and Marvel to be good tools with their powerful interaction creation function. complicate transitions, conditional layout etc, you name it. 

When it gets to usability, you need to focus on observing unalignment, Get a video recorder, point it to the screen from behind the participant and record everything so you can watch all of your participants back once you are finished, and you can use it when you need to demonstrate where your suggestion or choices come from.

Use your instinct. Yes, the participant might have completed the task correctly but that doesn't mean they don't have an opinion about another aspect of the website. Make sure they know that you are open to a discussion about any element of the website. Be ready for anything.

Target consumers: the test needs to be in front of the potential end users; Leon will never test his product on smartphone users (well actually he does use them for testing marketing messages but even for that, his audiences will be parents only), and i will not be talking to toddlers. 

To make sure the correct audiences come through the door, it is necessary you understand them, their behaviors  demographics and needs.And to conduct some pre-test research to make sure you’re getting the right people. This could be as simple as a quick telephone interview to ensure that they meet the criteria. 

Also, getting people to come into the lab and spend over an hour of their time is a big ask, so offering an incentive is the best way to get people interested. This could be cash, a voucher or any other valuable offer that will appeal to your participants.

You don’t need a consulting company to help you find the correct consumers. For a general consumer product, I used Userlystics to find the user for me but most of the time, i simply find friends or ask them to refer me to somebody who best fit my criteria. It has been working fine and the cost is significantly lower.  

Encourage them to talk a bit more: many times, we will not know how to fix if we don’t know why. It is better if you don’t structure the test too tight, so you have a little bit room to dig in and understand the logic behind and also the subject feels comfortable to share more; 

One thing to emphasize, nevertheless, is to give yourself the right tool to find the reason. Everybody has a different background, it is likely they don’t give you the answer you had expected (which maybe what you found to be the majority, what you had expected etc), in different situations, you can ask different follow up questions to find out the logic. For example, if testers can’t complete a function in the user test, you can ask why they did what they did, what usual software they are using that has similar function, what they had thought the correct path to be etc. 

Involve all teams: not until recently, I start exploring the benefit of involving development teams and testing teams in the user test, it used to be for product, user experience and management only. And many times technical teams even refuse to join because they think it is not useful for them. 

Well it is useful from 3 aspects: 1) it shows the team what may come next, take usability test for example, usually there are improvements to be made from these tests and developers can see it right away; 2) it allows developers to see how their products are actually used, many developers expect everybody to be as sophisticated as they are in using the product, seeing how users actually use it shows the reality and remind them to put more attention to details; 3) it brings development team to the same line of business and product, on what to focus on, what deliver the most value to the users; 

Reflect results: there is no need to reflect anything if no odds comes up, when it does,  we need to figure out what to change. Leon for example, deploy different advertisements on Google to see which one has better conversion rate, and then adjust the message or layout; I test interactions across different screens and use the result to adjust the transitions and gestures.

Not all findings will be reflected immediately, some can be fixed faster than others. Some will not be adopted, nobody wants advertisements for example, but we need to earn revenue.

It can affect multiple departments in the company which create the user journey together. 

It is never too early or too late to test.  


Pragmatic Product Hong Kong

Popular posts from this blog

4 Techniques to Make Your UX Review Meetings Successful

As a product manager, I often need to sit down with the executive management team to get their feedback on the new designs. It can be a frustrating process and many times I found that I cannot get things down in the time I am allowed to have.
Nevertheless, not having the sign-off from management is terrible for the team, we face high risk of having to re-work (yes, we always need to re-work, but it feels better if it is an improvement), schedule get delayed etc.
Over my 200+ review meetings, i've came to understand the reasons and learnt skills on how to stir the meeting towards an efficient completion, and I want to share them with you.
There are 4 key reasons of an unsuccessful product review meeting: 
1. Audience Lack the Background Knowledge: especially when introducing a new function, executives don’t know what they are looking at, or how the end users will be using such function, you may have sent the presentation before and again in the meeting invite, it doesn’t matter;

2. T…

How to Do Market Research and Competitor Benchmarking

Congratulation, you have validated your business idea from the people around you and it looks good in their eyes. You want to go ahead and start working on the idea.
No matter if you are a veteran in the industry and have good connections to help you start; or your idea came from your personal experience while you have little experience on the market.The first thing you will end up doing is market research.
Remember, you are doing the research to: Make sure the idea in a correct way;Identify what your end users are comparing toUnderstand what challenges you will face;See how you can make money;
You are not doing the research to: Copy what the other companies are doing Exhaust all potential competitors in the market
What I always to, is a 5+1 market analysis. In the beginning, I would start with a 5-day analysis with the following schedule:
Day 1: Find a list of companies that are targeting the same end users you want to approach 
For most of us, finding the list of companies that are target…

Indie Game: What Developers Should Do and PLayers Should Play

What does indie game stands for? What is the market and market growth for indie games?
Indie games are video games that are created without the support of a publisher (E.g. EA games; Microsoft etc). Indie game developers am to bring innovation to the market, they focus on digital channels (APP stores; online marketplace) to distribute the games.
Perhaps James Swirsky And Lisanne Pajot's movie "Indie Game" best visualize the community working on indie games. This is a group of entrepreneurs seeking to realize their ideal, keep refining the work they have done and hope to find others who believe the same.
I researched on the well recognized forums, bloggers, and platform operators (E.f. Steam) to construct a comprehensive list of 100+ popular indie games to try to see what criteria are important for a good indie game. 

My logic is: indie game mostly face serious gamers (people who spend a significant hours of their week playing games), there are certain special requirements fr…