Skip to main content

5 Things You can Do to Make Usability Test Part of Your Culture

Most of the times, the usability test requests come from the marketing team, shortly before a product or a major release is about to launch. 

It goes like this:

About the time when the product is about the launch, the marketing team comes to the executives, everybody is having these little butterflies in their stomach. We have all too business on everything we have to do, planning our marketing events, developing all the features, that we forget our users.

We did the user tests, well, focus groups actually, and we found a long list of problems users encounters when using our products. And we prioritize them based on the time needed from the development team to fix the issue: if it is the change of a label, priority high; if it is changing of an interaction, priority low etc.

Everybody is frustrated with the process: product team knew the test is too late for the current release, it should have been done much earlier; marketing themselves are upset as they are worried the product will not get user acceptance for all the problems we spotted from the test; development team are stressed as now they have more things to change and fix that they don’t know why (they are not invited to the team of course) on top of the zillion other things they have to do before the launch; 

And the CEO is shucked, s/he can’t believe the number of problems spotted from the test, some of them looks serious. 

Throughout my product manager life, I’ve seen the same story again and again. It is an inefficient and frustrating experience for most of the people involved in the process.

But why don’t we do something different, what stopped us from doing what we should do:

Executives Don’t Support Early Usability Test: cost is one of the main issues, hiring a professional testing company can be expensive, the last time we used TangUX in Shanghai for 4 test sessions on 12 people, it costed us more than half a million RMB (~80K USD). Privacy is another issue I faced, especially for new product, company can be very sensitive in any kind of test involving external parties even through the confidentiality agreement is signed.

Team Don’t Have the Skill to Execute: besides PM, the product team is actually not that busy, the requirements are given and assets produced, but they has very little knowledge on testing, we have brilliant business analysts and graphic designers, but they have no idea how to conduct user tests nor to analyze the findings; 

Those Who Knows Don’t Have Time to Teach Those Who Have Time: as a product managers, I have done all kinds of user test from Focus Group to Usability Benchmarking, but I am so busy trying to clean my to-do list and I have no time to write screener questions, prepare test scripts and do beautiful presentation for the executives. 

After being through the same process 3 times, I came to realize the daunting pattern:

Why is cost high, because we need to hire professionals, they are expensive;
Why can’t we do in-house, because we don’t have the skills;
Why don’t we have the skills, because we didn’t train the people;
Why didn’t you train the people, because I don’t have time;
Why didn’t you do it when you have time, because management don’t approve such a project at that time;
Why executives don’t approve such a project at that time, because they don’t see the need;
Why don’t they see to need then, because we are going to market now;
… …

Don’t we always want to go to market?

… …

I’ve came to realize we will just fall into the routine again and again unless I make an effort to broke the chain. And I’ve been doing 5 simple things which gives me tremendous benefits on making user tests part of the product management cycle:

Create Artifacts at the Best Time: the first 3 things I when I get my hands on a product is to prepare the screener questions, the focus group questions and the test scenarios. In the product envisioning stage, I will do screener questions and focus group questions, these are more generic questions to define the direction of a product; as I start developing each feature, I will write focus group questions and test scenarios, here the scope are limited to the respected feature; timing is very important because, in fact, I’ve done similar thinking and wrote them down in the requirement and PRD, so it just take me extra 10-15 minutes to create a Confluence page or Google Doc and format them for the use of user tests;

Beat the Competitors and We Buy You Lunch: everybody loves to look at competitors, it helps us know our place in the guard scheme of things, know what users recognize us as etc. So I decided the marry this need with my need for competitor research and organize bi-weekly brunch on competitor tests. I need to write script of course, but I need to do it as part of the competitor analysis anyway, and I got to use the office budget for team activities to provide lunch to everyone;

Piggyback on the Giants: look, executives don’t give a damn about products, they only care about revenue, ROI; but what they do care, are what other executives are doing; I subscribe to the search term “usability test” from Google Trends and I forward the interesting articles I found related to executives endorsing usability test to my executives. Be careful not to use the one that end up marketing their product though, you maybe misunderstood as trying to promote their products. Alternatively, you can always find tons of articles from the big consulting firms (McKinsey, Bain etc) that continue to come out with the “food for thought’ article that just so helpful for you to make a point. Don’t forget to write a comment using the key words they are (e.g. meet consumer needs; improve ROI; reduce cost etc). For example: “Great McKinsey article on how banks and large retailers using user tests to meet consumer needs

Outsource, Delegate and Automate: to do the above 3 I have a task list that needs to be done at different times:
  • Write screener questionnaire;
  • Write tester recruitment advertisement;
  • Write focus group questions;
  • Write test script;
  • Find a list of competitor products;
  • Collect and analyze results;
  • … ...

First I see if I can outsource or delegate to somebody else, maybe not now, but in the future; and what is needed to enable it. After that the list becomes sometime like:
  • Write screener questionnaire;
  • Write tester recruitment advertisement template;
  • Write focus group questions template;
  • Write test script template;
  • Review scripts;
  • Collect list of competitor products and scripts;
  • Read Google trend on usability ROI; 
  • … ...

Then I put the list in my calendar and reserve the time for them so I get to them and complete it.

If you can’t, well then read the next point.

The More People that Do it With You, The Easier it is: well, the last point is not really a serpent point, but it is one of the most important one, I consider myself a very disciplined person but even that, I get swapped by work and I just can’t continue, but that’s why we have a team right? My team member feeds me with competitor links, they share me articles that I can pass on to the executives; executives share good product that we test together; I can be a happy librarian centralizing the knowledge and pass them on.

As Kilian, Serrazin and Yeon stated at the end of Building a Design-driven Culture”: Customers increasingly expect products and services that are designed to meet their needs, delight them with unexpectedly great experiences, and address a heightened sense of aesthetics. Companies that meet those needs are rewarded with fierce brand loyalty and higher spending, which translates into fatter profit margins. But that kind of success only happens by design. 

And above all, test my words out, see it works for you. They have worked out for me, 

I would love you can drop a line or two in the comment section to give me some feedbacks. 



Building a Design-driven Culture, McKinsey

Popular posts from this blog

4 Techniques to Make Your UX Review Meetings Successful

As a product manager, I often need to sit down with the executive management team to get their feedback on the new designs. It can be a frustrating process and many times I found that I cannot get things down in the time I am allowed to have.
Nevertheless, not having the sign-off from management is terrible for the team, we face high risk of having to re-work (yes, we always need to re-work, but it feels better if it is an improvement), schedule get delayed etc.
Over my 200+ review meetings, i've came to understand the reasons and learnt skills on how to stir the meeting towards an efficient completion, and I want to share them with you.
There are 4 key reasons of an unsuccessful product review meeting: 
1. Audience Lack the Background Knowledge: especially when introducing a new function, executives don’t know what they are looking at, or how the end users will be using such function, you may have sent the presentation before and again in the meeting invite, it doesn’t matter;

2. T…

What Changes will Machines Bring to us - As Employees

Machine learning to the employment has been a topic in debate. Darrell West, in his paper titled "What happens if robots take the jobs? The impact of emerging technologies on employment and public policy” suggested a list of actions government should take to ensure people whose job has been replaced by machines can live a decent live. The general sentiment seems to suggest a turbulent era as work force transform.

Growing up in China during the time of State owned enterprise reform, I had real experience living through the time of large group of people being laid off because the jobs were suddenly gone. My parent’s generation had to learn new skills for a completely new industry at their 40s and 50s. Few of them made it and even became millionaire, many of them didn’t and the family suffered a lot. I followed the news of Detroit Car manufacturing industry lapse and it shows familiar traits. The fact is, jobs come and go all the time, employee as a group will constantly adapt while …

21 Tops on How to Write a Successful Blog

Hubspot and General Assembly came together to offer a 10 week planner for successful blog. The type of blog discussed in this plan are corporate blogs used to bring people to the site and explore what the company is doing, potentially generate a lead.

Identify Your Target Persona: talk to sales team and research contract dataStart Building Evergreen Content: start with evergreen contents that stay relevant though time; do keyword search to see what people are searchingChoose the Right Content Management Tool: a good tool is easy to use and allow users to track metrics such as conversion rate, page view, and where traffic come fromDesign Your Blog: consistent layout; Focus on Your Content Strategy: basically it is depending on what you wantSet Subscriber Path: there needs to be a workflow for emailing the subscriber, a subscription form and an unsubscribe form. Hubspot is towards the "don’t email your subscribers too often” group while I also heard UI Breakfast Jane Portman talked …