Market Research Insights for Technology Companies

One of the real strengths of focus groups – in person or online – is the opportunity for a bunch of people to see a live discussion, and even ask a few additional questions.  (Yes, Dimensional Research always leaves a few minutes at the end of a focus group session for the observers to ask a follow-on question or three.) 

It can be extremely powerful to expose people who work in corporate roles and don’t get out into the field – marcom managers, R&D, developers, etc. – to direct customer and prospect feedback. Often operational people (finance, legal) pick up something important by watching their target market discuss their jobs.

 Dimensional Research always encourages as many people as possible to listen into focus groups, or to watch the videos that we record when they’re done.

 However, sometimes “listening in” is a bad idea. It basically boils down to this:  If you only see part of a project, don’t assume that’s all there is. There is a reason why you conduct 8 focus groups, not just one.  Or why you conduct 25 interviews, not just 3 or 4.

Don’t let these scenarios happen to you:

• Attend two focus groups in New York and project that experience onto Chicago, Paris, Singapore, and Tokyo.
• Listen in on only one call of a 20-call interview project.

If you decide to listen in on market research, I strongly recommend the following:

1. Do read the final report and attend the presentation of the report.  You might as well enhance your limited experience with the full power of the overall project.
2. Don’t attend just one focus group or listen in on just one call!

Real life example

We recently conducted a series of 15 customer interviews about a client’s new initiative. It has been progressing for about a year and they wanted to know what messages their customers had absorbed. We spoke to 15 of their very best customers – the kind who spend lots of money every quarter, attend the user groups, and give references. It was a good study, and very helpful in finding out what parts of the new initiative were gaining traction and what parts needed even more evangelism.

There was one of the 15 interviews where the participant absolutely “got” it. He could have given the company’s pitch, including now and vision, with no problem at all.  It was delightful.  However, he was the ONLY one of the 15 participants who did that. The rest of the participants clearly struggled with some of the visionary aspects of the messaging. As luck would have it, that was the only interview that one of the project stakeholders listened in on. Unfortunately, during the report presentation he kept interrupting to talk about how the market “really got it.” We had to very strongly emphasize that the whole project needed to be considered – not just this one guy. The company had plenty of work to do to reach their entire customer base. They were not done.

Focus groups are ideal when you want to gain multiple perspectives in an interactive group setting.

However, one of the things that can happen in focus groups is “group think.” The first person starts off on a tangent – good or bad – and then the entire room goes along with that first opinion. This is a dynamic that every good researcher is aware of. 

So does groupthink invalidate a focus group? Of course not, and in fact it can be very useful as long as you know it’s happening and stop it when necessary. Here are a few tips:

1. First, establish if group think is a good or a bad thing for your project.  Depending on your goals, group think can be a very bad thing. For example, in message testing you’re usually evaluating the kinds of pain/benefit statements that someone experiences in isolation. Groupthink is very damaging in that kind of environment because you end up seeing only one participant’s uneducated, unguided reaction to messages.  In this case, maybe focus groups are not the right medium – in-depth interviews or a Web survey may give you better results.
2. Remember that group think can be very good.  If you’re trying to brainstorm an idea, having somebody to kick off the conversation and then see where the group goes with it is a very good thing. A smart moderator will encourage this kind of thinking for a while and see where it goes. The most creative ideas from focus groups I’ve been in did not come from one single individual. They came from the group members inspiring each other to build off of each other’s experiences and ideas and come up with something more powerful than any of the individual input. When one participants says “what if …”, and then the next participant says “cool idea, but how about this…” and then yet another participant says “or even better, you could take that idea and try…”.  Now you’re cooking!
3. Be prepared to funnel groupthink.  When you see it happening, a good moderator will challenge the group with the opposite assumption.  Prompt some “group think” around the positive ideas, and then redirect the group to talk about the negative ideas. Insist that the group gives you multiple perspectives.

Above all – don’t be scared of negative group think! It’s very good to hear every possible objection your target audience can make about your solution BEFORE you start selling, so you can be prepared with the products, messages, and objection handling you need to be successful.

Negative feedback is good for you. 

Seriously.

While it’s very human to only want to hear how wonderful your solution is, you need to learn about any objections to your product NOW. Not later, when you have to deal with an entire slipped pipeline, because you didn’t realize there was a deal-breaker.

You need honest feedback, and you need it now.

But how do you ensure feedback that is real? Very few people, no matter how stupid they think your idea is, will actually tell you that. You have to create an environment that allows people to tell you about the downside. 

One of the more painful exercises you can go through is to put a group of people together in a focus group and ask “what DON’T you like about this solution?” Once you ask them this question, and give them permission to be brutally honest, they WILL tell you about everything they don’t like. And the list of things they don’t like will be very long. And their list will inspire the other participants to think about even more things that they don’t like.

But as I said, this is GOOD FOR YOU!  If you know about the objections, you can prepare for them in advance. In the long run, this will save you significant amounts of time and money.

The answer is “enough to represent your market.” This number varies significantly according to the type of market research you’re conducting.

Web Surveys

For Web surveys this is a pretty straightforward question to answer. You will use quantitative methods to determine a sample size.

There are standard ways to calculate statistical validity, and a very easy-to-use calculator and descrition of the underlying statistics can be found here. You can use it to determine how many people you need to respond to a quantitative study in order to get results that reflect your target population.

You will need to know the population size of your audience - the total number of people  in the group your sample represents. Even if you don’t know the exact population size, this is not really a problem. The mathematics of probability allows you to make a pretty good guess as long as you have some kind of basic idea. The number of participants needed is not linear. As your audience gets much larger, your sample size doesn’t increase very much.

For example, to get a result that is accurate 95% of the time within 5%, you need:

Qualitative Studies – In-depth Interviews and Focus Groups

But enough statistics homework for now. The question that is less straightforward to answer, and so is asked all the time: “how many participants do I need for a qualitative study?” such as focus groups or a series of in-depth interviews.

The first thing to do when answering this question is to figure out segmentation. How many types of participants do you need to represent? This can include verticals, countries, roles, years of experience, customers vs. prospects vs. competitor customers vs. partners, and so on.  The most common segmentation when working with Corporate IT is to have two groups: “technology decision-makers” and “economic buyers”.

Once you figure this out, and map any overlap between these areas (for example, partners may also be end-users), you’re ready to go.

In my experience, with corporate IT you need about 8-10 participants of each “type,” with a minimum of 10 participants, to produce a valid study. The only exception is studies with competitor’s customers, where you typically need more participants.

Note that most market research companies, including Dimensional Research, base pricing for qualitative work on the number of participants, so doing a good job right-sizing your project will give you the most bang for your buck.

According to a recent survey from a specialist PlayStation 3 site, geeks make the best lovers.

Let me clarify right from the start that as someone with distinct geek-ish tendencies, who is married to a confirmed geek, I have no intentions of arguing with the study’s conclusions – just with its methodology.

The problem with the survey is that it was biased. Given the audience likely to peruse a PS3-specific site, it’s fair to assume the respondents skewed more toward the “geek” end of the spectrum.

When conducting research you must know who you’re talking to in order to interpret the conclusions correctly. If they had done the same survey with a running site, would they have found out that marathoners are the best lovers?  

In technology market research, identifying your research target audience is especially important.  Customers will give you different insights than prospects, employees, partners, or even the customers of direct competitors! 

If you’re talking to your customers, you need to acknowledge that by definition they have the pain that you’re selling to, or they wouldn’t be using your product. So, you shouldn’t do a customer survey and then announce, “100% of the MARKET does this.”  Of course, it’s completely fair to say, “100% of our CUSTOMERS do this.”

In comparison, if you’re talking to the general market who isn’t as fully educated on your solution, you’ll need to tailor your questions accordingly. If you ask questions that are too detailed and specific to your solution, you’ll get uninformed answers. The type of questions that are appropriate for customers who are familiar with your product, are not the same ones to use for the general market.

For example, if you’re asking about product features, you will get vastly different answers from existing customers who know your product, and from your competitor’s customers who know an alternative approach. Both of these will be different from the answers you’ll get from prospects who don’t have any product experiences.

As with everything in market research, it comes back to goals. Your very first step should be figuring out your business goals. Then, figure out what you need to know in order to achieve these goals. Next, figure out who has that knowledge, and who doesn’t. Only once you’ve answered all these questions, you can dive into the details of the market research project.

So, to be honest- it really depends.  But I know that’s not an answer to a very pragmatic question, so let me give more details here, bearing in mind that it really does depend on your specific project. 

A typical market research project takes six weeks. This is pretty standard for a small to mid-sized project, say 10-30 in-depth interviews or 4-8 focus groups.  Larger projects take longer, of course.  And we’ll talk about Web surveys later.

Here’s a pretty typical schedule for a single stage project with one series of focus groups or in-depth interviews - excluding unforeseen “bumps” or unique requirements:

Week 1: Project go-ahead. Write and approve recruiting guide. Identify source for recruiting participants - internal or external.

Week 2-3: Recruit participants. Write and approve interview guide or moderator’s guide.

Week 4-5: Conduct research. This may take less than two weeks, depending on your goals. Four to six focus groups all in the US are usually done in just one week.

 Week 6: Write and present market research report.

WARNING: The biggest schedule slippage that happens, aside from getting all the approvals in place for the project go-ahead of course, is with identifying internal participant contacts.  Using in-house lists or asking account managers to give us their contacts for recruiting may be time consuming and the schedule should be adjusted as needed.  

Web surveys are usually faster. A typical schedule for a web survey project is three weeks:  

Week 1: Project go-ahead. Determine goals. Identify lists for participation.  Write and approve questions. Upload them to survey tool.

Week 2: Field survey.

Week 3: Close survey. Do analysis including filtering and correlating findings. Write market research report.

If the survey is being conducted as a collateral piece to support outbound marketing efforts such as PR or lead gen, add another week for copy-editing and layout.

One of the most important skills in moderating any market research discussion is to make sure you’re not “guiding the witness” to tell you something that is not accurate.

I’ve discussed here the importance of asking market research participants open-ended questions such as “can you tell me more about that?” and “is there anything else?”  I also written about the importance of getting honest feedback, and suggested ways to make sure you do.

Today I’d like to emphasize the importance of giving the participants permission to NOT know.

Market research participants are smart, competent people who know what is going on in their environments. It’s natural that they’re only human and don’t want to come across as unknowledgeable. Nobody wants to admit, “I don’t know.” 

However, YOU need to know the truth. If they don’t know about a product, you need to know about that and adjust your marketing efforts accordingly. It’s just as important for you to uncover that participants are unfamiliar with a product or an idea as it is to know their opinion about it.

The key is to ask questions that allow a participant to tell you the truth without feeling like an idiot.

A question that is in many research guides is an awareness question. “Are you familiar with product X?  What do you know about it?”  I don’t like asking the question this way. It almost guarantees that some selection of the participants will answer, “Yes, I’ve heard of it. I read about it in a magazine” just so they don’t have to admit they have never heard about the product.

Instead, ask “Are you familiar with product X – and please feel free to say no, that’s just as important for me to know.” This will give the participant permission to not know without feeling stupid. 

Another option is to ask, “What do you know about the following product – if anything?” that also leaves the door open for participants to admit they don’t know without worrying about disappointing you.

Online focus groups are a highly effective market research tool.  My clients love them.  They save the time and cost of travel while making it easier for participants and observers.

Most importantly, these benefits don’t require you to tradeoff for lower quality research. Online focus groups still facilitate the “brainstorming” dynamic that you get at an in-person focus group, where one participant’s comment sparks a comment by another participant and so on. The brainstorming dynamic enables the group to dig deep into an issue, so that the client can get a very clear picture of market acceptance of their product and of any issues they need to deal with.

Participants love online focus groups too, because they give them an opportunity to listen to the feedback of others in the industry. This is an important benefit for participants, especially in customer advisory boards, which encourages them to join.

Online focus groups are conducted using conference calls and Web-based meeting technology such as WebEx, GotoMeeting, etc.  In many ways they’re similar to traditional face-to-face focus groups, except you’re not in one room. We recruit participants in the same way, set a time for the meeting, and prepare a moderator’s guide. If relevant, we can review a presentation with the participants. Just as in a traditional focus group, where the client can listen in through a one-way window, the client can listen to an online focus group on a muted phone line. And we can use Web technology to poll participants similarly to a visual “show of hands” or “nod of heads” in an in-person group.

Despite the many similarities between in-person focus groups and online focus groups, there are also a few differences. The most obvious one is that you don’t get the non-verbal clues from the participants. Because of this, the moderator needs to make an extra effort to draw out all the participants and balance the input. To facilitate this, we typically have a smaller group (4-6 participants per group rather than 6-8 for an in-person group) for the same amount of time. We may do more groups as a result – 5 instead of 4 for example. 

Of course, there is also the option of conducting video online focus groups.  As technology evolves, these are becoming more common and are easier to set up.

Another major benefit of online focus groups is that they reflect the majority of real-life interactions with customers. When selling to Corporate IT buyers, more and more meetings are being conducted online rather than in person, so online focus groups reflect that same scenario.  If your messages work in an online focus group, they will translate well into an online sales call or a Webinar.

Are you a technology professional? Do you like sharing your opinions? You might enjoy being part of a Dimensional Research Project.

The only real qualification is that you work with technology in your profession - we’re especially interested in people who work in corporate IT.  We welcome participation from around the globe!

Important information for participants:

• This is market research – no one will every try to sell you anything.
• Your input will be anonymous. Nothing you say will ever be attributed to you or your company in any way.

What do you have to do?

Register here (you may need to scroll down a bit). We’ll ask you a few questions about yourself. Then, as research projects come up that you seem to fit, we’ll contact you with some more detailed questions and information about the specific project.

What is involved?

The actual level of participation depends on the project. For our Web surveys, we’ll send you a link and you can go online to complete the survey. For in-depth interviews we’ll contact you and schedule a time to talk – usually about an hour. For focus groups we’ll let you know when (and where, if necessary) and see if you’re available – usually about 1½ – 2 hours.

What’s in it for you?

This depends on the project. For a Web survey, it may be a copy of the final report or a chance to win a cool prize. For a focus group or interview usually it is a cash stipend payable via Paypal or US dollar check. And of course, with all projects you have the knowledge that you’re influencing the future of technology in some way!

REGISTER HERE>>

We just completed an extensive series of in-depth interviews with great IT participants from around the globe. These participants were smart and articulate. They knew their job, and were very good at it.

But as smart and capable as these participants were, we discovered during the interviews that many of them – at least half -  held a completely false belief about my client’s product.

During the interviews with these participants, they kept repeating the same idea.  The details are confidential to that client, of course, but the important point is that this audience really believed that their perception of the products was true. This wasn’t a minor technical detail – it was a major assumption about environments the products operated in.  But the problem was IT WASN’T TRUE! This was verified by my client, by independent evaluations that we found while investigating the conflict, and by some of the other participants in the study.

So here you have a very well established fact about a product, but at least half of the customer population believes something else. In other words: even if something is not logical, and even if your customers and prospects are technology vendors who tend to be logical, they may still believe in something wrong. And it could negatively affect their buying decisions.

This type of dynamic is extremely important for vendors to understand. If there are things that your audience believes about your solution that are not true, you’d better know about it so you can take action.

You’ll notice that I’m referring to this as a “belief”. Beliefs are harder to respond to than competitive FUD or outright lies.  These are things that your target market thinks are true because they’ve never really thought about it.

A formal market research methodology is the best way to uncover and to root out these false beliefs across your target market. You can’t rely on day-to-day conversations with customers, because if these issues do surface during such a conversation, you tend to correct them immediately with that specific customer and then forget about them since it didn’t make any sense to you to think that way.