Category — Research Reports
InformationWeek did a very interesting article on Practical Analysis: User Habits And Making Tablets Seem More Like Beer. They did a survey showing IT prefers HP, then Dell for tablet computing and only then the infamous Apple iPad. But how can it be possible? “Everyone knows” the iPad is the market leader! The answer, according to the article, is that IT acts very differently than consumers do, and this was a survey of IT professionals.
We didn’t conduct the survey so we can’t comment on the methodology used (although as much of the methodology as the article reveals certainly sounds sound), but we do completely agree with this: When research delivers a finding different to core beliefs, you’d better be able to explain why.
There is almost always an explanation, although perhaps it’s not as obvious as the Infoweek example. We recently did a series of projects that included questions on how various IT professionals use vendor web sites when making purchase decisions. The participants in our study were adamant that they didn’t like videos – they weren’t scannable, tended to be full of “marketing fluff,” and were generally a waste of time. But our clients’ web analytics showed that videos were viewed a lot.
The instinct was for the web analytics people to assume that the research was wrong, because they had real-life data showing that videos were popular. Of course, we had to consider the possibility, since one of the well–documented challenges with research is that people don’t always act the way that they say they will act. But we were pretty confident in our methodologies, and our techniques for getting to actual behavior, so we decided to figure out why. And as they did in the Infoweek article, the first thing we did was look at our demographic. Was there something about the demographic we spoke to that behaved differently than the overall Web audience?
Once we started digging, the answer was obvious: Our study was with product-level decision-makers – the most technical of the technical, the “sharpest pencils in the box,” the brightest ones. They didn’t like videos because they understood things quickly and preferred to scan content. But we discovered these decision-makers also had to educate the rest of their teams – and they would frequently forward videos. Since there were multiple “less technical team members” that watched videos for every “technical decision maker” that didn’t, the behavior was lost in the analytics.
If this kind of “what’s the underlying explanation” problem fascinates you the way it does us, we recommend a great podcast from HBR: Strange-But-True Research Insights.
March 29, 2011 Comments Off on When “Everyone Knows” Contradicts the Research
The real result of any research project is not the final report – it’s the growth that happens as a result of business owners acting on the findings. Too often, great research ends up shelved because there is no action – or even outright disbelief. As researchers, it’s our job to present our findings so that they influence future decision making. Of course, working with your stakeholders to understand any preconceived ideas heading into the research will help to frame the discussion, but you’ll also want to understand the personas that will be consuming and acting on the research.
We have found that there are three types of research consumers that need special attention. Each should be communicated with differently. Knowing which one you have in the room will help considerably with presenting your findings in a way that would lead into action rather than disbelief and resistance.
1. The Impatient Executive
This is the guy with very little time and strongly preconceived ideas. This individual is very smart – but be careful, they are VERY confident in their view of the world. Communicating with this person is all about the executive summary. Make sure you get to your 3-4 top takeaways within the first 5 minutes of the meeting. Then watch carefully for any reaction. If your research matches their beliefs, no problem. Present the Executive Summary, they’ll agree, and will very likely leave the room early, happy that the rest of their team is getting a good validation of what is going on.
But keep an eye out for the place in the executive summary where the eyebrows come together, the blackberry is put down, or the ever-so-slight frown appears, then make sure you spend a little extra time on that area. Put a special emphasis on the anecdotes picked up in the research for that topic. Share the one opinion that was the outlier and then frame that as the clear exception.
2. The Data Lover
This persona is the researcher’s best friend. They will love you and your findings, and will spend hours drilling down into the subtle details of the findings. The trick with this persona is to make sure they don’t derail the meeting with minutia that will bore the other attendees, or even worse, prevent you from getting to the real meat of the study. If you know you have one of these people, a pre-brief with just them is a good idea. Offering to “Take it offline to do the deep dive in this subject” is also an option – one that will be deeply appreciated by the Impatient Executive.
3. The Skeptic
This persona simply doesn’t believe in research – any research. Their belief in the way the world works is based on years of experience, and a fact or two isn’t going to override that. The trick to communicating with this persona is to start with the parts of the project that do match their world view. If you can lead off with one or two things that the skeptic agrees with, they’ll have more faith in the research and be more open to hearing the next point that is out of line with their beliefs.
Of course, that still leaves you with the most challenging scenario – the meeting with a big group of people including all of these personas, plus a few more. Our best advice – do great research so you are confident in the meeting; get that executive summary in early; ask for questions to be held until the end if you’re getting off schedule; and then do your best to be dynamic and engaging.
August 4, 2010 4 Comments
You know this one, right? The market research project with a great recruit, good feedback, and actionable information. The segments you targeted were well represented, and a comprehensive view of the problem being studied was presented.
Then it happens. The curse strikes! The discussion during the report presentation focuses on just one participant: Some individual who was dynamic, funny, and perfectly represented a specific view of one of the study sponsors.
So why is that a curse? If the fascinating participant is an outlier who represents a view that is not in line with the majority of the study, then you have a problem. The opinion of one participant should never be over-weighted, no matter how articulate they are.
A couple of tips for how to handle this:
- Make sure the client observes enough participants to balance out any single point of view
- Look for cues that you have seen the “Fascinating Outlier”. Then, in the “post-research huddle” immediately put the input in perspective. If you’ve seen enough participants, strongly point out how unusual the participant’s point of view was. If it’s early in the study, set the stage for different input with a comment like “we’ll of course have to see what turns up during the rest of the study.”
- Specifically call out the outlier’s input in the report. Have a section labeled “Outlier Input” (or “Off-the-Island Input” if your audience isn’t market research savvy) that includes an analysis of the persona, why they were different, and why their feedback was not in line with the rest of the study.
Energetic, passionate, articulate research participants are great, and outlier opinions are important to balance any study. But following these tips will help put their feedback clearly in context.
May 26, 2010 1 Comment
Dimensional Research has just completed the fourth in a series of surveys sponsored by Dell KACE. Over 900 IT professionals completed this survey on the adoption of Windows 7.
The news is very good for Windows 7. More than half plan to deploy before the end of the year with many IT departments not planning to wait for SP1 to deploy. Concerns about performance and reliability have dropped significantly since the release of Windows 7. Key findings include:
- 87 percent of survey respondents plan to deploy Windows 7 compared to 47 percent who had plans to deploy Vista at a comparable point after its release;
- 46 percent of the total surveyed revealed they have plans to migrate even before the release of SP1;
- 86 percent reported concern about software compatibility when migrating to Windows 7;
- 25 percent expressed concerns about Windows 7 performance, down from 47 percent reported during the 2009 survey;
The study reveals an interesting opportunity here for products like those offered by Dell KACE since 72% of participants plan to do their Windows 7 migration manually or using free or point imaging tools.
The press has been covering this report, with some of my favorite stories here:
You can download a copy of the full report here.
March 17, 2010 3 Comments
If you’re not confident that your anti-virus software is keeping you safe, you’re not alone.
Dimensional Research recently completed a study on anti-virus and anti-malware software, sponsored by CoreTrace. The 226 IT professionals who completed the Web survey reported that Corporate IT believes the threat from malware is increasing, but they don’t have confidence in existing blacklisting approaches to protect them. Key findings include:
- 80% say threat from malware is increasing
- 74% do not have confidence in blacklisting anti-malware
- 66% concerned that blacklisting anti-malware is not effective on “day-zero” of a new attack
- 50% concerned about the impact of performance scans
- 80% say the idea whitelisting is compelling, but only 9% report using whitelisting approaches to anti-malware
You can find press coverage of the report from:
A full copy of the report is available for download here.
September 16, 2009 1 Comment
Dimensional Research completed a new study on Desktop Power Management, sponsored by KACE. I found this study particularly interesting because so much of the conversation about power management has talked about power management in the data center, with very little discussion about the impact of leaving desktop computers, monitors, and laptops powered on when not in use.
The study reveals an interesting opportunity here since most participants (93%) think desktop power management can reduce costs, but only 10% are using a commercially purchased solution to do this. The press has been covering this report, with some of my favorite stories here:
You can download a copy of the full report from: www.kace.com/resources/Desktop-Power-Management.
September 15, 2009 1 Comment
KACE – the leading systems management appliance company and a fabulous client – just announced the results of a new Dimensional Research Web survey that revealed 84 percent of IT staff polled do not have plans to upgrade existing Windows desktop and laptop systems to Windows 7 in the next year—despite early enthusiasm from beta testers of the new operating system
Other key findings include:
- 84 percent of survey respondents have no plans to upgrade existing Windows desktops and laptops to Windows 7 next year
- 72 percent indicated they are more concerned about upgrading to Windows 7 than staying with an outdated XP operating system
- 50 percent revealed they have considered moving from Windows to an alternative operating system, and 27 percent of those cited Mac OS as the top alternative
- Almost 60 percent of survey respondents do not presently have a tool in place that automates operating system migration
- Economic factors, such as budget freezes and staff reductions, were cited as other reasons to not immediately adopt Windows 7
You can download a full copy of the report, as well as detailed information on the methodology used at http://www.kace.com/resources/Windows-7-Adoption-Survey.
Here’s a few links to some news coverage of the survey:
April 14, 2009 Comments Off on New from Dimensional Research and KACE: Survey on Windows 7 Adoption
The way we present our research findings depends on the audience. People who understand and have experience with research – especially qualitative research where you have the opportunity to drill down and really understand subtleties – are great at their jobs because they understand the importance of detail.
As a result, these people prefer very detailed reports on exactly what happens in focus groups and interviews. Reports for this audience need to have lots of direct quotes, charts, and detailed summaries, organized by participant persona and market segmentation.
Recommendations need to be very descriptive with significant backup. It’s also beneficial if these people can attend or listen in on as many live or recorded interactions as possible, especially at the beginning of the project. This enables them to feed their observations back to the researcher to make sure they capture the subtleties. A report for this type of person is easily dozens of pages long!
Executives are different. They are too busy to read a long, thick report. They want to see the bottom line. So obviously, research reports should have an executive summary. An executive summary previews the key points of the in-depth report. It is written for people who don’t have time to read the main report. An executive summary should communicate independently of the report: after reading the summary, your audience should understand the main findings of the project.
How long should the summary be? It is almost impossible to be too brief for a busy executive. A good rule of thumb is to write the executive summary, then go back and edit out half of what you’ve written. Wait 24 hours, then go back to the report and edit out another half. Seriously: one slide with three bullets in a large font is a realistic summary for this audience! Once they have their ultra short summary, they can ask more questions. Needless to say, the researcher should always have the backup to present and defend the recommendations and findings presented in the report.
While it can be difficult for people who appreciate detail to go through this process of editing and re-editing the executive summary, it is a crucial step. Researchers need to realize that if they present their findings in a way that is not appealing to executives, they run the risk of seeing the project they worked so hard on make no impact on the business, simply because the report was too detailed and the executives never really got the message.
Hmm… Now let’s re-write this post as an executive summary:
1) Be brief – very brief
2) Be prepared with backup
February 9, 2009 Comments Off on Can an executive summary be too short?
Clients often tell me, “I have lots of happy customers but none of them will say anything about my products! How do I get the story about my products out?” This is a classic pain for anyone who sells into Corporate IT. You talk to your customers all the time and they tell you how happy they are, but when your corporate marketing team asks for those stories, even the happiest customers won’t let you tell them. They have their reasons, of course: they may have internal policies, or perhaps they just don’t have the confidence to work with their own corporate communications teams for permission.
So what you can do? Prospective buyers do their homework before investing in a new technology or product. They seek out third party validation: first-hand experiences from similar companies who have already implemented the same technology in similar real-world environments. Credible, unbiased information allows these decision makers to make confident purchase decisions.
Telling decision makers vague stories about your customer success without backing them up with credible proof just won’t cut it. Your stories simply aren’t that compelling for the IT executive or CFO of a public company that has been burned in the past with products that don’t work.
We interview your customers, understand their use of the product, get hard and soft data on ROI, and capture some of their best “tips and tricks”. Our technical background means we can understand and capture the real-life benefits your customers are receiving from your products. Then we can turn that into a report that does not name those customers or their companies. It gives the customer the opportunity to tell the story without jumping through hoops getting permission, and gives you a reliable third party validation that you can use as a sales tool.
January 26, 2009 Comments Off on Proving Your Customers’ Successes
A well-presented market research report is a vital part of a company’s decision making process. Collecting the information is just the start: we must make sure it is accurate, pertinent to the client’s needs, and as clear as possible.
Successful presentations are created, refined, and delivered by experienced professionals who exercise attention and care. A good report will enable the organization to reach conclusions and take meaningful action. A poorly crafted presentation that lacks focus will leave the client confused and unable to proceed. When an entire meeting is spent talking about a side issue that was not the focus of the project, you know the presentation isn’t very good and an otherwise fantastic research project will not impact the business.
Dimensional Research makes it a priority to present findings in a way that tells a clear story and inspires the company to take meaningful action. One of the best ways to achieve this is by allowing the voice of the participants to come through in the report.
Dimensional Research includes sample participant quotes in all our reports. Since participants are ensured full anonymity, we never share their identities, but we always include in our reports participant quotes that communicate overall findings very specifically using the same language as the target buyer.
The resulting report is highly valuable to the client, because participant quotes reveal subtle attitudes or perspectives that the summary findings don’t communicate effectively. They yield useful insights by clearly conveying the participants’ motivations or pain-points and by uncovering hidden issues.
A good example from our daily lives of how participant quotes enrich research findings is the Zagat restaurant guide. Take The French Laundry restaurant, for example. Glancing at the general findings, it’s clearly a very good restaurant, since it received high ratings. But when you read participant quotes such as “fantastic enough to halt all conversation” and “wildly imaginative”, you get a true feel for the spectacular experience the restaurant offers.
In the same way, including participant quotes in a technology market research report allows companies an important glimpse into the issues. Last week I had two conversations with IT professionals who were concerned about managing their virtual environments. If the report said just that, it would be accurate, but consider what the tone of the participants actually communicated:
FIRST PARTICIPANT: “That’s a great question. I don’t know. We should be concerned, of course.”
SECOND PARTICIPANT: “We’re very concerned about virtualization. When everything is hidden, diagnostics is much harder.”
There is as great opportunity to sell immediately to the second participant. The first participant needs education first.
January 6, 2009 1 Comment