Category — IT Participants

There are many ways to make sure your online survey is efficient and effective. One of them, of course, is to avoid asking bad questions.  

Tip 1: Craft your questions carefully to avoid unwanted results 

This tip was inspire d by Seth Godin. He explains that “Every question you ask changes the way your users think. If you ask, ‘which did you hate more…’ then you’ve planted a seed.” 

Mr. Godin makes a great point. I recently booked a trip with Travelocity. It was a great trip. I was happy.  I did have a small issue with the airport transfer getting home, so I filed a complaint to see if I could get a refund.  It was a small matter – about $30 credit - so I wasn’t too worried about it. Travelocity’s reply email was pretty typical, asking me for more information (which I had to get by opening up the email THEY sent me, so a bit annoying). 

BUT… Then they sent me a survey asking about my experience with Travelocity.  One of the questions on the survey was “Do you know about Travelocity’s guarantee that your booking will be right, or we’ll work with our partners to make it right, right away?” I actually didn’t know, but this question clearly did NOT describe the experience I just had. 

Travelocity did make it right – it took them about 30 days to do so – but that wasn’t what their guarantee said and their survey pointed that out to me.  By including that question in the survey, they planted a seed that they didn’t want to plant and I ended up being less happy overall than I was before the survey.

Tip 2: Ask at least one question that participants actually WANT to answer

It’s important to ask the questions your customers are actually interested in answering.  Too often, marketing departments are so focused on the company’s newest offerings that they ignore the products their customers have come to depend on.

I use QuickBooks for my business. I have LOTS of feedback for Intuit on the core QuickBooks product, but they never ask me about that.  They constantly survey me, asking if I want to buy checks or do payroll or take credit cards, but there is no “Thanks for your time, is there anything else you’d like to tell us?” that would enable me to give them the feedback I WANT to give them.  

Another example from my own professional life: I use Zoomerang for my surveys. It is a GREAT product – with a few caveats.  One of the problems I have with the product is that they have a horrible interface for “choose the answer that most closely applies.”  It’s a small button that’s almost impossible to see. On the other hand, their interface for “choose all that apply” is great – a nice square with a big check mark.  I want to tell them about this issue, but they keep sending me web surveys about other things and I’ve never had a chance to give them important feedback that I really want to give them. Maybe they’ll read my blog and I’ll get to them that way?  <CORRECTION:  They did in fact just send me a survey this week that allowed me to give that feedback.  I’ll wait and see if they act on the feedback.>

Tip 3: Reward your participants, wisely

This tip is inspired by Patricio Robles: “offer users who respond to a survey a discount, an entry in a drawing for a prize, something of value. It will boost response rates and make them feel like they’re investing their time wisely.”

While I generally agree, I would add that when offering a reward to participants, it’s important to consider your target market. A gift that is too nice motivates people who aren’t qualified to complete the survey. Then you have to wade through junk or set up lots of qualifying questions to weed them out. If the reward is nice enough, some people will game the system and try to guess what you’re looking for, so the filtering questions won’t always work. 

When you’re doing a technology web survey, this is especially important, since people who don’t know your topic can really throw off the results, as they are not informed.  If possible, giving a copy of the final report on a topic is a great incentive, since only people who know and care about the topic will respond to this type of reward, keeping your input very clean.

Dimensional Research clearly stands apart from other market research firms because we are technology experts, performing market research for technology companies.

But what exactly does it mean to be a “technology expert” when it comes to research? Obviously it doesn’t mean we’re deep technical experts in every possible area of corporate IT. We don’t write code or manage developers. We don’t manage data centers (although we do our own IT and can commiserate with some of the pain!). We don’t personally install patches or upgrades or test new releases of enterprise applications.

What we’re experts on is Corporate IT and how it works. We are very familiar with Corporate IT and its processes and can talk comfortably and knowledgeably about:

1. Technology. Not the actual coding and implementation, but the types of technology and the tradeoffs.  We understand the perceived benefits and potential downsides of a service-oriented architecture.  We know what cloud computing is (or at least, know all the ways the industry uses the word) and what is real and what is vision.  We understand the challenges with data and the effort that is needed to generate simple graphs and reports.  We know the TLAs (Three letter acronyms) – from IDE to BSM to SDL to SAN (Both the storage one and the security one!)

2. Processes. We know that ISVs and SAAS companies purchase differently than financial service organizations and that those are different than government and non-profits.  We know the software development lifecycle, the people involved, and the products used. We are familiar with ITIL and know how some organizations treat it as a guiding principle, others as one of many guidelines, and some just ignore it.  We know the new focus on costs and services in ITIL v3.  We also know IT has other systems – Six Sigma, COBIT, and more that need to be considered

3. People. We know the common organizational interactions that take place between IT and the business owners; between applications and operations groups; between development and testing; and between server and network teams.  We love talking to engineers and technologists about their challenges, and appreciate the creative side of a job that is often considered to be purely technical.

Most importantly, because we have this foundation, we can quickly come up to speed on any new technology. 

• On a recent security project we had to learn new vocabulary like “pen testing” and “cross-site scripting”. It didn’t take us long, because we already knew the development and testing processes and quickly understood where security fit into those.
• On our first data center automation project, we had to learn about different types of automation.  But the products automated processes we were already familiar with, so it was a very straightforward learning curve. 
• When we did our first cloud computing project, our foundational understanding of virtualization, systems management, and application management meant we were able to be productive right away.

Being technology experts means we “get it.” Whatever technology our clients are developing, whatever processes they use, we are very comfortable talking with IT using professional terminology, asking the right follow-on questions, and gaining the respect of the participants. The result: an efficient and focused market research process that generates a highly effective analysis.

Getting your customers to participate in a customer advisory board isn’t too difficult: customers love to sit on advisory boards.  They appreciate not only the opportunity to be heard and to have a relationship with a key vendor, but also the opportunity to hear what’s happening with their peers and to make important industry connections with your other users. 

However, ongoing advisory board programs do require a level of commitment in order to develop a great group dynamic. Typically we ask for a one-year commitment which includes four quarterly meetings.  My personal favorite is to kick the year off with an in-person customer advisory board meeting and then conduct three online meetings.

The best way to keep customers engaged with such an ongoing advisory board program is to show them that their feedback is actually impacting the company and its products. One way to do that is to start all customer advisory board meetings, except for the first one, with a summary of how the feedback from the previous meeting has impacted the company and its products.

But you can go even further and engage your customers even more. When I ran marketing at Freshwater Software, we had a product-focused advisory board (the “ThinkTank”) that was held bi-annually.  Participants were asked to come prepared with their top three product requests. During the meetings, we spent time reviewing those requests and prioritizing them. 

We used to end the meetings by voting on the top three features the group wanted.  We committed to customers that if they took the time to participate, we would guarantee that at least one of their top three features would get into the product. We were often able to get more than just one of their top features into the product. But the important point is that this kind of commitment drove active and passionate participation in the advisory board meetings.

“Thank you, I really enjoyed our conversation.”  This is the typical way in which customers end a market research call or customer advisory board meeting.

It still amazes me how many people love being involved in market research – especially customers.  They don’t see it as a chore or just another thing on the “to do” list. On the contrary, they love being involved and appreciate the opportunity to air their opinions.  Even when I talk to unhappy customers or to participants who clearly don’t connect to the pain we’re testing, they often end a call with sincere gratitude for being given an opportunity to be heard.

This is particularly important because people are often worried about asking their customers to participate in research.  Your company probably asks a lot of its customers already. They are asked to be sales references, PR references, and beta testers. Many companies are hesitant to add a research project on top of everything else they ask their customers to do.

But companies need to realize that customers LOVE being asked to do market research. Not all of them of course, but the ones that are not interested will say no quickly.  Most customers feel that being involved is empowering. They appreciate the fact that they get to voice their opinions anonymously and that there is no preparation on their side and no action items. In fact, involving customers in market research is a great way to increase customer loyalty.

Another incentive for customers is the participant stipend or honorarium – the money we give participants in appreciation of their time. Cash is always nice of course, but Dimensional Research also gives research participants the opportunity to direct their stipend to a charity of their choice. Just in the past month I’ve learned about several great causes through the choices of our research participants.

Happy New Year! 

I’d like to kick off the year by giving a special shout out to the research participants in 2008 who chose to donate their honorarium to a charity.  Dimensional Research gives all research participants the opportunity to direct their stipend (typically $50-$250, depending on the project) to a charity of their choice.

2008 was a great year for philanthropy among the IT professionals we had the pleasure to work with. From the big charities we all know, to the operations manager who just said “I like dogs, you pick”, here are a few of the organizations that we had the pleasure of learning about through our participants in 2008:

• St. Jude Chidren’s Research Hospital
• American Cancer Society
• Autism Society of America
• Boys & Girls Club of America
• United Way
• American Heart Association
• Capital Area Humane Society
• Camp Smile-a-Mile
• National Resources Defence Council
• Open Voting Consortium
• UNICEF
• Sunnyvale Community Services
• Heifer International
• UK Dogs Trust