Surveys: Design for Insights and Action

It started as a simple tweet: can you help and fill out this survey?  We’ll share results to all who participate.  Interested in the topic, I clicked on the link only to find a very short, poorly designed survey.  The survey had seven questions (not counting the email capture for results sharing) of which several were closed ended essentially binary questions — i.e, “yes”, “no” and “maybe” were the only answer options.  There were two open ends to collect unaided recall of favorite cause marketing programs and past participation in cause marketing programs.

I quit the survey.  I am interested in learning about best practices for corporate/non-profit partnerships.  But, this survey didn’t address that question. It wasn’t designed to gain insights and inform companies or non-profits on best practices as its introduction suggested.  There were no questions about effective strategies/practices used by corporations and non-profits in cause marketing campaigns;  no sense of how effective these partnerships are or could be; no lessons learned from these partnerships; or even whether the respondent was from a company or a non-profit.  That kind of survey would have been meaningful and would have uncovered the necessary information to report on best practices.

Instead, this survey was essentially a poll to gauge the temperature of its respondents on several cause marketing issues.

I’m not opposed to polls.  In fact, I find them very useful to take a temperature.  And, when well designed, can also provide great insight.  For instance, this morning on Twitter, I learned of this poll on geolocation services:

But, sadly, the survey on cause marketing best practices won’t even be able to provide this level of insight because too many questions were worded with “yes,” “no” and “maybe” responses.  And, more fundamentally, the survey design did not align with the stated survey goals — improving cause marketing efforts and informing companies on cause marketing best practices.

With the proliferation of DIY survey sites like Survey Monkey, Zoomerang and Survey Gizmo and social networks, anyone can write a survey and send it globally.  But, before doing so, it’s helpful to design for insight and decisioning.  Specifically:

• Be clear on your objective.  In other words, what is the purpose of the survey? What is the business problem or question you are trying to answer?  And, what will you do with the results?
• Collect information that meets your objective. That is, what information do you need to answer the business question? For instance, if you want to establish an understanding of best practices, you need to collect information on best practices — e.g., what are people doing, why does it works, who is doing it, etc.  This is information forms the backbone of your survey and your analysis.  It will be the building blocks of the story when the data comes back.
• Align your questions with your research objective and information needs.  If the survey question doesn’t add to your understanding of the business question, don’t ask it.  And, ask the right kind of question.  Think about when you meet a new person and you want to draw them out.  If you only ask  ”yes” or “no” questions, you won’t learn that much about a person. It’s the same for surveys.
• Survey the right people. Sample design is an area that many people ignore.  They think they can get a response from anyone.  If you want to know what any one thinks about a topic, you can certainly ask any one.  On the other hand, if you are trying to make a business decision that impacts only a segment of the total population, your survey should interview that segment.  That means, you’ll need a screener to identify that you have the right survey respondent.  Not all of these DIY survey tools allow you to screen so its a good idea to check.  If you are in doubt about the value of surveying the right respondents, just look at political polls.  Some polls — particularly from news organizations, ask anyone who is 18 years or older; others ask if the respondent is registered to vote; the best at predicting election outcome are the ones that survey likely voters.

Of course, there are many other best practices for an effective survey.  These four elements — objective, information requirements, question design and sample — though are the most critical.  Otherwise, as an early research mentor told me:  ”garbage in; garbage out.”

And, you?  What advice would you provide?