Online surveys? They seem to be taking over as a principle market research data collection method. Market survey research design tools, survey software, and market research web hosting suppliers make it relatively easy for most anyone with an understanding of marketing research design issues to implement an online survey.  The issue is whether web marketing surveys are valid in their design, measurements and sampling.  The key to online survey quality is to look at principles of other self-administered survey methods.

An Online Survey Story

To everyone, the online survey looked too easy.  I've written scores of survey questionnaires, so a short questionnaire helped make it go fast.  The capabilities of our online survey software did make it flow well.   The point is that available technology makes for ease of writing questions, presenting visual material, and implementing piping and questionnaire logic.   The speed advantage of implementing and fielding an online survey market research study can be a seedbed for sprouting quality problems.

Genealogy of Online Surveys

• Online surveys are in the class of self-administered surveys. 
• Prior to internet and web-based online surveys, self-administered surveys usually meant some form of "paper and pencil" questionnaire. 
• In marketing research, a popular method in this class was the direct mail survey. So, a useful guideline comes from the marketing research literature developed around mail survey methods. Online surveys, in a sense, have applied internet and HTML capability to the now seemingly old-fashioned mail survey. 
• An expert Paul Erdos wrote the bible on direct mail surveys, and his book, Professional Mail Surveys, lays out principles which remain valid today for online surveys. 
• As the personal computer took over the desktop in business, disk-by-mail questionnaires evolved as a hybrid of mail and computer technology. 
• Here, we employed the same approach as a mail survey, what with waves of advance letters, monetary incentives, reminders, and "thank-you" letters. Except for one thing: the paper questionnaire was replaced by a floppy disk which the respondent used to display an "offline questionnaire", respond, and then mail back in a provided pre-paid envelope. 

Quality Determinants for Self-Administered and Online Surveys

From this market research history of self-administered surveys using mail and offline surveys, we uncover some common environmental principles that apply to online surveys as well.   We found that a self-administered methodology worked well in these situations:

• Moderate or high involvement by the target population
• Identifiable and quality sampling frame, and sample list
• Relevant and interesting survey topics
• Uniform non-response bias (people did not opt-out, or opt-in,  because of the survey method itself)
• Literacy and verbal skills were not an issue
• Respondents saw some benefit to themselves by the act of responding to the survey

Online Survey Design and Implementation Issues

Here are some more topics specific, then, to online surveys today:

• Research design considerations.
• Disadvantages.
• Advantages.
• Appropriate use.
• Method considerations.
• Implementation Considerations

Determining the Appropriate Methodology

• Information needs and objectives
• Research objectives
• Sampling frame (Target audience)
• Budget
• Time
• Complexity
• Projectability to the target population

There has been much ballyhoo of late about online surveys. While we clearly believe that they have their place, and a growing one at that, in the arsenal of research methodologies, we believe it would be folly to launch into a online survey without first going through the aforementioned steps. That said, let's examine the pluses and minuses of online surveys, in the context of quality research design.

Disadvantages of online surveys

Sample Randomness & Projectability

More and more people are connected to and using the Internet.  At first, Internet surveys only could be used in rare cases since a relatively small proportion of the population used web.   Further, the population of web surfers, at first, was not representative of the general population.

Target Audience

It is very difficult to reach certain target audiences with online surveys. 

Advantages of online surveys

Speed

Online surveys can be conducted much more quickly that mail surveys, which take several weeks for returns to come in. Up-front design time is comparable to that of programming a CATI (computer aided telephone interviewing) survey to be administered by telephone. Data entry time (on the part of the research firm) is negligible, as it is with CATI. Data collection time is often faster than CATI.

Data-entry accuracy

Data entry errors (on the part of the research firm) are non-existent, however, we rely on (and cannot measure) data entry quality by  respondents.

Security

Using a secure server (the same technology used to process secure credit card transactions), security is not an issue. Anyone who attempts to intercept a transmission made to a secure server will not be able to understand the information they see, as it is encoded.

With written surveys, respondents may question whether their handwriting will give a clue to their identities. This issue does not exist for Internet surveys.

Control

Internet surveys enable much more control than do paper surveys. A few examples of this include:

• It is possible to make certain questions required.
• Algorithms can be written to ensure that in a ranking question, no rank is given more than once.
• Through the use of cookies, people can be prevented from taking a survey multiple times.

Cost

For medium to large size surveys, online surveys will have a cost advantage over telephone and mail. It remains to be seen whether the prices of banner ads will increase to the point where this cost advantage remains.

Ability to gather Complex Data

A well designed and written online survey can gather large amounts of data and to use complex skip, jumps, and piping patterns.   Display of visual material such as ads, concept visuals, and conjoint analysis product concepts is also facilitated by online surveys.

Online survey methodology

1. Banner ads
2. E-Mail
3. Pre-recruited "market research panels"

Banner ads also can be placed in an array of non-search engine sites. In a survey of Web news viewers, ads could be placed in an array of news sites.

The use of e-mail to attract surfers to a online survey should be considered only when people would not consider the e-mail to be junk mail. This is a rare circumstance. However, there are certain conditions which would render e-mail superior to banner ads in attracting people to the survey. This would include a customer satisfaction survey to be conducted for an ISP (Internet Services Provider), or an employee survey to be conducted among employees of a company who conduct most communications over the internet and give all employees internet access (assuming that the dissemination of information is web-page, as well as e-mail based.)

Implementation Considerations

Speed of page loading should be minimized. This means graphics, if any, should be kept to a minimum.

If possible, limit each person who takes the survey to complete it only once.

It sometimes is desirable to allow people to return to the survey and complete it later if they stop it before finishing. If this is done, they should be allowed to pick up at the point where they stopped.

Collect demographic information to enable the data to be weighted, and to determine the degree to which you are reaching the target audience with the survey.