• • • • • •

JCMC Home Submit Issues Author Index Editors About JCMC

Wright, K. B. (2005). Researching Internet-based populations: Advantages and disadvantages of online survey research, online questionnaire authoring software packages, and web survey services. Journal of ComputerMediated Communication, 10(3), article 11. http://jcmc.indiana.edu/vol10/issue3/wright.html

Researching Internet-Based Populations: Advantages and Disadvantages of Online Survey Research, Online Questionnaire Authoring Software Packages, and Web Survey Services
Kevin B. Wright Department of Communication University of Oklahoma

þÿ

Abstract
This article examines some advantages and disadvantages of conducting online survey research. It explores current features, issues, pricing, and limitations associated with products and services, such as online questionnaire features and services to facilitate the online survey process, such as those offered by web survey businesses. The review shows that current online survey products and services can vary considerably in terms of available features, consumer costs, and limitations. It is concluded that online survey researchers should conduct a careful assessment of their research goals, research timeline, and financial situation before choosing a specific product or service.

Introduction

The past decade has seen a tremendous increase in internet use and computer-mediated communication (Fox, Rainie, Larsen, Horrigan, Lenhart, Spooner, & Carter, 2001; Horrigan, 2001; Nie & Erbring, 2000; Nie,

Hillygus, & Erbring, 2002). As an increasing amount of communicative activity takes place through this new medium, there has likewise been a significant increase in primary research on virtual communities, online relationships, and a variety of other aspects of computer-mediated communication (Flaherty, Pearce, & Rubin, 1998; Matheson, 1991; Nonnecke, Preece, Andrews, & Voutour, 2004; Preece, 1999; Preece & Ghozati, 2001; Walther, 1996; Walther & Boyd, 2002; Wood & Smith, 2001; Wright, 2000a, 2002a, 2002b, 2004). Studies of online populations have led to an increase in the use of online surveys, presenting scholars with new challenges in terms of applying traditional survey research methods to the study of online behavior and Internet use (Andrews, Nonnecke, & Preece, 2003; Bachmann & Elfrink, 1996; Stanton, 1998; Witmer, Colman, & Katzman, 1999; Yun & Trumbo, 2000).

The technology for online survey research is young and evolving. Until recently, creating and conducting an online survey was a time-consuming task requiring familiarity with web authoring programs, HTML code, and scripting programs. Today, survey authoring software packages and online survey services make online survey research much easier and faster. Yet many researchers in different disciplines may be unaware of the advantages and disadvantages associated with conducting survey research online. Advantages include access to individuals in distant locations, the ability to reach difficult to contact participants, and the convenience of having automated data collection, which reduces researcher time and effort. Disadvantages of online survey research include uncertainty over the validity of the data and sampling issues, and concerns surrounding the design, implementation, and evaluation of an online survey.

This article considers and evaluates the advantages and disadvantages related to conducting online surveys identified in previous research. In addition, it reviews the current state of available web survey software packages and services, various features of these software packages and services, and their advantages and limitations. The purpose of the article is to provide an overview of issues and resources in order to assist researchers in determining if they would benefit from using online surveys, and to guide them in the selection and use of online survey techniques. To facilitate these goals, which are both methodological and applied, the author draws on published research dealing with online survey methods, as well as his experience conducting

Wellman & Haythornthwaite. 2002). Wellman. Bordia. Haythornthwaite. and hundreds of thousands of people regularly participate in discussions about almost every conceivable issue and interest (Horrigan. These organizations not only offer information to consumers. McGrath. Irmer. & Chang. communities and groups exist only in cyberspace. many of them aggressively promoting their presence through the use of search engines. Thousands of groups and organizations have moved online. health (Rice & Katz. and the popularity of the Internet increases. family members. Flanagin & Metzger. few meet . more segments of society are using the Internet for communication and information (Fox et al. they also present opportunities for researchers to access a variety of populations who are affiliated with these groups. 1999. 2002). Wright. & O'Connor.1998. 2001) have been studied using online surveys. 1993. Tidwell & Walther. organizational (Ahuja & Carley. Virtual communities have flourished online. 1998).. and banner advertisements. Areas as diverse as interpersonal (Parks & Floyd. online stock trading. and co-workers. 1997. 2002. and the pros and cons of virtual dating. Wellman. to reach through other channels (Garton. As the cost of computer hardware and software continues to decrease. Access to Unique Populations One advantage of online survey research is that it takes advantage of the ability of the Internet to provide access to groups and individuals who would be difficult. 2004). Nie et al.. For example. 2001. 2001. Communication researchers may find the Internet an especially rich domain for conducting survey research. email lists.. it would be difficult to find a large. Wright. 2002). 1997). group (Hollingshead. Advantages of Online Survey Research Researchers in a variety of disciplines may find the Internet a fruitful area for conducting survey research. if not impossible.more than 10 online surveys. and mass communication (Flaherty et al. 2000a). concentrated group of people conducting face-to-face discussions of topics such as cyber-stalking. 2001. While people certainly discuss such issues among friends. & Wellman. 1996. In many cases. Hobman.

Wright. As already noted.. For example. 2000c). 2000). 1996. 2004). the Internet enables communication among people who may be hesitant to meet face-to-face. eating disorders. Garton et al. In contrast. 2000. Individuals with these conditions and diseases are often difficult to reach because they are stigmatized offline. Taylor. and message board communities. Another example is the case of individuals with diseases or conditions. and values regarding an issue. problem. individuals with unpopular political views may hesitate to express themselves openly. 2000a. These individuals and groups often can be reached on the Internet in larger numbers than would be possible using face-to-face research methods. Waldron.face-to-face in large groups to discuss them. and attitudes in one location. 2003. & Finn. Time A second advantage is that Internet-based survey research may save time for researchers. Health communication researchers have been able to go online to study these populations. Wright. including examining how features of the computer medium help people cope with the social stigma of their condition (Braithwaite. One advantage of virtual communities as sites for research is that they offer a mechanism through which a researcher can gain access to people who share specific interests. . A researcher interested in surveying hard-to-reach populations can quickly gain access to large numbers of such individuals by posting invitations to participate to newsgroups. and physical disabilities. such as HIV. 1989. online surveys allow a researcher to reach thousands of people with common characteristics in a short amount of time. Yun & Trumbo. More generally. interests. chat rooms. 2000b). or activity. with traditional survey research methods it may be more difficult to reach a large number of demographically-similar older people who are interested in computers. and groups of individuals such as ArabAmericans may feel uncomfortable talking about anti-Arab sentiment in public places (Muhtaseb. 1999. beliefs. researchers can find a concentrated number of older individuals who use computers on the Internet-based community SeniorNet (Furlong. it would take considerably longer-if it were possible at all-to find an equivalent number of people with specific attributes. despite possibly being separated by great geographic distances (Bachmann & Elfrink. For example. attitudes. In the face-to-face research environment.

Researchers had to "cut and paste" responses from the email responses into statistical software programs such as SAS and SPSS. and data entry (Llieva et al. 1999. either by email. however. Witmer et al. More recently.Online surveys may also save time by allowing researchers to collect data while they work on other tasks (Llieva. and later used products such as Macromedia's Dreamweaver. In addition. 2002). such as those incurred through postage. conducting online interviews. and the costs of a traditional large-scale survey using mailed questionnaires can be enormous. researchers may collect data while working on other projects (Andrews et al.. and analyze and interpret data. travel. Yun & Trumbo. generate samples. transcription costs can be avoided since online responses are automatically documented. printing.. Once an invitation to participate in a survey is posted to the website of a community of interest. or in a synchronous "chat" format. & Healey. even when using a relatively small sample. which involved creating online survey forms using word processing software. emailed to people through a listserv service. Some of the newer software packages and web-based services are detailed below. Watt. First generation online survey researchers often used email-based surveys. and the telephone can be eliminated. Couper. online survey creation software packages provide a variety of templates to create and implement online surveys more easily. or distributed through an online survey research service. This allows researchers to conduct preliminary analyses on collected data while waiting for the desired number of responses to accumulate (Llieva et al. . Paper surveys tend to be costly. or posted to an HTML document or database file. 1996. Responses to online surveys can be transmitted to the researcher immediately via email. 2000). 2002). as well as to export data to statistical software packages.. The use of online surveys circumvents this problem by eliminating the need for paper and other costs. 2002. 2003). Baron. a number of online survey services provide survey design assistance. offers cost savings advantages. 1999). Cost Online survey researchers can also save money by moving to an electronic medium from a paper format (Bachmann & Elfrink. Costs for recording equipment. Moreover. 2002. 2000... Llieva et al. Newer online survey creation software and web survey services costs can vary from very little to thousands of dollars depending upon the types of features and services selected. Similarly.

and even this information may be questionable (Dillman. relatively little may be known about the characteristics of people in online communities. Some offer access to specialized populations based on data from previous surveys. not all members of virtual groups and organizations allow their email addresses to be listed. some are unique to the computer medium.this is relatively inexpensive compared to the cost of traditional paper-and-pencil surveys. if the data were self-reported. it is possible to email an online survey invitation and link to every member on the list.. and some may not allow administrators to provide their email addresses to researchers. there are also disadvantages that should be considered by researchers contemplating using online survey methodology. However. Theoretically. this can give researchers a sampling frame. Howard. Sampling Issues When conducting online research. problems such as multiple email . 1998). A number of recent web survey services provide access to certain populations by offering access to email lists generated from other online surveys conducted through the web survey service. For example. Although many of the problems discussed in this section are also inherent in traditional survey research. investigators can encounter problems as regards sampling (Andrews et al. there is no guarantee that participants from previous surveys provided accurate demographic or characteristics information. Stanton. Disadvantages Associated with Online Survey Research As discussed above. Rainie. Once an email list is obtained. online surveys offer many advantages over traditional surveys. This makes accurately sizing an online population difficult. 2001). & Jones. However. However. aside from some basic demographic variables. However. 2000. 2003. Generating Samples from Virtual Groups and Organizations Some virtual groups and organizations provide membership email lists that can help researchers establish a sampling frame.

. while others only participate intermittently. "lurkers. This feature helps to reduce multiple responses. may complete an online survey even though they are not visible to the rest of the community. Participants are required to submit their email address in order to complete the survey." who may make daily contributions to discussions. 2003. not fees. & Edwards. Some researchers attempt to establish a sampling frame by counting the number of participants in an online community. and little information is required when registering to use these communities. such as community bulletin boards and chat rooms. 2000). Nonnecke. or the published number of members. Studies have found that in some online communities lurkers represent a high percentage (between 45% and 99%) of community members. & Andrews. the ebb and flow of communication in online communities can make it difficult to establish an accurate sampling frame. this makes it difficult to obtain an . Furthermore. Generating a Sample from an Online Community Establishing a sampling frame when researching an online community presents a number of challenges. participation in online communities may be sporadic depending on the nature of the group and the individuals involved in discussions. while other studies have found few lurkers (Preece et al. the survey program remembers the participant's email address and does not allow anyone using that email address access to the survey. Another solution that some newer web survey programs offer is response tracking. However. Membership is based on common interests. 2004).. Some people are "regulars. although someone could potentially complete the survey a second time using a secondary email address (Konstan." or individuals who read posts but do not send messages. 2004). over a given period of time. Because lurkers do not make their presence known to the group. One solution is for researchers to require participants to contact them to obtain a unique code number (and a place to include this code number on the online questionnaire) prior to completing a survey. Ross. Rosser. multiple responses from participants. Stanton. if registration is required at all. For example. requiring this extra step may significantly reduce the response rate. The presence of lurkers in online communities appears to be highly variable (Preece. Couper. 2005). Once they have completed the survey. In either case. do not typically provide participant email addresses.addresses for the same person. many online communities. Unlike membership-based organizations. and invalid/inactive email addresses make random sampling online a problematic method in many circumstances (Andrews et al.

Other community administrators might ask participants for information about interests. 2003). The researchers attempted to assess whether the online responses were skewed in any way by comparing the responses from both subsamples. For example. Some communities are willing to share participant information with researchers as a validation technique by comparing the survey sample characteristics with those of the online community in general.accurate sampling frame or an accurate estimate of the population characteristics. This may make it difficult to assess whether the observed differences are due to factors such as participant deception or actual differences due to characteristics associated with computer and non-computer users. some community administrators are beginning to compile statistics on their community's participants. When possible. Stanton. Query and Wright (2003) used a combination of online and paper surveys to study older adults who were caregivers for loved ones with Alzheimer's disease. using both online and traditional paper surveys helps to assess whether individuals responding to the online version are responding in systematically different ways from those who completed the paper version. Yet. basic demographic information such as age and gender. Many communities require a person to register with the community in order to participate in discussions. Sanders. Registration typically involves asking for the individual's name. While no significant differences between the two subsamples were found in this particular study. income level. 1998. real differences in responses between Internet users and non-Internet users might exist in other populations. Thompson. and some communities are willing to provide researchers with statistics about community membership (at least in aggregate form). because individuals easily can lie about any information they report to community administrators. and email address. these findings may be questionable because non-response rate tracking is . As internet communities become more stable. 1995. Other Sampling Concerns Although some studies of online survey methods have found that response rates in email surveys are equal to or better than those for traditional mailed surveys (Mehta & Sivadas. Surface. Martin. there is no guarantee of accuracy. etc. education.

Wittmer et al. For researchers interested only in conducting nonprobability research. Researchers who use nonprobability samples assume that they will not be able to estimate population parameters. In short. may be more effective and more credible. he or she does not know for certain whether the recipient of the mailed survey is the person who actually completes and returns it (Schmidt.difficult to ascertain in most large online communities (Andrews et al. 1997). 2003). et al. Many Internet communities pay for community operations with advertising. However. Many of the problems discussed here are not unique to online survey research. in turn.. Internet users frequently encounter bogus lotteries and other "get rich quick" schemes online. which presents the greatest threat to conducting probability research.e.. a lottery. offering a financial incentive may increase multiple responses to the survey as participants try to "stack the deck" to increase their chances of winning (Konstan. Individuals who participate in the survey are given a chance to win a prize or gift certificate. 1998. While a researcher may have a person's mailing address. These sampling issues inhibit researchers' ability to make generalizations about study findings. there are undoubtedly some individuals who are more likely than others to complete an online survey. there is a tendency of some individuals to respond to an invitation to participate in an online survey. This can desensitize participants to worthwhile survey requests posted on the website. limits their ability to estimate population parameters. Straight incentives such as a coupon redeemable for real merchandise. e.. This. gender... Moreover. books. Thompson et al. In addition. One relatively inexpensive technique used by market researchers to increase response rates is to offer some type of financial incentive. 2003. these issues are somewhat less of a concern. while others ignore it. In any given Internet community. i. Mailed surveys suffer from the same basic limitations. this technique is not without problems.. Self-selection bias is another major limitation of online survey research (Stanton. respondents to mailed surveys can misrepresent their age. 2005). level of . 1999). leading to a systematic bias.g. so a lottery approach to increasing response rates could potentially undermine the credibility of the survey. and the winner is selected randomly from the pool of respondents.

with an explanation of the importance of . Although accessing some online communities can be extremely challenging. Access Issues Some researchers access potential participants by posting invitations to participate in a survey on community bulletin boards. or the researcher may be inundated with emails from irate members of the community. and community sponsors may reject the researcher's request despite his or her attempts to convey the possible benefits of the study for the community (Andrews et al. 1996).. With some diplomatic dialogue initiated by the researcher. or consider this type of posting to be "spam" (Andrews et al. especially when members are interested in how their community is perceived by others. Nonetheless. seeking permission from the community and taking time to explain the purpose of the study might help a researcher to gain access. An unwanted email advertisement is often considered an invasion of privacy. discussion groups. 2003). or the researcher may receive email from participants complaining about it. it is often possible to work with web community administrators and participants when proposing a study idea (Reid. it may take a long time before receiving a response to a request. Researchers might apologize in advance for the potentially unwanted posting.. However. and a host of other variables as easily as a person can in an online survey. people can still respond in socially desirable ways or misrepresent their identity or their true feelings about the content of the survey. 2003).education. This is a more ethnographic approach. The best defense against deception that researchers may have is replication. and chat rooms. The invitation for the survey may be deleted. Some participants in Internet communities actually welcome studies by researchers. Only by conducting multiple online surveys with the same or similar types of Internet communities can researchers gain a reliable picture of the characteristics of online survey participants. 2004). Even when the precise characteristics of a sample are known by the researcher. A community moderator may delete the unwanted post. Researchers using email invitations to participate in a survey may face similar rejection. members of online communities often find this behavior rude or offensive (Hudson & Bruckman.

and have community administrators post a link to the page on the community web site. and points to . Researchers can sometimes become the targets of abusive individuals who resent the invasion of privacy when they encounter an online survey. SeniorNet administrators created a special link to this web page so that the participants in the study (as well as other SeniorNet members) could learn about the results and their possible implications. One way to do this is to create a study report. highlighting the most interesting results to the online community audience. Hate email or worse types of abuse can occur if some individuals on the Internet find online surveys offensive. It is important for researchers to include contact information. The above does not necessarily constitute an exhaustive review of the advantages and disadvantages of conducting online surveys. and something about their credentials when creating an invitation to participate in a survey. However.conducting the research and possible benefits to members. although it represents experiences encountered by many researchers. In addition to being a requirement of most institutional research review boards in universities in the United States. providing researcher contact information has its downside. this helps to enhance the credibility of the survey and it can create opportunities for email interaction between the researcher and participants. information about the study. Researchers can foster "good will" between themselves and community participants by offering to provide information about the results of their study to the community. This is valuable. 2000a). especially when participants have questions. (2003) point out. post it on a web page. Study results should be presented so that audience members can understand them. the author of this article created a summary of research findings for the SeniorNet community after completing a study of social support among participants (Wright. While such incidents tend to be rare. A man once called the author's home phone number repeatedly and left threatening messages on his voice mail after obtaining the number from his department secretary (the department number appeared on the informed consent for the online survey). For example. the possibility of irate responses is something to consider. as Andrews et al.

com/activewebsurvey/ Apian Software http://www.com Perseus www.com InfoPoll www.apian.activewebsoftwares.pollpro. Table 1 lists 20 of the more prominent packages and services.net/ CreateSurvey www. Active Websurvey www.perseus. along with their web addresses. The next section surveys current web survey software packages and online survey-related services available to researchers who may be interested in conducting online survey research.com HostedSurvey www.quask.infopoll. today's researchers have help with online survey work.com PollPro www.keysurvey. Current Web Survey Software Packages and Online Survey-Related Services As noted above.com Quask www.net/ InstantSurvey www.netreflector.factors that should be taken into consideration in deciding to use and designing an online survey.formsite.com KeySurvey www.com EZSurvey www.createsurvey.com .hostedsurvey.raosoft.com FormSite www. There are currently dozens of online survey software packages and web survey services available to researchers willing to pay for them.

Company Name/Product Features Pricing Service Limitations Active Websurvey Unlimited surveys.websurveyor.com Table 1.com SurveyMonkey www.Ridgecrest www. pricing issues. and limitations.surveycrafter.zoomerang.com Zoomerang www. as well as to identify current trends in the online survey product and services market. Table 2 presents a comparison of features. free email support Survey housed on company server for a set amount of time .ridgecrestsurveys. Twenty web survey companies The author examined each of the websites for these 20 online survey product and service companies in order to assess current features. educational discount $99 a month for unlimited surveys and responses.com SurveySite www.surveysite.com/ SuperSurvey www. customer charged for technical support Customer required to purchase software CreateSurvey Standard features. limited to 9 question formats Apian Software Full service web design and hosting available $1195 up to $5995 depending on number of software users.supersurvey.com SumQuest www.com SurveyCrafter www. and limitations of the 20 online product and service companies.sumquest.surveymonkey. software automatically generates HTML codes for survey forms Information unavailable on website Customer required to purchase software. pricing.com WebSurveyor www.

95 up to $99. limited number of response per month HostedSurvey Standard features. additional software is extra. limited customer support. mobile survey technology available. telephone training is $150 an hour Customer required to purchase software FormSite Weekly survey traffic report. access to PollPro server is an additional fee Customer required to purchase software Quask Supports multimedia $199 for basic software. but works best on InfoPoll server. mobile survey technology available Information unavailable on website. first 250 response are free. free 30 day trial Survey housed on company server for only a set amount of time. educational discount $399 for basic software.95 per Survey housed on company month depending on server for only a set amount of desired number of response time. training available for a fee Software can be downloaded free. educational discount $54. multiple language support $9. free 30 day trial Survey housed on company server for only a set amount of time KeySurvey Online focus group feature. free 30 day trial Survey housed on company server for only a set amount of time PollPro Standard features. supports multimedia Information unavailable on website. then around $20 every 50 responses. limited to 1000 responses for basic package . access to Quask server for an additional fee Customer required to purchase software. educational discount Charge is per number of responses.EZSurvey Unlimited surveys. customers appear to be charged for using InfoPoll server InstantSurvey Standard features. Survey housed on company server for only a set amount of time InfoPoll Standard features. limited to 2000 responses Perseus Educational discount. Software can be downloaded for free Information unavailable on website. unlimited surveys $670 per year for a basic subscription. more advanced features only come with higher priced software Ridgecrest Standard features. unlimited surveys $249 for single user.95 for 30 days Survey housed on company server for only a set amount of time.

limitations. free and unlimited technical support Customer required to purchase software SurveyMonkey Standard features. Comparison of online survey software and services This is not. user guidebook for creating questionnaire available $495 to purchase software.SumQuest Standard features. educational discount $495 for basic software package. Expanded Services The businesses listed in Tables 1 and 2 offer researchers two basic options for creating and conducting online . online focus group feature Information unavailable on website Company staff rather than customer create and conduct survey WebSurveyor Standard features. an exhaustive list of online survey software and service businesses. and the implications of using these products and services for online survey research. data collection and analysis. free email support Survey housed on company server for a set amount of time. educational discount $599 for software Customer required to purchase software Table 2. However. 2000 response per week limit SurveyCrafter Standard features. it represents a good cross-section of the types of online survey products and services currently available to researchers. unlimited surveys $1. The following sections consider some of the current features of online survey products and services. Survey housed on company server for only a set amount of time. Current Features Survey Creation Software vs. of course. unlimited surveys $20 a month for a basic subscription. limited to 1000 initial responses SurveySite Company helps with all aspects of survey design.495 per year for software license Customer required to purchase software Zoomerang Standard features. pricing issues. free unlimited telephone support Customer required to purchase software SuperSurvey Standard features $149 per week for basic package.

The major features and problems with each option are discussed below. An example of this option is EZ Survey. participant tracking. which allows researchers to run the software on their own computer and a server of their choice. Many of these companies also offer customer support. This may be an attractive choice for researchers who have access to free server space on their university or research organization server. sampling and data collection services. which are computer programs that researchers use to create and conduct online surveys on their own computer and server. Given rapid advances in software development. and several data tracking and analysis options. online questionnaire development. This transmits the survey responses to the researcher. per time period (e. however.survey research. by the month). including many different templates to help first-time web survey researchers. After completing an online survey. including help with design. Other companies offer a wider range of services for conducting online surveys. and data analysis and interpretation services. only the . The companies that offer such packages also provide options for customer support. A "form" is an interactive type of web page that allows Internet users to send information across computer networks. this software may be outdated in a relatively short period of time. One disadvantage of owning the software is that customers have to pay to upgrade software. and data analysis. by number of responses. participants click on a "submit" button on the webpage. or by some combination of these options. including research design. Each of the online survey products reviewed offered some type of online form to collect data from participants. Purchasing Software Option Some companies (see Table 2) require customers to purchase online survey creation software. by survey length. Customers who have purchased software receive discounted upgrades.. One option is the online survey software packages.g. Owning the software enables researchers to create multiple online surveys of any length as opposed to being charged per survey. data collection. Online survey questions are the same types as on a traditional paper/pencil questionnaire. server space for the online survey (in some cases). Online Questionnaire Features The businesses listed in Tables 1 and 2 offer a wide array of options for creating online surveys.

. but this type of work can be cumbersome for a busy researcher. However. semantic differential scales. The businesses listed in Tables 1 and 2 typically offered a range of question types. Common Gateway Interface (CGI) scripting. including multimedia can increase download times and may be frustrating to participants who must download media players or other types of programs in order to participate in the survey (Andrews et al. In addition. Multimedia can also be useful when targeting low literacy populations. some products offer more complex question-type options. so as to vary the order of question responses and thus reduce question order bias. the technology exists to easily construct a . All of the reviewed companies offering online survey products provide a variety of useful questionnaire options. and filter questions (to tailor surveys to individual characteristics of survey respondents). and a userfriendly process to develop online questionnaires. Additionally. asking participants to respond to a video or audio clip. Basic question options usually include Likert-type scales. a type of computer language that is run on the Web server where an online survey is housed. For example. Some products support multiple language versions of an online survey and versions for visually impaired respondents. 2003). the majority of the reviewed products offer randomized answer choices for participants. such as multiple response matrices and the ability to use multimedia. Most Internet users are quite familiar with Web forms since search engines. although the number of options varied from business to business. textboxes (for qualitative responses).participants submit the information over the Internet rather than return questionnaires in person or by mail. are sophisticated forms. especially one who is not technologically proficient. checklists..e. researchers who want to measure participant perceptions of a political candidate's positions on foreign policy could include a video clip from a recent speech. i. since video and audio messages guide participants through an online survey. Writing scripts for processing forms can be done manually. including Yahoo! and Google. A multimedia video or audio clip can be used to jog the memories of respondents or as a reference point for participant responses. Taking the use of multimedia a bit further. tells the server how to process information that is submitted. drop-down menus (for nominal or categorical items).

and interpretation of results. It is also possible to develop computer scripts that randomly send participants to one of several other web pages. Basic survey process features include tracking of survey respondent email. Researchers who are interested in more sophisticated designs such as these would probably benefit from selecting a business that offers a greater degree of consulting and technical support. email response notification. especially for key survey measures. questionnaire creation. SurveySite offers consultation throughout the entire survey research process. Zoomerang offers access to tailored email lists and multisource recruiting for sampling. as well as customer support. Although most of the reviewed companies offer free technical support. which prevents survey data submission unless certain items are responded to. after a participant submits the data to the researcher. enabling the random assignment of participants to different levels of an independent variable. Other companies will help researchers collect data by advertising the survey on certain websites. In addition. including method design. All respondents (regardless of which condition they were assigned to) could then be linked to an online questionnaire containing dependent variable measures. researchers are generally charged a fee for extensive consultations and/or training. Most online survey companies offer a redirect feature to display a "thank you for participating" web page. most of the reviewed businesses offer features that aid the data collection and analysis processes. or any web page a researcher chooses. enabling research teams with members at different locations to share survey results. allowing researchers to target specific demographic groups within a population of interest. most of the reviewed companies offer a required answer feature. Some companies . For example. data collection. Data Collection and Analysis Features In addition to helping researchers create online surveys. Each web page could contain a different audio and video stimulus.web page that uses video and/or audio clips as stimuli for online experimental and quasi-experimental designs. This reduces missing data. and the ability to export survey responses to statistical software packages such as SAS and SPSS. These range from basic features to more in-depth involvement by company consultants. Other basic features include the ability to share data with other researchers. real time tracking of item responses. data analysis.

or socioeconomic groups that do not have access to computers or the Internet. have the capability to merge computer technology with traditional survey methods. unsubscribe respondents from an email list after they have completed a survey. Researchers can interact with participants via chat room applications or webcam and audio teleconferencing technologies. leading to richer results. such as KeySurvey and SurveySite. Pricing . and businesses such as SurveyMonkey offer pop-up advertising to aid in recruiting participants. Using a wireless device (as opposed to a laptop computer). Several of the companies offer researchers even more sophisticated options for conducting survey research. They offer telephone survey capabilities where participants use a touch tone phone to enter responses. For example. individuals in rural settings. The Internet allows researchers to include participants from multiple geographic locations in the same focus group. Mobile Internet surveys offer a number of advantages to researchers. researchers can bring a survey to otherwise inaccessible populations in the non-virtual world. EZ Survey offers a free sample size calculator. and/or text in real time from remote locations. such as patients in a busy healthcare setting. some companies. The responses of one participant can trigger ideas and responses among other participants. Perseus can conduct mobile surveys. These qualitative focus groups are often used as a precursor to developing a quantitative survey to reach broader numbers of individuals. Participants view the same video.offer other types of features to aid with the survey research process. Realtime computer applications are important in focus groups because researchers want participants to interact with the focus group facilitator and with each other at the same time. provide the ability to conduct online focus groups. In addition. This allows researchers to conduct face-to-face interviews with participants while using the wireless device to store and transmit responses to the survey to a database. such as InstantSurvey. such as Perseus. Other companies. audio. using wireless handheld devices like Palm pilots. Data are sent through wireless technology to a server (similar to other online survey forms) where the information is posted to a database file. which may help to reduce multiple responses from the same participant. Some companies.

In general. such as the limitation of 1000 responses per month. and services similar to SurveyMonkey for $500 to $2000 per business quarter (depending upon number of users and number of responses desired). such as Perseus and SurveyCrafter. As previously stated. For example. In general. and services. Pricing for the businesses reviewed here varied considerably even though they offered similar products. Other businesses. the more it will cost. Moreover. SurveyMonkey provides authoring tools. less expensive approaches may be sufficient. However. Perseus charges $2000 per day for personalized training. Researchers should . server space for surveys. then businesses that include these services in the price should be avoided. advertise educational discounts on a wider variety of services. features. the more features and services needed from a web survey company. these basic. such as KeySurvey. Other companies charge researchers by the survey. while most companies offer free technical support. SuperSurvey offers products. and customer support for around $350 (about $100 less than the regular price for service). Companies that charge less typically do not recruit participants for customers and do not provide consultation throughout all stages of the research process. for many web survey researchers. paying more does not necessarily mean more services. Moreover.Costs of survey products and their services vary. Other business. data analysis. SurveyMonkey charges an additional 5 cents per survey response over the 1000 response limit. researchers are generally charged extra for extended training and consultation. many of the reviewed business websites offer educational discounts for academics. but also offers discounts for group consultation and training. Zoomerang offers educators one year access to their online web survey creation services. and interpretation of results. charge substantially more ($670 per year for a basic subscription) for products and services similar to those offered by SurveyMonkey. consultation can be expensive. For example. However. In some cases. if sample generation or help with analyzing data is not needed. as opposed to only $20 a month. For instance. there are limitations. as well as consultations and other support services. especially for those experienced in conceptualizing survey projects. or else these services should be negotiated out of the price. it is a "buyer beware" situation. features. including discounts on software. and simple automated survey analysis for about $20 a month ($240 per year). however. server space. For example. Basic features can be purchased for a relatively small amount of money.

Implications of Using Web Survey Products and Services Current web survey products and services have greatly facilitated the process of creating and conducting online surveys. in contrast. Researchers who use a company's email lists to generate a sample are limited by the quality of this type of sampling frame. space. generally does not have space or response number restrictions. and this could negatively impact response rates. If a researcher wants to keep a survey on the company's server for an extended period of time (such as more than a year). In addition. Limitations As noted above. Customer support may be available for minor technical problems and customer questions. compared to the time that it would take most researchers .inquire about these special discounts since they may help to reduce the overall cost of purchasing web survey software or services. Researchers can save considerable time by utilizing the products and services that are offered by many of the businesses highlighted in this article. minor questions can be answered for free via telephone. Some specific limitations include issues of time. email. There are also generally limitations to the amount of free customer support a researcher can obtain. companies such as SurveyMonkey and SuperSurvey will host an online survey for a set amount of time. Typically. Purchased software. this costs extra. but a researcher may be charged for extensive training. but customers are generally charged extra for extended consultations and training. there may be limitations associated with using web survey products and services. In cases where a company uses the same lists again and again for different clients. and number of responses allowed for a given price. the individuals who receive the advertisements about a survey on these lists may become weary of being targeted by multiple surveys. such as learning advanced web page creation techniques or data analysis instruction. For example. or chat applications. some companies often charge more for longer surveys and for a number of respondents exceeding a certain amount (generally over 1000).

using these products and services does not necessarily circumvent the disadvantages of online surveys. J. & Preece. References Ahuja. these products and services are not without limitations. & Elfrink. MarketingResearch.. . Acknowledgments The author would like to thank the anonymous reviewers for their helpful and insightful suggestions for improving this manuscript. 31-35. response rates. While attractive. and access to populations. K. As with purchasing any product or service. 185-210. researchers should assess their research needs. B.html Andrews. 8 (2). International Journal of Human-Computer Interaction. and research time frame. budget. K. Bachmann. J. participant deception. thanks to a variety of attractive features offered..g. Network structure in virtual organizations. Electronic survey methodology: A case study in reaching hard-to-involve Internet users. Retrieved April 18. D. Furthermore.. however. (1996). and comparison shop when deciding on which business to use.. As we have seen. (2003). The cost of these products and services varies depending on the types of features and services a researcher desires. In short. M. features of the surveys themselves (such as multimedia) and the services (e. 2005 from http://jcmc. D. Tracking the progress of e-mail versus snail-mail. 3 (4). & Carley.edu/vol3/issue4/ahuja.. including issues related to sampling frames.indiana. (1998).to create an online survey themselves using a web authoring program. M. researchers should view current web survey products and services as another research tool that-like the online survey itself-has its own unique advantages and disadvantages. 16 (2). Journal of Computer-Mediated Communication. using company email lists to generate samples) offered by web survey business can affect the quality of data in a variety of ways. Nonnecke.

& Finn. (2000). A. J. M. D. (2002).. A. Couper. Pearce. J. Internet and face-to-face communication: Not functional alternatives. Flanagin. Irmer. P.. Jones (Ed. Retrieved May 4th 2005 from: http://www. The expression of conflict in computer-mediated and . M. & Rubin. Lenhart. 250-268. V. Health Communication.. Flaherty. 64 (4). R. Dillman.. B. P. The Pew Internet and American Life Project. 27 (1). Public Opinion Quarterly. Human Communication Research. Fox.pewinternet. Hamilton. Rainie. In S. D. 46 (3). (2000). & Metzger. Garton. Communication Quarterly. 46 (15). & Wellman. C.org/pdfs/PIP_Wired_Seniors_Report. (1999). R. & Chang.. V. Internet use in contemporary media environment. Web-based surveys: A review of issues and approaches. New York: John Wiley & Sons.. Communication of social support in computer-mediated groups for people with disabilities. A. CA: Sage. Larsen. E. S. Studying on-line social networks. J. 39 (3). R. 75-105). 11 (2).. S. J. Doing Internet Research: Critical Issues and Methods for Examining the Net (pp. 153-181. L. The Chronicle of Higher Education. (2001). 145-153. (2001). Wired Seniors. E. Journal of Communication. C.. K. Haythornthwaite.. & Carter. L. M.pdf Furlong. 123-151.Braithwaite. C... T. (1989)..). (1999). 464-494. Horrigan... Bordia. B6-7. L. (1999). An electronic community for older adults: The SeniorNet network. Mail and Internet Surveys: The Tailored Design Method. Waldron. A. M. The ethics of conducting social science research on the Internet. B. Thousand Oaks. O. Spooner... Hobman. (1998).

439-465. 44 (3). A. Y. 383-404. N. American Behavioral Scientist. B. Social cues in computer-mediated negotiation: Gender makes a difference. A.. & Bruckman. Gibson (Eds. 2004 from http://www. B. (2005). L. A.indiana. Ross. Rainie. B. M. 361-367. Stanton. W. Rosser. B. (1996). Horrigan. Jacobson. 10 (2). E. M.. W. Small Group Research.. (1993). M. NJ: Hampton Press. Pew Internet and American Life Project. J.. 24 (3). 45 (3). "Go away:"? Participant objections to being studied and the ethics of chatroom research. Computers in . Online surveys in marketing research: Pros and cons. & Edwards. 2005 from http://jcmc.. S. Retrieved October 1. Hollingshead. article 11. Llieva. J. & S. International Journal of Market Research. J. Cresskill. R. K. 33 (4).. Konstan. 275296). Journal of Computer-Mediated Communication. Retrieved April 25. The story of subject naught: A cautionary but optimistic tale of Internet survey research..)..asp?Report=47 Howard. Small Group Research. In L.. P. Days and nights on the Internet: The impact of a diffusing technology. R. 307-333. & O'Connor. M. (1991). Hudson. J. (2002). Strate.html Lee. Online communities: Networks that nurture long-distance relationships and local ties. The Information Society. 127-139. M. K. J. McGrath. Communication and Cyberspace: Social Interaction in an Electronic Environment (pp. Baron.. (2004). & Jones. Charting the codes of cyberspace: A rhetoric of electronic mail.. J.pewInternet. & Healey. 20 (2).face-to-face groups. (2001). Group task performance and communication technology: A longitudinal study of computer-mediated versus face-to-face work groups. Matheson.org/reports/toc.edu/vol10/issue2/konstan. S. J. (2001). S.

The Internet as an Alternative Source of Information and Alternative Forum of Expression for Arab Americans.. Empathetic communities: Balancing emotional and factual communication. (2000). (1999). Andrews... R. In R. K.. (2003). J.Human Behavior.). J. Parks. Journal of the Market Research Society. E. dissertation. Preece. Rice & J. N. 201-223. Preece. Paper presented at the 2004 Americas Conference on Information Systems. (2004). Preece. (2001). M. R. B. The top five reasons for lurking: Improving community experiences for everyone. K. & Erbring. University of Memphis. (1995). 63-77. Nonnecke. B. Experiencing empathy on-line.. Internet and Society: A Preliminary Report. Hillygus.. R. interpersonal relations and sociability: Findings from a detailed time diary study. August). (1996). L.. Mutahseb. J. Internet use. J.. In B.D. CA: Sage. London: Blackwell Publishers. Nie. Mehta. N. 46 (1). 137-145. Unpublished Ph. NY. E. Computers in Human Behavior. 237-260). 7 (3). CA: Stanford Institute for the Quantitative Study of Society Press. (2004). Jr. L. E. & Floyd. L. S. Comparing response rates and response content in mail versus electronic mail surveys. 215-243). & Andrews.. & Voutour. Katz (Eds.. & Ghozati. Thousand Oaks. & Suvadas. 80-97. (2004. Journal of Communication. Interacting with Computers: The Interdisciplinary Journal of Human-Computer Interaction. Online Lurkers Tell Why. D. & Wright. New York City. Nie. J. & Erbring. (2002). Query. Wellman (Ed. B...). 37 (4). Palo Alto. Nonnecke. Assessing communication competence in an on-line study: Toward . D. 429-439. A. 20 (2). Making friends in cyberspace. 12 (1). Preece. The Internet and Health Communication: Experiences and Expectations (pp. H. The Internet in Everyday Life (pp. J. K.

A. B. & Katz. C. Human Communication Research. Schmidt. and hyperpersonal interaction. potential problems. their lay caregivers. G.. World-Wide Web survey research: Benefits. Computer-mediated communication: Impersonal. E. S. 23 (1). An empirical assessment of data collection using the Internet. Personnel Psychology. E. Tidwell. 56 (1). 42 (1). Informed consent in the study of on-line communities: A reflection on the effects of computer mediated research. M. 29 (2). B. interpersonal. 51-63.. Rice. Health Communication. (2003). 15 (2). (2000). impressions and interpersonal evaluations: Getting to know one another a bit at a time. R. (2002). The Internet and Health Communication: Experiences and Expectations. In C. 3-43. Thompson. From paper to pixels: Moving personnel surveys to the Web. International Journal of Market Research. Lin & D. & Boyd.. L. Instruments and Computers. Martin. F. 709-725. 153-188).. E. 12. CA: Sage. W. & Walther. (2002). J. (1997).). Thousand Oaks. 197-227. Personnel Psychology. Taylor. J. Reid. Cresskill. Does Internet research work? Comparing electronic survey results with telephone survey. L. H. (1998). The Information Society. J. 317-348. and peers. and solutions. C. & Sanders.informing subsequent interventions among older adults with cancer. NJ: . D. Walther. M. E. L. Stanton. Communication Technology and Society: Audience Adoption and Uses (pp. 169-174. Walther. Computer-mediated communication effects on disclosure. (1996). B. Behavior Research Methods. Attraction to computer-mediated social support. J. J. 205-219. 274-279.. Surface. 51 (3). 28 (3).. Atkin (Eds. Communication Research. (1996). A. (2001).

100-118. Moral. H. In S. K. Wellman. The Internet in Everyday Life.). 145-161). Thousand Oaks. Communication Quarterly. M. In S. J. 277-295). .. Doing Internet Research: Critical Issues and Methods for Examining the Net (pp. older adults. Oxford. (1999). 402-414. 47 (4). (2000a). F. and Practical Implications (pp. Mahwah. B. Information Technologies in Evaluation: Social. (1997). (Eds.).). Colman. Wellman. J. B. & Katzman. J. L. In S. UK: Blackwell. & Haythornthwaite. C. W. and coping strategies. Email overload: Exploring personal information management of email. In G. Kiesler (Ed. Wright. D. & Burgoon. Mahwah. San Francisco: Jossey-Bass. Wright. F. Jones (Ed. Culture of the Internet (pp. Watt. & Sidner.) (2002). perceived stress. 23-44). Relational communication in computer-mediated interaction. Whittaker. and coping. J. From paper-and-pencil to screen-and-keyboard: Toward a methodology for survey research on the Internet.. Kiesler (Ed. Culture of the Internet (pp. NJ: Lawrence Erlbaum Associates. Journal of Communication. Witmer.. K. (1992). Gay & T. Walther.). CA: Sage. Mahwah. NJ: Lawrence Erlbaum. Wood.. C.Hampton Press. and Culture. Internet systems for evaluation research. Computer-mediated social support. K. (1997). A. (1999).. An electronic group is virtually a social network. Bennington (Eds. S.. Identity. B. & Smith. NJ: Lawrence Erlbaum Associates. On-line Communication: Linking Technology. Computer-mediated support groups: An examination of relationships among social support. L. R. B. 179-205). 50-88. S. (1999). 50 (3). 19 (1). Epistemological. B. Human Communication Research. (2001).

Wright. K. and online survey methodology. and motives for using the community. on-line communication apprehension. Social support satisfaction.indiana. and perceived life stress within computer-mediated support groups. Qualitative Research Reports in Communication. Yun. & Trumbo. Communication Studies. communication and social support within on-line support groups. K. 33-43. 6 (1). 19 (1). source credibility. (2000c). Social support within an on-line cancer community: An assessment of emotional support. K. G. Motives for communication within on-line support groups and antecedents for interpersonal use. 89-98. (2002a). B. Communication Research Reports. C. and web form. Wright. Wright (Ph. W. 418-432. B. B. 48 (1). B. 30 (3). email.html About the Author Kevin B. K.D. K. 139-147. On-line relational maintenance strategies and perceptions of partners within exclusively Internet-based and primarily Internet-based relationships. The communication of social support within an on-line community for older adults: A qualitative analysis of the SeniorNet community. interpersonal communication issues within the context of computer-mediated interaction. 2005 from http://jcmc. 195-209. (2002b). Perceptions of on-line support providers: An examination of perceived homophily. Communication Research Reports. 55 (2). 1 (2).. K. Retrieved April 18. (2000d). (2000). Wright. Journal of Computer-Mediated Communication. The majority of his research has focused on computermediated support groups and health outcomes for people with health concerns.Wright. B. 17 (2). perceptions of advantages and disadvantages. More information . Wright. Communication Quarterly. Comparative response to a survey executed by post.edu/vol6/issue1/yun. W. University of Oklahoma 1999) is an assistant professor in the Department of Communication at the University of Oklahoma. (2000b). Journal of Applied Communication Research. B. 44-59. Wright. (2004).

is available at: http://www. University of Oklahoma.edu/cas/deptcomm/facpages/wright. OK 73019 USA © 2005 Journal of Computer-Mediated Communication . Room 101. 610 Elm Avenue.ou.html Address: Department of Communication. Norman.

Master your semester with Scribd & The New York Times

Special offer for students: Only $4.99/month.

Master your semester with Scribd & The New York Times

Cancel anytime.