Abstract
Conducting qualitative research online is becoming more prominent and popular as a way of collecting insights and information as companies discover the advantages, learn the methodologies, and understand the most appropriate uses. Examines various aspects of the two most utilized methodologies for conducting online qualitative research and discusses each phase of designing and implementing such a study. Specific recommendations for conducting virtual groups and avoiding technical obstacles are offered for those new to the online venue.
Keywords
Citation
Sweet, C. (2001), "Designing and conducting virtual focus groups", Qualitative Market Research, Vol. 4 No. 3, pp. 130-135. https://doi.org/10.1108/13522750110393035
Publisher
:MCB UP Ltd
Copyright © 2001, MCB UP Limited
Qualitative research has become one of the many beneficiaries of the electronic revolution and evolution, specifically in the development of online focus groups. The rapidly increasing number of consumers and business professionals online has made it possible and important for this newer research methodology to move forward. Conducting qualitative research online is becoming more prominent and popular as a way of collecting insights and information as companies discover the advantages, learn the methodologies, and understand the most appropriate uses.
Online qualitative studies have been conducted to evaluate online and offline advertising, evaluate mock Websites, critique existing Websites, test and evaluate new products (product mailed in advance of the groups), uncover competitive Website information, evaluate training programs, explore decision‐making, uncover imagery, evaluate concepts, evaluate package visuals, generate ideas, and ascertain customer and employee satisfaction.
At present, the two most utilized online methodologies are real‐time virtual focus group rooms where six to eight respondents participate simultaneously, and asynchronous to online bulletin boards with, ideally, 12 to 20 respondents lasting over a period of days. Both venues allow respondents to participate from their desired location and, with the bulletin board, at a time that is most convenient for them. There are a variety of real‐time and asynchronous virtual facilities offering different formats. The specific features, capabilities, and sophistication will vary according to which provider is selected.
The use of either methodology requires a re‐thinking and re‐application of qualitative design and techniques. Some activities and techniques used with in‐person groups work as well (e.g. sentence completion, brain dumps) or better (e.g. pantry recall/checks, brand verification) with online groups. Other activities and techniques need to be adjusted (e.g. brainstorming, imagery exercises), do not work as well (e.g. mind mapping, picture sorts), or do not work at all with technology that is currently available (e.g. controlled sensory tests, paired assignments).
The design and development activities necessary to conduct successful online groups are similar to offline groups, specifically: establishment of objectives, screeners, discussion guides, moderation, analysis and report writing. However, there are crucial differences, reflecting the influence of the technology, that require expertise beyond the skills required for in‐person groups. Below is a summary of some of the similarities and differences between online and in‐person qualitative research. A glossary of terms used is given in the Appendix.
Similarities and differences
Screeners, recruitment and virtual facilities
Recruiting online groups requires specially crafted screeners that are similar in content and depth to those used for in‐person groups. As always, the quality of the screener can determine the quality of the respondents. Since many of the screeners are administered electronically, some questions are worded differently to disguise qualifying and disqualifying answers. A well‐written screener, in combination with a professional online facility, will thank and release all disqualified respondents without them knowing why. This, as well as putting a block on their electronic address, discourages them from re‐trying to qualify by logging back in or from sharing information about the specific screener questions with friends.
Online groups (real‐time or asynchronous) can be initiated by contacting a moderator or company that specializes in online focus groups. Most will provide turnkey solutions and have a virtual facility they work with regularly that provides virtual rooms as well as recruitment services. Respondents can be recruited electronically from established panels, compiled online lists, targeted Websites, or client‐provided lists. Sometimes telephone recruiting is utilized to recruit audiences less likely to respond online, highly valued audiences where “high touch” interactions are necessary, or to confirm and obtain e‐mail addresses.
View for the client observers
If all is going well, most of the moderating elements discussed below will be transparent to the research observers. In fact, it may even seem slow for them as they passively sit in front of their computer watching the interaction. It is important to point out that the optimal way for the client to interact with the moderator is through one designated client liaison. Similar to in‐person groups where notes are passed to the moderator, the designated liaison decides what is important to pursue and approves questions given to the moderator. These “notes” may be submitted to the moderator in private message form or entered in the backroom dialogue stream for the moderator to see. The method of communication between the client and moderator may vary depending on the virtual facility being used and the software capabilities.
Invitations and preparation
Respondents and observers who are invited to the group receive invitations with passwords and passnames, instructions, dates, and times. The invitation requests that they sign on to the site in advance of the group, using the computer they will use during the group, to guarantee that all technology is compatible. If there are any complications or questions, the respondents can contact tech support in advance to resolve them. They can also contact tech support during the group for online support, as can the moderator and client observers.
Discussion guide development and design
The content and structure of the inquiry resembles in‐person groups. The major difference is in the actual presentation of questions that are mostly written in full sentence form in advance. The main topic questions must be written clearly and completely, or respondents may find it necessary to ask for clarification and this will delay the response process.
Real‐time online groups tend to last 90 minutes though shorter or longer ones are possible. Ideally, 40‐45 questions are written in advance for input during the discussion. With a 90‐minute group this usually leaves enough time to insert additional spontaneous probes into the dialogue stream.
Bulletin board groups can last three to five days or more, depending on the amount of material to be covered. At the beginning of each day there are a set number of questions posted that all respondents answer. The number of daily questions depends on the complexity of the issues, relatedness of the topics, attention span of the respondents, and level of respondent involvement in the subject. During the response process throughout the day, additional probes can be interjected to request elaboration on topics or new subjects can be added. (Note: there is also an e‐mail version of asynchronous groups that tends to be less sophisticated and less dynamic in its interaction; however, this type is not covered in this discussion.)
Show rates and selection
Show rates can vary dramatically based on a number of factors, including: the target respondent (i.e. as with in‐person groups, some types of respondents are more reliable); origination of the respondent (e.g. online database, established panel, Website intercept, client customer or employee lists, etc.); confirmation procedures (e.g. online only, phone only, or phone and online combined); respondent comfort and familiarity with online venue in general (i.e. respondents should possess at least adequate keyboard skills); and the typical kinds of other personal/business commitments that can inhibit attendance. One positive influence is that the weather, traffic, and transportation have less of a negative impact on show rates since the respondents are participating from a variety of locations and not encountering the same delays.
For both kinds of online group the number of respondents recruited is roughly 50 per cent to 100 per cent more than the desired attendance. The final selection of respondents is based on the rescreener information and final screener spreadsheet. The moderator and client select the respondents together, similar to in‐person groups.
Moderating
The excitement and pace of moderating an online group, for the moderator, can be likened more to a roller coaster ride than an in‐person group. Ideally, the discussion guide is downloaded directly onto the site so the moderator can, with one click, enter a question into the dialogue stream. Another method that is more frequently available and workable (though requiring more time, concentration and actions by the moderator), is having the discussion guide document loaded in a separate window behind the virtual room to use for cutting and pasting each question.
To begin a group, the moderator introduces the purpose of the group and lays the ground rules. This includes a personal introduction, purpose, timeline, instructions for entering responses, encouragement to be candid and honest, and instructions for signing back on if they accidentally drop off. Respondents are also encouraged to “feel free to agree, disagree, or ask questions of each other that relate to the subjects being discussed” and are told that this interaction will help bring the discussion to life.
Online groups demand that a moderator possess strong and fast keyboard skills or be willing to hire an assistant who does. There are no unused moments during a group to accommodate slow typers on the moderator side. While a group is under way, there may be technical problems with respondents and clients requiring telephone calls back and forth to resolve. Simultaneously, the moderator is reading and interpreting the dialogue stream, responding to client notes, composing probes and entering questions and all the while (potentially) dealing with various technical issues.
Also, moderating online groups requires someone who relates to the online venue and recognizes that respondents are adept at developing relationships in this medium. Many respondents have participated in chat rooms and feel comfortable relating online. At the same time, it is the responsibility of the moderator to help make the less experienced respondents feel comfortable and valuable.
The strategy of online moderating resembles in‐person moderating; that is, the moderator follows the discussion guide to the extent that it is eliciting the desired information. If a subject that was supposed to be covered later in the group is brought up earlier by the respondents, those questions can be inserted where the moderator deems it most appropriate. In addition, if topics not covered in the guide are introduced, the moderator can choose to interject a new line of questioning.
View for the respondents
Respondents see all of the moderator’s questions and the comments of other respondents as they are input. Each respondent has a name, often a pseudonym, that identifies them and sometimes the responses are color coded. They do not see any of the comments from the observers or observer notes to the moderator.
Analysis and reporting
Analysis and reporting is similar to in‐person groups, with the exception that transcripts are quickly available for every group. The analysis will be very inclusive and reflect the input of most respondents since most of them answer every question. In the absence of visual and verbal cues, analysis of some areas, such as appeal, will be based on an interpretation of respondent statements and any (qualitative) ratings they use to indicate levels of appeal. The final reports cover areas similar to other qualitative reports such as objectives, methodology, conclusions, and key findings. They can be in topline, executive summary, or full report form. Typically, reports can be turned around quicker due to the immediate availability of the transcripts.
Unanswered questions
In considering the use of these new online research tools, there are a number of unanswered questions for most researchers. The following are tips to address some common issues faced by moderators who are new to online work and answers to frequently asked questions.
Ten tips for conducting virtual groups the first time are:
- 1.
(1) Similar to in‐person groups, begin with an introduction that discusses the purpose of the group, gives guidelines to follow, and establishes the moderator as the discussion leader. (Assume respondents are not fluent in this methodology.)
- 2.
(2) Moderator needs to thoroughly know the software platform and its capabilities.
- 3.
(3) Estimate approximately two minutes a question, or 40‐45 questions for a 90 minute group (leaves room for 10‐15 follow‐up probes during the group).
- 4.
(4) Clearly define target segments and closely monitor screening and recruitment; rescreening can be conducted in the waiting room prior to the group.
- 5.
(5) Educate observers and offer to demonstrate and explain technical aspects prior to the groups.
- 6.
(6) Designate one client contact to communicate with moderator.
- 7.
(7) Complete discussion guide and all visuals one day before the groups and test to assure full functioning.
- 8.
(8) Design discussion guide questions that are complete and concise and rewrite any question that might prompt a respondent to ask for clarification.
- 9.
(9) When conducting special tasks during the group (e.g. reviewing whiteboard or Website materials) instruct respondents to indicate their completion of the task with a greeting and to withhold all comments until everyone returns and moderator resumes the discussion.
- 10.
(10) Schedule group sessions based on the time zones of participants.
Ten ways to avoid technical obstacles are:
- 1.
(1) Moderator should have excellent keyboard skills and expect multiple and simultaneous inputs from clients and respondents.
- 2.
(2) Qualify and accept respondents who have adequate keyboard skills.
- 3.
(3) In the introduction, instruct participants and observers on procedures if their connection is lost.
- 4.
(4) Moderator signs on at least 20‐30 minutes prior to the group to establish connection, conduct last minute tests, observe waiting room, and confirm that the software is fully functioning.
- 5.
(5) Schedule groups at least 30 minutes apart.
- 6.
(6) Assign numbers to each discussion guide question and instruct respondents to use them in front of each of their answers.
- 7.
(7) Moderator submits questions in uppercase to facilitate easy recognition of moderator input during the group. This will also assist in reviewing the transcripts later.
- 8.
(8) Check to make sure all browsers are compatible with the software platform.
- 9.
(9) For any special tasks during the group (e.g. whiteboards, site visits) give thorough and clear directions before the task begins.
- 10.
(10) Establish procedures as to how a respondent can be “uninvited” during the group should this rare measure become necessary.
Frequently asked questions
Who and what are being researched in online focus groups?
Online focus groups are ideal for locating and researching markets that are hard to recruit, have low incidence, touch on sensitive topics, are online based, and geographically dispersed. For instance, high level executives may be willing to participate in an online group but would never consider expending the amount of time required to attend an in‐person group. Or, respondents connected to sensitive topics (e.g. HIV) may feel more comfortable and willing to share in the anonymous online environment. Others who have certain low incidence characteristics or unusual purchase behavior may be impossible to recruit within a geographic region, but across the US or in a wider regional area they can be found. Lastly, audiences whose relationships and interactions with companies have been strictly online are good candidates for online focus group research.
Some of the topics that are particularly appropriate are: e‐commerce inquiries regarding purchase behavior and interest; Website evaluation, development, and offers; sensitive topics requiring anonymity; and, high‐tech products and services. In some cases, online groups are used as pre‐ or post‐phases in conjunction with online quantitative research.
Some companies are beginning to experiment and show excitement about the possibilities of dovetailing in‐person and online group methodologies to collect wider geographic input, gain experience in conducting combined studies, and develop comparison measures. To date, it is preliminarily suggested that online and in‐person results are more similar than dissimilar.
Are online groups going to replace traditional focus groups?
Not in many situations. The demand for online groups is expanding the research arena and adding a new tool to the qualitative toolbox, not shrinking the demand for traditional groups. An eight inch research pie is now growing into a ten inch pie and will continue to grow as demand builds and new applications are created.
In some cases, online groups may be used in conjunction with in‐person groups or may replace some of the in‐person groups. This might be done to obtain a wider geographic representation, uncover similarities/differences from the online point of view, or conserve on travel expenses.
However, many in‐person projects will not be appropriate for online groups because the visual and vocal cues of respondents or visual review and evaluation by respondents are critical to the evaluation of the package, product, promotions, or other communications. Additionally, real‐time online groups may not always provide the depth of response necessary.
Are respondents who they say they are?
If a database is used to recruit, the answers respondents submit can be compared to the database information to confirm their identity. In cases where little is known about the potential respondents, it requires a well‐written, thorough screener, similar to in‐person groups. Just prior to the group, respondents can be rescreened online to verify their identity. Or, rescreening can take place in advance over the phone as an additional check. So far, it appears that respondents are who they say they are and the threat of cheaters and repeaters is minimal. Similar to in‐person facilities, each virtual facility can choose to establish their own database to document past participation to be used for comparison purposes during the selection and confirmation process.
Do respondents in online groups really interact with each other?
Yes, but it is not exactly the same as with in‐person groups, mainly because it is perceived but not seen. Many of the individuals who participate in online groups have been online for at least a year and have participated in chat rooms so they are comfortable and adept at creating online relationships quickly. The guided discussion draws participants out and personalities begin to emerge, thereby creating a dynamic that develops during the group and varies just like in‐person groups. Some suggest that the absence of sight and sound strengthens the use of perceptual senses by not relying on subjective visual and vocal judgments and interpretations.
In bulletin board groups the interaction between respondents can be extensive and on‐going throughout the days. Respondents choose the best time to participate and often spend much more time than required answering questions and interacting with others. It is not unusual for them to pose additional insightful questions for others to consider and answer. In fact, this involved level of participation is encouraged from the outset.
In comparing real‐time groups with in‐person groups it would be fair to say that the interaction in real‐time, online groups can be more limited. The amount of interaction between respondents in this fast‐paced environment can vary and may be influenced by the topic and moderator. At the same time, unrelated small talk and long stories are far less likely.
One interesting and overall difference online is that it is much more difficult for someone to dominate a group. An overbearing respondent does not have the same power or influence with words in the dialogue stream. All respondents are composing their responses simultaneously and not waiting for others to respond. The online environment can create a more even playing field and can mitigate other influencing factors such as gender, age, ethnicity, accents, physical appearance or condition, and shyness, not to mention the weather. Also, there tends to be less inhibition online and respondents are often more direct in stating their opinions and less likely to edit their thoughts because time is of the essence and input takes time.
Conclusion
The future promises many imaginable advancements in online focus groups including sophisticated visual aids, real‐time video and sound, accurate voice recognition, and videoconferencing of individual respondents. If the next five years bring the magnitude of changes as the past five years, there will be many more advancements, unimaginable at this time.
Qualitative caveat
Results from online groups depend on the expertise and qualifications of the professional who is conducting them. All of the information provided in this article assumes that the most knowledgeable and qualified professionals to conduct online groups are qualitative researchers who have research and marketing expertise as well as experience with managing group interactions. Sometimes “techies” attempt to do groups because they are comfortable with the technology and mechanics and some even have experience with chat groups. However, they often lack research, analysis, moderating, and marketing expertise and the results can suffer from these deficiencies.
Glossary of terms
Brand verification: identification of specific brands of products known, purchased, or used by participants.
Final screener spreadsheet: list of individuals expected to participate in study which includes breakdown of demographic, psychographic, or behavioral variables.
“High touch” interactions: extra care or attention used to communicate with certain individuals to ensure desired results and minimize possibility of misunderstanding (e.g. executive level contacts, cultural differences).
Imagery exercises: verbal, written, or schematic illustrations of imagined traits or characteristics of a product or brand.
Mind mapping: schematic representation of one’s thoughts, feelings, or beliefs.
Pantry recall/checks: respondents are asked to recall or take an inventory of products in their home.
Picture sorts: placing pictures into two or more groups according to perceived characteristics.
Rescreener information: questionnaire administered immediately prior to participation in focus group to verify that respondents meet required criteria.
Screener: questionnaire designed to qualify potential respondents based on specified demographic, psychographic, or behavioral criteria.
Techies: computer and Internet technology professionals.
Website intercept: visitor to a Website is presented with request, often via pop‐up window, to participate in an activity (e.g. answer questionnaire, enter contest, request sales information).