Traditionally, printed questionnaires distributed by paper mail have been used to collect data in epidemiological studies. Potential problems with printed questionnaires, such as incomplete or unreasonable answers, may be reduced by the use web-based questionnaires. Web-based questionnaires have several advantages compared with traditional printed questionnaires that make the data collection more efficient, but there are also disadvantages that need to be considered(Reference Couper1–Reference Bälter and Bälter Augustsson3). One of the main challenges with web questionnaires is to convince people to voluntarily sit in front of a computer for a long period of time.
However, many people already spend a lot of their spare time in front of the computer playing computer games. The idea of taking inspiration from computer game design into other applications than games per se, especially when the goal of a task is routine or boring, was first presented by Malone in 1981(Reference Malone4). Today the idea of interactivity has been put to use in education(Reference Jenkins, Klopfer and Squire5), training, health, public policy, political campaigns(Reference Zaphiris and Siang Ang6), to counteract dementia(7), in computer-supported cooperative work, learning and human–computer interaction in general(Reference Davis and Carini8).
We believe that interactivity in the form of feedback in web questionnaires makes filling out the questionnaire a more pleasurable process and thereby motivates the study participants to complete the questionnaire and stay in the study. In short, personalized feedback can be given based on the participant's answer to one or several questions; for example, questions about physical activity. After the participant answers questions on physical activity, the questionnaire gives back information regarding the person's energy expenditure (see Bälter(Reference Bälter9) for live examples).
Here we report from a prospective follow-up study aimed at studying the effect of lifestyle factors, such as physical activity, diet and stress, on incidence of upper respiratory tract infections (URTI). URTI is by far the most frequent disease in the industrialized world, being the number one reason for staying home from work as well as attending work while sick, possibly spreading the disease(Reference Adams, Gerry and Hendershot10). At baseline in the present study, we used an interactive web questionnaire with feedback. URTI is a short-lived and recurrent disease, and the main challenge was to assess incidence of URTI in the study population. Therefore, we used email to administer short web-based questionnaires asking about incidence of infections and symptoms every third week. In order to achieve high compliance, i.e. the willingness to continue answer questions and stay in the study, we introduced interactive personalized feedback in the baseline questionnaire as well as feedback based on answers from the whole group in the follow-up questionnaires as non-economic incentives. The aims of the present study were to show the proportion of participants who chose to view personalized feedback, if there were any associations between actively choosing personalized feedback in the baseline questionnaire and completing the follow-up questionnaires, which type of feedback was mostly used, and if there were any age and gender differences.
Participants and methods
The study Lifestyle and Immune function (LIME) is a population-based cohort study, and results regarding lifestyle factors and URTI have been described previously(Reference Fondell, Christensen and Bälter11, Reference Fondell, Trolle Lagerros and Sundberg12). Here we focus on feedback as a feature in the web questionnaires and compliance for the follow-up questionnaires. Compliance is defined as the number of responders to the follow-up questionnaire divided by the number of eligible participants for the follow-up questionnaire. The study population consists of men and women, aged 20–60 years, residing in a middle-sized county in Sweden with 80 % of the inhabitants living in a city and 20 % in rural areas. The study was approved by the Ethics Committee at the Karolinska Institutet.
In total, 5000 individuals were randomly selected from the Swedish population registry and invited to participate in the study. An information letter about the study was sent out via regular paper mail on 9 January 2004, and the second letter including information on how to access the web questionnaire, details on use of a web browser, the URL to our web questionnaire and an individual password was sent out on 23 January. One reminder letter was sent out to non-respondents on 9 February.
Baseline questionnaire
The interactive web-based baseline questionnaire included questions about demographic, anthropometric and lifestyle questions. Food habits were assessed using a validated ninety-six-item FFQ, stress was assessed using the Perceived Stress Scale by Cohen et al.(Reference Cohen, Kamarck and Mermelstein13) translated into Swedish, and physical activity was assessed using a method developed and validated by Trolle Lagerros et al.(Reference Trolle Lagerros, Mucci and Bellocco14). The respondents were given the option of getting personalized feedback based on their self-reported answers and the feedback was given shortly after the respondents had finished the questions that were linked to the feedback. In order to get feedback, the respondent actively had to answer, ‘Yes, I would like to get feedback based on this question’ or decline feedback by selecting ‘No’. Feedback was given in four areas and in the following order: (i) BMI; (ii) energy expenditure based on total physical activity; (iii) intake of nutrients (vitamin C, Ca, Fe and fibre); and (iv) meal composition. Apart from the participant's outcome for these variables, they were given information about their result compared with national recommendations. For example, their intake of vitamin C was compared with the Nordic Nutritional Recommendations regarding intake of vitamin C. If their intake was lower than recommended, information on how to improve the intake (or encouragement if they already were within the recommended limits) was given. The feedback was given as numbers with descriptive text. Due to a technical problem in the web questionnaire, data regarding who chose feedback on BMI are missing. The question of who wanted feedback about BMI was given the same question identification number as another question that came later in the questionnaire; therefore only data from the latter question were stored. This was an unfortunate technical error in the web questionnaire system used for this study, but the error did not affect any other questions. Therefore, we only show data regarding feedback on energy expenditure, intake of nutrients and meal composition.
In the baseline questionnaire we also asked for the participant's email address, and all communication with the participants was thereafter done by email.
Follow-up questionnaires
Out of the 1805 respondents who completed the baseline web questionnaire, sixty were excluded from further analyses on follow-up. The main reason for exclusion was that respondents were unable to receive emails from us. Seventeen participants did not have an email address and another twenty respondents chose not to disclose it. Twenty-three respondents were excluded because our emails with follow-up questionnaires were filtered as junk mail and therefore they did not get the questionnaires. All of these twenty-three individuals worked at one of the largest working places in the county, and it was after one of them contacted us that we found out about the problem with filtering. This might have been a potential problem for other participants as well, but we were not aware of it. After exclusions at baseline, 1745 respondents (56 % women) were invited to the follow-up questionnaires.
Follow-up questionnaires asking about incidence of respiratory tract infections were sent out via email every 3 weeks, on 23 February, 15 March, 1 April, 26 April and 17 May. One reminder to non-responders was sent out via email 10 days after each follow-up letter.
Feedback was included in the follow-up questionnaires as well. However, this feedback was not on an individual level but rather on a group level. Each time we contacted the participants, i.e. every third week, we informed them about the results from the previous follow-up. The feedback was expressed as ‘In the previous follow-up, 23 % of all the participants reported that they had, or had had, a respiratory tract infection’.
Case ascertainment
Self-reported incidence of URTI was assessed at baseline and in all five follow-up questionnaires during the 15-week study. Participants were asked if they currently had an infection or if they had had a new infection during the last 3 weeks, or since the last questionnaire. Follow-up questions about symptoms were given to all participants who reported an infection in order to aid self-diagnosis. These symptoms were runny nose, cough, sore throat, headache, fever, malaise and other non-specified symptoms. In total, there were 1181 self-reported cases of URTI and, in total, 16 985 person-weeks of follow-up.
Statistical analyses
Descriptive statistical analyses were performed using the STATA Intercooled statistical software package version 10·1 (Stata Corporation, College Station, TX, USA).
Results
Baseline characteristics of the study population by age are shown in Table 1. Increasing age was associated with higher BMI, a higher proportion of former smokers and fewer never smokers, a lower level of education and lower levels of stress, whereas there were no trends regarding prevalence of asthma and intakes of energy, carbohydrate, protein or saturated fat.
Table 1 Baseline characteristics of the participants by age: Swedish men and women, Lifestyle and Immune function (LIME) study
*As assessed using the Perceived Stress Scale of Cohen et al.(Reference Cohen, Kamarck and Mermelstein13): low, score <23 (below median); high, score >23 (above median).
Out of the 1805 participants who responded to the baseline questionnaire, 88 % actively chose to view personalized feedback in the questionnaire, whereas 32 % took advantage of all three possibilities to get feedback. In the subgroup of participants that later on completed all five follow-up questionnaires, 91 % used feedback at least once and 34 % used all three feedback options (Table 2).
Table 2 Number of participants who chose personalized feedback in the baseline questionnaire among all participants and among those who completed all five follow-up questionnaires: Swedish men and women, Lifestyle and Immune function (LIME) study
*Excluding sixty participants who were unable to participate in the follow-up questionnaires because they did not have an email address (n 17), did not want to disclose it (n 20) or worked at a workplace that filtered our emails with follow-up questionnaires as junk mail (n 23).
Table 3 shows the initial response rate for the baseline questionnaire and the interest in feedback by age and gender. The initial response rate for the baseline questionnaire was higher for women than for men in corresponding age groups. The type of feedback that was mostly used was energy expenditure, followed by intake of selected nutrients and then meal composition, and there were no differences with regard to age or gender. Nor were there any large differences between men and women with regard to completing all five follow-ups, except for the highest age group, where the compliance was 48 % in men and 70 % in women.
Table 3 Initial response rate for the baseline questionnaire, interest in personalized feedback and compliance by age and gender: Swedish men and women, Lifestyle and Immune function (LIME) study
*Excluding sixty participants who were unable to participate in the follow-up questionnaires because they did not have an email address (n 17), did not want to disclose it (n 20) or worked at a workplace that filtered our emails with follow-up questionnaires as junk mail (n 23).
The compliance, i.e. the proportion completing each follow-up questionnaire, ranged between 83 % and 84 % for the whole group. In total 1286 responded to all five follow-ups, which means that the proportion completing all five follow-ups was 74 %. Table 4 shows compliance for the follow-up questionnaires depending on the use of feedback in the baseline questionnaire. The compliance for completing all five follow-ups among the respondents who chose feedback three times was 76 % and 58 % among those who did not use feedback (P < 0·05). A similar trend was seen for each of the individual follow-up questionnaires.
Table 4 Compliance for the follow-up questionnaires depending on the amount of feedback chosen in the baseline questionnaire: Swedish men and women, Lifestyle and Immune function (LIME) study
Discussion
We found that almost 90 % of the participants actively chose to take part in the personalized feedback that was offered to them in the baseline questionnaire. The type of feedback that was most frequently chosen was feedback regarding energy expenditure from physical activity, followed by intake of nutrients and then finally meal composition. Among all the participants eligible for follow-up, the compliance for each follow-up questionnaire was 83–84 %, and 74 % completed all five follow-up questionnaires. The group that chose feedback all three times in the baseline questionnaire had higher overall compliance for the follow-up questionnaires compared with the group that never used feedback. There were no differences with respect to age or gender and the use of feedback.
The personalized feedback used in our study gives the respondent something that has merit outside the survey situation. It can therefore be seen as a non-economic incentive for the respondent to keep answering the questionnaire. However, one could argue that we have only shown that respondents who already are motivated to complete a series of questionnaires also appreciate personalized feedback. If we had given the respondents information about feedback from the beginning, the feedback per se could have influenced the recruitment of respondents. However, we did not inform the participants about feedback by the time they were invited to the study, and the participants were therefore recruited independently of their interest in getting personalized feedback.
Feedback based on the respondents’ own answers must always be carefully considered, adapted and tested within each specific study population, and should always be a free choice for the respondent. The ethics committee must approve the feedback that is given in the same manner as for any other part of a study. The feedback should be in form of gentle advice that can be helpful for the participant's future decisions regarding her or his lifestyle. It is important to inform the participants that the feedback given is not to be considered as a diagnosis and that further questions or concerns should be discussed with a physician(Reference Lorimer, Gray and Hunt15). Also, when offering feedback on questionnaire data, one should mention that the feedback given, although based on the participant's own responses, may not truly reflect his/her status. For example, an FFQ does not capture the entire diet. In our study we used text-based feedback, but a previous study showed that visual feedback using illustrations was preferred by a test panel compared with text-based feedback(16). This opens up future experimentation regarding how the feedback is presented and perceived by the participants.
The response rate for web-based questionnaires in epidemiological studies is of great concern. In a population-based mixed-mode study by us, the response rate was 50 % in the group that was invited to fill out a web questionnaire without personalized feedback. The corresponding response rate was 65 % in the group that was invited to fill out the same questionnaire as a printed questionnaire. A multicentre study in Europe found that the response rate for a web-based questionnaire on food habits was high, even though no feedback or any other incentive was offered(Reference Illner, Harttig and Tognon17). The participants were recruited from existing cohort studies and the average response rate response was 65 %, but ranged between 37 % and 87 % for different countries. However, in cohort studies, the initial response rate is of less importance as compared with compliance, i.e. the willingness to stay in the study, when it comes to the validity of the study(Reference Rothman18). The high compliance (83–84 %) at each follow-up, despite only one reminder and no economic incentive, may be due to the feedback as well as the feature of sending emails with a link to the follow-up questionnaire. To our knowledge, no one has made a similar study of common cold before using multiple follow-ups. One reason is probably that the logistics of handling a large study with many follow-ups in such a short time is just too burdensome if it is done in a traditional way with printed questionnaires and too expensive if personal telephone interviews are used for the follow-up.
Potential drawbacks with using web-based questionnaires include the fact that some people will be excluded due to lack of Internet access or knowledge. Higher demands on the respondents’ computers will exclude even more people; therefore, the balance between a potential increase in compliance and a decrease in the initial response rate must be considered carefully. In order to compare how representative the 1805 respondents in the present study were of the corresponding age group in the general population in Sweden, the study participants’ characteristics were compared with national statistics(19). We found high similarities between the participants in our study and national statistics. For example, we found identical prevalences of asthma (8 %), smoking among men (17 %) and overweight in men (49 % with BMI≥25 kg/m2). There was a slight under-representation of smoking women (16 % v. 18 %) and overweight women (28 % v. 35 % with BMI≥25 kg/m2). As often occurs in web-based studies, there was an over-representation of high education level (≥2 years at university level). We found that 48 % of the men and women in the study reported high education level, as compared with 17 % for men and 20 % for women in the general population in Sweden. However, the county in which the study was conducted has a large university and many academic workplaces, which contributes to the large proportion of well-educated people in the study. Taken together, this shows that it is possible to use web-based questionnaires in a population-based study and get a representative sample to participate.
Ideally, we would have liked to enrol a control group in the study, i.e. a group that did not get feedback, in order to fully explore the effect of feedback per se on compliance. However, to do a parallel study without feedback would have been both expensive and burdensome and beyond the resources at hand at the start of the study. Since we have not found any other study designed in a similar fashion (online, general population, no economic incentive), it is difficult to compare the compliance in our study with the compliance in other studies. A review of the literature shows that, with incentives totalling £460 per respondent, Hurling et al.(Reference Hurling, Catt and Boni20) reached an average compliance of 75 % for the last five weeks out of a 9-week study of a physical activity programme. Verheijden et al. (Reference Verheijden, Jans and Hildebrandt21) reached a compliance of 11 % with one email reminder in a follow-up three months after baseline in a web-based health behaviour change programme. Glasgow et al. (Reference Glasgow, Nelson and Kearney22) reported a compliance of 22 % after the second email follow-up, twelve months after enrolling to an Internet-mediated weight-loss programme, where a $US 10 gift certificate was offered for each follow-up. A paper mail questionnaire sent to the non-respondents along with $US 10 in cash reached 64 %. Anhøj and Møldrup(Reference Anhøj and Møldrup23) used SMS (Short Messaging System) messages in a 2-month study of asthma with twelve patients and reached a mean answering rate of 69 %. Tate et al.(Reference Tate, Wing and Winett24) reached 68 % in an Internet weight-loss programme among ninety-one respondents with incentives totalling $US 35 per person. However, an economic incentive is not feasible in large epidemiological studies, and the effect of a non-economic incentive on compliance is therefore of great interest to explore further.
One of the best compliance rates that we are aware of is in the Health Professionals’ Follow-up Study (HPFS)(Reference Michaud, Augustsson and Rimm25) in the USA. It started in 1986 and included 51 529 men at baseline. After more than 20 years of follow-up, the overall response rate for disease follow-up is approximately 94 % every second year among survivors. In addition, about 75 % of the surviving cohort completes a detailed questionnaire on lifestyle factors every fourth year. Comparison with rates in our study could be inappropriate as their follow-up questionnaire is much longer and most participants fill out a printed version of the questionnaire. On the other hand, the respondents were highly motivated from the beginning, being health professionals, as compared with the general population as in our study. Also, six reminder letters are sent for each follow-up in the HPFS and the final reminder letter has a hand-written address in order to increase the response rate.
Conclusions
The results of the present study show that it is possible to use feedback in web questionnaires and that it has the potential to increase compliance. Future research should focus on improving the design of the feedback, for example using visual feedback instead of text-based feedback, which may ultimately result in even higher compliance.
Acknowledgements
The study was financed in part by a grant from the Osher Center for Integrative Medicine, Karolinska Institutet. Additional funds were received from the Swedish Council for Working Life and Social Research and the Swedish Research Council. The authors declare no conflict of interest. All authors contributed to the study design. K.B. supervised the study. K.B. and E.F obtained funding. K.B., O.B. and E.F. obtained the data. E.F. performed data analysis and K.B., O.B. and E.F. drafted the manuscript. All authors participated in critically revising the manuscript for important intellectual content. The authors would also like to thank Netsurvey for cooperation with the web system and infrastructure.
