FFQ are often used to measure diet in observational studies because they provide a standardised and cost-efficient approach for collecting data on usual food intake. Many FFQ have been developed and used in a variety of ways, ranging from capturing usual intake among large, population-based samples(Reference Subar, Thompson and Kipnis1) to tailoring the questionnaire to measure intake of a particular nutrient, food, or food group in small, specialised samples. Validation studies have found weak correlations between FFQ and other dietary assessment measures, including food records(Reference Freedman, Potischman and Kipnis2), 24 h recalls(Reference Schatzkin, Kipnis and Carroll3) and biomarkers(Reference Subar, Kipnis and Troiano4). The research community is actively debating whether future nutrition research should incorporate FFQ(Reference Kristal, Peters and Potter5–Reference Willett and Hu8). Given that few suitable alternatives currently exist for large studies, it is worthwhile to consider potential improvements in the analysis and administration of FFQ.
Administration of FFQ is typically paper-based rather than web-based. Web-based administration of the questionnaire may impact the data quality in several ways. Participants with Internet access can complete an FFQ at any time from any location with centralised monitoring of participant completion. Missing data can be minimised by adding alerts to users and automating skip or branching logic. Data processing can be facilitated by eliminating scanning of paper forms. Web-based administration can also offer aids such as illustrations of food portions to improve portion size estimation and recognition of the food. The use of food pictures may also reduce the reading level needed to complete an FFQ. The use of computer-based questionnaires, however, may exclude segments of the population without access to or ability to use computers.
Web-based versions of two of the more widely used questionnaires have been developed and are available(9, 10), but no published data estimate how the aforementioned factors associated with web-based administration influence relative validity and repeatability of dietary assessment. Though other paper-based FFQ include pictures to depict portion sizes, the impact of pictures on FFQ validity has not been previously reported.
A web-based pictorial diet history questionnaire (Web-PDHQ) was developed by adding pictures to the National Cancer Institute’s DHQ(Reference Subar, Thompson and Kipnis1) to represent portion sizes. The hypothesis was that participants would be able to provide more accurate and complete accounts of usual intake using the Web-PDHQ since portion size estimates should be improved using a pictorial form of a diet history. A randomised, controlled trial (n 218) was conducted to determine whether the Web-PDHQ provided an improved estimation of dietary intake compared to a non-pictorial, paper-based version (Paper-DHQ) of the same questionnaire.
Methods
Development of the Web-PDHQ
A registered dietitian obtained, prepared and measured foods described in the National Cancer Institute’s DHQ using standard measuring cups and a portion control scale. Foods were placed in the centre of a plate with common utensils to provide perspective. Each food portion was professionally photographed using a high-quality digital camera from an angle and distance comparable to the view of these foods while sitting at a table. The resulting photographs were reviewed and prepared in jpeg format to be used on the web site. Photographs were linked with the 124 items from the National Cancer Institute’s DHQ(Reference Subar, Thompson and Kipnis1) asking users to indicate the frequency and amount of consumption of each particular food over the past year. In accordance with the DHQ, users were first asked to identify the frequency of consumption for an individual food. Once it was identified that the food was consumed, users were presented with a display of the food in the portion sizes specified on the DHQ (typically two portion sizes) and asked to select the amount typically consumed at one sitting.
Study participants
Participants were recruited from advertisements placed in The Washington Post. Adults aged 18 years or older who were computer literate, defined as using the Internet at least three times per week, were eligible to participate in the study. The study was approved by the PICS Institutional Review Board and each participant provided written, informed consent.
Study design
The study was conducted between April and July 2006 at the PICS office in Reston, VA, USA. All the 218 participants were assigned to complete two versions of the DHQ spaced 4 weeks apart: the Internet-based, pictorial version (Web-PDHQ) and the traditional paper and pencil version (Paper-DHQ). The order of the administration was randomly assigned using the random numbers generator located at www.random.org.
Study procedures
Participants were screened via phone and scheduled for an initial appointment. At the first visit, participants reviewed and signed informed consent, completed a demographic questionnaire and were weighed using a Tanita digital scale (model WB-110A).
Based on the randomisation assignment, the participant was asked to complete either the self-administered Web-PDHQ or the Paper-DHQ. During the subsequent 4-week period, the dietary intake of participants was assessed using a 4 d food record and two 24 h dietary recalls. Timing of food records was assigned as four consecutive days to include one weekend day and was determined during the initial visit. The 24 h recalls were administered on two non-consecutive days, which were randomly assigned by the research assistant based upon a randomisation table. The assigned research assistant attempted to contact the participant in the morning and afternoon, if necessary, on the assigned day to collect 24 h recall data. If they were not able to collect the information on that day, additional calls would be made for two consecutive days in an attempt to collect data for the 24 h prior. Each participant was provided with a 24 h recall kit containing measurement tools to facilitate portion size estimation. Procedures for administering 24 h recalls were adapted for use without an automated system from the US Department of Agriculture's multiple pass format (quick list, time, occasion, and place, forgotten foods list, food details and review)(Reference Conway, Ingwersen and Moshfegh11, Reference Conway, Ingwersen, Vinyard and Moshfegh12). A registered dietitian trained research assistants to conduct telephone interviews. Data collection was completed on a standardised form using standardised language across all participants and participants were asked to refer to their food portion estimation handouts throughout the interview. Data collection began with the first thing the participant ate or drank when they awoke the morning before through 24 h later.
Fifty participants who received the Web-PDHQ were randomly selected to assess repeatability of the Web-PDHQ. One week after the initial visit, these participants were asked to repeat the Web-PDHQ. Test–retest participants who did not have access to a high-speed Internet connection were asked to return to the PICS office for an additional visit to complete this repeat administration.
At the conclusion of the 4-week period, all participants returned to the research office and the other DHQ was administered. Food records were reviewed with participants to ensure writing was legible and records were complete. Participants rated usability of both the Web-PDHQ and Paper-DHQ. Using 10-point Likert scales, participants were asked to rate ease of use, ability to change answers and ability to accurately estimate portion sizes. For the Web-PDHQ only, participants rated the usefulness of the pictures of each portion size.
Statistical analysis
Both the Paper-DHQ and Web-PDHQ were processed using the National Cancer Institute’s Diet*Calc software which provides raw energy and nutrient intake estimates(13). The data were ported into both SPSS (SPSS Inc., Chicago, IL, USA) and STATA Version 8·2 (STATA, College Station, TX, USA) for analysis. Food intake obtained from the food records and 24 h recalls were entered and analysed using ESHA FoodProcessor SQL version 9·5·0 (ESHA, Salem, OR, USA). A registered dietitian coded all food records, and trained research assistants coded 24 h recalls. No formal quality control measures were implemented to assess intra-individual variability in the coding of 24 h recalls.
Descriptive statistics were computed to describe participant characteristics and nutrient intake measured by the four nutrient assessment methods. Due to non-normality, Box–Cox transformations were applied to energy and nutrient intakes.
To assess concurrent validity, nutrient values obtained from the Web-PDHQ were compared to the Paper-DHQ, food record and the average of the two 24 h recalls using Pearson correlations. Repeatability was measured using Pearson correlation coefficients between Web-PDHQ 1 and 2 among the subset of fifty participants asked to complete the questionnaire twice. Data were not corrected for attenuation due to random error associated with within-person variability (errors between the comparison dietary assessment methods are correlated, thereby violating one of the assumptions underlying this approach). Correlations were adjusted for energy intake using the residual approach. Overall and sex-specific estimates were calculated, but data were combined by sex for presentation since stratified estimates were similar by gender.
Results
Study participants
Of the 218 participants enrolled in the study, 217 completed the web-based PDHQ and 215 completed the paper-based DHQ (Fig. 1). Four Web-PDHQ were excluded from the analysis due to a procedural problem resulting in data loss. Three Paper-DHQ were missing due to procedural errors. All 217 participants who completed the study returned a completed food record. Three participants did not complete one of the two recalls, but all 218 participants completed at least one recall. Two of the fifty participants randomly assigned to repeat the Web-PDHQ 1 week later declined to complete the second administration.
Fig. 1 Study participant flow in the web-based, pictorial diet history questionnaire (PDHQ) trial
The majority of the study participants were female, White, and all reported at least 12 years of education (Table 1). Consistent with the mean age and education level, 49 % were employed with 34 % retired, 11 % homemakers, 3 % unemployed, 2 % disabled and 1 % students. Sixty-four per cent were married or living with a partner.
Table 1 Characteristics of study participants in the Web-PDHQ Validity Trial (n 218)
Web-PDHQ, web-based pictorial diet history questionnaire.
Mean height was 167·6 (sd 10·2) cm (66 (sd 4) in.) and mean weight was 76·2 (sd 19·1) kg, range 45·8–160·1 kg) (168 (sd 42) lb, range 101–353 lb). Thirty-one per cent (n 67) of the study sample was overweight (BMI 25–30 kg/m2) and 27 % (n 59) were obese (BMI > 30 kg/m2). Twenty-nine per cent reported having received some form of prior instruction in food portion estimation.
Comparison between dietary assessment measures
Table 2 presents medians and inter-quartile ranges obtained from the Web-PDHQ, Paper-DHQ, 4 d food record and the average of the two 24 h recalls for each of the food label nutrients. Median values for the Web-PDHQ were similar to the Paper-DHQ as well as the other two dietary assessment measures, but the Web-PDHQ produced slightly higher energy and nutrient values compared to the Paper-DHQ.
Table 2 Summary of energy and nutrient estimates by dietary assessment method
Paper-DHQ, paper version of diet history questionnaire; Web-PDHQ, web-based pictorial diet history questionnaire; RE, retinol equivalents; α-TE, α-tocopherol equivalents.
Data are median (interquartile range; 25th percentile, 75th percentile).
Concurrent validity
Unadjusted correlations between Box–Cox transformed values obtained from the Web-PDHQ and Paper-DHQ ranged from 0·60 for zinc to 0·81 for vitamin A (Table 3). Energy-adjusted correlations were lower, ranging from 0·28 to 0·73. Moderate mean correlations (unadjusted 0·41 and 0·38; energy-adjusted 0·41 and 0·34) were obtained between both the Web-PDHQ and Paper-DHQ with the 4 d food record on energy and nutrients, but the correlations between the Web-PDHQ and Paper-DHQ with the 24 h recalls, were modest (unadjusted 0·31 and 0·29; energy-adjusted 0·37 and 0·26). Paper-DHQ correlations with the 24 h recall and food record were similar to the Web-PDHQ correlations.
Table 3 Pearson correlations between dietary assessment measures, unadjusted and adjusted for energy intake by the residual method
Paper-DHQ, paper version of diet history questionnaire; Web-PDHQ, web-based pictorial diet history questionnaire; RE, retinol equivalents; α-TE, α-tocopherol equivalents.
Repeatability
Mean reported intake was generally higher for the first Web-PDHQ administration compared to the second (Table 4). Mean unadjusted correlation between the two Web-PDHQ administrations within a 4-week period was 0·82.
Table 4 Summary of energy and nutrient estimates for Web-PDHQ administrations spaced 1 week apart
Web-PDHQ, web-based pictorial diet history questionnaire; RE, retinol equivalents; α-TE, α-tocopherol equivalents.
Data are median (interquartile range; 25th percentile, 75th percentile).
Usability ratings
Subjective ratings were on a scale of 1–10 with 10 being the most favourable. Mean ratings for ease of completion were 9·4 (sd 1·2) for the Web-PDHQ and 9·2 (sd 1·4) for the Paper-DHQ out of 10, for both. Participants rated the Paper-DHQ as more amenable to modifying responses compared to the Web-PDHQ (mean 9·4, sd 2·8 v. 8·7, sd 2·2, P = 0·003). Participants rated their ability to accurately estimate portion sizes slightly higher with the Web-PDHQ (mean 8·2, sd 1·6) compared to the Paper-DHQ (mean 7·8, sd 2·0; P = 0·01). For the Web-PDHQ, 66 % of participants (n 214) relied on the pictures for at least half of the questions with only 9 % stating that they did not use the pictures at all.
Discussion
These data support comparable validity and repeatability of the web-based, pictorial DHQ compared to the traditional paper-based DHQ. Mean unadjusted correlation of energy and the twenty-five examined nutrients between the paper and web-based versions of the DHQ administered one month apart was 0·71. Mean energy-adjusted correlation between the two DHQ was lower (r = 0·51). This is contrary to other FFQ validation studies, as energy adjustment typically does not affect or improves correlations(Reference Subar, Thompson and Kipnis1). However, previous validation studies excluded individuals reporting extreme energy intakes. We chose not to exclude data based on reported energy intake as we were concerned this would lead to an inflated estimate of relative validity and repeatability.
In the sub-sample of individuals (n 48) repeating the Web-PDHQ 1 week later, the correlations between the two Web-PDHQ administrations ranged from 0·77 to 0·85. These estimates are higher than those reported for other FFQ (0·5–0·7)(Reference Cade, Thompson, Burley and Warm14). Since the two FFQ were administered 1 week apart, intake likely did not vary very much, resulting in high correlations.
With the notable exception of energy, the correlations between the Web-PDHQ and both the food record and 24 h recalls were also consistent with what has been reported in the literature(Reference Subar, Thompson and Kipnis1). This is likely due to the differences in approach for data with extreme energy intakes. Similar to the current study, the Eating at America’s Table Study reported deattenuated, unadjusted correlations ranging from 0·4 to 0·6 between the DHQ and four 24 h recalls(Reference Subar, Thompson and Kipnis1). Though not directly comparable to the present study because four 24 h recalls over a 1-year period were used as the criterion measure rather than food records, the similarity in the correlation coefficients between the two studies supports the consistency of the measurement properties of the Web-PDHQ and the paper-based DHQ.
Our approach also differed from other validation studies in that we did not adjust for measurement error using deattenuated correlations. Adjusting for measurement error requires two assumptions: errors in self-reported dietary intake are dependent on true intake and errors in 24 h recalls are not independent of errors in the FFQ. Evidence from biomarker studies suggests these assumptions are violated, and adjusting for measurement error likely overestimates the true correlation between instruments(Reference Neuhouser, Tinker and Shaw15). The correlations between the Web-PDHQ and 24 h recalls were lower than those using the Paper-DHQ or food records as the comparison measure. Two days of dietary recalls in a 1-month period may have been insufficient to account for the intra-individual variation in food intake being reported over a 1-year period for both forms of the DHQ. In addition to intra-individual variation, other potential sources of error include portion size estimation errors, staff experience conducting 24 h recalls and nutrient analysis differences between the ESHA FoodProcessor Database and Diet*Calc Software.
Although entry criteria were broad in an attempt to improve generalisability, our study population was predominantly white, female, older and more educated than the general US adult population. As a result, one of the potential advantages of using a pictorial DHQ, improved understanding and assessment in those with limited reading ability, likely had limited impact on this highly educated sample. Future research should assess if a pictorial DHQ increases the repeatability and validity of responses from low-literacy participants.
The current study provides evidence to support the equivalence of a new administration method for a cognitively based FFQ to the paper-based approach(Reference Thompson, Subar and Brown16). We had hypothesised that pictures of actual food portions would improve the accuracy of the DHQ by reducing the measurement error due to food portion estimation errors, but the addition of the food pictures did not appear to improve the relationship of the DHQ to other food intake measures. This could be because food portion reporting was not improved by the pictorial representations or because the combination of prior experience with portion estimation in this sample (29 % received prior training) and the portion estimation training required to complete the 24 h recalls and food records for this study reduced the effect of the pictorial representations on food portion estimation. Although the Web-PDHQ does not appear to be superior to the DHQ on concurrent validity with other food intake measures, there are practical advantages of a web-based DHQ including remote administration, immediate nutrient analyses and potential reductions in missing responses, which may be of value for some research purposes. The Web-PDHQ does appear to be strongly associated with the Paper-DHQ and has similar psychometric properties as the Paper-DHQ, indicating that these two forms of DHQ administration produce comparable results.
Acknowledgements
Funding support:This research was supported by NIDDK (DK061079) and Department of Health and Human Services, NIH NEI Training grant Number EY 07127 Clinical Trials Training Program in Vision Research.
Conflict of interest:The authors have no conflicts of interest. The authors were previously employed at PICS when this study was conducted. PICS has commercial interest in the programme described in this article, but the authors do not.
Authors’ contributions:All authors were involved in the design of the study, A.D. served as project manager for the trial, J.M.B. performed the data analysis and drafted the manuscript and W.T.R. secured funding and contributed to writing the manuscript.
