It was a pleasure to read and reflect on the thought-provoking paper by three emerging scholars from Baylor (Upenieks et al., Reference Upenieks, Bonhag and McDowan2022). I was especially interested in reading this piece because I was the Principal Investigator on the grants that funded the data they analyze.
I begin by commending the authors for exploring the influence that relationships with pastors have on the psychological well-being of rank-and-file church members. There is a good deal of research on pastors, but there are too few studies on the role that pastors play in the wider web of social relationships in the church. Unfortunately, the measures I devised for the data set the authors used were not fully up to the task of fully evaluating pastoral support. A pastor can provide support in at least four different social contexts: (1) sermons and other facets of formal worship services may convey valuable informational support as well as motivation to adhere to religious precepts; (2) support may be provided by pastors in Bible study and prayer groups, (3) pastors may provide support in formal counseling session; and (4) pastors may provide support during informal interaction that takes place during church social gatherings or hallway conversations. The nature, meaning, and ultimately the impact of pastoral support may vary across these settings. The authors of the current paper could obviously not address these issues with the data that are on hand, but pointing out important next steps plays a meaningful role in the research process. The authors briefly mention the influence of formal pastoral counseling, but as I have just shown, there are other potentially useful contexts to explore outside this one. We need to know which context(s) have the greatest impact on health and well-being.
There are, however, a couple of issues that were under the control of the authors and that merit closer scrutiny. At least three points in the paper, the authors partition continuous measures into ordinal measures: (1) the continuous religious doubt measures are divided into four categories (stable low doubt, high stable religious doubt, increasing religious doubt, and decreasing doubt), (2) the binary church attendance measure contrasts weekly attendance with less frequent attendance, and (3) the binary frequency of private prayer measure contrasts those who pray daily with all others.
The problems with partitioning continuous measures are well-known in the literature. For example, Butts and Ng (Reference Butts, Ng, Lance and Vanderberg2009) bluntly state that, “… the act of chopping up continuous data for subsequent analytic purposes is an unwise practice. The many methodological and statistical problems with such an approach have been repeatedly voiced…” (p. 362). Here is how I think about this issue. The goal of a multiple regression analysis is to explain why scores vary. Chopping up continuous measures, by definition, reduces this variance. This is generally not a good thing because researchers who partition continuous measures literally have less variance to explain. In other words, precision is lost in partitioned analyses.
In addition to problems arising from restricting variance, determining where to divide continuous measures can be challenging. Consider, for example, the category in the current study that assesses stable high religious doubt. The individuals in this category are defined as study participants who, “…did not change their religious doubt scores over time but had scores greater than 1 on the doubt scale at both time points” (Upenieks et al., Reference Upenieks, Bonhag and McDowan2022, p. 10). So, if a person had a score of 2 on the doubt scale at Time 1 and a score of 2 on the doubt scale at Time 2, they are considered to have stable high religious doubt. Why does a score of 2 at both times denote high religious doubt? This issue is complicated by the fact that the authors averaged responses to all the doubt items – I am not sure why this was done nor are the advantages of doing so evident. How might the analyses have changed if different cut points been used (e.g., high stable religious doubt based on a score of 3 on both measurement occasions)?
The issue of determining the proper cut point aside, I would have analyzed the doubt measures in their original continuous format. Please do not misunderstand me. I am not saying that the authors did the “wrong” thing in the way they configured the doubt measures. Partitioned data are frequently found in the medical sciences. Instead, I am saying the authors may not have used the “optimal” procedure.
I want to flag a minor issue in the process of considering the measurement of religious doubt. The authors discuss how they created the four doubt categories. They begin by noting that, “… we followed the exact procedure employed by previous research…” (Upenieks et al., Reference Upenieks, Bonhag and McDowan2022). If I understand this statement correctly, they cite my paper with Chris Ellison as one of these previous studies that use their strategy (Krause and Ellison, Reference Krause and Ellison2009). We used continuous doubt measures throughout our paper: we looked at a continuous measure of doubt at Time 1 and (separately) a continuous measure of doubt at Time 2 on changes in well-being outcomes over time.
The issue of partitioning data is evident elsewhere in the paper by Ulenieks et al. (Reference Upenieks, Bonhag and McDowan2022). The authors are interested in studying gender differences in the relationship between support from a pastor, religious doubt, and depressive symptoms. This is a worthwhile issue to pursue. But in the process of exploring this issue, they partition the sample into two groups: one consisting of men and the other group consisting of women. They subsequently estimate the statistical interaction between pastoral support and the doubt categories within each gender subgroup. I would not have partitioned the data in this manner. Instead, I would have worked with the full sample and tested for a three-way interaction between sex, pastoral support, and doubt on change in depressive symptoms. Among other things, doing so would preserve the statistical power of the analyses.
There are other reasons for avoiding subgroup analyses that may not be readily apparent. The authors look at the two-way interaction between pastoral support and doubt within each sex subgroup after controlling for the effects of several other measures (e.g., age, education, married, race/ethnicity, etc.). By following this strategy, the authors are unwittingly assuming that there is a statistical interaction between each of these measures and sex on depression. I’m not sure it is wise to make this assumption across the full range of independent variables.
I have three more suggestions that might round out the analyses in the Upenieks et al. (Reference Upenieks, Bonhag and McDowan2022) study. First, I think it would be very useful to assess the relationship between pastoral support and religious doubt. There are a range of possibilities here. For example, does support from a pastor lower religious doubt? If pastoral support is effective, it seems that it should do so. How might this affect the analyses in the current study? Alternatively, do some pastors find rank-and-file parishoners with high levels of religious doubt to be repugnant, and as a result, do pastors subtly withdraw from them? These (and other) issues regarding the relationship between pastors and doubters could be handled in a Supplementary Analysis section. If there is insufficient space to do this in the current paper, it might be useful to write a separate paper on these issues. If the authors are interested in delving into this issue in a separate paper, they should look at the classic paper by Wheaton (Reference Wheaton1985). He provides five different ways to model the interface between stress, social support, and psychological distress. Some intriguing possibilities are likely to arise when “pastoral support” is used in place of “social support” in his models.
My second suggestion comes from the paper I wrote with Chris Ellison (Krause and Ellison, Reference Krause and Ellison2009). In a way, the gist of this paper was to show that religious doubt is neither inherently bad nor inherently good. Instead, the effect of doubt might vary depending upon how people cope with it. For some, doubt can be a growth-inducing process. In contrast, for others, religious doubt is something that is shameful and should be suppressed. When all doubt is treated as undesirable (as it is assumed in the Upenieks et al., Reference Upenieks, Bonhag and McDowan2022 paper), the effects of doubt on a health-related outcome are likely to be attenuated.
My third suggestion builds upon the two previous suggestions. Might support from a pastor lower religious doubt by helping regular church members see that doubt can be a growth experience? Evaluating this issue empirically can help close the gap between social psychological research on religious doubt and the needs of pastors who offer formal counseling sessions.
When viewed at the broadest level, my comments identify contingencies in doubt, pastoral support, and psychological distress models. For example, I asked whether these relationships vary by the context in which pastoral support is provided and whether they vary by the way study participants cope with religious doubt. This raises a vexing issue that I have yet to resolve in my own work. How can researchers deal with the seemingly limitless list of factors that may influence the relationship between two variables? In other words, how much minutia (i.e., qualifiers or contingencies) are researchers willing to tolerate in process of trying to explain a relationship adequately? There is no hard and fast answer to this question. Instead, the solution is likely to emerge from the fine art of feeling one’s way through a set of data analysis. This challenge is complicated by the fact that the page limits are often placed on journal articles make it difficult to deal with substantively meaningful supplementary issues.
I hope my overall positive reaction to the Upenieks et al. (Reference Upenieks, Bonhag and McDowan2022) paper is not lost in the process of considering my musings. As I indicated at the outset, the authors did an admirable job with a challenging set of relationships. The social and behavioral sciences are indeed a cumulative enterprise, and I believe their paper adds significantly to the advancement of research on pastoral support, religious doubt, and psychological distress. Now it is time to take additional steps.
You have
Access
It was a pleasure to read and reflect on the thought-provoking paper by three emerging scholars from Baylor (Upenieks et al., Reference Upenieks, Bonhag and McDowan2022). I was especially interested in reading this piece because I was the Principal Investigator on the grants that funded the data they analyze.
I begin by commending the authors for exploring the influence that relationships with pastors have on the psychological well-being of rank-and-file church members. There is a good deal of research on pastors, but there are too few studies on the role that pastors play in the wider web of social relationships in the church. Unfortunately, the measures I devised for the data set the authors used were not fully up to the task of fully evaluating pastoral support. A pastor can provide support in at least four different social contexts: (1) sermons and other facets of formal worship services may convey valuable informational support as well as motivation to adhere to religious precepts; (2) support may be provided by pastors in Bible study and prayer groups, (3) pastors may provide support in formal counseling session; and (4) pastors may provide support during informal interaction that takes place during church social gatherings or hallway conversations. The nature, meaning, and ultimately the impact of pastoral support may vary across these settings. The authors of the current paper could obviously not address these issues with the data that are on hand, but pointing out important next steps plays a meaningful role in the research process. The authors briefly mention the influence of formal pastoral counseling, but as I have just shown, there are other potentially useful contexts to explore outside this one. We need to know which context(s) have the greatest impact on health and well-being.
There are, however, a couple of issues that were under the control of the authors and that merit closer scrutiny. At least three points in the paper, the authors partition continuous measures into ordinal measures: (1) the continuous religious doubt measures are divided into four categories (stable low doubt, high stable religious doubt, increasing religious doubt, and decreasing doubt), (2) the binary church attendance measure contrasts weekly attendance with less frequent attendance, and (3) the binary frequency of private prayer measure contrasts those who pray daily with all others.
The problems with partitioning continuous measures are well-known in the literature. For example, Butts and Ng (Reference Butts, Ng, Lance and Vanderberg2009) bluntly state that, “… the act of chopping up continuous data for subsequent analytic purposes is an unwise practice. The many methodological and statistical problems with such an approach have been repeatedly voiced…” (p. 362). Here is how I think about this issue. The goal of a multiple regression analysis is to explain why scores vary. Chopping up continuous measures, by definition, reduces this variance. This is generally not a good thing because researchers who partition continuous measures literally have less variance to explain. In other words, precision is lost in partitioned analyses.
In addition to problems arising from restricting variance, determining where to divide continuous measures can be challenging. Consider, for example, the category in the current study that assesses stable high religious doubt. The individuals in this category are defined as study participants who, “…did not change their religious doubt scores over time but had scores greater than 1 on the doubt scale at both time points” (Upenieks et al., Reference Upenieks, Bonhag and McDowan2022, p. 10). So, if a person had a score of 2 on the doubt scale at Time 1 and a score of 2 on the doubt scale at Time 2, they are considered to have stable high religious doubt. Why does a score of 2 at both times denote high religious doubt? This issue is complicated by the fact that the authors averaged responses to all the doubt items – I am not sure why this was done nor are the advantages of doing so evident. How might the analyses have changed if different cut points been used (e.g., high stable religious doubt based on a score of 3 on both measurement occasions)?
The issue of determining the proper cut point aside, I would have analyzed the doubt measures in their original continuous format. Please do not misunderstand me. I am not saying that the authors did the “wrong” thing in the way they configured the doubt measures. Partitioned data are frequently found in the medical sciences. Instead, I am saying the authors may not have used the “optimal” procedure.
I want to flag a minor issue in the process of considering the measurement of religious doubt. The authors discuss how they created the four doubt categories. They begin by noting that, “… we followed the exact procedure employed by previous research…” (Upenieks et al., Reference Upenieks, Bonhag and McDowan2022). If I understand this statement correctly, they cite my paper with Chris Ellison as one of these previous studies that use their strategy (Krause and Ellison, Reference Krause and Ellison2009). We used continuous doubt measures throughout our paper: we looked at a continuous measure of doubt at Time 1 and (separately) a continuous measure of doubt at Time 2 on changes in well-being outcomes over time.
The issue of partitioning data is evident elsewhere in the paper by Ulenieks et al. (Reference Upenieks, Bonhag and McDowan2022). The authors are interested in studying gender differences in the relationship between support from a pastor, religious doubt, and depressive symptoms. This is a worthwhile issue to pursue. But in the process of exploring this issue, they partition the sample into two groups: one consisting of men and the other group consisting of women. They subsequently estimate the statistical interaction between pastoral support and the doubt categories within each gender subgroup. I would not have partitioned the data in this manner. Instead, I would have worked with the full sample and tested for a three-way interaction between sex, pastoral support, and doubt on change in depressive symptoms. Among other things, doing so would preserve the statistical power of the analyses.
There are other reasons for avoiding subgroup analyses that may not be readily apparent. The authors look at the two-way interaction between pastoral support and doubt within each sex subgroup after controlling for the effects of several other measures (e.g., age, education, married, race/ethnicity, etc.). By following this strategy, the authors are unwittingly assuming that there is a statistical interaction between each of these measures and sex on depression. I’m not sure it is wise to make this assumption across the full range of independent variables.
I have three more suggestions that might round out the analyses in the Upenieks et al. (Reference Upenieks, Bonhag and McDowan2022) study. First, I think it would be very useful to assess the relationship between pastoral support and religious doubt. There are a range of possibilities here. For example, does support from a pastor lower religious doubt? If pastoral support is effective, it seems that it should do so. How might this affect the analyses in the current study? Alternatively, do some pastors find rank-and-file parishoners with high levels of religious doubt to be repugnant, and as a result, do pastors subtly withdraw from them? These (and other) issues regarding the relationship between pastors and doubters could be handled in a Supplementary Analysis section. If there is insufficient space to do this in the current paper, it might be useful to write a separate paper on these issues. If the authors are interested in delving into this issue in a separate paper, they should look at the classic paper by Wheaton (Reference Wheaton1985). He provides five different ways to model the interface between stress, social support, and psychological distress. Some intriguing possibilities are likely to arise when “pastoral support” is used in place of “social support” in his models.
My second suggestion comes from the paper I wrote with Chris Ellison (Krause and Ellison, Reference Krause and Ellison2009). In a way, the gist of this paper was to show that religious doubt is neither inherently bad nor inherently good. Instead, the effect of doubt might vary depending upon how people cope with it. For some, doubt can be a growth-inducing process. In contrast, for others, religious doubt is something that is shameful and should be suppressed. When all doubt is treated as undesirable (as it is assumed in the Upenieks et al., Reference Upenieks, Bonhag and McDowan2022 paper), the effects of doubt on a health-related outcome are likely to be attenuated.
My third suggestion builds upon the two previous suggestions. Might support from a pastor lower religious doubt by helping regular church members see that doubt can be a growth experience? Evaluating this issue empirically can help close the gap between social psychological research on religious doubt and the needs of pastors who offer formal counseling sessions.
When viewed at the broadest level, my comments identify contingencies in doubt, pastoral support, and psychological distress models. For example, I asked whether these relationships vary by the context in which pastoral support is provided and whether they vary by the way study participants cope with religious doubt. This raises a vexing issue that I have yet to resolve in my own work. How can researchers deal with the seemingly limitless list of factors that may influence the relationship between two variables? In other words, how much minutia (i.e., qualifiers or contingencies) are researchers willing to tolerate in process of trying to explain a relationship adequately? There is no hard and fast answer to this question. Instead, the solution is likely to emerge from the fine art of feeling one’s way through a set of data analysis. This challenge is complicated by the fact that the page limits are often placed on journal articles make it difficult to deal with substantively meaningful supplementary issues.
I hope my overall positive reaction to the Upenieks et al. (Reference Upenieks, Bonhag and McDowan2022) paper is not lost in the process of considering my musings. As I indicated at the outset, the authors did an admirable job with a challenging set of relationships. The social and behavioral sciences are indeed a cumulative enterprise, and I believe their paper adds significantly to the advancement of research on pastoral support, religious doubt, and psychological distress. Now it is time to take additional steps.