Let us briefly summarize the two main points Lyle Bachman made in his discussion paper. The first was that he was concerned about the variety of abilities, or as he put it “inconsistency” of subscores, that make up the overall score in each cell of the multitrait multimethod matrix; second, he wondered whether this lack of consistency in the subscores might account for the factor analyses in the large-scale proficiency study not revealing the traits we were looking for, and whether it might not be useful to look further at the detailed correlations among the 40+ subscores and try to find more consistent factors that would perhaps cut across the three traits we had originally proposed. As he noted, we even hinted in the second-year report (Allen et al. 1983) that we were going ahead with something like the factor analysis of the detailed scores that he is proposing.

To take the second point first, the reason we did not pursue the analysis of the detailed scores was simply that the subscores did not appear reliable enough to warrant such an analysis. Actually, what we had planned to do was slightly different from what Bachman is proposing – we were going to try to find evidence for our original constructs in the detailed analysis, whereas Bachman is suggesting that we might look for evidence of a new, more consistent set of constructs that would cut across our original set.