Recipient orinentation in verbal report protocols: Methodological issues in concurrent think-alaud

Abstract

Verbal report protocols have been widely used as a research methodology to gain information about cognitive processes. Although such self-reports have been considered direct representations of cognitive processes if elicited under appropriate conditions (Ericsson & Simon, 1984, 1993), some researchers (e.g., Witte & Cherry, 1994; Smagorinsky, 1998, 2001) have argued that verbal reports are socially situated constructs rather than merely representations of the thought processes unfolding in the individual mind. The present study examined the social nature of verbal reports, particularly focusing on whether and in what ways concurrent think-aloud (TA) protocol data in interlanguage pragmatics research are recipient designed. Verbal reports were collected from eight native speakers of Japanese engaged in answering a rating scale instrument on refusal strategies. The data were audio-taped, coded, and analyzed qualitatively. These data suggest that verbal reports elicited by means of TA procedures do contain interactive and social features, that the participants orient to a listener while carrying out the protocol, and that they are selective about what information to report while carrying out the task. The verbal report data gathered in Japanese revealed much information that is not available from data in English. The results of this study suggest that treating verbal report protocols as solely cognitive products under-represents what they actually reveal. Protocols are socially and interactively constituted, and this fact has to be taken into consideration when analyzing TA data. The strong orientations to the listener observed in this study suggest that a different recipient might evoke different content or types of protocol, just as other social factors could influence the data. Therefore, researchers need to take into consideration that verbal report is a socially situated activity when they collect, analyze, and interpret protocol data.