Investigating Chinese Second Language Pragmatic Competence in Interaction Using Paired Speaking Tests

Abstract

Second language pragmatic competence, the ability of language learners to understand and perform the pragmatic functions of target languages in social interactions (Taguchi & Roever, 2017), develops over time and is an important research area. Youn (2015) defines L2 pragmatic competence in interaction as the ability of interactive participants to use different pragmatic and interactive resources to achieve pragmatic meaning and conduct actions in organized sequences. In the current study, the peer-topeer paired speaking test, considered as a way of classroom assessment, was employed to investigate Chinese learners’ second language (L2) pragmatic competence in interaction in the personal language use domain. An analytical rubric was developed based on related conversational organizations and interaction relevant studies for raters to award scores. Mixed method design was employed to analyze the data - test takers’ in-test discourse. The results indicate that open role-play and situational topic discussion (extended discourse) tasks were effective in eliciting interactions for assessing the construct. The test content was based on the needs analysis of the most commonly used situations, topics, and language functions in this domain. When using the analytical rubric to assess test takers’ in-test discourse, inter-rater reliability did not meet established thresholds. The detailed results of DA for 12 excerpts of in-test discourse not only identified additional components (language use and situation response), but also distinguished new interactional features within the three major interactional rating categories (turn-taking organization, sequence organization and topic management). DA revealed that all the rating categories were distinguishable across three different competence levels, a finding that was confirmed via quantitative analyses (descriptive statistics and repeated measures ANOVA). Based on the general interactional features summarized from in-test discourse, the developmental trajectory of Chinese learners’ L2 pragmatic competence in interaction was summarized by five elements: frequency, proactivity, complexity, content and coherence. Specifically, as L2 pragmatic competence in interaction develops, learners are more active, and their cognitive abilities are more capable of dealing with faster turn-takings, more complex structures, and the more coherent delivery of deeper content. The findings from the mixed method approach were strengthened and could help to revise the analytical rating rubric and improve future rater training. In summary, the findings offer the potential to contribute to the future assessment of Chinese learners’ L2 pragmatic competence in interaction.