The criterion-based rubric: Always a good assessment method? A comparison of different methods for evaluating paraverbal elements in the oral productions of 11–12-year-old students

Abstract

This article compares three methods for assessing paraverbal aspects in oral productions: the holistic absolute method (overall score), the analytical absolute method (criterion-based rubric), and the holistic comparative method (Comproved software). The study addresses three key questions: 1) What is the inter-rater reliability of each evaluation method ? 2) What is the correlation between these methods ? 3) What differences in scoring can be observed among these methods ? Each method was used to evaluate oral productions based on paraverbal criteria such as intonation, volume, and pauses. The results reveal that, contrary to expectations, the holistic absolute method demonstrated the highest inter-rater reliability. Although significant correlations were found between the methods, notable discrepancies remained in the grading of the same productions. These findings question the systematic use of criterion-based rubrics and emphasize the need to adapt evaluation methods to specific objectives, particularly for assessing paraverbal aspects of oral productions.