Evaluating the kit-build concept mapping process using sub-map scoring

Abstract

Concept mapping allows learners to visually represent their knowledge by connecting nodes (concepts) and links (relations between concepts). The kit-build (KB) concept map framework enhances this process by enabling learners to recompose a concept map from provided nodes and links, leading to improved learning outcomes. Additionally, KB employs an automatic assessment method called “Full Map Scoring (FMS)”, which evaluates the learner’s understanding based on the recomposed concept map. However, FMS only evaluates the final product of the recomposition activity, neglecting the process itself. This is a potential limitation because different processes leading to the same result could reflect different levels of understanding among learners. Therefore, it is crucial to incorporate process analysis into learner assessment. To address this issue, our research proposes a new assessment procedure termed “Sub-Map Scoring (SMS)”. A concept map is generally composed of several sub-maps with each sub-map representing a set of meanings. We hypothesize that if a learner comprehends the meaning of a sub-map, the learner will recompose the sub-map as a continuous activity. Therefore, SMS evaluates the recomposition process of each sub-map from the viewpoint of continuity, and the overall SMS score is derived from these evaluations. To verify the effectiveness of SMS, we compared SMS and FMS scores using data from a practical use of the KB framework. A multiple linear regression analysis confirmed that the SMS score was a more precise predictor of learning gain than the FMS score.