Theoretical and practical assessments over SSH

Abstract

During the COVID-19 pandemic, universities worldwide were forced to close, causing a shift from presential to remote classes. This situation motivated teachers to find suitable tools to evaluate students remotely, fairly, and accurately. However, currently available systems are either survey or exercise evaluation based, not suitable for competency-based assessments. Faced with this context and limitations of available evaluation systems, we developed TestsOverSSH, a system to devise, deliver, and automatically correct assessments performed in a Command Line Interface (CLI) environment. Unique assessments are generated per student when they access the proposed system via Secure SHell (SSH). TestsOverSSH is composed of shell scripts that orchestrate a series of tools and services that come pre-installed in Linux distributions. It can be used to construct multiple-choice or direct answer questions while also requiring students to perform tasks in the environment per se, namely computer programming or CLI manipulation-related assignments. We present examples of the question types in this system, explaining question formats and operating guidelines. Since the assessments are directly performed in the system, logs and command history can be easily retrieved while keeping information within student devices uncollected. We performed evaluations using this system in a real context and obtained student feedback through a custom survey and the System Usability Scale (SUS). Survey results and SUS score suggest that TestsOverSSH is an intuitive evaluation tool, with eased access and usage, making it applicable for e-learning.