An Exploratory Study on Students' Perceptions and Experiences of Assertion Messages in Test Cases
Date
2024
Authors
Contributor
Advisor
Department
Instructor
Depositor
Speaker
Researcher
Consultant
Interviewer
Narrator
Transcriber
Annotator
Journal Title
Journal ISSN
Volume Title
Publisher
Volume
Number/Issue
Starting Page
Ending Page
Alternative Title
Abstract
Unit testing, or the process of evaluating code at its smallest functional level, is an essential step in ensuring the overall quality of software projects. Core to unit testing are assertion statements developers use to verify that an expected condition has been met (i.e., check a unit of code for correctness.) JUnit, one of the most popular unit test and assertion libraries for Java projects, offers multiple assertion method types to easily and repeatedly check the code. The reason for the failure of a test can be found in the error log report within the integrated development environment (IDE) or from the command line. Furthermore, developers can add their own custom error messages to explain a more detailed cause for the failure of the test. While there is prior research on unit testing and assertions, there is a lack of research on custom assertion error messages.This study investigates the impact of assertion messages on university students' abilities to troubleshoot bugs in common Java string utility methods. Eighty-seven students participated in the study and were randomly assigned to one of three study groups: Group A (no messages), Group B (detailed messages of failure), and Group C (messages that were intended to be less descriptive than those of Group B). The study found that the presence of messages had little impact on the students' abilities to fix the code. Most students could fix all code errors, regardless of academic standing. Students with at least a moderate familiarity in unit testing were
significantly more likely to fix more than 4 tests. These findings indicate that familiarity with unit testing had some impact on students' ability to solve the errors in the code. Students were able to assess the quality of messages and students noticed a difference in the quality of the messages between Groups B and C. This study reports features and potential patterns in messages perceived by students to be more or less informative. This study also highlights a discrepancy between understandability and helpfulness, where students could find a message understandable but not necessarily useful in understanding the purpose of the test case.
Description
Keywords
Computer science, assert messages, software quality assurance, unit testing
Citation
Extent
119 pages
Format
Geographic Location
Time Period
Related To
Related To (URI)
Table of Contents
Rights
All UHM dissertations and theses are protected by copyright. They may be viewed from this source for any purpose, but reproduction or distribution in any format is prohibited without written permission from the copyright owner.
Rights Holder
Local Contexts
Collections
Email libraryada-l@lists.hawaii.edu if you need this content in ADA-compliant format.