Can Industrial-Strength Static Analysis Be Used to Help Students Who Are Struggling to Complete Programming Activities?

Date
2019-01-08
Authors
Edwards, Stephen
Spacco, Jaime
Hovemeyer, David
Contributor
Advisor
Department
Instructor
Depositor
Speaker
Researcher
Consultant
Interviewer
Annotator
Journal Title
Journal ISSN
Volume Title
Publisher
Volume
Number/Issue
Starting Page
Ending Page
Alternative Title
Abstract
Static analysis tools evaluate source code to identify potential problems or issues beyond typical compiler errors. Prior work has shown a statistically significant relationship between the correctness of a student's work and statically identifiable flaws or "code smells" that are likely to indicate programming errors. This paper presents a comprehensive study of this relationship in the context of small programming exercises intended for use in student skill building. We use FindBugs, a static analysis tool that identifies program features that are likely to represent actual bugs in professional software. Our goal is to identify the extent to which FindBugs warnings might help novices struggling to solve short programming exercises. In this study, we ran FindBugs against 149,054 answers submitted by 516 students on 57 drill-and-practice coding exercises. We identify the specific FindBugs warnings that are inversely correlated with correctness. We confirm that presence of these warnings is significantly associated with struggling on an exercise, as indicated by taking more time, making more submissions, and receiving lower scores. Finally, every exercise exhibited answers that trigger these warnings, and 92.4% of students would experience these warnings over a full semester. Our results indicate that static analysis with tools designed for use in industry offers an untapped opportunity to provide hints or suggestions to students who are measurably struggling.
Description
Keywords
Vision and Novel Delivery, Software Engineering Education and Training, CS Education, static analysis, drill-and-practice, errors, bugs, hints, CodeWorkout, FindBugs, CS1, CS2
Citation
Extent
10 pages
Format
Geographic Location
Time Period
Related To
Proceedings of the 52nd Hawaii International Conference on System Sciences
Table of Contents
Rights
Attribution-NonCommercial-NoDerivatives 4.0 International
Rights Holder
Local Contexts
Email libraryada-l@lists.hawaii.edu if you need this content in ADA-compliant format.