Software Sustainability: Research on Usability, Maintainability, and Reproducibility

Permanent URI for this collection

https://hdl.handle.net/10125/110016

Browse

Recent Submissions

Now showing 1 - 5 of 5
  • Thumbnail Image
    Item
    "We provide our resources in a dedicated repository": Surveying the Transparency of HICSS Publications
    (2025-01-07) Pekaric, Irdin; Apruzzese, Giovanni
    Every day, new discoveries are made by researchers from all across the globe and fields. HICSS is a flagship venue to present and discuss such scientific advances. Yet, the activities carried out for any given research can hardly be fully contained in a single document of a few pages the "paper.'' Indeed, any given study entails data, artifacts, or other material that is crucial to truly appreciate the contributions claimed in the corresponding paper. External repositories (e.g., GitHub) are a convenient tool to store all such resources so that future work can freely observe and build upon them---thereby improving transparency and promoting reproducibility of research as a whole. In this work, we scrutinize the extent to which papers recently accepted to HICSS leverage such repositories to provide supplementary material. To this end, we collect all the 5579 papers included in HICSS proceedings from 2017--2024. Then, we identify those entailing either human subject research (850) or technical implementations (737), or both (147). Finally, we review their text, examining how many include a link to an external repository and, inspect its contents. Overall, out of 2028 papers, only 3% have a functional and publicly available repository that is usable by downstream research. We release all our tools.
  • Thumbnail Image
    Item
    Making Software Work Sustainable for the Academic Research Group: a Comparative Case Study
    (2025-01-07) Sutherland, Will; Neang, Andrew; Lee, Charlotte
    Studies of research software development have focused on how to promote or encourage the adoption of software engineering practices, but we do not have a good empirical understanding of strategies that researchers have already begun to take in order to integrate those practices into research work in sustainable ways. We conduct a comparative case study of two research groups in different fields, and characterize two approaches that they have taken to get research software engineering work done: practice integration and differentiating expertise. From these findings we argue that examining outcomes of change in research software development practice is critical for understanding sustainability and the ramifications of such changes for scientific work.
  • Thumbnail Image
    Item
    AI Readiness: A Reusability Study of Popular AI Algorithms
    (2025-01-07) Quick, Rob; Kasula, Mohith
    The FAIR Data Principles of findability, accessibility, interoperability, and reusability provide a roadmap to reusing data analysis findings and reproducibility of AI-based data analysis. However, the work done during this research project has identified an issue that impacts AI reproducibility before code and data interoperability can be considered. Namely, code reusability when attempting to recreate the hardware and system-level software or the “runtime environment.” While attempting to determine the metadata needed to FAIRly couple datasets with AI algorithms, the research team determined that the problem of recreating the runtime environment of published state-of-the-art algorithms from the website Papers with Code provided a hurdle that must be overcome before automated data-algorithm coupling can be considered. While containerization solutions such as Docker or Singularity are created to address the issue of inconsistent runtime environments, few AI algorithm developers have embraced publishing containers alongside their AI codes, opting for documenting software dependencies, which only tell part of the runtime story. Additionally, containers are software, and many issues affecting the recreation of runtime environments can also affect orchestrated container solutions. This work describes the process employed to survey 75 openly available AI algorithms, as recorded by Papers with Code, spanning the machine learning areas of computer vision, audio analysis, and natural language processing. It also makes a case that merely publishing the algorithm software repository and datasets used to benchmark the accuracy of the analysis is not enough to enable the reproducibility of results or reuse of AI algorithms. Finally, it identifies the gap in runtime environment reusability between code repositories like GitHub and commercial services like Hugging Face to focus future work. It proposes providing solutions like a container library, enhanced documentation, and other methods to allow reproducible and reusable research and a roadmap for continuing toward a review of enhancing AI-ready data.
  • Thumbnail Image
    Item
  • Thumbnail Image
    Item
    How to Position Your Gateway for Failure: The Ten Don’ts of Gateway Design
    (2025-01-07) Gesing, Sandra; Kee, Kerk; Brandt, Steven; Cleveland, Sean; Bradley, Shannon; Smith, Jack; Villegas Martinez, Braulio M.
    How to Position Your Gateway for Failure:The Ten Don’ts of Gateway DesignAbstractScience gateways are accelerators for science and education, providing user-friendly access to powerful computational resources and data analysis tools. Sustained science gateways frameworks such as Hubzero, Tapis, and Galaxy demonstrate the potential for gateways to revolutionize scientific exploration.However, despite initial promise, many gateway projects struggle to transition from prototypes to sustainable, long-term services. Well-intentioned, yet ultimately unsuccessful, gateways are part of the scientific landscape. This raises a critical question: what factors contribute to the demise of science gateways, and how can we avoid these pitfalls to ensure the success of future endeavors?This paper delves into the ten most common pitfalls that lead to science gateway failure. By analyzing these roadblocks, we aim to equip new and developing gateway initiatives with suggestions for long-term success. Our research draws on the collective experiences of numerous gateway projects.We identified critical areas where focused attention and strategic planning are essential. This knowledge will enable the development of good practices that nurture vibrant gateway communities and ensure the long-term sustainability of these valuable research tools.