Don't Buy the Pig in a Poke: Benchmarking DNNs Inference Performance before Development
Files
Date
2024-01-03
Contributor
Advisor
Department
Instructor
Depositor
Speaker
Researcher
Consultant
Interviewer
Narrator
Transcriber
Annotator
Journal Title
Journal ISSN
Volume Title
Publisher
Volume
Number/Issue
Starting Page
7790
Ending Page
Alternative Title
Abstract
As deep neural networks (DNNs) are increasingly used in practical settings, developers often adopt pre-existing DNN architectures from a vast collection of well-established models. However, in industrial environments, factors beyond simply achieving high accuracy are becoming important. The runtime performance is critical, as delays can lead to conveyor stops or equipment damage. Currently, determining the runtime performance of a DNN requires multiple iterations and testing specific configurations, whereas existing methods for benchmarking DNNs mainly compare different hardware or runtime parameters. We present tritonPerf, an approach to obtain and compare the runtime (latency and throughput) for a broad range of settings and existing DNN architectures in the final application environment. It allows data scientists to evaluate the performance and compare the results for a wide range of models before time and resource-intensive hyperparameter tuning is performed. We demonstrate the gain of tritonPerf in an extensive field study using an industrial setting, where we benchmark and compare the runtime of 57 different models.
Description
Keywords
Software Technology and Software Development, benchmark framework, computer vision, intelligent edge computing
Citation
Extent
10 pages
Format
Geographic Location
Time Period
Related To
Proceedings of the 57th Hawaii International Conference on System Sciences
Related To (URI)
Table of Contents
Rights
Attribution-NonCommercial-NoDerivatives 4.0 International
Rights Holder
Local Contexts
Collections
Email libraryada-l@lists.hawaii.edu if you need this content in ADA-compliant format.