Reusing Deep Learning Models: Challenges and Directions in Software Engineering

Reusing Deep Learning Models: Challenges and Directions in Software Engineering#

Vision Paper JVA 2023 PTM Reuse

Authors#

James C. Davis
Purvish Jajal
Wenxin Jiang
Taylor R. Schorlemmer
Nicholas M. Synovic
George K. Thiruvathukal

Note

All authors are listed in alphabetical order as equal contributors to this manuscript. Please direct correspondence to davisjam@purdue.edu and gthiruvathukal@luc.edu.

Abstract#

Deep neural networks (DNNs) achieve state-of-the art performance in many areas, including computer vision, system configuration, and question-answering. However, DNNs are expensive to develop, both in intellectual effort (e.g., devising new architectures) and computational costs (e.g., training). Reusing DNNs is a promising direction to amortize costs within a company and across the computing industry. As with any new technology, however, there are many challenges in re-using DNNs. These challenges include both missing technical capabilities and missing engineering practices.

This vision paper describes challenges in current approaches to DNN re-use. We summarize studies of re-use failures across the spectrum of re-use techniques, including conceptual (e.g., reusing based on a research paper), adaptation (e.g., re-using by building on an existing implementation), and deployment (e.g., direct re-use on a new device). We outline possible advances that would improve each kind of re-use.

Artifacts#

Todo

  • Add the paper preprint

  • Add the poster

  • Add link to the source code

  • Update the bibtex

Paper Preprint

Download

Published Paper

View

Poster

Download

Source Code

View

BibTex
@inproceedings{synovic_snapshot_2023,
   address = {New York, NY, USA},
   series = {{ASE} '22},
   title = {Snapshot {Metrics} {Are} {Not} {Enough}: {Analyzing} {Software} {Repositories} with {Longitudinal} {Metrics}},
   isbn = {978-1-4503-9475-8},
   shorttitle = {Snapshot {Metrics} {Are} {Not} {Enough}},
   url = {https://dl.acm.org/doi/10.1145/3551349.3559517},
   doi = {10.1145/3551349.3559517},
   abstract = {Software metrics capture information about software development processes and products. These metrics support decision-making, e.g., in team management or dependency selection. However, existing metrics tools measure only a snapshot of a software project. Little attention has been given to enabling engineers to reason about metric trends over time—longitudinal metrics that give insight about process, not just product. In this work, we present PRIME (PRocess MEtrics), a tool to compute and visualize process metrics. The currently-supported metrics include productivity, issue density, issue spoilage, and bus factor. We illustrate the value of longitudinal data and conclude with a research agenda. The tool’s demo video can be watched at https://bit.ly/ase2022-prime. Source code can be found at https://github.com/SoftwareSystemsLaboratory/prime.},
   urldate = {2023-09-06},
   booktitle = {Proceedings of the 37th {IEEE}/{ACM} {International} {Conference} on {Automated} {Software} {Engineering}},
   publisher = {Association for Computing Machinery},
   author = {Synovic, Nicholas M. and Hyatt, Matt and Sethi, Rohan and Thota, Sohini and Shilpika and Miller, Allan J. and Jiang, Wenxin and Amobi, Emmanuel S. and Pinderski, Austin and Läufer, Konstantin and Hayward, Nicholas J. and Klingensmith, Neil and Davis, James C. and Thiruvathukal, George K.},
   month = jan,
   year = {2023},
   keywords = {Empirical software engineering, Software metrics},
   pages = {1--4},
}

Note

I have continued to iterate upon the PRIME tool in a new repository. If want to learn more, visit NicholasSynovic/prime.

Video#

Contents