Transfer learning is an effective technique for tuning a deep learning model when training data or computational resources are limited. Instead of training a new model from scratch, the parameters of an existing “base model” are adjusted for a new task. The accuracy of such a fine-tuned model depends on choosing an appropriate base model. Model search automates the selection of such a base model by evaluating the suitability of candidate models for a specific task. This entails inference with each candidate model on task-specific data. With thousands of models available through model stores, the computational cost of model search is a major bottleneck for efficient transfer learning. In this work, we present Alsatian, a novel model search system. Based on the observation that many candidate models overlap to a significant extent and based on a careful bottleneck analysis, we propose optimization techniques that are applicable to many model search frameworks. These optimizations include: (i) splitting models into individual blocks that can be shared across models, (ii) caching of intermediate inference results and model blocks, and (iii) selecting a beneficial search order for models to maximize sharing of cached results. In our evaluation on state-of-the-art deep learning models from computer vision and natural language processing, we show that Alsatian outperforms baselines by up to ~14×.
Nils is a PhD student in the Database Group at the Hasso Plattner Institute (HPI) in Potsdam, under the supervision of Tilmann Rabl. His research focuses on ML systems, particularly ML model management and search. In addition to his research, he contributes to the lecture on big data systems, leads seminars on ML systems, and supervises master’s theses. Before starting his PhD, he earned a master’s degree in IT-Systems Engineering from HPI and a bachelor’s degree in Computer Science from the University of Hamburg. As part of his studies, he completed a six-month internship at SAP Labs France in Sophia Antipolis and spent a semester at ETH Zurich.