The wide adoption of machine learning (ML) in diverse application domains is resulting in an explosion of available models described by, and stored in model repositories. In application contexts where inference needs are dynamic and subject to strict execution constraints – such as in video processing – the manual selection of an optimal set of models from a large model repository is a nearly impossible task and practitioners typically settle for models with a good average accuracy and performance. In this talk we present a query optimizer for ML model selection and predicate ordering. Given a model repository with ML models of different accuracy and performance characteristics, the optimizer aims at picking the best models to apply to a predicate given an optimization goal and a constraint on either accuracy or execution time. We focus on object detection queries against video and image data and present three query optimizers with varying sophistication: greedy, model selection optimal using mixed integer linear programming, and execution order optimal. To evaluate our approach, we construct a model repository of 125 models and evaluate the optimizers on queries of varying complexity against the COCO dataset.
Ziyu Li is a PhD Student at Delft University of Technology (TU Delft), under the supervision of Asterios Katsifodimos. Her research lies at the intersection of Database and Machine Learning (ML), mainly focusing on ML inference over multimedia data and ML model management. Ziyu obtained her Master degree in Computer Science from TU Delft.