Prediction queries are widely used across industries to perform advanced analytics and draw insights from data. They include a data processing part (e.g., for joining, filtering, cleaning, featurizing the datasets) and a machine learning (ML) part invoking one or more trained models to perform predictions. These parts have so far been optimized in isolation, leaving significant opportunities for optimization unexplored. In this talk, I will present Raven, a production-ready system for optimizing prediction queries. Raven follows the enterprise architectural trend of collocating data and ML runtimes, and it relies on a unified intermediate representation to unlock two families of optimizations. First, it employs logical optimizations that pass information between the data part (and the properties of the underlying data) and the ML part to optimize each other. Second, it introduces logical-to-physical transformations that allow operators to be executed on different runtimes (relational, ML, and DNN) and hardware (CPU, GPU). Our evaluation shows that Raven is able to improve performance of prediction queries on Apache Spark and SQL Server by orders of magnitude over both CPU and GPU. Using the same architecture, I will also discuss recent results on offloading relational and graph operations to the ML accelerators that are collocated with the data engines.
Konstantinos Karanasos is a Principal Scientist Manager at Microsoft’s Gray Systems Lab (GSL), Azure Data’s applied research group. He is the manager of the Bay Area branch of GSL and tech lead for systems-for-ML efforts in the group. Konstantinos’ work at Microsoft previously focused on resource management for the company’s production analytics clusters. This work was deployed in over 300K machines across Microsoft and was key to enable the company to operate the world’s largest YARN clusters. He has also contributed big part of his work at Microsoft to open source projects: he is a committer and member of the Project Management Committee (PMC) of Apache Hadoop, and a contributor to ONNX Runtime. Before joining Microsoft, he was a postdoctoral researcher at IBM Almaden Research Center. Konstantinos holds a PhD from Inria and the University Paris-Sud, France, and a Diploma in Electrical and Computer Engineering from the National Technical University of Athens, Greece.