Mixture of Linear Models Co-supervised by Deep Neural Networks

Jia Li (Pennsylvania State University)

Wednesday 6th March 13:00-14:00 Maths 311B

Abstract

Deep neural networks (DNNs) often achieve state-of-the-art prediction accuracy across many applications. However, their adoption in certain domains is hindered by the inherent complexity of DNN models, which poses significant challenges to their interpretability. In contrast, linear models, such as logistic regression, are considered highly interpretable but tend to have lower accuracy. Our goal is to develop mechanisms for balancing interpretability and accuracy to bridge the gap between explainable linear models and black-box models. Specifically, we propose a new approach termed Mixture of Linear Models (MLM) for regression or classification, whose estimation is guided by a pre-trained DNN, acting as a proxy for the optimal prediction function. Visualization methods and quantitative approaches have been developed for interpretation. Experiments show that this new method can effectively balance interpretability and accuracy. In some instances, MLM achieves comparable accuracy to DNNs but significantly enhances interpretability. I will also briefly discuss our more recent work on an EM-type algorithm to estimate MLM and its potential to improve logistic regression for small datasets.

Add to your calendar

Download event information as iCalendar file (only this event)