Understanding machine learning models through Shapley values and the associated uncertainties

Date
2021
Journal Title
Journal ISSN
Volume Title
Publisher
Producer
Director
Performer
Choreographer
Costume Designer
Music
Videographer
Lighting Designer
Set Designer
Crew Member
Funder
Rehearsal Director
Concert Coordinator
Moderator
Panelist
Alternative Title
Department
Haverford College. Department of Computer Science
Type
Thesis
Original Format
Running Time
File Format
Place of Publication
Date Span
Copyright Date
Award
Language
eng
Note
Table of Contents
Terms of Use
Rights Holder
Access Restrictions
Dark Archive until 2022-01-01, afterwards Haverford users only.
Tripod URL
Identifier
Abstract
To explain black-box machine learning models, one popular method is to calculate the feature importance using Shapley value, a solution to distribute the payoff among the players in a coalition game. Though popular, Shapley-based feature importance methods suffer from interpretation issues when there are correlated features and out-of-distribution perturbed data points. Kumar et al. 2020b introduces Shapley residuals to quantify the information lost during the computation of Shapley values, e.g. feature interaction in the model; meanwhile, Slack et al. 2020b introduces Bayesian SHAP to calculate the variance of the Shapley values. With these uncertainty measures, one important question arises: how do practitioners interpret the Shapley values and does that interpretation match the underlying machine learning model? This thesis aims to answer that question and contribute to the literature by (1) introducing a linear mental model that represents how practitioners interpret Shapley values, and (2) introducing deviation as a way to measure how much this mental model deviates from the machine learning model. Our experiments show that this deviation measure is related to Shapley values and sampling variance, but also captures another aspect of the model: its nonlinearity.
Description
Subjects
Citation
Collections