AI, ML & Data Science

Part 4 of the Explainable AI Blog Series: Using SHAP to Understand Model Decisions: Exploring SHAP for Global and Local Interpretability

by Kinshuk Dutta
This entry is part 4 of 5 in the series Explainable AI
wp-content/uploads/2024/11/iStock-1227226348-scaled.jpg
Explainable AI

📝 This Blog is Part 4 of the Explainable AI Blog Series

In the previous blog, we used LIME to explain individual predictions in our loan approval model, focusing on local interpretability. Now, we’ll dive into SHAP (SHapley Additive exPlanations), a powerful tool that provides both global and local interpretability. SHAP’s ability to quantify feature contributions across the model makes it invaluable for understanding model behavior and detecting potential biases.

By the end of this blog, you’ll:

  1. Understand how SHAP works and why it’s important.
  2. Use SHAP to analyze global feature importance.
  3. Explain individual predictions with SHAP visualizations.
  4. Apply SHAP to answer real-world business questions in our loan approval model.

🔍 What is SHAP?

SHAP (SHapley Additive exPlanations) is an explainability framework grounded in game theory. It assigns each feature in a model an importance score based on its contribution to the prediction. SHAP stands out for:

  • Global Interpretability: Identifying features that impact the model’s predictions across the dataset.
  • Local Interpretability: Explaining individual predictions with detailed contributions of each feature.

🧮 Key Formula: Shapley Value

The Shapley value for a feature ii is calculated as:

Where:

  • SS: A subset of all features except ii.
  • NN: The set of all features.
  • v(S)v(S): The model prediction using only features in SS.

This formula ensures fairness by considering all possible feature subsets, calculating their marginal contributions.

🏦 Applying SHAP to the Loan Approval Model

We’ll continue using the loan approval model built in previous blogs to demonstrate SHAP’s capabilities. If you haven’t followed along, refer to Part 2 for instructions on setting up the model.

Step 1: Install and Import SHAP

Ensure SHAP is installed in your environment. If it’s not, install it:

bash
pip install shap

Import SHAP and other necessary libraries in your script:

python
import shap
import pandas as pd
import joblib
import matplotlib.pyplot as plt

Step 2: Load the Trained Model and Test Data

Load the logistic regression model and test dataset from the previous implementation:

python
# Load the trained model and test data
model = joblib.load('loan_model.pkl')
X_test = pd.read_csv('X_test.csv')

Step 3: Initialize the SHAP Explainer

Create an explainer object using SHAP’s LinearExplainer for logistic regression models:

python
# Initialize SHAP explainer
explainer = shap.Explainer(model.predict, X_test)

Step 4: Generate SHAP Values

Compute SHAP values for the test dataset:

python
# Generate SHAP values
shap_values = explainer(X_test)

Step 5: Visualize Global Feature Importance

5.1 Summary Plot

The summary plot provides a high-level view of feature importance across the dataset.

python
# Plot summary of SHAP values
shap.summary_plot(shap_values, X_test, feature_names=['ApplicantIncome', 'LoanAmount', 'Credit_History'])

Output: A horizontal bar chart where:

  • The length of each bar indicates the overall importance of the feature.
  • Colors show whether the feature values are high (red) or low (blue).

5.2 Industry Example: Bank Loan Approvals

For example, the summary plot may reveal:

  • Credit_History: The most influential feature, with good credit history strongly linked to approvals.
  • LoanAmount: Higher loan amounts are negatively associated with approvals.

Step 6: Explain Individual Predictions

6.1 Force Plot

The force plot explains why a specific applicant was approved or denied by visualizing how each feature contributed to the prediction.

python
# Force plot for the first applicant in the test set
shap.force_plot(
explainer.expected_value[1],
shap_values[1][0],
X_test.iloc[0],
matplotlib=True
)

Output: A horizontal plot showing:

  • The baseline prediction (e.g., the average approval likelihood).
  • Feature contributions pushing the prediction toward approval or denial.

6.2 Answering “Why Was Applicant A Denied?”

For an applicant with:

  • ApplicantIncome: $2,000
  • LoanAmount: $180,000
  • Credit_History: Poor

The force plot reveals:

  • Credit_History and LoanAmount had strong negative contributions, outweighing the positive contribution of ApplicantIncome.

Step 7: SHAP Interaction Values (Optional)

SHAP can analyze interactions between features, such as how Credit_History interacts with LoanAmount.

python
# Compute interaction values
interaction_values = shap.Explanation.shap_interaction_values(shap_values)

# Visualize interactions
shap.summary_plot(interaction_values, X_test)

Output: A detailed plot showing which feature pairs have the strongest interactions.

🌟 Real-Life Impact: Using SHAP to Improve Transparency

Use Case 1: Customer Communication

Banks can use SHAP force plots to explain to applicants:

  • Why their loan was approved or denied.
  • What changes (e.g., improving credit history) could increase approval chances.

Use Case 2: Regulatory Compliance

Global SHAP insights ensure that decisions align with ethical guidelines by highlighting potential biases.

Use Case 3: Model Debugging

SHAP identifies features that may have unintended influence, guiding model refinement.

🔜 What’s Next in This Series?

This blog is Part 4 of the Explainable AI series. In the next blog (Part 5), we’ll:

  • Detect and mitigate biases using insights from both LIME and SHAP.
  • Ensure fairness in AI models for real-world applications.

Stay tuned for the next blog, and let us know how you’re using XAI in your projects! 🚀

Series Navigation<< Applying LIME for Local InterpretabilityDetecting and Mitigating Bias with XAI Tools >>