https://github.com/SelfExplainML/PiML-Toolbox Skip to content Navigation Menu Toggle navigation Sign in * Product + GitHub Copilot Write better code with AI + Security Find and fix vulnerabilities + Actions Automate any workflow + Codespaces Instant dev environments + Issues Plan and track work + Code Review Manage code changes + Discussions Collaborate outside of code + Code Search Find more, search less Explore + All features + Documentation + GitHub Skills + Blog * Solutions By company size + Enterprises + Small and medium teams + Startups By use case + DevSecOps + DevOps + CI/CD + View all use cases By industry + Healthcare + Financial services + Manufacturing + Government + View all industries View all solutions * Resources Topics + AI + DevOps + Security + Software Development + View all Explore + Learning Pathways + White papers, Ebooks, Webinars + Customer Stories + Partners * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles Repositories + Topics + Trending + Collections * Enterprise + Enterprise platform AI-powered developer platform Available add-ons + Advanced Security Enterprise-grade security features + GitHub Copilot Enterprise-grade AI features + Premium Support Enterprise-grade 24/7 support * Pricing Search or jump to... Search code, repositories, users, issues, pull requests... Search [ ] Clear Search syntax tips Provide feedback We read every piece of feedback, and take your input very seriously. [ ] [ ] Include my email address so I can be contacted Cancel Submit feedback Saved searches Use saved searches to filter your results more quickly Name [ ] Query [ ] To see all available qualifiers, see our documentation. Cancel Create saved search Sign in Sign up Reseting focus You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert {{ message }} SelfExplainML / PiML-Toolbox Public * Notifications You must be signed in to change notification settings * Fork 118 * Star 1.1k PiML (Python Interpretable Machine Learning) toolbox for model development & diagnostics selfexplainml.github.io/piml-toolbox License Apache-2.0 license 1.1k stars 118 forks Branches Tags Activity Star Notifications You must be signed in to change notification settings * Code * Issues 15 * Pull requests 0 * Discussions * Actions * Projects 0 * Security * Insights Additional navigation options * Code * Issues * Pull requests * Discussions * Actions * Projects * Security * Insights SelfExplainML/PiML-Toolbox main BranchesTags [ ] Go to file Code Folders and files Name Name Last commit Last commit message date Latest commit History 435 Commits datasets datasets docs docs examples examples .gitignore .gitignore Acknowledgements.md Acknowledgements.md LICENSE.md LICENSE.md PiML Low-code Example PiML Low-code Example Run.ipynb Run.ipynb README.md README.md View all files Repository files navigation * README * Apache-2.0 license drawing An integrated Python toolbox for interpretable machine learning pip install PiML Dec 1, 2023: V0.6.0 is released with enhanced data handling and model analytics. May 4, 2023: V0.5.0 is released together with PiML user guide. October 31, 2022: V0.4.0 is released with enriched models and enhanced diagnostics. July 26, 2022: V0.3.0 is released with classic statistical models. June 26, 2022: V0.2.0 is released with high-code APIs. May 4, 2022: V0.1.0 is launched with low-code UI/UX. PiML (or p-ML, /'paI*'em*'el/) is a new Python toolbox for interpretable machine learning model development and validation. Through low-code interface and high-code APIs, PiML supports a growing list of inherently interpretable ML models: 1. GLM: Linear/Logistic Regression with L1 [?] L2 Regularization 2. GAM: Generalized Additive Models using B-splines 3. Tree: Decision Tree for Classification and Regression 4. FIGS: Fast Interpretable Greedy-Tree Sums (Tan, et al. 2022) 5. XGB1: Extreme Gradient Boosted Trees of Depth 1, with optimal binning (Chen and Guestrin, 2016; Navas-Palencia, 2020) 6. XGB2: Extreme Gradient Boosted Trees of Depth 2, with effect purification (Chen and Guestrin, 2016; Lengerich, et al. 2020) 7. EBM: Explainable Boosting Machine (Nori, et al. 2019; Lou, et al. 2013) 8. GAMI-Net: Generalized Additive Model with Structured Interactions (Yang, Zhang and Sudjianto, 2021) 9. ReLU-DNN: Deep ReLU Networks using Aletheia Unwrapper and Sparsification (Sudjianto, et al. 2020) PiML also works for arbitrary supervised ML models under regression and binary classification settings. It supports a whole spectrum of outcome testing, including but not limited to the following: 1. Accuracy: popular metrics like MSE, MAE for regression tasks and ACC, AUC, Recall, Precision, F1-score for binary classification tasks. 2. Explainability: post-hoc global explainers (PFI, PDP, ALE) and local explainers (LIME, SHAP). 3. Fairness: disparity test and segmented analysis by integrating the solas-ai package. 4. WeakSpot: identification of weak regions with high residuals by slicing techniques. 5. Overfit: identification of overfitting regions according to train-test performance gap. 6. Reliability: assessment of prediction uncertainty by split conformal prediction techniques. 7. Robustness: evaluation of performance degradation under covariate noise perturbation. 8. Resilience: evaluation of performance degradation under different out-of-distribution scenarios. Installation | Examples | Usage | Citations Installation pip install PiML Low-code Examples Click the ipynb links to run examples in Google Colab: 1. BikeSharing data: [L] ipynb 2. CaliforniaHousing data: [L] ipynb 3. TaiwanCredit data: [L] ipynb 4. Fairness_SimuStudy1 data: [L] ipynb 5. Fairness_SimuStudy2 data: [L] ipynb 6. Upload custom data in two ways: [L] ipynb 7. Deal with external models: [L] ipynb Begin your own PiML journey with this demo notebook. High-code Examples The same examples can also be run by high-code APIs: 1. BikeSharing data: [L] ipynb 2. CaliforniaHousing data: [L] ipynb 3. TaiwanCredit data: [L] ipynb 4. Model saving: [L] ipynb 5. Results return: [L] ipynb Low-code Usage on Google Colab Stage 1: Initialize an experiment, Load and Prepare data from piml import Experiment exp = Experiment() exp.data_loader() [data_loade] exp.data_summary() [data_summa] exp.data_prepare() [data_prepa] exp.data_quality() [data_quali] exp.feature_select() [feature_se] exp.eda() [data_eda] Stage 2: Train intepretable models exp.model_train() [model_trai] Stage 3. Explain and Interpret exp.model_explain() [model_expl] exp.model_interpret() [model_inte] Stage 4. Diagnose and Compare exp.model_diagnose() [model_diag] exp.model_compare() [model_comp] exp.model_fairness() [model_fair] exp.model_fairness_compare() [model_fair] Arbitrary Black-Box Modeling For example, train a complex LightGBM with depth 7 and register it to the experiment: from lightgbm import LGBMClassifier exp.model_train(LGBMClassifier(max_depth=7), name='LGBM-7') Then, compare it to inherently interpretable models (e.g. XGB2 and GAMI-Net): exp.model_compare() [model_comp] Citations PiML, ReLU-DNN Aletheia and GAMI-Net --------------------------------------------------------------------- "PiML Toolbox for Interpretable Machine Learning Model Development and Diagnostics" (A. Sudjianto, A. Zhang, Z. Yang, Y. Su and N. Zeng, 2023) arXiv link @article{sudjianto2023piml, title={PiML Toolbox for Interpretable Machine Learning Model Development and Diagnostics}, author={Sudjianto, Agus and Zhang, Aijun and Yang, Zebin and Su, Yu and Zeng, Ningzhou}, year={2023} } "Designing Inherently Interpretable Machine Learning Models" (A. Sudjianto and A. Zhang, 2021) arXiv link @article{sudjianto2021designing, title={Designing Inherently Interpretable Machine Learning Models}, author={Sudjianto, Agus and Zhang, Aijun}, journal={arXiv preprint:2111.01743}, year={2021} } "Unwrapping The Black Box of Deep ReLU Networks: Interpretability, Diagnostics, and Simplification" (A. Sudjianto, W. Knauth, R. Singh, Z. Yang and A. Zhang, 2020) arXiv link @article{sudjianto2020unwrapping, title={Unwrapping the black box of deep ReLU networks: interpretability, diagnostics, and simplification}, author={Sudjianto, Agus and Knauth, William and Singh, Rahul and Yang, Zebin and Zhang, Aijun}, journal={arXiv preprint:2011.04041}, year={2020} } "GAMI-Net: An Explainable Neural Network based on Generalized Additive Models with Structured Interactions" (Z. Yang, A. Zhang, and A. Sudjianto, 2021) arXiv link @article{yang2021gami, title={GAMI-Net: An explainable neural network based on generalized additive models with structured interactions}, author={Yang, Zebin and Zhang, Aijun and Sudjianto, Agus}, journal={Pattern Recognition}, volume={120}, pages={108192}, year={2021} } Other Interpretable ML Models --------------------------------------------------------------------- "Fast Interpretable Greedy-Tree Sums (FIGS)" (Tan, Y.S., Singh, C., Nasseri, K., Agarwal, A. and Yu, B., 2022) @article{tan2022fast, title={Fast interpretable greedy-tree sums (FIGS)}, author={Tan, Yan Shuo and Singh, Chandan and Nasseri, Keyan and Agarwal, Abhineet and Yu, Bin}, journal={arXiv preprint arXiv:2201.11931}, year={2022} } "Accurate intelligible models with pairwise interactions" (Y. Lou, R. Caruana, J. Gehrke, and G. Hooker, 2013) @inproceedings{lou2013accurate, title={Accurate intelligible models with pairwise interactions}, author={Lou, Yin and Caruana, Rich and Gehrke, Johannes and Hooker, Giles}, booktitle={Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining}, pages={623--631}, year={2013}, organization={ACM} } "Purifying Interaction Effects with the Functional ANOVA: An Efficient Algorithm for Recovering Identifiable Additive Models" (Lengerich, B., Tan, S., Chang, C.H., Hooker, G. and Caruana, R., 2020) @inproceedings{lengerich2020purifying, title={Purifying interaction effects with the functional anova: An efficient algorithm for recovering identifiable additive models}, author={Lengerich, Benjamin and Tan, Sarah and Chang, Chun-Hao and Hooker, Giles and Caruana, Rich}, booktitle={International Conference on Artificial Intelligence and Statistics}, pages={2402--2412}, year={2020}, organization={PMLR} } "InterpretML: A Unified Framework for Machine Learning Interpretability" (H. Nori, S. Jenkins, P. Koch, and R. Caruana, 2019) @article{nori2019interpretml, title={InterpretML: A Unified Framework for Machine Learning Interpretability}, author={Nori, Harsha and Jenkins, Samuel and Koch, Paul and Caruana, Rich}, journal={arXiv preprint:1909.09223}, year={2019} } About PiML (Python Interpretable Machine Learning) toolbox for model development & diagnostics selfexplainml.github.io/PiML-Toolbox Topics low-code interpretable-machine-learning model-diagnostics ml-workflow Resources Readme License Apache-2.0 license Activity Custom properties Stars 1.1k stars Watchers 23 watching Forks 118 forks Report repository Releases 19 V0.6.0 Latest Nov 30, 2023 + 18 releases Packages 0 No packages published Contributors 6 * @ajzhanghk * @ZebinYang * @CnBDM-Su * @simoncos * @Eveyz * @parulnith Languages * Jupyter Notebook 100.0% Footer (c) 2024 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact * Manage cookies * Do not share my personal information You can't perform that action at this time.