Main Article Content
In line with new international financial supervision directives (IFRS9), banks should look at a new set of analytical tools, such as machine learning. The introduction of these methods into banking practice requires reformulation of business goals, both in terms of the accuracy of predictions and the definition of risk factors. The article compares methods for selecting variables and assigning "importance" in statistical and algorithmic models. The calculations were carried out using the example of financial data classification for loan default. The effectiveness of various machine learning algorithms on selected sets of variables was compared. The results of the analyzes indicate the need to revise the concept of the "importance" of a variable so that it does not depend on the structure of the model.
Article Details
Adler A. I., Painsky A. (2022) Feature Importance in Gradient Boosting Trees with Cross-Validation Feature Selection. Entropy, 24(5), 687. https://doi.org/10.3390/e24050687 (Crossref)
Bag S., Gupta K., Deb S. (2022) A Review and Recommendations on Variable Selection Methods in Regression Models for Binary Data. https://arxiv.org/pdf/2201.06063
Ben Jabeur S., Stef N., Carmona P. (2023) Bankruptcy Prediction using the XGBoost Algorithm and Variable Importance Feature Engineering. Computational Economics, 61, 715-741. https://doi.org/10.1007/s10614-021-10227-1 (Crossref)
Breiman L. (2001) Statistical Modeling: The Two Cultures. Statistical Science, 16(3), 199-215.
Dorogush A.V., Ershov V., Gulin A. (2017) CatBoost: Gradient Boosting with Categorical Features Support. Workshop on ML Systems at NIPS 2017.
De Sa C. R. (2019) Variance-Based Feature Importance in Neural Networks. [in:] Kralj Novak P., Šmuc T., Džeroski S. (eds) Discovery Science, Lecture Notes in Computer Science. 11828, Springer, Cham. https://doi.org/10.1007/978-3-030-33778-0_24 (Crossref)
Engelmann B., Rauchmeier R. (2011) The Basel II: Risk Parameters. Estimation, Validation, Stress Testing - with Applications to Loan Risk Management. Springer Berlin, Heidelberg, https://doi.org/10.1007/978-3-642-16114-8 (Crossref)
Elith J., Leathwick J. R., Hastie T. (2008) A Working Guide to Boosted Regression Trees. Journal of Animal Ecology, 77, 802-813. https://doi.org/10.1111/j.1365-2656.2008.01390.x (Crossref)
Firth D. (1993) Bias Reduction of Maximum Likelihood Estimates. Biometrika, 80(10), 27-38. https://doi.org/10.2307/2336755 (Crossref)
Gajowniczek K. et al. (2022) HOFS: Higher Order Mutual Information Approximation for Feature Selection in R. SoftwareX, 19, 1-9. https://doi.org/10.1016/j.softx.2022.101148 (Crossref)
Hastie T., Tibshirani R., Friedman J. (2008) The Elements of Statistical Learning (2nd ed.). Springer.
Hastie T., Tibshirani R., Wainwright M. (2015) Statistical Learning with Sparsity. The Lasso and Generalizations. New York Chapman & Hall/CRC. https://doi.org/10.1201/b18401 (Crossref)
Hopf K., Sascha R. (2021) Filter Methods for Feature Selection in Supervised Machine Learning Applications-Review and Benchmark. arXiv preprint arXiv:2111.12140, 2021
Jia W., Sun M., Lian J. et al. (2022) Feature Dimensionality Reduction: a Review. Complex Intell. Syst., 8, 2663-2693. https://doi.org/10.1007/s40747-021-00637-x (Crossref)
Kohavi R, John G. H. (1997) Wrappers for Feature Subset Selection. Artificial Intelligence, 97(1-2), 273-324. https://doi.org/10.1016/S0004-3702(97)00043-X (Crossref)
Lal T. N., Chapelle O., Weston J., Elisseeff A. (2006) Embedded Methods. [in:] Guyon I., Nikravesh M., Gunn S., Zadeh L. A. (eds) Feature Extraction. Studies in Fuzziness and Soft Computing, 207, Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-35488-8_6 (Crossref)
Li J., Cheng K., Wang S., Morstatter F. (2017) Feature Selection: A Data Perspective ACM Computing Surveys, 50(6), Article 94, 1-45. https://doi.org/10.1145/3136625 (Crossref)
Olden J., Joy M., Death R. (2004) An Accurate Comparison of Methods for Quantifying Variable Importance in Artificial Neural Networks using Simulated Data. Ecological Modelling, 178(3-4), 389-397. https://doi.org/10.1016/j.ecolmodel.2004.03.013 (Crossref)
Ostroumova L., Gusev G., Vorobev A., Dorogush A.V., Gulin A. (2017) CatBoost: Unbiased Boosting with Categorical Features. NIPS'18: Proceedings of the 32nd International Conference on Neural Information Processing Systems December 2018, Pages 6639–6649. https://arxiv.org/pdf/1706.09516
Priyatno A.,Widiyaningtyas T. (2024) A Systematic Literature Review: Recursive Feature Elimination Algorithms. JITK (Jurnal Ilmu Pengetahuan dan Teknologi Komputer). 9. 196-207. https://doi.org/10.33480/jitk.v9i2.5015 (Crossref)
Pudjihartono N., Fadason T., Kempa-Liehr A. W., O’Sullivan J. M. (2022) A Review of Feature Selection Methods for Machine Learning-Based Disease Risk Prediction. Frontiers in Bioinformatics, 2(927312). https://doi.org/10.3389/fbinf.2022.927312 (Crossref)
Puhr R, Heinze G, Nold M, Lusa L, Geroldinger A. (2017) Firth's Logistic Regression with Rare Events: Accurate Effect Estimates and Predictions? Stat Med. 36(14), 2302–2317. https://doi.org/10.1002/sim.7273 (Crossref)
Sánchez-Maroño N., Alonso-Betanzos, A., Tombilla-Sanromán, M. (2007) Filter Methods for Feature Selection – A Comparative Study. [in:] Yin, H., Tino, P., Corchado, E., Byrne, W., Yao, X. (eds) Intelligent Data Engineering and Automated Learning - IDEAL 2007. IDEAL 2007. Lecture Notes in Computer Science, 4881, Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-77226-2_19 (Crossref)
Sauerbrei W., Perperoglou A., Schmid M. et al. (2020) State of the Art in Selection of Variables and Functional Forms in Multivariable Analysis-Outstanding Issues. Diagn Progn Res 4, 3. https://doi.org/10.1186/s41512-020-00074-3 (Crossref)
Taylor J., Tibshirani R. J. (2015) Statistical Learning and Selective Inference. Proc Natl Acad Sci U S A, 112(25), 7629-34. https://doi.org/10.1073/pnas.1507583112 (Crossref)
Wojtas M., Chen K. (2020) Feature Importance Ranking for Deep Learning. 34th Conference on Neural Information Processing Systems (NeurIPS 2020), Vancouver, Canada. https://doi.org/10.48550/arXiv.2010.08973
Vergara J. R., Estévez P.A. (2014) A Review of Feature Selection Methods based on Mutual Information, Neural Comput & Applic, 24, 175-186. https://doi.org/10.1007/s00521-013-1368-0 (Crossref)
Zebari R., Abdulazeez A., Zeebaree D., Zebari D., Saeed, J. (2020) A Comprehensive Review of Dimensionality Reduction Techniques for Feature Selection and Feature Extraction, Journal of Applied Science and Technology Trends, 1(2), 56-70. https://doi.org/10.38094/jastt1225
Downloads
- Urszula Grzybowska, Ludwik Wicki, Wykorzystanie metody DEA do analizy zmian produktywności i efektywności w zakresie produkcji zbóż według województw w latach 1997-2019 , Metody Ilościowe w Badaniach Ekonomicznych: Tom 23 Nr 4 (2022)
- Urszula Grzybowska, Marek Karwański, Selekcja zmiennych metodami statystycznymi i uczenia maszynowego. Porównanie podejść na przykładzie danych finansowych , Metody Ilościowe w Badaniach Ekonomicznych: Tom 24 Nr 4 (2023)
- Bolesław Borkowski, Marek Karwański, Wiesław Szczesny, PORÓWNANIE SKUTECZNOŚCI DWÓCH KULTUR ANALITYCZNYCH , Metody Ilościowe w Badaniach Ekonomicznych: Tom 22 Nr 4 (2021)
- Marek Karwański, Urszula Grzybowska, APPLICATION OF L-MOMENTS IN HOMOGENEITY EXAMINATION FOR GROUPS OF PRODUCTION COMPANIES DISTINGUISHED BY DEA , Metody Ilościowe w Badaniach Ekonomicznych: Tom 18 Nr 3 (2017)
- Marek Karwański, Krzysztof Zmarzłowski, MODELE PROGNOZ EKONOMETRYCZNYCH , Metody Ilościowe w Badaniach Ekonomicznych: Tom 16 Nr 4 (2015)
- Urszula Grzybowska, Marek Karwański, Wykorzystanie miar matematycznych i biznesowych do porównania modeli macierzy migracji stosowanych w analizie ryzyka kredytowego , Metody Ilościowe w Badaniach Ekonomicznych: Tom 12 Nr 2 (2011)
- Marek Karwański, Urszula Grzybowska, Vassilis Kostoglou, Ewa Mierzejewska, Katarzyna Szamotulska, Application of ITransformers to Predicting Preterm Birth Rate. Comparison with the ARIMA Model , Metody Ilościowe w Badaniach Ekonomicznych: Tom 25 Nr 3 (2024)
- Urszula Grzybowska, Marek Karwański, FAMILIES OF CLASSIFIERS – APPLICATION IN DATA , Metody Ilościowe w Badaniach Ekonomicznych: Tom 15 Nr 2 (2014)
- Urszula Grzybowska, Marek Karwański, APPLICATION OF MIXED MODELS AND FAMILIES OF CLASSIFIERS TO ESTIMATION OF FINANCIAL RISK PARAMETERS , Metody Ilościowe w Badaniach Ekonomicznych: Tom 16 Nr 1 (2015)
- Urszula Grzybowska, Marek Karwański, APPLICATION OF MIGRATION MATRICES TO RISK EVALUATION AND THEIR IMPACT ON PORTFOLIO VALUE , Metody Ilościowe w Badaniach Ekonomicznych: Tom 14 Nr 1 (2013)
Utwór dostępny jest na licencji Creative Commons Uznanie autorstwa – Użycie niekomercyjne 4.0 Międzynarodowe.
Publikowane artykuły dostępne są na warunkach Open Access na zasadach licencji Creative Commons CC BY-NC – do celów niekomercyjnych udostępnione materiały mogą być kopiowane, drukowane i rozpowszechniane. Autorzy ponoszą opłatę za opublikowanie artykułu.