Optimizing Automated Machine Learning for Ensemble Performance and Overfitting Mitigation
Article Sidebar
-
Systematic Literature Review, AutoML, Ensemble Learning, Overfitting Mitigation, Enhancing Diversity
Abstract
Automated Machine Learning (AutoML) has revolutionized model development, but its impact on ensemble diversity and overfitting reduction remains underexplored. This Systematic Literature Review (SLR) analyzes 107 studies published between 2020 and 2024 to explore how AutoML enhances ensemble diversity, mitigates overfitting, and the challenges hindering its integration. Unlike previous reviews focusing on AutoML or ensemble methods independently, this study synthesizes their intersection and identifies key research trends. The findings reveal that AutoML improves ensemble robustness through automated hyperparameter tuning, meta-learning, and algorithmic blending while facing trade-offs in computational cost and interpretability. Four main themes emerge, integration mechanisms (19.6%), overfitting mitigation (26.2%), performance trade-offs (28.6%), and integration barriers (26.2%). Empirical results indicate that AutoML ensembles outperform traditional models by 22–41% in accuracy but require approximately 3.2 times higher computational resources. Hybrid AutoML and Explainable AI frameworks are recommended to balance accuracy and transparency. Theoretically, this study advances understanding of the synergy between AutoML and ensemble learning, while practically providing guidance for deploying reliable AI systems in sectors like healthcare, finance, and digital business. Policy implications align with the EU AI Act and the US Executive Order on trustworthy AI, supporting Sustainable Development Goals 9 and 8.
Full text article
References
[1] J. Guo, Z. Chen, Y. Ji, L. Zhang, D. Luo, Z. Li, and Y. Shen, “Uniautoml: A human-centered framework for unified discriminative and generative automl with large language models,” arXiv preprint arXiv:2410.12841, 2024.
[2] F. Mohr and M. Wever, “Naive automated machine learning,” Machine Learning, vol. 112, no. 4, pp. 1131–1170, 2023.
[3] S. Wijono, U. Rahardja, H. D. Purnomo, N. Lutfiani, and N. A. Yusuf, “Leveraging machine learning models to enhance startup collaboration and drive technopreneurship,” Aptisi Transactions on Technopreneurship (ATT), vol. 6, no. 3, pp. 432–442, 2024.
[4] F. ¨Ozbayrak, J. T. Foster, and M. J. Pyrcz, “Spatial bagging for predictive machine learning uncertainty quantification,” Computers & Geosciences, p. 105947, 2025.
[5] V. B. Kamble, K. Pisal, P. Vaidya, and S. Gaikwad, “Enhancing upi fraud detection: A machine learning approach using stacked generalization,” International Journal of Multidisciplinary on Science and Management, vol. 2, no. 1, pp. 69–83, 2025.
[6] M. Baratchi, C. Wang, S. Limmer, J. N. Van Rijn, H. Hoos, T. B¨ack, and M. Olhofer, “Automated machine learning: past, present and future,” Artificial intelligence review, vol. 57, no. 5, p. 122, 2024.
[7] P. Gijsbers, M. L. Bueno, S. Coors, E. LeDell, S. Poirier, J. Thomas, B. Bischl, and J. Vanschoren, “Amlb: an automl benchmark,” Journal of Machine Learning Research, vol. 25, no. 101, pp. 1–65, 2024.
[8] J. G. Hernandez, A. K. Saini, A. Ghosh, and J. H. Moore, “The tree-based pipeline optimization tool: Tackling biomedical research problems with genetic programming and automated machine learning,” Patterns, vol. 6, no. 7, 2025.
[9] N. Hasebrook, F. Morsbach, N. Kannengießer, M. Z¨oller, J. Franke, M. Lindauer, F. Hutter, and A. Sunyaev, “Practitioner motives to select hyperparameter optimization methods,” arXiv preprint arXiv:2203.01717, 2022.
[10] S. Anggoro and A. Nuche, “Leadership configurations supporting togaf-based information system architecture at jenderal achmad yani university,” International Journal of Cyber and IT Service Management (IJCITSM), vol. 5, no. 2, pp. 134–143, 2025.
[11] M. A. Pava, R. Groh, and A. M. Kist, “Eg-enas: Efficient and generalizable evolutionary neural architecture search for image classification,” in AutoML 2025 Methods Track, 2025.
[12] S. Wan, “Automatic optimization method for database indexing by integrating monte carlo tree search and graph neural network,” Procedia Computer Science, vol. 262, pp. 831–839, 2025.
[13] N. Alangari, M. El Bachir Menai, H. Mathkour, and I. Almosallam, “Exploring evaluation methods for interpretable machine learning: A survey,” Information, vol. 14, no. 8, p. 469, 2023.
[14] U. Sarmah, P. Borah, and D. K. Bhattacharyya, “Ensemble learning methods: An empirical study,” SN Computer Science, vol. 5, no. 7, p. 924, 2024.
[15] J. Siswanto, Hendry, U. Rahardja, I. Sembiring, E. Sediyono, K. D. Hartomo, and B. Istiyanto, “Deep learning-based lstm model for number of road accidents prediction,” in AIP Conference Proceedings, vol. 3234, no. 1. AIP Publishing LLC, 2025, p. 050004.
[16] J. Friedman, “Greedy function approximation: a gradient boosting machine. accessed july 9,” JSTOR, 2023.
[17] S.-C. Chen, I. Yati, and E. A. Beldiq, “Advancing production management through industry 4.0 technologies,” Startupreneur Business Digital (SABDA Journal), vol. 3, no. 2, pp. 181–192, 2024.
[18] R. Bounab, K. Zarour, B. Guelib, and N. Khlifa, “Enhancing medicare fraud detection through machine learning: Addressing class imbalance with smote-enn,” IEEE Access, vol. 12, pp. 54 382–54 396, 2024.
[19] M. Rahmaty, “Machine learning with big data to solve real-world problems,” Journal of Data Analytics, vol. 2, no. 1, pp. 9–16, 2023.
[20] X. Liu, D. McDuff, G. Kovacs, I. Galatzer-Levy, J. Sunshine, J. Zhan, M.-Z. Poh, S. Liao, P. Di Achille, and S. Patel, “Large language models are few-shot health learners,” arXiv preprint arXiv:2305.15525, 2023.
[21] P. C. Sauer and S. Seuring, “How to conduct systematic literature reviews in management research: a guide in 6 steps and 14 decisions,” Review of Managerial Science, vol. 17, no. 5, pp. 1899–1933, 2023.
[22] L. Cazenille, “Ensemble feature extraction for multi-container quality-diversity algorithms,” in Proceedings of the Genetic and Evolutionary Computation Conference, 2021, pp. 75–83.
[23] A. H. Aditiya, H. Hamdan, S. N. W. Putra, S. Visiana, and R. Thakkar, “Transforming education with genai: Case study on chatgpt, midjourney, and policy changes,” Sundara Advanced Research on Artificial Intelligence, vol. 1, no. 1, pp. 20–27, 2025.
[24] N. Rane, S. Choudhary, and J. Rane, “Leading-edge technologies for architectural design: a comprehensive review,” Available at SSRN 4637891, 2023.
[25] S. Kambhampati, K. Valmeekam, L. Guan, M. Verma, K. Stechly, S. Bhambri, L. P. Saldyt, and A. B. Murthy, “Position: Llms can’t plan, but can help planning in llm-modulo frameworks,” in Forty-first International Conference on Machine Learning, 2024.
[26] S. Zaidi, A. Zela, T. Elsken, C. C. Holmes, F. Hutter, and Y. Teh, “Neural ensemble search for uncertainty estimation and dataset shift,” Advances in Neural Information Processing Systems, vol. 34, pp. 7898–7911, 2021.
[27] R. Xin, H. Liu, P. Chen, and Z. Zhao, “Robust and accurate performance anomaly detection and prediction for cloud applications: a novel ensemble learning-based framework,” Journal of Cloud Computing, vol. 12, no. 1, p. 7, 2023.
[28] L. Chen and H. Shi, “Dexdeepfm: Ensemble diversity enhanced extreme deep factorization machine model,” ACM Transactions on Knowledge Discovery from Data (TKDD), vol. 16, no. 5, pp. 1–17, 2022.
[29] Y. Wang, Q. Zhang, G.-G. Wang, and H. Cheng, “The application of evolutionary computation in generative adversarial networks (gans): a systematic literature survey,” Artificial Intelligence Review, vol. 57, no. 7, p. 182, 2024.
[30] Y. Liu, X. Wang, X. Xu, J. Yang, and W. Zhu, “Meta hyperparameter optimization with adversarial proxy subsets sampling,” in Proceedings of the 30th ACM International Conference on Information & Knowledge Management, 2021, pp. 1109–1118.
[31] A. M. Vincent and P. Jidesh, “An improved hyperparameter optimization framework for automl systems using evolutionary algorithms,” Scientific Reports, vol. 13, no. 1, p. 4737, 2023.
[32] S. A. Sibagariang, N. Septiani, and A. Rodriguez, “Enhancing educational management through social media and e-commerce-driven branding,” International Journal of Cyber and IT Service Management (IJCITSM), vol. 5, no. 2, pp. 235–245, 2025.
[33] N. Rost, T. M. Br¨uckl, N. Koutsouleris, E. B. Binder, and B. M¨uller-Myhsok, “Creating sparser prediction models of treatment outcome in depression: a proof-of-concept study using simultaneous feature selection and hyperparameter tuning,” BMC medical informatics and decision making, vol. 22, no. 1, p. 181, 2022.
[34] S. Horv´ath, A. Klein, P. Richt´arik, and C. Archambeau, “Hyperparameter transfer learning with adaptive complexity,” in International conference on artificial intelligence and statistics. PMLR, 2021, pp. 1378–1386.
[35] Q. Wu, C. Wang, and S. Huang, “Frugal optimization for cost-related hyperparameters,” in Proceedings of the AAAI conference on artificial intelligence, vol. 35, no. 12, 2021, pp. 10 347–10 354.
[36] A. V. Luong, T. T. Nguyen, K. Han, T. H. Vu, J. McCall, and A. W.-C. Liew, “Defeg: deep ensemble with weighted feature generation,” Knowledge-Based Systems, vol. 275, p. 110691, 2023.
[37] Y.-R. Choi and D.-J. Lim, “Ddes: A distribution-based dynamic ensemble selection framework,” IEEE Access, vol. 9, pp. 40 743–40 754, 2021.
[38] D. Jonas, H. D. Purnomo, A. Iriani, I. Sembiring, D. P. Kristiadi, and Z. Nanle, “Iot-based community smart health service model: Empowering entrepreneurs in health innovation,” Aptisi Transactions on Technopreneurship (ATT), vol. 7, no. 1, pp. 61–71, 2025.
[39] A. J. Preto, P. Matos-Filipe, J. Mour˜ao, and I. S. Moreira, “Synpred: prediction of drug combination effects in cancer using different synergy metrics and ensemble learning,” GigaScience, vol. 11, p. giac087, 2022.
[40] N. P. L. Santoso, R. Nurmala, and U. Rahardja, “Corporate leadership in the digital business era and its impact on economic development across global markets,” IAIC Transactions on Sustainable Digital Innovation (ITSDI), vol. 6, no. 2, pp. 188–195, 2025.
[41] N. Bosch et al., “Automl feature engineering for student modeling yields high accuracy, but limited interpretability,” Journal of Educational Data Mining, vol. 13, no. 2, pp. 55–79, 2021.
[42] H. Eldeeb and R. Elshawi, “Empowering machine learning with scalable feature engineering and interpretable automl,” IEEE Transactions on Artificial Intelligence, vol. 6, no. 2, pp. 432–447, 2024.
[43] T. S. Goh, D. Jonas, B. Tjahjono, V. Agarwal, and M. Abbas, “Impact of ai on air quality monitoring systems: A structural equation modeling approach using utaut,” Sundara Advanced Research on Artificial Intelligence, vol. 1, no. 1, pp. 9–19, 2025.
[44] C. Rudin, C. Chen, Z. Chen, H. Huang, L. Semenova, and C. Zhong, “Interpretable machine learning: Fundamental principles and 10 grand challenges,” Statistic Surveys, vol. 16, pp. 1–85, 2022.
[45] K. Zhou, Y. Yang, Y. Qiao, and T. Xiang, “Domain adaptive ensemble learning,” IEEE Transactions on Image Processing, vol. 30, pp. 8008–8018, 2021.
[46] P. Sarajcev, A. Kunac, G. Petrovic, and M. Despalatovic, “Power system transient stability assessment using stacked autoencoder and voting ensemble. energies 2021, 14, 3148,” 2021.
[47] X. Yang, W. Yan, Y. Yuan, M. B. Mi, and R. T. Tan, “Semantic segmentation in multiple adverse weather conditions with domain knowledge retention,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 7, 2024, pp. 6558–6566.
[48] W. Liu and C. Zhao, “Etm: effective tuning method based on multi-objective and knowledge transfer in image recognition,” IEEE Access, vol. 9, pp. 47 216–47 229, 2021.
[49] D. Wuisan, J. W. Manurung, C. Wantah, and M. E. Yuliana, “Entrepreneurial self-employment and work engagement in msmes through autonomy and rewards,” Aptisi Transactions on Technopreneurship (ATT), vol. 7, no. 1, pp. 264–281, 2025.
[50] R. Barbudo, S. Ventura, and J. R. Romero, “Eight years of automl: categorisation, review and trends,” Knowledge and Information Systems, vol. 65, no. 12, pp. 5097–5149, 2023.
[51] R. Zheng, L. Qu, B. Cui, Y. Shi, and H. Yin, “Automl for deep recommender systems: A survey,” ACM Transactions on Information Systems, vol. 41, no. 4, pp. 1–38, 2023.
[52] S. K. Karmaker, M. M. Hassan, M. J. Smith, L. Xu, C. Zhai, and K. Veeramachaneni, “Automl to date and beyond: Challenges and opportunities,” Acm computing surveys (csur), vol. 54, no. 8, pp. 1–36, 2021.
[53] L. Quaranta, K. Azevedo, F. Calefato, and M. Kalinowski, “A multivocal literature review on the benefits and limitations of industry-leading automl tools,” Information and Software Technology, vol. 178, p. 107608, 2025.
[54] T. W. House, “Removing barriers to american leadership in artificial intelligence,” https://www.whitehouse.gov/presidential-actions/2025/01/removing-barriers-to-american-leadership-in-artificial-intelligence/, January 2025, accessed: 2025-10-21.
[55] E. Commission, “Ai act enters into force,” https://commission.europa.eu/news-and-media/news/ai-act-enters-force-2024-08-01 en, August 2024, accessed: 2025-10-21.
[56] O. for Economic Co-operation and Development, “Oecd ai principles,” https://www.oecd.org/en/topics/sub-issues/ai-principles.html, 2024, accessed: 2025-10-21.
Authors
Copyright (c) 2025 Migunani, Adi Setiawan, Irwan Sembiring
This work is licensed under a Creative Commons Attribution 4.0 International License.
This journal permits and encourages authors to post items submitted to the journal on personal websites while providing bibliographic details that credit its publication in this journal.
Authors are permitted to post their work online in institutional/disciplinary repositories or on their own websites. Pre-print versions posted online should include a citation and link to the final published version in Journal of Librarianship and Scholarly Communication as soon as the issue is available; post-print versions (including the final publisher's PDF) should include a citation and link to the journal's website.