{"id":38749193,"url":"https://github.com/joshuapowell/preparing-your-mainframe-data-for-machine-learning","last_synced_at":"2026-01-17T11:48:28.085Z","repository":{"id":262832969,"uuid":"888497887","full_name":"joshuapowell/preparing-your-mainframe-data-for-machine-learning","owner":"joshuapowell","description":"Mainframe Data Wrangling: Preparing Your Mainframe Data for Machine Learning","archived":false,"fork":false,"pushed_at":"2025-04-21T13:40:38.000Z","size":182,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-09-05T07:28:27.848Z","etag":null,"topics":["api-usability","data-engineering","data-wrangling","large-scale-computing","machine-learning","mainframe"],"latest_commit_sha":null,"homepage":"","language":"TeX","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/joshuapowell.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-11-14T13:58:47.000Z","updated_at":"2025-04-21T13:40:41.000Z","dependencies_parsed_at":null,"dependency_job_id":"7992e6f5-3048-4564-bd1d-dd7ebaa377c1","html_url":"https://github.com/joshuapowell/preparing-your-mainframe-data-for-machine-learning","commit_stats":null,"previous_names":["joshuapowell/preparing-your-mainframe-data-for-machine-learning"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/joshuapowell/preparing-your-mainframe-data-for-machine-learning","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joshuapowell%2Fpreparing-your-mainframe-data-for-machine-learning","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joshuapowell%2Fpreparing-your-mainframe-data-for-machine-learning/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joshuapowell%2Fpreparing-your-mainframe-data-for-machine-learning/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joshuapowell%2Fpreparing-your-mainframe-data-for-machine-learning/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/joshuapowell","download_url":"https://codeload.github.com/joshuapowell/preparing-your-mainframe-data-for-machine-learning/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joshuapowell%2Fpreparing-your-mainframe-data-for-machine-learning/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28508459,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-17T10:25:30.148Z","status":"ssl_error","status_checked_at":"2026-01-17T10:25:29.718Z","response_time":85,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api-usability","data-engineering","data-wrangling","large-scale-computing","machine-learning","mainframe"],"created_at":"2026-01-17T11:48:27.402Z","updated_at":"2026-01-17T11:48:28.076Z","avatar_url":"https://github.com/joshuapowell.png","language":"TeX","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Mainframe Data Wrangling: Preparing Your Data for Use in Machine Learning Models\n\nMainframe computing continues to drive the global economy, with\nforty-five of the world's top fifty banks \\[1\\] handling critical\ntransaction data through the IBM Z mainframe platform. While recent\nresearch highlights the importance of mainframe modernization rather\nthan replacement \\[2\\], enterprises struggle to effectively utilize\nmainframe data for automation and optimization due to data-driven and\ncommunication-driven failures \\[3\\]. This challenge creates a\nsignificant gap between available mainframe capabilities and realized\nbusiness value \\[4\\].\n\nTo address this gap, we conducted semi-structured interviews with\neighteen participants across three roles: mainframe subject matter\nexperts (SME) \\[n=6\\], mainframe individual contributor end-users\n\\[n=9\\], and mainframe people manager end-users \\[n=3\\]. The study,\nconducted between \\[dates\\], investigated two research questions:\n\\[RQ1\\] What are the primary use cases in which mainframe network and\nnetwork security data impacts mean-time-to-resolution (MTTR) in top\nfifty banks? And \\[RQ2\\] What mainframe data sources and methods do\nend-users employ to resolve these network and network security issues?\n\nAnalysis revealed that 91% of participants \\[5\\] encountered data\nquality or completeness issues that impeded network problem resolution.\nThis tutorial demonstrates how end-users can leverage exploratory data\nanalysis techniques \\[6\\], mainframe data APIs \\[7\\], and open source\ndata science tools \\[8\\] to prepare data for advanced analytics and\nmachine learning applications. The presented methodology aims to reduce\nMTTR by addressing identified data-driven and communication-driven\nfailure points \\[9\\], with specific focus on network security use cases\n\\[10-13\\].\n\n**Keywords:** Data Engineering, Data Wrangling, API Usability, API\nOnboarding, Mainframe, Large-scale Computing, Machine Learning\n\n## References\n1. IBM (International Business Machines Corporation). (2023). IBM 2023 Annual Report.\n2. Wishart-Smith, H. (2024, November 13). Mainframes: the backbone of the\nworldwide economy. Forbes.\n3. Ryseff, J., de Bruhl, B., \u0026 Newberry, S. J. (2024). The Root Causes of\nFailure for Artificial Intelligence Projects and How They Can Succeed:\nAvoiding the Anti-Patterns of AI.\n4. IBM z/OS operating system. (Accessed: October 28, 2024).\nhttps://www.ibm.com/products/zos\n5. Mcgregor, S. E. (2022). Practical Python Data Wrangling and Data\nQuality. http://oreilly.com\n6. Powell, J.I., Broadcom Mainframe Software Division, Internal Study,\nApril 2024\n7. Alam, A., Bales, R., Dumir, V., Kunze, N., Li, J., Mishra, S., Rivera,\nE., Wan, M., \u0026 Yu, Y. (2024). Turning Data into Insight with Machine\nLearning for IBM z/OS (First). International Business Machines\nCorporation.\n8. Broadcom Mainframe Developer Portal. (Accessed: October 28, 2024).\nhttps://integration.mainframe.broadcom.com/\n9. Harrell, M. (2024). Mainframe Application Developer Study.\n10. Kanvar, V., Tamilselvam, S., \u0026 Raghunath, K. N. (2024, August 8).\nEnabling communication via APIs for mainframe applications. arXiv.org.\nhttps://arxiv.org/abs/2408.04230\n11. Dau, A. T., V., Dao, H. T., Nguyen, A. T., Tran, H. T., Nguyen, P. X., \u0026\nBui, N. D. Q. (2024, August 5). XMainframe: a large language model for\nmainframe modernization. arXiv.org. https://arxiv.org/abs/2408.04660\n12. Raju, J., Modernizing Mainframe Workloads in Banking: Embracing the\nPower of Hyperscalers, International Journal of Computer Engineering and\nTechnology (IJCET), 15(5), 2024, pp. 366-374.\n13. Raju, J., AI-Driven Transformation of Mainframe Environments: A\nComprehensive Framework for Operational Resilience, International\nJournal of Engineering and Technology Research (IJETR), 9(2), 2024, pp.\n420--433.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjoshuapowell%2Fpreparing-your-mainframe-data-for-machine-learning","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjoshuapowell%2Fpreparing-your-mainframe-data-for-machine-learning","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjoshuapowell%2Fpreparing-your-mainframe-data-for-machine-learning/lists"}