{"id":19659785,"url":"https://github.com/maicius/intelligentmachine","last_synced_at":"2025-10-06T08:13:17.930Z","repository":{"id":82734721,"uuid":"115812687","full_name":"Maicius/IntelligentMachine","owner":"Maicius","description":"天池工业AI大赛-智能制造质量预测，排名89/2539","archived":false,"fork":false,"pushed_at":"2018-10-11T00:41:20.000Z","size":68491,"stargazers_count":36,"open_issues_count":0,"forks_count":16,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-28T20:46:11.683Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Maicius.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-12-30T17:41:09.000Z","updated_at":"2024-12-12T08:36:38.000Z","dependencies_parsed_at":null,"dependency_job_id":"d09f064c-7871-4f2f-b098-d4089311bcea","html_url":"https://github.com/Maicius/IntelligentMachine","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Maicius/IntelligentMachine","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Maicius%2FIntelligentMachine","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Maicius%2FIntelligentMachine/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Maicius%2FIntelligentMachine/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Maicius%2FIntelligentMachine/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Maicius","download_url":"https://codeload.github.com/Maicius/IntelligentMachine/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Maicius%2FIntelligentMachine/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278577929,"owners_count":26009703,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-06T02:00:05.630Z","response_time":65,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-11T15:44:22.220Z","updated_at":"2025-10-06T08:13:17.924Z","avatar_url":"https://github.com/Maicius.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003e\n1.比赛链接：[天池工业AI大赛-智能制造质量预测](https://tianchi.aliyun.com/competition/rankingList.htm?season=0\u0026raceId=231633\u0026pageIndex=2)，最终排名89/2529\n\n\u003e 2.Github链接：[IntelligentMachine](https://github.com/Maicius/IntelligentMachine)\n\n\u003e 3.博客地址：[记第一次参加的数据挖掘比赛](http://www.xiaomaidong.com/?p=565)\n\n## 比赛中所用的算法和技术回顾\n\n### 特征工程\n\n\u003e 所有代码都包含在了github的项目中，都写为了独立的函数\n\n- 去除错误列\n\n\t\u003e 去除数据中缺失值过多的列或行，参数得自己调整。\n\n- 去除错误行\n\n\t\u003e 对于不符合正态分布3 sigmoid的行（根据正太分布原理，数值分布在（μ—3σ,μ+3σ)中的概率为0.9974），超过一定数量就删除该行。这部分代码如下:\n\n\u003e\n\t\n\t# 去除不符合正太分布的行\n\tdef remove_wrong_row(data):\n\t\t# 计算每一列数据的上界\n    \tupper = data.mean(axis=0) + 3 * data.std(axis=0)\n    \t# 计算每一列数据的下界\n    \tlower = data.mean(axis=0) - 3 * data.std(axis=0)\n\t\t# 计算每一行中超过上界的数值的数量\n    \twrong_data1 = (data \u003e upper).sum(axis=1).reset_index()\n    \twrong_data1.columns = ['row', 'na_count']\n    \t# 参数是经过调试的\n    \t# 记录数量超过一定值的行数\n    \twrong_row1 = wrong_data1[wrong_data1.na_count \u003e= 40].row.values\n    \t# 计算每一行中超过下界的数值的数量\n    \twrong_data2 = (data \u003c lower).sum(axis=1).reset_index()\n    \twrong_data2.columns = ['row', 'na_count']\n    \twrong_row2 = wrong_data2[wrong_data2.na_count \u003e= 95].row.values\n    \twrong_row = np.concatenate((wrong_row1, wrong_row2))\n\t\t# 去除不符合正太分布的行\n    \tdata.drop(wrong_row, axis=0, inplace=True)\n    \treturn data\n\n\t\n\t\n\n- 填补缺失值\n\t\n\t\u003e 由于缺失的值比较多，单纯的使用fillna()的效果不是很好，在满绩王的启发下，写了KNN 近邻填充。就是利用欧式距离上距离每一行最近的K行在某一列的平均值来填充该行在该列的缺失值。具体实现代码如下（我已经尽量使用矩阵的算法来计算距离，避免使用for循环，但代码中还是使用了一层for循环，如果哪位大佬知道更好的方式，请告诉我）：\n\t\n\t\n\t\tdef knn_fill_nan(data, K):\n\n    \t\t# 计算每一行的空值，如有空值则进行填充，没有空值的行用于做训练数据\n    \t\tdata_row = data.isnull().sum(axis=1).reset_index()\n    \t\tdata_row.columns = ['raw_row', 'nan_count']\n    \t\t# 空值行（需要填充的行）\n    \t\tdata_row_nan = data_row[data_row.nan_count \u003e 0].raw_row.values\n\n    \t\t# 非空行 原始数据\n    \t\tdata_no_nan = data.drop(data_row_nan, axis=0)\n\n    \t\t# 空行 原始数据\n    \t\tdata_nan = data.loc[data_row_nan]\n    \t\tfor row in data_row_nan:\n        \t\tdata_row_need_fill = data_nan.loc[row]\n        \t\t# 找出空列，并利用非空列做KNN\n        \t\tdata_col_index = data_row_need_fill.isnull().reset_index()\n        \t\tdata_col_index.columns = ['col', 'is_null']\n        \t\tis_null_col = data_col_index[data_col_index.is_null == 1].col.values\n        \t\tdata_col_no_nan_index = data_col_index[data_col_index.is_null == 0].col.values\n        \t\t# 保存需要填充的行的非空列\n        \t\tdata_row_fill = data_row_need_fill[data_col_no_nan_index]\n\n        \t\t# 广播，矩阵 - 向量\n        \t\tdata_diff = data_no_nan[data_col_no_nan_index] - data_row_need_fill[data_col_no_nan_index]\n        \t\t# 求欧式距离\n        \t\t# data_diff = data_diff.apply(lambda x: x**2)\n        \t\tdata_diff = (data_diff ** 2).sum(axis=1)\n        \t\tdata_diff = data_diff.apply(lambda x: np.sqrt(x))\n        \t\tdata_diff = data_diff.reset_index()\n        \t\tdata_diff.columns = ['raw_row', 'diff_val']\n        \t\tdata_diff_sum = data_diff.sort_values(by='diff_val', ascending=True)\n        \t\tdata_diff_sum_sorted = data_diff_sum.reset_index()\n        \t\t# 取出K个距离最近的row\n        \t\ttop_k_diff_row = data_diff_sum_sorted.loc[0:K - 1].raw_row.values\n        \t\t# 根据row 和 col值确定需要填充的数据的具体位置（可能是多个）\n        \t\t# 填充的数据为最近的K个值的平均值\n        \t\ttop_k_diff_val = data.loc[top_k_diff_row][is_null_col].sum(axis=0) / K\n\n        \t\t# 将计算出来的列添加至非空列\n        \t\tdata_row_fill = pd.concat([data_row_fill, pd.DataFrame(top_k_diff_val)]).T\n        \t\t# print(data_no_nan.shape)\n        \t\tdata_no_nan = data_no_nan.append(data_row_fill, ignore_index=True)\n        \t\t# print(data_no_nan.shape)\n    \t\tprint('填补缺失值完成')\n    \t\treturn data_no_nan\n\t\n\n- 去除日期列\n\n\t\u003e 在本比赛环境中，日期对生产误差应该没有影响，但是会影响我们的线性模型，所以删除\n\t\n- 将非浮点数列转化为浮点数列或直接删除\n\n\t\u003e 主要是将数据值为字母、单词等列转换为浮点数，这些列主要是生产工具的名称，可能会有影响，在经过尝试之后，发现直接删除这些列效果更好一丢丢。\n\t\n\n- 去除皮尔森系数在[-0.1, 0.1]之间的特征列\n\n\t\u003e 上过概率统计的应该还记得它，它描述了两个数据之间的线形相关性，值属于[-1, 1]，-1表示完全负相关，1表示完全相关，0表示完全不相关。这里去除了绝对值小于0.1的特征，也就是几乎没什么关系的列。\n\t\n- 特征选择\n\n\t\u003e 这是特征工程中最重要的一步，看了很多博客里的方法，做了很多尝试，最终选择使用了模型融合（在复赛A榜中效果较好）。使用了RandomForestRegressor()/ AdaBoostRegressor()/ ExtraTreesRegressor()对每一列数据进行评分，选择评分最好的100列，再进行融合（其实到这里我已经感觉很玄学了）。使用了这个方法之后，我跑一遍程序的时间够我看一部电影了。代码如下：\n\t\n\t\tdef ensemble_model_feature(X, Y, top_n_features):\n    \t\tfeatures = list(X)\n    \t\t# 随机森林\n    \t\trf = ensemble.RandomForestRegressor()\n    \t\trf_param_grid = {'n_estimators': [900], 'random_state': [2, 4, 6, 8]}\n    \t\trf_grid = GridSearchCV(rf, rf_param_grid, cv=10, verbose=1, n_jobs=25)\n    \t\trf_grid.fit(X, Y)\n    \t\ttop_n_features_rf = get_top_k_feature(features=features, model=rf_grid, top_n_features=top_n_features)\n    \t\tprint('RF 选择完毕')\n    \t\t# Adaboost\n    \t\tabr = ensemble.AdaBoostRegressor()\n    \t\tabr_grid = GridSearchCV(abr, rf_param_grid, cv=10, n_jobs=25)\n    \t\tabr_grid.fit(X, Y)\n    \t\ttop_n_features_bgr = get_top_k_feature(features=features, model=abr_grid, top_n_features=top_n_features)\n    \t\tprint('Adaboost 选择完毕')\n    \t\t# ExtraTree\n    \t\tetr = ensemble.ExtraTreesRegressor()\n    \t\tetr_grid = GridSearchCV(etr, rf_param_grid, cv=10, n_jobs=25)\n    \t\tetr_grid.fit(X, Y)\n    \t\ttop_n_features_etr = get_top_k_feature(features=features, model=etr_grid, top_n_features=top_n_features)\n    \t\tprint('ETR 选择完毕')\n    \t\t# 融合以上三个模型\n    \t\tfeatures_top_n = pd.concat([top_n_features_rf, top_n_features_bgr, top_n_features_etr],\n                               ignore_index=True).drop_duplicates()\n    \t\tprint(features_top_n)\n    \t\tprint(len(features_top_n))\n    \t\treturn features_top_n\n \n- 数据规范化（Normalization）\n\n\t\u003e 这是一项基本操作，由于数据与数据之间值的差距特别大，为了减少误差，须通过规范化，将所有数据的值都映射到同一范围内。规范化的具体实现有很多种，我最终使用的是sklearn.preprocess.scale()。另外，如果自己实现的话，代码如下：\n\t\n\t\t# 自定义规范化数据\n\t\tdef normalize_data(data):\n\t\t\t# 最大最小值规范化\n    \t\treturn data.apply(lambda x: (x - np.min(x)) / (np.max(x) - np.min(x)))\n    \t\t# z-socre 规范化\n    \t\t# return data.apply(lambda x: (x - np.average(x)) / np.std(x))\n \n \n- 至此，特征工程的所有工作就完成了。最终特征的维数在170左右，根据参数的不同会有一些变化。\n\n\n### 机器学习\n\n\u003e 一开始在技术圈看到有人讨论，很多人都说这是一个线性模型，所以我就主要关注线性模型了。我尝试了很多模型，关于每种模型我就简单解释一下了，原理都可以自己检索。\n\n- 最小二乘线性拟合， sklearn.linear_model.LinearRegression\n\n\u003e 最常规的线性回归模型，使用最小二乘法做拟合，效果一般\n\n- 最小二乘线性拟合 + L2正则，sklearn.linear_model.Ridge\n\n\u003e 引入了L2正则的线性拟合，因为数据特征过多，样本太少，过拟合一直都是整个比赛中最大的问题。为了避免过拟合，首先想到的就是L2正则。效果确实好多了。\n\n- Ridge + Bagging 集成学习\n\n\u003e 使用Ridge为基学习器，使用Bagging做集成学习，由于基础模型的选择有限，所以这里选择了Bagging做融合。Bagging是从同一数据集中随机抽取不同的数据做训练，讲道理非常适合这种样本非常少的情况。本地线下交叉验证的结果与单纯的Ridge差不多，但是感觉更稳定。初赛最终提交的就是这种算法产出的数据。\n\n- xgboost及调参\n\n\u003e xgboost在很多文章中都被推崇，初赛中也使用了xgboost，但是还不怎么会调参，所以效果不是很好。在复赛A榜阶段，使用xgboost取得了不错的成果，并通过将xgboost与Bagging进一步融合，再将以前提提交的线上验证比较好的答案融合，得到了A榜最好的一次成绩（其实也很糟糕...)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaicius%2Fintelligentmachine","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmaicius%2Fintelligentmachine","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaicius%2Fintelligentmachine/lists"}