ML之xgboost：利用xgboost算法(sklearn+3Split+调参曲线)训练mushroom蘑菇数据集(22+1,6513+1611)来预测蘑菇是否毒性(二分类预测)

ML之xgboost：利用xgboost算法(sklearn+3Split+调参曲线)训练mushroom蘑菇数据集(22+1,6513+1611)来预测蘑菇是否毒性(二分类预测)

---

输出结果

设计思路

核心代码

eval_set = [(X_train_part, y_train_part), (X_validate, y_validate)]
bst.fit(X_train_part, y_train_part, eval_metric=["error", "logloss"], eval_set=eval_set, verbose=True)

preds = bst.predict(X_test)
predictions = [round(value) for value in preds]
test_accuracy = accuracy_score(y_test, predictions)
print("【max_depth=2,lr=0.1】Test Accuracy: %.2f%%" % (test_accuracy * 100.0))

results = bst.evals_result() 

更多输出

X_train： (6513, 126)
X_test： (1611, 126)
After split(33%)，X_train_part： (4363, 126)
After split(33%)，X_validate： (2150, 126)
[0]validation_0-error:0.045611validation_0-logloss:0.614637validation_1-error:0.048372validation_1-logloss:0.615401
[1]validation_0-error:0.041256validation_0-logloss:0.549907validation_1-error:0.042326validation_1-logloss:0.550696
[2]validation_0-error:0.045611validation_0-logloss:0.49543validation_1-error:0.048372validation_1-logloss:0.496777
[3]validation_0-error:0.041256validation_0-logloss:0.449089validation_1-error:0.042326validation_1-logloss:0.450412
[4]validation_0-error:0.041256validation_0-logloss:0.409231validation_1-error:0.042326validation_1-logloss:0.410717
[5]validation_0-error:0.041256validation_0-logloss:0.373748validation_1-error:0.042326validation_1-logloss:0.375653
[6]validation_0-error:0.023378validation_0-logloss:0.343051validation_1-error:0.023256validation_1-logloss:0.344738
[7]validation_0-error:0.041256validation_0-logloss:0.315369validation_1-error:0.042326validation_1-logloss:0.317409
[8]validation_0-error:0.041256validation_0-logloss:0.290912validation_1-error:0.042326validation_1-logloss:0.292587
[9]validation_0-error:0.023378validation_0-logloss:0.269356validation_1-error:0.023256validation_1-logloss:0.271103
[10]validation_0-error:0.00573validation_0-logloss:0.249593validation_1-error:0.006512validation_1-logloss:0.251354
[11]validation_0-error:0.01719validation_0-logloss:0.228658validation_1-error:0.017674validation_1-logloss:0.230144
[12]validation_0-error:0.01719validation_0-logloss:0.210442validation_1-error:0.017674validation_1-logloss:0.21167
[13]validation_0-error:0.01719validation_0-logloss:0.194562validation_1-error:0.017674validation_1-logloss:0.19555
[14]validation_0-error:0.01719validation_0-logloss:0.1807validation_1-error:0.017674validation_1-logloss:0.181463
[15]validation_0-error:0.01719validation_0-logloss:0.168585validation_1-error:0.017674validation_1-logloss:0.169138
[16]validation_0-error:0.01719validation_0-logloss:0.157988validation_1-error:0.017674validation_1-logloss:0.158345
[17]validation_0-error:0.01719validation_0-logloss:0.149407validation_1-error:0.017674validation_1-logloss:0.149731
[18]validation_0-error:0.0259validation_0-logloss:0.140835validation_1-error:0.024651validation_1-logloss:0.140979
[19]validation_0-error:0.022003validation_0-logloss:0.133937validation_1-error:0.020465validation_1-logloss:0.13405
[20]validation_0-error:0.022003validation_0-logloss:0.126967validation_1-error:0.020465validation_1-logloss:0.126914
[21]validation_0-error:0.022003validation_0-logloss:0.121386validation_1-error:0.020465validation_1-logloss:0.121303
[22]validation_0-error:0.022003validation_0-logloss:0.115692validation_1-error:0.020465validation_1-logloss:0.115456
[23]validation_0-error:0.022003validation_0-logloss:0.111147validation_1-error:0.020465validation_1-logloss:0.110881
[24]validation_0-error:0.022003validation_0-logloss:0.106477validation_1-error:0.020465validation_1-logloss:0.10607
[25]validation_0-error:0.022003validation_0-logloss:0.102434validation_1-error:0.020465validation_1-logloss:0.102319
[26]validation_0-error:0.022003validation_0-logloss:0.098434validation_1-error:0.020465validation_1-logloss:0.09819
[27]validation_0-error:0.022003validation_0-logloss:0.094875validation_1-error:0.020465validation_1-logloss:0.094824
[28]validation_0-error:0.022003validation_0-logloss:0.091579validation_1-error:0.020465validation_1-logloss:0.091784
[29]validation_0-error:0.013294validation_0-logloss:0.086202validation_1-error:0.013488validation_1-logloss:0.086807
[30]validation_0-error:0.022003validation_0-logloss:0.083247validation_1-error:0.020465validation_1-logloss:0.083741
[31]validation_0-error:0.022003validation_0-logloss:0.080496validation_1-error:0.020465validation_1-logloss:0.080924
[32]validation_0-error:0.022003validation_0-logloss:0.077298validation_1-error:0.020465validation_1-logloss:0.077394
[33]validation_0-error:0.015815validation_0-logloss:0.074507validation_1-error:0.016279validation_1-logloss:0.074765
[34]validation_0-error:0.022003validation_0-logloss:0.071848validation_1-error:0.020465validation_1-logloss:0.071811
[35]validation_0-error:0.010543validation_0-logloss:0.069488validation_1-error:0.009302validation_1-logloss:0.069385
[36]validation_0-error:0.001834validation_0-logloss:0.067147validation_1-error:0.002326validation_1-logloss:0.067341
[37]validation_0-error:0.001834validation_0-logloss:0.06504validation_1-error:0.002326validation_1-logloss:0.065406
[38]validation_0-error:0.001834validation_0-logloss:0.062898validation_1-error:0.002326validation_1-logloss:0.063381
[39]validation_0-error:0.001834validation_0-logloss:0.060837validation_1-error:0.002326validation_1-logloss:0.061088
[40]validation_0-error:0.001834validation_0-logloss:0.058894validation_1-error:0.002326validation_1-logloss:0.059039
[41]validation_0-error:0.001834validation_0-logloss:0.057112validation_1-error:0.002326validation_1-logloss:0.057326
[42]validation_0-error:0.001834validation_0-logloss:0.055391validation_1-error:0.002326validation_1-logloss:0.05543
[43]validation_0-error:0.001834validation_0-logloss:0.053745validation_1-error:0.002326validation_1-logloss:0.053871
[44]validation_0-error:0.001834validation_0-logloss:0.052198validation_1-error:0.002326validation_1-logloss:0.052235
[45]validation_0-error:0.001834validation_0-logloss:0.050776validation_1-error:0.002326validation_1-logloss:0.051033
[46]validation_0-error:0.001834validation_0-logloss:0.049351validation_1-error:0.002326validation_1-logloss:0.04973
[47]validation_0-error:0.001834validation_0-logloss:0.047848validation_1-error:0.002326validation_1-logloss:0.048287
[48]validation_0-error:0.001834validation_0-logloss:0.046406validation_1-error:0.002326validation_1-logloss:0.046702
[49]validation_0-error:0.001834validation_0-logloss:0.045141validation_1-error:0.002326validation_1-logloss:0.045492
[50]validation_0-error:0.001834validation_0-logloss:0.043917validation_1-error:0.002326validation_1-logloss:0.044133
[51]validation_0-error:0.001834validation_0-logloss:0.042729validation_1-error:0.002326validation_1-logloss:0.042999
[52]validation_0-error:0.001834validation_0-logloss:0.041608validation_1-error:0.002326validation_1-logloss:0.041807
[53]validation_0-error:0.001834validation_0-logloss:0.040493validation_1-error:0.002326validation_1-logloss:0.040855
[54]validation_0-error:0.001834validation_0-logloss:0.039457validation_1-error:0.002326validation_1-logloss:0.039871
[55]validation_0-error:0.001834validation_0-logloss:0.038452validation_1-error:0.002326validation_1-logloss:0.038755
[56]validation_0-error:0.001834validation_0-logloss:0.037478validation_1-error:0.002326validation_1-logloss:0.037717
[57]validation_0-error:0.001834validation_0-logloss:0.036439validation_1-error:0.002326validation_1-logloss:0.036777
[58]validation_0-error:0.001834validation_0-logloss:0.035552validation_1-error:0.002326validation_1-logloss:0.035936
[59]validation_0-error:0.001834validation_0-logloss:0.034694validation_1-error:0.002326validation_1-logloss:0.034984
[60]validation_0-error:0.001834validation_0-logloss:0.033826validation_1-error:0.002326validation_1-logloss:0.034132
[61]validation_0-error:0.001834validation_0-logloss:0.032959validation_1-error:0.002326validation_1-logloss:0.033348
[62]validation_0-error:0.001834validation_0-logloss:0.032192validation_1-error:0.002326validation_1-logloss:0.032526
[63]validation_0-error:0.001834validation_0-logloss:0.031476validation_1-error:0.002326validation_1-logloss:0.031754
[64]validation_0-error:0.001834validation_0-logloss:0.030756validation_1-error:0.002326validation_1-logloss:0.031081
[65]validation_0-error:0.001834validation_0-logloss:0.030038validation_1-error:0.002326validation_1-logloss:0.030377
[66]validation_0-error:0.001834validation_0-logloss:0.029332validation_1-error:0.002326validation_1-logloss:0.029594
[67]validation_0-error:0.001834validation_0-logloss:0.028703validation_1-error:0.002326validation_1-logloss:0.029079
[68]validation_0-error:0.001834validation_0-logloss:0.028064validation_1-error:0.002326validation_1-logloss:0.028391
[69]validation_0-error:0.001834validation_0-logloss:0.027404validation_1-error:0.002326validation_1-logloss:0.027725
[70]validation_0-error:0.001834validation_0-logloss:0.026824validation_1-error:0.002326validation_1-logloss:0.027187
[71]validation_0-error:0.001834validation_0-logloss:0.026268validation_1-error:0.002326validation_1-logloss:0.026565
[72]validation_0-error:0.001834validation_0-logloss:0.025679validation_1-error:0.002326validation_1-logloss:0.025982
[73]validation_0-error:0.001834validation_0-logloss:0.025153validation_1-error:0.002326validation_1-logloss:0.025413
[74]validation_0-error:0.001834validation_0-logloss:0.02461validation_1-error:0.002326validation_1-logloss:0.024927
[75]validation_0-error:0.001834validation_0-logloss:0.0241validation_1-error:0.002326validation_1-logloss:0.02446
[76]validation_0-error:0.001834validation_0-logloss:0.023615validation_1-error:0.002326validation_1-logloss:0.023921
[77]validation_0-error:0.001834validation_0-logloss:0.023118validation_1-error:0.002326validation_1-logloss:0.023423
[78]validation_0-error:0.001834validation_0-logloss:0.022671validation_1-error:0.002326validation_1-logloss:0.023015
[79]validation_0-error:0.001834validation_0-logloss:0.022244validation_1-error:0.002326validation_1-logloss:0.022538
[80]validation_0-error:0.001834validation_0-logloss:0.021793validation_1-error:0.002326validation_1-logloss:0.022087
[81]validation_0-error:0.001834validation_0-logloss:0.021396validation_1-error:0.002326validation_1-logloss:0.021654
[82]validation_0-error:0.001834validation_0-logloss:0.020948validation_1-error:0.002326validation_1-logloss:0.021198
[83]validation_0-error:0.001834validation_0-logloss:0.020559validation_1-error:0.002326validation_1-logloss:0.020806
[84]validation_0-error:0.001834validation_0-logloss:0.020144validation_1-error:0.002326validation_1-logloss:0.020388
[85]validation_0-error:0.001834validation_0-logloss:0.019775validation_1-error:0.002326validation_1-logloss:0.020057
[86]validation_0-error:0.001834validation_0-logloss:0.019029validation_1-error:0.002326validation_1-logloss:0.019235
[87]validation_0-error:0.001834validation_0-logloss:0.018672validation_1-error:0.002326validation_1-logloss:0.018823
[88]validation_0-error:0.001834validation_0-logloss:0.018313validation_1-error:0.002326validation_1-logloss:0.018507
[89]validation_0-error:0.001834validation_0-logloss:0.017989validation_1-error:0.002326validation_1-logloss:0.01815
[90]validation_0-error:0.001834validation_0-logloss:0.017376validation_1-error:0.002326validation_1-logloss:0.01748
[91]validation_0-error:0.001834validation_0-logloss:0.017087validation_1-error:0.002326validation_1-logloss:0.017189
[92]validation_0-error:0.001834validation_0-logloss:0.016778validation_1-error:0.002326validation_1-logloss:0.016877
[93]validation_0-error:0.001834validation_0-logloss:0.016458validation_1-error:0.002326validation_1-logloss:0.016527
[94]validation_0-error:0.001834validation_0-logloss:0.015932validation_1-error:0.002326validation_1-logloss:0.015956
[95]validation_0-error:0.001834validation_0-logloss:0.015645validation_1-error:0.002326validation_1-logloss:0.015665
[96]validation_0-error:0.001834validation_0-logloss:0.015379validation_1-error:0.002326validation_1-logloss:0.015397
[97]validation_0-error:0.001834validation_0-logloss:0.015116validation_1-error:0.002326validation_1-logloss:0.01513
[98]validation_0-error:0.001834validation_0-logloss:0.014883validation_1-error:0.002326validation_1-logloss:0.014893
[99]validation_0-error:0.001834validation_0-logloss:0.01464validation_1-error:0.002326validation_1-logloss:0.014624

【max_depth=2,lr=0.1】Test Accuracy: 99.81%
{'validation_0': {'error': [0.045611, 0.041256, 0.045611, 0.041256, 0.041256, 0.041256, 0.023378, 0.041256, 0.041256, 0.023378, 0.00573, 0.01719, 0.01719, 0.01719, 0.01719, 0.01719, 0.01719, 0.01719, 0.0259, 0.022003, 0.022003, 0.022003, 0.022003, 0.022003, 0.022003, 0.022003, 0.022003, 0.022003, 0.022003, 0.013294, 0.022003, 0.022003, 0.022003, 0.015815, 0.022003, 0.010543, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834], 'logloss': [0.614637, 0.549907, 0.49543, 0.449089, 0.409231, 0.373748, 0.343051, 0.315369, 0.290912, 0.269356, 0.249593, 0.228658, 0.210442, 0.194562, 0.1807, 0.168585, 0.157988, 0.149407, 0.140835, 0.133937, 0.126967, 0.121386, 0.115692, 0.111147, 0.106477, 0.102434, 0.098434, 0.094875, 0.091579, 0.086202, 0.083247, 0.080496, 0.077298, 0.074507, 0.071848, 0.069488, 0.067147, 0.06504, 0.062898, 0.060837, 0.058894, 0.057112, 0.055391, 0.053745, 0.052198, 0.050776, 0.049351, 0.047848, 0.046406, 0.045141, 0.043917, 0.042729, 0.041608, 0.040493, 0.039457, 0.038452, 0.037478, 0.036439, 0.035552, 0.034694, 0.033826, 0.032959, 0.032192, 0.031476, 0.030756, 0.030038, 0.029332, 0.028703, 0.028064, 0.027404, 0.026824, 0.026268, 0.025679, 0.025153, 0.02461, 0.0241, 0.023615, 0.023118, 0.022671, 0.022244, 0.021793, 0.021396, 0.020948, 0.020559, 0.020144, 0.019775, 0.019029, 0.018672, 0.018313, 0.017989, 0.017376, 0.017087, 0.016778, 0.016458, 0.015932, 0.015645, 0.015379, 0.015116, 0.014883, 0.01464]}, 'validation_1': {'error': [0.048372, 0.042326, 0.048372, 0.042326, 0.042326, 0.042326, 0.023256, 0.042326, 0.042326, 0.023256, 0.006512, 0.017674, 0.017674, 0.017674, 0.017674, 0.017674, 0.017674, 0.017674, 0.024651, 0.020465, 0.020465, 0.020465, 0.020465, 0.020465, 0.020465, 0.020465, 0.020465, 0.020465, 0.020465, 0.013488, 0.020465, 0.020465, 0.020465, 0.016279, 0.020465, 0.009302, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326], 'logloss': [0.615401, 0.550696, 0.496777, 0.450412, 0.410717, 0.375653, 0.344738, 0.317409, 0.292587, 0.271103, 0.251354, 0.230144, 0.21167, 0.19555, 0.181463, 0.169138, 0.158345, 0.149731, 0.140979, 0.13405, 0.126914, 0.121303, 0.115456, 0.110881, 0.10607, 0.102319, 0.09819, 0.094824, 0.091784, 0.086807, 0.083741, 0.080924, 0.077394, 0.074765, 0.071811, 0.069385, 0.067341, 0.065406, 0.063381, 0.061088, 0.059039, 0.057326, 0.05543, 0.053871, 0.052235, 0.051033, 0.04973, 0.048287, 0.046702, 0.045492, 0.044133, 0.042999, 0.041807, 0.040855, 0.039871, 0.038755, 0.037717, 0.036777, 0.035936, 0.034984, 0.034132, 0.033348, 0.032526, 0.031754, 0.031081, 0.030377, 0.029594, 0.029079, 0.028391, 0.027725, 0.027187, 0.026565, 0.025982, 0.025413, 0.024927, 0.02446, 0.023921, 0.023423, 0.023015, 0.022538, 0.022087, 0.021654, 0.021198, 0.020806, 0.020388, 0.020057, 0.019235, 0.018823, 0.018507, 0.01815, 0.01748, 0.017189, 0.016877, 0.016527, 0.015956, 0.015665, 0.015397, 0.01513, 0.014893, 0.014624]}}