{"id":1236,"date":"2023-07-27T04:52:33","date_gmt":"2023-07-27T04:52:33","guid":{"rendered":"https:\/\/statorials.org\/cn\/r-%e4%b8%ad%e7%9a%84-xgboost\/"},"modified":"2023-07-27T04:52:33","modified_gmt":"2023-07-27T04:52:33","slug":"r-%e4%b8%ad%e7%9a%84-xgboost","status":"publish","type":"post","link":"https:\/\/statorials.org\/cn\/r-%e4%b8%ad%e7%9a%84-xgboost\/","title":{"rendered":"R \u4e2d\u7684 xgboost\uff1a\u5206\u6b65\u793a\u4f8b"},"content":{"rendered":"

<\/p>\n

\n

Boosting<\/a>\u662f\u4e00\u79cd\u673a\u5668\u5b66\u4e60\u6280\u672f\uff0c\u5df2\u88ab\u8bc1\u660e\u53ef\u4ee5\u751f\u6210\u5177\u6709\u9ad8\u9884\u6d4b\u51c6\u786e\u6027\u7684\u6a21\u578b\u3002<\/span><\/p>\n

\u5728\u5b9e\u8df5\u4e2d\u5b9e\u73b0 boosting \u6700\u5e38\u89c1\u7684\u65b9\u6cd5\u4e4b\u4e00\u662f\u4f7f\u7528XGBoost<\/strong> \uff0c\u5b83\u662f\u201c\u6781\u9650\u68af\u5ea6\u63d0\u5347\u201d\u7684\u7f29\u5199\u3002<\/span><\/p>\n

\u672c\u6559\u7a0b\u63d0\u4f9b\u4e86\u5982\u4f55\u4f7f\u7528 XGBoost \u5728 R \u4e2d\u62df\u5408\u589e\u5f3a\u6a21\u578b\u7684\u5206\u6b65\u793a\u4f8b\u3002<\/span><\/p>\n

\u7b2c1\u6b65\uff1a\u52a0\u8f7d\u5fc5\u8981\u7684\u5305<\/span><\/strong><\/h3>\n

\u9996\u5148\uff0c\u6211\u4eec\u5c06\u52a0\u8f7d\u5fc5\u8981\u7684\u5e93\u3002<\/span><\/p>\n

 library<\/span> (xgboost) #for fitting the xgboost model<\/span>\nlibrary<\/span> (caret) #for general data preparation and model fitting<\/span>\n<\/strong><\/pre>\n
\u7b2c2\u6b65\uff1a\u52a0\u8f7d\u6570\u636e<\/strong><\/span><\/h3>\n
\u5bf9\u4e8e\u6b64\u793a\u4f8b\uff0c\u6211\u4eec\u5c06\u4f7f\u7528MASS<\/strong>\u5305\u4e2d\u7684\u6ce2\u58eb\u987f<\/strong>\u6570\u636e\u96c6\u62df\u5408\u6539\u8fdb\u7684\u56de\u5f52\u6a21\u578b\u3002<\/span><\/p>\n
\u8be5\u6570\u636e\u96c6\u5305\u542b 13 \u4e2a\u9884\u6d4b\u53d8\u91cf\uff0c\u6211\u4eec\u5c06\u4f7f\u7528\u5b83\u4eec\u6765\u9884\u6d4b\u540d\u4e3amdev<\/strong>\u7684\u54cd\u5e94\u53d8\u91cf<\/a>\uff0c\u8be5\u53d8\u91cf\u8868\u793a\u6ce2\u58eb\u987f\u5468\u56f4\u4e0d\u540c\u4eba\u53e3\u666e\u67e5\u533a\u7684\u623f\u5c4b\u4e2d\u503c\u3002<\/span><\/p>\n
 #load the data\n<\/span>data = MASS::Boston\n\n#view the structure of the data\n<\/span>str(data) \n\n'data.frame': 506 obs. of 14 variables:\n $ crim: num 0.00632 0.02731 0.02729 0.03237 0.06905 ...\n $ zn : num 18 0 0 0 0 0 12.5 12.5 12.5 12.5 ...\n $ indus: num 2.31 7.07 7.07 2.18 2.18 2.18 7.87 7.87 7.87 7.87 ...\n $chas: int 0 0 0 0 0 0 0 0 0 0 ...\n $ nox: num 0.538 0.469 0.469 0.458 0.458 0.458 0.524 0.524 0.524 0.524 ...\n $rm: num 6.58 6.42 7.18 7 7.15 ...\n $ age: num 65.2 78.9 61.1 45.8 54.2 58.7 66.6 96.1 100 85.9 ...\n $ dis: num 4.09 4.97 4.97 6.06 6.06 ...\n $rad: int 1 2 2 3 3 3 5 5 5 5 ...\n $ tax: num 296 242 242 222 222 222 311 311 311 311 ...\n $ptratio: num 15.3 17.8 17.8 18.7 18.7 18.7 15.2 15.2 15.2 15.2 ...\n $ black: num 397 397 393 395 397 ...\n $ lstat: num 4.98 9.14 4.03 2.94 5.33 ...\n $ medv: num 24 21.6 34.7 33.4 36.2 28.7 22.9 27.1 16.5 18.9 ...\n<\/span><\/span><\/strong><\/pre>\n
\u7b2c\u4e09\u6b65\uff1a\u51c6\u5907\u6570\u636e<\/strong><\/span><\/h3>\n
\u63a5\u4e0b\u6765\uff0c\u6211\u4eec\u5c06\u4f7f\u7528 caret \u5305\u4e2d\u7684createDataPartition()<\/strong>\u51fd\u6570\u5c06\u539f\u59cb\u6570\u636e\u96c6\u62c6\u5206\u4e3a\u8bad\u7ec3\u96c6\u548c\u6d4b\u8bd5\u96c6\u3002<\/span><\/p>\n
\u5bf9\u4e8e\u8fd9\u4e2a\u4f8b\u5b50\uff0c\u6211\u4eec\u5c06\u9009\u62e9\u4f7f\u7528 80% \u7684\u539f\u59cb\u6570\u636e\u96c6\u4f5c\u4e3a\u8bad\u7ec3\u96c6\u7684\u4e00\u90e8\u5206\u3002<\/span><\/p>\n
\u8bf7\u6ce8\u610f\uff0cxgboost \u5305\u4e5f\u4f7f\u7528\u77e9\u9635\u6570\u636e\uff0c\u56e0\u6b64\u6211\u4eec\u5c06\u4f7f\u7528data.matrix()<\/strong>\u51fd\u6570\u6765\u4fdd\u5b58\u9884\u6d4b\u53d8\u91cf\u3002<\/span><\/p>\n
 #make this example reproducible\n<\/span>set.seed(0)\n\n#split into training (80%) and testing set (20%)\n<\/span>parts = createDataPartition(data$medv, p = .8<\/span> , list = F<\/span> )\ntrain = data[parts, ]\ntest = data[-parts, ]\n\n#define predictor and response variables in training set\n<\/span>train_x = data. matrix<\/span> (train[, -13])\ntrain_y = train[,13]\n\n#define predictor and response variables in testing set\n<\/span>test_x = data. matrix<\/span> (test[, -13])\ntest_y = test[, 13]\n\n#define final training and testing sets\n<\/span>xgb_train = xgb. DMatrix<\/span> (data = train_x, label = train_y)\nxgb_test = xgb. DMatrix<\/span> (data = test_x, label = test_y)\n<\/span><\/span><\/strong><\/pre>\n
\u7b2c\u56db\u6b65\uff1a\u8c03\u6574\u6a21\u578b<\/strong><\/span><\/h3>\n
\u63a5\u4e0b\u6765\uff0c\u6211\u4eec\u5c06\u4f7f\u7528xgb.train()<\/strong>\u51fd\u6570\u8c03\u6574 XGBoost \u6a21\u578b\uff0c\u8be5\u51fd\u6570\u663e\u793a\u6bcf\u4e2a\u63d0\u5347\u5468\u671f\u7684\u8bad\u7ec3\u548c\u6d4b\u8bd5 RMSE\uff08\u5747\u65b9\u8bef\u5dee\uff09\u3002<\/span><\/p>\n
\u8bf7\u6ce8\u610f\uff0c\u6211\u4eec\u9009\u62e9\u5728\u672c\u793a\u4f8b\u4e2d\u4f7f\u7528 70 \u8f6e\uff0c\u4f46\u5bf9\u4e8e\u66f4\u5927\u7684\u6570\u636e\u96c6\uff0c\u4f7f\u7528\u6570\u767e\u751a\u81f3\u6570\u5343\u8f6e\u5e76\u4e0d\u7f55\u89c1\u3002\u8bf7\u8bb0\u4f4f\uff0c\u56de\u5408\u6570\u8d8a\u591a\uff0c\u8fd0\u884c\u65f6\u95f4\u8d8a\u957f\u3002<\/span><\/p>\n
\u53e6\u8bf7\u6ce8\u610f\uff0c max. Degree<\/strong>\u53c2\u6570\u6307\u5b9a\u5404\u4e2a\u51b3\u7b56\u6811\u7684\u5f00\u53d1\u6df1\u5ea6\u3002\u6211\u4eec\u901a\u5e38\u9009\u62e9\u76f8\u5f53\u4f4e\u7684\u6570\u5b57\uff0c\u4f8b\u5982 2 \u6216 3\uff0c\u4ee5\u4fbf\u79cd\u690d\u8f83\u5c0f\u7684\u6811\u3002\u4e8b\u5b9e\u8bc1\u660e\uff0c\u8fd9\u79cd\u65b9\u6cd5\u5f80\u5f80\u4f1a\u4ea7\u751f\u66f4\u51c6\u786e\u7684\u6a21\u578b\u3002<\/span><\/p>\n
 #define watchlist\n<\/span>watchlist = list(train=xgb_train, test=xgb_test)\n\n#fit XGBoost model and display training and testing data at each round\n<\/span>model = xgb.train(data = xgb_train, max.depth = 3<\/span> , watchlist=watchlist, nrounds = 70<\/span> )\n\n[1] train-rmse:10.167523 test-rmse:10.839775 \n[2] train-rmse:7.521903 test-rmse:8.329679 \n[3] train-rmse:5.702393 test-rmse:6.691415 \n[4] train-rmse:4.463687 test-rmse:5.631310 \n[5] train-rmse:3.666278 test-rmse:4.878750 \n[6] train-rmse:3.159799 test-rmse:4.485698 \n[7] train-rmse:2.855133 test-rmse:4.230533 \n[8] train-rmse:2.603367 test-rmse:4.099881 \n[9] train-rmse:2.445718 test-rmse:4.084360 \n[10] train-rmse:2.327318 test-rmse:3.993562 \n[11] train-rmse:2.267629 test-rmse:3.944454 \n[12] train-rmse:2.189527 test-rmse:3.930808 \n[13] train-rmse:2.119130 test-rmse:3.865036 \n[14] train-rmse:2.086450 test-rmse:3.875088 \n[15] train-rmse:2.038356 test-rmse:3.881442 \n[16] train-rmse:2.010995 test-rmse:3.883322 \n[17] train-rmse:1.949505 test-rmse:3.844382 \n[18] train-rmse:1.911711 test-rmse:3.809830 \n[19] train-rmse:1.888488 test-rmse:3.809830 \n[20] train-rmse:1.832443 test-rmse:3.758502 \n[21] train-rmse:1.816150 test-rmse:3.770216 \n[22] train-rmse:1.801369 test-rmse:3.770474 \n[23] train-rmse:1.788891 test-rmse:3.766608 \n[24] train-rmse:1.751795 test-rmse:3.749583 \n[25] train-rmse:1.713306 test-rmse:3.720173 \n[26] train-rmse:1.672227 test-rmse:3.675086 \n[27] train-rmse:1.648323 test-rmse:3.675977 \n[28] train-rmse:1.609927 test-rmse:3.745338 \n[29] train-rmse:1.594891 test-rmse:3.756049 \n[30] train-rmse:1.578573 test-rmse:3.760104 \n[31] train-rmse:1.559810 test-rmse:3.727940 \n[32] train-rmse:1.547852 test-rmse:3.731702 \n[33] train-rmse:1.534589 test-rmse:3.729761 \n[34] train-rmse:1.520566 test-rmse:3.742681 \n[35] train-rmse:1.495155 test-rmse:3.732993 \n[36] train-rmse:1.467939 test-rmse:3.738329 \n[37] train-rmse:1.446343 test-rmse:3.713748 \n[38] train-rmse:1.435368 test-rmse:3.709469 \n[39] train-rmse:1.401356 test-rmse:3.710637 \n[40] train-rmse:1.390318 test-rmse:3.709461 \n[41] train-rmse:1.372635 test-rmse:3.708049 \n[42] train-rmse:1.367977 test-rmse:3.707429 \n[43] train-rmse:1.359531 test-rmse:3.711663 \n[44] train-rmse:1.335347 test-rmse:3.709101 \n[45] train-rmse:1.331750 test-rmse:3.712490 \n[46] train-rmse:1.313087 test-rmse:3.722981 \n[47] train-rmse:1.284392 test-rmse:3.712840 \n[48] train-rmse:1.257714 test-rmse:3.697482 \n[49] train-rmse:1.248218 test-rmse:3.700167 \n[50] train-rmse:1.243377 test-rmse:3.697914 \n[51] train-rmse:1.231956 test-rmse:3.695797 \n[52] train-rmse:1.219341 test-rmse:3.696277 \n[53] train-rmse:1.207413 test-rmse:3.691465 \n[54] train-rmse:1.197197 test-rmse:3.692108 \n[55] train-rmse:1.171748 test-rmse:3.683577 \n[56] train-rmse:1.156332 test-rmse:3.674458 \n[57] train-rmse:1.147686 test-rmse:3.686367 \n[58] train-rmse:1.143572 test-rmse:3.686375 \n[59] train-rmse:1.129780 test-rmse:3.679791 \n[60] train-rmse:1.111257 test-rmse:3.679022 \n[61] train-rmse:1.093541 test-rmse:3.699670 \n[62] train-rmse:1.083934 test-rmse:3.708187 \n[63] train-rmse:1.067109 test-rmse:3.712538 \n[64] train-rmse:1.053887 test-rmse:3.722480 \n[65] train-rmse:1.042127 test-rmse:3.720720 \n[66] train-rmse:1.031617 test-rmse:3.721224 \n[67] train-rmse:1.016274 test-rmse:3.699549 \n[68] train-rmse:1.008184 test-rmse:3.709522 \n[69] train-rmse:0.999220 test-rmse:3.708000 \n[70] train-rmse:0.985907 test-rmse:3.705192 \n<\/span><\/span><\/strong><\/pre>\n
\u56e0\u6b64\uff0c\u6211\u4eec\u5c06\u6700\u7ec8\u7684 XGBoost \u6a21\u578b\u8bbe\u7f6e\u4e3a\u4f7f\u7528 56 \u8f6e\uff1a<\/span><\/p>\n
 #define final model\n<\/span>final = xgboost(data = xgb_train, max.depth = 3<\/span> , nrounds = 56<\/span> , verbose = 0<\/span> )<\/span><\/span><\/strong><\/pre>\n
\u6ce8\u610f\uff1a verbose=0<\/strong>\u53c2\u6570\u544a\u8bc9 R \u4e0d\u8981\u663e\u793a\u6bcf\u8f6e\u7684\u8bad\u7ec3\u548c\u6d4b\u8bd5\u9519\u8bef\u3002<\/span><\/p>\n
\u7b2c 5 \u6b65\uff1a\u4f7f\u7528\u6a21\u578b\u8fdb\u884c\u9884\u6d4b<\/strong><\/span><\/h3>\n
\u6700\u540e\uff0c\u6211\u4eec\u53ef\u4ee5\u4f7f\u7528\u6700\u7ec8\u7684\u6539\u8fdb\u6a21\u578b\u6765\u9884\u6d4b\u6d4b\u8bd5\u96c6\u4e2d\u6ce2\u58eb\u987f\u623f\u5c4b\u7684\u4e2d\u503c\u3002<\/span><\/p>\n
\u7136\u540e\u6211\u4eec\u5c06\u8ba1\u7b97\u6a21\u578b\u7684\u4ee5\u4e0b\u51c6\u786e\u5ea6\u6307\u6807\uff1a<\/span><\/p>\n
\u4ece\u7ed3\u679c\u4e2d\u6211\u4eec\u53ef\u4ee5\u770b\u51fa\uff0c\u572856<\/strong>\u8f6e\u65f6\u8fbe\u5230\u4e86\u6700\u5c0f\u6d4b\u8bd5 RMSE\u3002\u8d85\u8fc7\u8fd9\u4e00\u70b9\uff0c\u6d4b\u8bd5 RMSE \u5f00\u59cb\u589e\u52a0\uff0c\u8868\u660e\u6211\u4eec\u8fc7\u5ea6\u62df\u5408\u8bad\u7ec3\u6570\u636e<\/a>\u3002<\/span><\/p>\n
\u6211\u4eec\u53ef\u4ee5\u770b\u5230\u6570\u636e\u96c6\u603b\u5171\u5305\u542b 506 \u4e2a\u89c2\u6d4b\u503c<\/a>\u548c 14 \u4e2a\u53d8\u91cf\u3002<\/span><\/p>\n