{"id":1233,"date":"2023-07-27T04:52:33","date_gmt":"2023-07-27T04:52:33","guid":{"rendered":"https:\/\/statorials.org\/ja\/r%e3%81%aexgboost\/"},"modified":"2023-07-27T04:52:33","modified_gmt":"2023-07-27T04:52:33","slug":"r%e3%81%aexgboost","status":"publish","type":"post","link":"https:\/\/statorials.org\/ja\/r%e3%81%aexgboost\/","title":{"rendered":"R \u306e xgboost: \u6bb5\u968e\u7684\u306a\u4f8b"},"content":{"rendered":"

<\/p>\n

\n

\u30d6\u30fc\u30b9\u30c6\u30a3\u30f3\u30b0\u306f<\/a>\u3001\u9ad8\u3044\u4e88\u6e2c\u7cbe\u5ea6\u306e\u30e2\u30c7\u30eb\u3092\u751f\u6210\u3059\u308b\u3053\u3068\u304c\u8a3c\u660e\u3055\u308c\u3066\u3044\u308b\u6a5f\u68b0\u5b66\u7fd2\u624b\u6cd5\u3067\u3059\u3002<\/span><\/p>\n

\u5b9f\u969b\u306b\u30d6\u30fc\u30b9\u30c6\u30a3\u30f3\u30b0\u3092\u5b9f\u88c5\u3059\u308b\u6700\u3082\u4e00\u822c\u7684\u306a\u65b9\u6cd5\u306e 1 \u3064\u306f\u3001\u300cextreme gradient boosting\u300d\u306e\u7565\u79f0\u3067\u3042\u308bXGBoost<\/strong>\u3092\u4f7f\u7528\u3059\u308b\u3053\u3068\u3067\u3059\u3002<\/span><\/p>\n

\u3053\u306e\u30c1\u30e5\u30fc\u30c8\u30ea\u30a2\u30eb\u3067\u306f\u3001XGBoost \u3092\u4f7f\u7528\u3057\u3066 R \u306e\u62e1\u5f35\u30e2\u30c7\u30eb\u3092\u9069\u5408\u3055\u305b\u308b\u65b9\u6cd5\u306e\u6bb5\u968e\u7684\u306a\u4f8b\u3092\u793a\u3057\u307e\u3059\u3002<\/span><\/p>\n

\u30b9\u30c6\u30c3\u30d7 1: \u5fc5\u8981\u306a\u30d1\u30c3\u30b1\u30fc\u30b8\u3092\u30ed\u30fc\u30c9\u3059\u308b<\/span><\/strong><\/h3>\n

\u307e\u305a\u3001\u5fc5\u8981\u306a\u30e9\u30a4\u30d6\u30e9\u30ea\u3092\u30ed\u30fc\u30c9\u3057\u307e\u3059\u3002<\/span><\/p>\n

 library<\/span> (xgboost) #for fitting the xgboost model<\/span>\nlibrary<\/span> (caret) #for general data preparation and model fitting<\/span>\n<\/strong><\/pre>\n
\u30b9\u30c6\u30c3\u30d7 2: \u30c7\u30fc\u30bf\u3092\u30ed\u30fc\u30c9\u3059\u308b<\/strong><\/span><\/h3>\n
\u3053\u306e\u4f8b\u3067\u306f\u3001\u6539\u826f\u3055\u308c\u305f\u56de\u5e30\u30e2\u30c7\u30eb\u3092MASS<\/strong>\u30d1\u30c3\u30b1\u30fc\u30b8\u306e\u30dc\u30b9\u30c8\u30f3<\/strong>\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u306b\u9069\u5408\u3055\u305b\u307e\u3059\u3002<\/span><\/p>\n
\u3053\u306e\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u306b\u306f\u3001\u30dc\u30b9\u30c8\u30f3\u5468\u8fba\u306e\u3055\u307e\u3056\u307e\u306a\u56fd\u52e2\u8abf\u67fb\u5730\u57df\u306e\u4f4f\u5b85\u306e\u4e2d\u592e\u5024\u3092\u8868\u3059mdev<\/strong>\u3068\u547c\u3070\u308c\u308b\u5fdc\u7b54\u5909\u6570<\/a>\u3092\u4e88\u6e2c\u3059\u308b\u305f\u3081\u306b\u4f7f\u7528\u3059\u308b 13 \u306e\u4e88\u6e2c\u5909\u6570\u304c\u542b\u307e\u308c\u3066\u3044\u307e\u3059\u3002<\/span><\/p>\n
 #load the data\n<\/span>data = MASS::Boston\n\n#view the structure of the data\n<\/span>str(data) \n\n'data.frame': 506 obs. of 14 variables:\n $ crim: num 0.00632 0.02731 0.02729 0.03237 0.06905 ...\n $ zn : num 18 0 0 0 0 0 12.5 12.5 12.5 12.5 ...\n $ indus: num 2.31 7.07 7.07 2.18 2.18 2.18 7.87 7.87 7.87 7.87 ...\n $chas: int 0 0 0 0 0 0 0 0 0 0 ...\n $ nox: num 0.538 0.469 0.469 0.458 0.458 0.458 0.524 0.524 0.524 0.524 ...\n $rm: num 6.58 6.42 7.18 7 7.15 ...\n $ age: num 65.2 78.9 61.1 45.8 54.2 58.7 66.6 96.1 100 85.9 ...\n $ dis: num 4.09 4.97 4.97 6.06 6.06 ...\n $rad: int 1 2 2 3 3 3 5 5 5 5 ...\n $ tax: num 296 242 242 222 222 222 311 311 311 311 ...\n $ptratio: num 15.3 17.8 17.8 18.7 18.7 18.7 15.2 15.2 15.2 15.2 ...\n $ black: num 397 397 393 395 397 ...\n $ lstat: num 4.98 9.14 4.03 2.94 5.33 ...\n $ medv: num 24 21.6 34.7 33.4 36.2 28.7 22.9 27.1 16.5 18.9 ...\n<\/span><\/span><\/strong><\/pre>\n
\u30b9\u30c6\u30c3\u30d7 3: \u30c7\u30fc\u30bf\u3092\u6e96\u5099\u3059\u308b<\/strong><\/span><\/h3>\n
\u6b21\u306b\u3001\u30ad\u30e3\u30ec\u30c3\u30c8 \u30d1\u30c3\u30b1\u30fc\u30b8\u306ecreateDataPartition()<\/strong>\u95a2\u6570\u3092\u4f7f\u7528\u3057\u3066\u3001\u5143\u306e\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u3092\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0 \u30bb\u30c3\u30c8\u3068\u30c6\u30b9\u30c8 \u30bb\u30c3\u30c8\u306b\u5206\u5272\u3057\u307e\u3059\u3002<\/span><\/p>\n
\u3053\u306e\u4f8b\u3067\u306f\u3001\u5143\u306e\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u306e 80% \u3092\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0 \u30bb\u30c3\u30c8\u306e\u4e00\u90e8\u3068\u3057\u3066\u4f7f\u7528\u3059\u308b\u3053\u3068\u3092\u9078\u629e\u3057\u307e\u3059\u3002<\/span><\/p>\n
 xgboost \u30d1\u30c3\u30b1\u30fc\u30b8\u306f\u884c\u5217\u30c7\u30fc\u30bf\u3082\u4f7f\u7528\u3059\u308b\u306e\u3067\u3001 data.matrix()<\/strong>\u95a2\u6570\u3092\u4f7f\u7528\u3057\u3066\u4e88\u6e2c\u5b50\u5909\u6570\u3092\u4fdd\u6301\u3059\u308b\u3053\u3068\u306b\u6ce8\u610f\u3057\u3066\u304f\u3060\u3055\u3044\u3002<\/span><\/p>\n
 #make this example reproducible\n<\/span>set.seed(0)\n\n#split into training (80%) and testing set (20%)\n<\/span>parts = createDataPartition(data$medv, p = .8<\/span> , list = F<\/span> )\ntrain = data[parts, ]\ntest = data[-parts, ]\n\n#define predictor and response variables in training set\n<\/span>train_x = data. matrix<\/span> (train[, -13])\ntrain_y = train[,13]\n\n#define predictor and response variables in testing set\n<\/span>test_x = data. matrix<\/span> (test[, -13])\ntest_y = test[, 13]\n\n#define final training and testing sets\n<\/span>xgb_train = xgb. DMatrix<\/span> (data = train_x, label = train_y)\nxgb_test = xgb. DMatrix<\/span> (data = test_x, label = test_y)\n<\/span><\/span><\/strong><\/pre>\n
\u30b9\u30c6\u30c3\u30d7 4: \u30e2\u30c7\u30eb\u3092\u8abf\u6574\u3059\u308b<\/strong><\/span><\/h3>\n
\u6b21\u306b\u3001 xgb.train()<\/strong>\u95a2\u6570\u3092\u4f7f\u7528\u3057\u3066 XGBoost \u30e2\u30c7\u30eb\u3092\u8abf\u6574\u3057\u307e\u3059\u3002\u3053\u306e\u95a2\u6570\u306f\u3001\u5404\u30d6\u30fc\u30b9\u30c6\u30a3\u30f3\u30b0 \u30b5\u30a4\u30af\u30eb\u306e\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u3068\u30c6\u30b9\u30c8\u306e RMSE (\u5e73\u5747\u4e8c\u4e57\u8aa4\u5dee) \u3092\u8868\u793a\u3057\u307e\u3059\u3002<\/span><\/p>\n
\u3053\u306e\u4f8b\u3067\u306f 70 \u30e9\u30a6\u30f3\u30c9\u3092\u4f7f\u7528\u3059\u308b\u3053\u3068\u3092\u9078\u629e\u3057\u307e\u3057\u305f\u304c\u3001\u306f\u308b\u304b\u306b\u5927\u304d\u306a\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u306e\u5834\u5408\u306f\u3001\u6570\u767e\u3001\u3055\u3089\u306b\u306f\u6570\u5343\u306e\u30e9\u30a6\u30f3\u30c9\u3092\u4f7f\u7528\u3059\u308b\u3053\u3068\u3082\u73cd\u3057\u304f\u306a\u3044\u3053\u3068\u306b\u6ce8\u610f\u3057\u3066\u304f\u3060\u3055\u3044\u3002\u30e9\u30a6\u30f3\u30c9\u6570\u304c\u591a\u3044\u307b\u3069\u3001\u5b9f\u884c\u6642\u9593\u304c\u9577\u304f\u306a\u308b\u3053\u3068\u306b\u6ce8\u610f\u3057\u3066\u304f\u3060\u3055\u3044\u3002<\/span><\/p>\n
\u307e\u305f\u3001 max.degree<\/strong>\u5f15\u6570\u306f\u3001\u500b\u3005\u306e\u30c7\u30b7\u30b8\u30e7\u30f3 \u30c4\u30ea\u30fc\u306e\u958b\u767a\u306e\u6df1\u3055\u3092\u6307\u5b9a\u3059\u308b\u3053\u3068\u306b\u3082\u6ce8\u610f\u3057\u3066\u304f\u3060\u3055\u3044\u3002\u901a\u5e38\u3001\u3088\u308a\u5c0f\u3055\u306a\u6728\u3092\u80b2\u3066\u308b\u305f\u3081\u306b\u3001\u3053\u306e\u6570\u5024\u306f 2 \u307e\u305f\u306f 3 \u306a\u3069\u3001\u304b\u306a\u308a\u4f4e\u3044\u6570\u5024\u3092\u9078\u629e\u3057\u307e\u3059\u3002\u3053\u306e\u30a2\u30d7\u30ed\u30fc\u30c1\u306f\u3001\u3088\u308a\u6b63\u78ba\u306a\u30e2\u30c7\u30eb\u3092\u751f\u6210\u3059\u308b\u50be\u5411\u304c\u3042\u308b\u3053\u3068\u304c\u793a\u3055\u308c\u3066\u3044\u307e\u3059\u3002<\/span><\/p>\n
 #define watchlist\n<\/span>watchlist = list(train=xgb_train, test=xgb_test)\n\n#fit XGBoost model and display training and testing data at each round\n<\/span>model = xgb.train(data = xgb_train, max.depth = 3<\/span> , watchlist=watchlist, nrounds = 70<\/span> )\n\n[1] train-rmse:10.167523 test-rmse:10.839775 \n[2] train-rmse:7.521903 test-rmse:8.329679 \n[3] train-rmse:5.702393 test-rmse:6.691415 \n[4] train-rmse:4.463687 test-rmse:5.631310 \n[5] train-rmse:3.666278 test-rmse:4.878750 \n[6] train-rmse:3.159799 test-rmse:4.485698 \n[7] train-rmse:2.855133 test-rmse:4.230533 \n[8] train-rmse:2.603367 test-rmse:4.099881 \n[9] train-rmse:2.445718 test-rmse:4.084360 \n[10] train-rmse:2.327318 test-rmse:3.993562 \n[11] train-rmse:2.267629 test-rmse:3.944454 \n[12] train-rmse:2.189527 test-rmse:3.930808 \n[13] train-rmse:2.119130 test-rmse:3.865036 \n[14] train-rmse:2.086450 test-rmse:3.875088 \n[15] train-rmse:2.038356 test-rmse:3.881442 \n[16] train-rmse:2.010995 test-rmse:3.883322 \n[17] train-rmse:1.949505 test-rmse:3.844382 \n[18] train-rmse:1.911711 test-rmse:3.809830 \n[19] train-rmse:1.888488 test-rmse:3.809830 \n[20] train-rmse:1.832443 test-rmse:3.758502 \n[21] train-rmse:1.816150 test-rmse:3.770216 \n[22] train-rmse:1.801369 test-rmse:3.770474 \n[23] train-rmse:1.788891 test-rmse:3.766608 \n[24] train-rmse:1.751795 test-rmse:3.749583 \n[25] train-rmse:1.713306 test-rmse:3.720173 \n[26] train-rmse:1.672227 test-rmse:3.675086 \n[27] train-rmse:1.648323 test-rmse:3.675977 \n[28] train-rmse:1.609927 test-rmse:3.745338 \n[29] train-rmse:1.594891 test-rmse:3.756049 \n[30] train-rmse:1.578573 test-rmse:3.760104 \n[31] train-rmse:1.559810 test-rmse:3.727940 \n[32] train-rmse:1.547852 test-rmse:3.731702 \n[33] train-rmse:1.534589 test-rmse:3.729761 \n[34] train-rmse:1.520566 test-rmse:3.742681 \n[35] train-rmse:1.495155 test-rmse:3.732993 \n[36] train-rmse:1.467939 test-rmse:3.738329 \n[37] train-rmse:1.446343 test-rmse:3.713748 \n[38] train-rmse:1.435368 test-rmse:3.709469 \n[39] train-rmse:1.401356 test-rmse:3.710637 \n[40] train-rmse:1.390318 test-rmse:3.709461 \n[41] train-rmse:1.372635 test-rmse:3.708049 \n[42] train-rmse:1.367977 test-rmse:3.707429 \n[43] train-rmse:1.359531 test-rmse:3.711663 \n[44] train-rmse:1.335347 test-rmse:3.709101 \n[45] train-rmse:1.331750 test-rmse:3.712490 \n[46] train-rmse:1.313087 test-rmse:3.722981 \n[47] train-rmse:1.284392 test-rmse:3.712840 \n[48] train-rmse:1.257714 test-rmse:3.697482 \n[49] train-rmse:1.248218 test-rmse:3.700167 \n[50] train-rmse:1.243377 test-rmse:3.697914 \n[51] train-rmse:1.231956 test-rmse:3.695797 \n[52] train-rmse:1.219341 test-rmse:3.696277 \n[53] train-rmse:1.207413 test-rmse:3.691465 \n[54] train-rmse:1.197197 test-rmse:3.692108 \n[55] train-rmse:1.171748 test-rmse:3.683577 \n[56] train-rmse:1.156332 test-rmse:3.674458 \n[57] train-rmse:1.147686 test-rmse:3.686367 \n[58] train-rmse:1.143572 test-rmse:3.686375 \n[59] train-rmse:1.129780 test-rmse:3.679791 \n[60] train-rmse:1.111257 test-rmse:3.679022 \n[61] train-rmse:1.093541 test-rmse:3.699670 \n[62] train-rmse:1.083934 test-rmse:3.708187 \n[63] train-rmse:1.067109 test-rmse:3.712538 \n[64] train-rmse:1.053887 test-rmse:3.722480 \n[65] train-rmse:1.042127 test-rmse:3.720720 \n[66] train-rmse:1.031617 test-rmse:3.721224 \n[67] train-rmse:1.016274 test-rmse:3.699549 \n[68] train-rmse:1.008184 test-rmse:3.709522 \n[69] train-rmse:0.999220 test-rmse:3.708000 \n[70] train-rmse:0.985907 test-rmse:3.705192 \n<\/span><\/span><\/strong><\/pre>\n
\u3057\u305f\u304c\u3063\u3066\u3001\u6700\u7d42\u7684\u306a XGBoost \u30e2\u30c7\u30eb\u306f 56 \u30e9\u30a6\u30f3\u30c9\u3092\u4f7f\u7528\u3059\u308b\u3088\u3046\u306b\u8a2d\u5b9a\u3057\u307e\u3059\u3002<\/span><\/p>\n
 #define final model\n<\/span>final = xgboost(data = xgb_train, max.depth = 3<\/span> , nrounds = 56<\/span> , verbose = 0<\/span> )<\/span><\/span><\/strong><\/pre>\n
\u6ce8: verbose=0<\/strong>\u5f15\u6570\u306f\u3001\u5404\u30e9\u30a6\u30f3\u30c9\u306e\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u304a\u3088\u3073\u30c6\u30b9\u30c8\u306e\u30a8\u30e9\u30fc\u3092\u8868\u793a\u3057\u306a\u3044\u3088\u3046\u306b R \u306b\u6307\u793a\u3057\u307e\u3059\u3002<\/span><\/p>\n
\u30b9\u30c6\u30c3\u30d7 5: \u30e2\u30c7\u30eb\u3092\u4f7f\u7528\u3057\u3066\u4e88\u6e2c\u3092\u884c\u3046<\/strong><\/span><\/h3>\n
\u6700\u5f8c\u306b\u3001\u6700\u7d42\u7684\u306b\u6539\u826f\u3055\u308c\u305f\u30e2\u30c7\u30eb\u3092\u4f7f\u7528\u3057\u3066\u3001\u30c6\u30b9\u30c8 \u30bb\u30c3\u30c8\u5185\u306e\u30dc\u30b9\u30c8\u30f3\u306e\u4f4f\u5b85\u306e\u4e2d\u592e\u5024\u306b\u3064\u3044\u3066\u306e\u4e88\u6e2c\u3092\u884c\u3046\u3053\u3068\u304c\u3067\u304d\u307e\u3059\u3002<\/span><\/p>\n
\u6b21\u306b\u3001\u30e2\u30c7\u30eb\u306e\u6b21\u306e\u7cbe\u5ea6\u30e1\u30c8\u30ea\u30af\u30b9\u3092\u8a08\u7b97\u3057\u307e\u3059\u3002<\/span><\/p>\n
\u7d50\u679c\u304b\u3089\u3001\u6700\u5c0f\u30c6\u30b9\u30c8 RMSE \u306f56<\/strong>\u30e9\u30a6\u30f3\u30c9\u3067\u9054\u6210\u3055\u308c\u308b\u3053\u3068\u304c\u308f\u304b\u308a\u307e\u3059\u3002\u3053\u306e\u70b9\u3092\u8d85\u3048\u308b\u3068\u3001\u30c6\u30b9\u30c8 RMSE \u304c\u5897\u52a0\u3057\u59cb\u3081\u3001 \u30c8\u30ec\u30fc\u30cb\u30f3\u30b0 \u30c7\u30fc\u30bf\u304c\u904e\u5270\u9069\u5408\u3057\u3066\u3044\u308b<\/a>\u3053\u3068\u3092\u793a\u3057\u307e\u3059\u3002<\/span><\/p>\n
\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u306b\u306f\u5408\u8a08 506 \u500b\u306e\u89b3\u6e2c\u5024<\/a>\u3068 14 \u500b\u306e\u5909\u6570\u304c\u542b\u307e\u308c\u3066\u3044\u308b\u3053\u3068\u304c\u308f\u304b\u308a\u307e\u3059\u3002<\/span><\/p>\n