在XGBoost.XGBRegressor中创建自定义目标函数 [英] Creating a Custom Objective Function in for XGBoost.XGBRegressor

问题描述

因此，我对使用python的ML/AI游戏还是比较陌生的，我目前正在研究围绕XGBoost的自定义目标函数的实现的问题.

So I am relatively new to the ML/AI game in python, and I'm currently working on a problem surrounding the implementation of a custom objective function for XGBoost.

我的微分方程知识非常生锈，因此我创建了一个带有梯度和粗麻布的自定义obj函数，该函数对作为XGBRegressor中默认目标函数运行的均方误差函数建模，以确保我正在执行所有操作这是正确的.问题是模型的结果(错误输出很接近，但在大多数情况下是不相同的(在某些方面有些偏离).我不知道我做错了什么，或者如果我做错了怎么办正确地计算事物.如果大家都可以看一下，也许可以洞悉我的错误之处，那就太好了！

My differential equation knowledge is pretty rusty so I've created a custom obj function with a gradient and hessian that models the mean squared error function that is ran as the default objective function in XGBRegressor to make sure that I am doing all of this correctly. The problem is, the results of the model (the error outputs are close but not identical for the most part (and way off for some points). I don't know what I'm doing wrong or how that could be possible if I am computing things correctly. If you all could look at this an maybe provide insight into where I am wrong, that would be awesome!

没有自定义功能的原始代码是:

    import xgboost as xgb

    reg = xgb.XGBRegressor(n_estimators=150, 
                   max_depth=2,
                   objective ="reg:squarederror", 
                   n_jobs=-1)

    reg.fit(X_train, y_train)

    y_pred_test = reg.predict(X_test)

，我为MSE定制的目标函数如下:

    def gradient_se(y_true, y_pred):
        #Compute the gradient squared error.
        return (-2 * y_true) + (2 * y_pred)

    def hessian_se(y_true, y_pred):
        #Compute the hessian for squared error
        return 0*(y_true + y_pred) + 2

   def custom_se(y_true, y_pred):
        #squared error objective. A simplified version of MSE used as
        #objective function.

        grad = gradient_se(y_true, y_pred)
        hess = hessian_se(y_true, y_pred)
        return grad, hess

文档参考位于此处

谢谢！

推荐答案

根据文档，则库将按顺序传递预测值(在您的情况下为y_pred)和基本真理值(在您的情况下为y_true).

According to the documentation, the library passes the predicted values (y_pred in your case) and the ground truth values (y_true in your case) in this order.

您可以在custom_se(y_true, y_pred)函数中以相反的顺序将y_true和y_pred值传递给gradient_se和hessian_se函数.对于hessian来说，这没有什么区别，因为hessian应该为所有x值返回2，而您已经正确地做到了.

You pass the y_true and y_pred values in reversed order in your custom_se(y_true, y_pred) function to both the gradient_se and hessian_se functions. For the hessian it doesn't make a difference since the hessian should return 2 for all x values and you've done that correctly.

对于gradient_se功能，您对y_true和y_pred的符号不正确.

For the gradient_se function you've incorrect signs for y_true and y_pred.

正确的实现如下:

    def gradient_se(y_pred, y_true):
        #Compute the gradient squared error.
        return 2*(y_pred - y_true)

    def hessian_se(y_pred, y_true):
        #Compute the hessian for squared error
        return 0*y_true + 2

   def custom_se(y_pred, y_true):
        #squared error objective. A simplified version of MSE used as
        #objective function.

        grad = gradient_se(y_pred, y_true)
        hess = hessian_se(y_pred, y_true)
        return grad, hess

这篇关于在XGBoost.XGBRegressor中创建自定义目标函数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！