归一化在优化中是否有用/必需? [英] Is normalization useful/necessary in optimization?
问题描述
我正在尝试使用Matlab优化工具箱(精确地使用 fmincon 函数)优化设备设计.为了快速阐明我的观点，我提供了一个小的变量集{l_m，r_m，l_c，r_c}，其初始值等于{4mm，2mm，1mm，0.5mm}.
I am trying to optimize a device design using Matlab optimization toolbox (using the fmincon function to be precise). To get my point across quickly I am providing a small variable set {l_m, r_m, l_c, r_c} which at it's starting value is equal to {4mm, 2mm, 1mm, 0.5mm}.
尽管Matlab不特别建议标准化输入变量，但我的教授建议我将变量标准化为{l_m，r_m，l_c，r_c}的最大值.因此，变量现在将采用从 0 到 1 的值(而不是在 l_m 的情况下说 3mm 到 4.5mm).当然，我必须修改目标函数以将其转换回适当的值，然后进行计算.
Though Matlab doesn't specifically recommend normalizing the input variables, my professor advised me to normalize the variables to the maximum value of {l_m , r_m, l_c, r_c}. Thus the variables will now take values from 0 to 1 (instead of say 3mm to 4.5mm in the case of l_m). Of course I have to modify my objective function to convert it back to the proper values and then do the calculations.
我的问题是:像 fmincon 这样的优化函数是否关心输入变量是否已规范化?归因于标准化，期望性能变化是否合理?要考虑的问题是优化器如何改变变量，例如l_m，在一种情况下，可以将其从4mm更改为4.1mm，在另一种情况下，可以将其从0.75更改为0.76.
My question is: do optimization functions like fmincon care if the input variables are normalized? Is it reasonable to expect change in performance on account of normalization? The point to be considered is how the optimizer varies variables like say l_m — in one case it can change it from 4mm to 4.1mm and in the other case it can change it from 0.75 to 0.76.
推荐答案
当对输入进行规范化时，通常更容易进行优化.您可以期望在收敛速度和输出准确性方面都有改进.
It is usually much easier to optimize when the input is normalized. You can expect an improvement in both speed of convergence and in the accuracy of the output.
For instance, As you can see on this article ( http://www-personal.umich.edu/~mepelman/teaching/IOE511/Handouts/511notes07-7.pdf ), the convergence rate of gradient descent is better bounded when the ratio of largest and smallest eigenvalues of the Hessian is small. Typically, when your data is normalized, this ratio is 1 (optimal).
这篇关于归一化在优化中是否有用/必需?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
