计算多维降维技术的 mse [英] Calculating mse for multiple dimensionality reduction technique

问题描述

我正在尝试寻找一个指标来比较类似于这篇博文中所做的多维降低技术pca-vs-autoencoders-for-Dimensionity-reduction...

I'm trying to find a metric to compare multiple dimensionality reduction techniques similar to what was done in this blog post pca-vs-autoencoders-for-dimensionality-reduction...

特别是这部分对比

# pCA reconstruction
pca.recon <- function(pca, x, k){
  mu <- matrix(rep(pca$center, nrow(pca$x)), nrow = nrow(pca$x), byrow = T)
  recon <- pca$x[,1:k] %*% t(pca$rotation[,1:k]) + mu
  mse <- mean((recon - x)^2)
  return(list(x = recon, mse = mse))
}

xhat <- rep(NA, 10)
for(k in 1:10){
  xhat[k] <- pca.recon(pca, x_train, k)$mse
}

ae.mse <- rep(NA, 5)
for(k in 1:5){
  modelk <- keras_model_sequential()
  modelk %>%
    layer_dense(units = 6, activation = "tanh", input_shape = ncol(x_train)) %>%
    layer_dense(units = k, activation = "tanh", name = "bottleneck") %>%
    layer_dense(units = 6, activation = "tanh") %>%
    layer_dense(units = ncol(x_train))

  modelk %>% compile(
    loss = "mean_squared_error", 
    optimizer = "adam"
  )

  modelk %>% fit(
    x = x_train, 
    y = x_train, 
    epochs = 5000,
    verbose = 0
  )

  ae.mse[k] <- unname(evaluate(modelk, x_train, x_train))
}

df <- data.frame(k = c(1:10, 1:5), mse = c(xhat, ae.mse), method = c(rep("pca", 10), rep("autoencoder", 5)))
ggplot(df, aes(x = k, y = mse, col = method)) + geom_line()

我想添加其他技术，例如 Rtsne 包中的 TSNE、umap 包中的 UMAP 和 ivis 包中的 IVIS(目前不在 CRAN 上，但可以像这样安装 ->

i will like to add other techniques to the mix such as TSNE from Rtsne package, UMAP from the umap package and IVIS from the ivis package (currently not on CRAN but can be installed like so ->

devtools::install_github("beringresearch/ivis/R-package")
library(ivis)
install_ivis()

所有技术的数据输入和处理都是相似的，但似乎其中一些技术已经将 mse 确定融入到它们的功能中(例如自动编码器).我想知道是否有人对我正在尝试做的事情有经验.

The data input and processing for all the techniques are similar but it seems some of them already have mse determination baked into their functions (e.g. autoencoder). I'm wondering if anyone has experience with what i'm trying to do.

推荐答案

不同的分解方法可以看作是统计机器中可互换的齿轮，这很有用给你，创造者.

Different decomposition methods can be regarded as interchangeable gears in a statistical machine, that serves to be useful to you, the creator.

要选择最佳齿轮，您评估的指标不一定与齿轮有关，而是机器在分别插入每个齿轮时的整体性能如何.

To pick the best gear, the metric you evaluate by should not necessarily concern the gears, but how well the machine performs overall with each gear inserted respectively.

忽略齿轮规格:你有几个齿轮，它们都带有来自他们工厂(包)的验证规范.这些数字/摘要/规格可能不是您想要的.可能的齿轮不会提供相同的指标，因此很难进行公平的比较.此外，这些指标将完全与齿轮有关，而与您的特定机器无关.不要像博客建议的那样将机器指标与 pca.recon() 中的齿轮混合.让齿轮成为齿轮，将度量评估延迟到机器层面.

Disregard the gear specs: You have a couple of gears which all come with their own validation specs from their factories (packages). Those numbers/summeries/specs are likely not what you want. Likely gears will not be provided with the same metrics, so it is hard to do a fair comparison. Also, those metrics will be all about the gears and not about your specific machine anyways. Do not do as the blog suggest and mix machine metrics with the gears in the pca.recon(). Let the gears be gears, and delay the metric evaluation onto the machine level.

齿轮完全适合吗?:您需要检查您的特定机器，所有候选齿轮实际上都适合里面.您的合成/重建机器的齿轮必须能够双向转动.t-sne 只是为了向前和做分解而设计的，因此无法进行有意义的评估. UMAP 也是如此.也许整个重建损失基准测试并不是您最初想要使用的实际机器.也许只是为另一台机器挑选齿轮的一个副项目......如果你的机器要制作漂亮的地块，那么很难获得好的定量基准测试.如果您的机器是一些与简单分类器混合的初始分解，那么 t-sne 齿轮将非常适合，并且一些预测精度指标可能对选择齿轮有用.

Does the gear fit at all?: You need to check for your specific machine, that all your candidate gears will actually fit inside. Your gears for your composition/reconstruction machine, must be able to turn both ways. The t-sne is only designed to turn forward and do decomposition, so it is not possible to do a meaningful evaluation. Likewise for the UMAP. 

Maybe the whole reconstruction loss benchmarking thing was not the actual machine you wanted to use in the first place. Maybe just a side project to pick gears for another machine... 
If your machine is to make beautiful plots then good quantitative benchmarking is hard to come by. If your machine is some initial decomposition mixed with a simple classifier, then the t-sne gear will fit just fine and some prediction accuracy metric is likely useful to pick gears with

.

连接各种齿轮:这些齿轮实际上并不适合开箱即用的机器，因为尺寸和形式不同.每个齿轮都需要单独适应.您可能很想将机器重新安装到齿轮上，这对一些齿轮也适用.那将是字面复制粘贴您的机器代码，插入并调整每个齿轮.一种更具可扩展性的方法是仅将齿轮连接起来，这样您就可以将它们放在机器旁边的袋子中，让机器人一次插入一个齿轮并给您写报告.这是 sklearn、caret 和 keras 等框架的主要卖点.你也可以自己编码.这是一个简单的例子:

Interfacing various gears: The gears will actually not fit out-of-the-box into your machine because the size and form is not the same. Each gear needs individual adaptation. You may be tempted to re-fit the machine to the gear and that will do for a few gears. That would be to literal copy-paste your machine code, insert and adapt each gear. A more scalable way is to just interface the gears, such you can leave them in a bag next to the machine and let a robot insert one gear at the time and write you a report. That is a main selling point of frameworks such as sklearn, caret and keras. You could also code it your self. Here is a simple example

:

rm(list=ls())
#some data
X <- iris[,c("Sepal.Length","Sepal.Width","Petal.Length","Petal.Width")]

#my_gear, prcomp wrapped in an interface
#any gear must have the gear(X, N, ...) signature
pca_decompose <- function(X, N=2, ...) {
  
  #implement gear forward (decompose)
  pca <- prcomp(
    X, rank. = N,
    scale = FALSE #must be false, beacuse reconstructor below does not support re-scaling, because I'm lazy.
  )
  
  #implement gear backward (reconstruct)
  reconstruct <- function(Xnew = pca$x) {
    
    # a pca reconstructor implementation similar to function from the blog, pca already in closure
    # I think the blog mistankenly referred to pca$x instead of x sometimes
    pca.recon <- function(x, k){
      x_recon <- x[,1:k] %*% t(pca$rotation[,1:k])
      #slightly more effecient way to reapply center
      for(i in seq_along(pca$center)) x_recon[,i] <- x_recon[,i] + pca$center[i] 
      return(x_recon)
    }
    X_rc <- pca.recon(Xnew, k=N)
    return(X_rc)
  }
  
  #wrap up the interface
  self <- list(
    X_decomposed = pca$x,  # any decomposition must be named X_dc
    reconstruct = reconstruct
  )
  
  class(self) <- c("my_pca","my_universal_gear")
  return(self)
}

#define a machine with the relevant use case
my_machine <- function(gear, data, ...) {
 dc_obj <- gear(data, ...)
 data_rc <- dc_obj$reconstruct(dc_obj$X_decomposed)
}

#define the most useful metric
my_metric <- function(X,Y) {
  # this 'multivariate' mse, is not commonly used I think.
  # but whatever floats the boat
  mean((X-Y)^2) 
}

#define how to evaluate.
#try the gear in the mahine and meassure outcome with metric
my_evaluation <- function(gear, machine, data, metric, ...) {
  data <- as.matrix(data)
  output <- machine(gear,data, ...)
  my_metric(data,output)
}

#useful syntactic sugar
set_params <- function(gear, ...) {
  params = list(...)
  function(...) do.call(gear,c(list(...),params))
}

#evaluate a gear
my_evaluation(
  gear = pca_decompose,
  machine = my_machine,
  data = X,
  #gear params
  N=2
)

#the same as
my_evaluation(
  gear = set_params(pca_decompose,N=2), #nice to preset gear params
  machine = my_machine,
  data = X
)

#define all gears to evaluate
#the gearbag could also in another usecase be a grid search of optimal hyper-parameters
my_gearbag = list(
  pca_dc_N1 = set_params(pca_decompose,N=1),
  pca_dc_N2 = set_params(pca_decompose,N=2),
  pca_dc_N3 = set_params(pca_decompose,N=3),
  pca_dc_N4 = set_params(pca_decompose,N=4)
  #put also autoencoder or what ever in the gearbag
)

my_robot <- function(evaluation, machine, gearbag, data) {
  results <- sapply(
    X = gearbag, #this X is not the data put placeholder for what to iterate
    FUN = evaluation,
    machine = machine,
    data = X
  )
  
  report = list(
    README = "metric results for gears",
    results = results
  )
}


my_report <- my_robot(my_evaluation, my_machine, my_gearbag, data)

print(my_report)

打印输出

$README
[1] "metric results for gears"

$results
   pca_dc_N1    pca_dc_N2    pca_dc_N3    pca_dc_N4 
8.560431e-02 2.534107e-02 5.919048e-03 1.692109e-31 

这篇关于计算多维降维技术的 mse的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！