使用预测中的 accuracy() 测量 VAR 准确性 [英] Measuring VAR accuracy using accuracy() from forecast

问题描述

我正在尝试使用 R 中的 vars 包来学习向量自回归模型.这个包没有任何方法可以测量返回模型的准确性.

I'm trying to learn a vector autoregressive model using the vars package in R. This package doesn't have any way to measure the accuracy of the returned model.

具体来说，我想使用 R 中 forecast 包中的 accuracy 函数中定义的 MASE，将使用 VAR 的预测与使用 Arima 模型对每个组件时间的预测进行比较系列(我使用 4 个可能相关的时间序列).accuracy 无法识别 vars 返回的 varest 对象.如何获得每个预测组件的 MASE?我想计算样本内和样本外的准确性

Specifically, I want to use MASE as defined in the accuracy function from the forecast package in R to compare forecasting with VAR with forecasting using Arima models on each component time series (I'm using 4 possibly correlated time series). accuracy doesn't recognize the varest object returned by vars. How can I get the MASE for each forecasted component? I want to calculate both in-sample and out-of-sample accuracy

代码示例:

library(vars)
library(forecast)
data(Canada)
v<- VAR(window(Canada, end=c(1998,4)), p=2)
accuracy(v$varresult[[1]])

accuracy 的参数是一个 lm 对象，返回序列 1 的训练准确度为:

The argument of accuracy is an lm object and returns the training accuracy for series 1 as:

                       ME      RMSE       MAE           MPE      MAPE       MASE
Training set 1.536303e-15 0.3346096 0.2653946 -1.288309e-05 0.0281736 0.03914555

我想使用类似的方法来获得样本外测试的准确性(不完全是这样，因为需要指定预测期):

I want to get the out-of-sample test accuracy using something like (not exactly this, since the forecast period needs to be specified):

 accuracy(v$varresult[[1]], window(Canada[,1], start=c(1999,1)))

但 lm 对象不支持此操作并返回错误

but this is not supported for lm objects and returns the error

 Error in testaccuracy(f, x, test) : Unknown list structure

如果我按如下方式直接使用这些值，我不会得到 MASE，它需要有关训练集的信息.这也容易出现一对一错误，因为使用的是值而不是 ts 对象，accuracy 将直接匹配存储的时间:

And if I use the values directly as follows, I don't get the MASE, which needs information about the training set. This is also prone to off-by-one errors because values are used and not ts objects, for which accuracy will match the stored times directly:

 p<-predict(v, n.ahead=8)
 accuracy(p$fcst[[1]][,"fcst"],window(Canada[,1], start=c(1999,1)))

             ME      RMSE       MAE         MPE       MAPE      ACF1 Theil's U
Test set -0.1058358 0.8585455 0.7385238 -0.01114099 0.07694492 0.5655117  1.359761

理想情况下，我想将其预测为:

Ideally, I should like to forecast it as:

fr<-forecast(v$varresult[[1]], h=8)

但这行不通，因为它需要其他系列进行预测，并给出:

but this can't work because it needs the other series for the prediction, and gives:

Error in eval(expr, envir, enclos) : object 'datamat' not found

我可以尝试复制 forecast.Arima 等的功能并尝试编写 forecast.varresult 包，但有没有更简单的方法出来了吗?

I could try copying the functionality of forecast.Arima etc and try writing a forecast.varresult package, but is there a simpler way out?

推荐答案

你为什么不尝试阅读文档.这是关于第一个参数 f 的内容:

Why don't you try reading the documentation. Here is what it says about the first argument, f:

预测"类的对象，或包含预测.如果 x 是，它也适用于 Arima、ets 和 lm 对象省略 - 在这种情况下，返回样本内准确度度量.

An object of class "forecast", or a numerical vector containing forecasts. It will also work with Arima, ets and lm objects if x is omitted – in which case in-sample accuracy measures are returned.

VAR 不返回预测"类的对象，但您可以计算一个包含预测的数值向量.

VAR does not return an object of class "forecast", but you can compute a numerical vector containing forecasts.

现在阅读第二个参数，x.

Now read about the second argument, x.

一个可选的数值向量，包含与对象长度相同的实际值，或时间序列与 f 的时间重叠.

An optional numerical vector containing actual values of the same length as object, or a time series overlapping with the times of f.

好的，这很简单.只需给它 x 中的实际值和 f 中的预测值.

OK, that's pretty straightforward. Just give it the actual values in x and the forecast values in f.

但这不会为您提供 MASE，因为它在帮助页面的下方解释说MASE 计算是使用非季节性时间序列的样本内朴素预测的 MAE 缩放的，季节性的样本内季节性朴素预测非时间序列数据的时间序列和样本内平均预测."所以它不能在没有历史数据的情况下进行计算，除非你传递一个预测"类的对象，否则它不会知道它们.

But that won't give you the MASE as further down the help page it explains that the "MASE calculation is scaled using MAE of in-sample naive forecasts for non-seasonal time series, in-sample seasonal naive forecasts for seasonal time series and in-sample mean forecasts for non-time series data." So it can't do that calculation without the historical data, and unless you are passing an object of class 'forecast' it won't know about them.

然而，欺骗它给予你想要的东西并不难.下面是一些代码:

However, it is not hard to trick it into giving what you want. Here is some code that does it:

trainingdata <- window(Canada, end=c(1998,4))
testdata <- window(Canada, start=c(1999,1))
v <- VAR(trainingdata, p=2)
p <- predict(v, n.ahead=8)
res <- residuals(v)
fits <- fitted(v)
for(i in 1:4)
{
  fc <- structure(list(mean=p$fcst[[i]][,"fcst"], x=trainingdata[,i],
     fitted=c(NA,NA,fits[,i])),class="forecast")
  print(accuracy(fc,testdata[,i]))
}

这篇关于使用预测中的 accuracy() 测量 VAR 准确性的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！