使用Keras顺序模型进行“强化学习"的问题会出现. R中的包装 [英] Problem Using Keras Sequential Model for "reinforcelearn" Package in R

问题描述

我正在尝试使用keras(2.2.50版)神经网络/顺序模型根据此插图使用reinforcelearn包(0.2.1版)在强化学习环境中创建一个简单的代理: https://cran.r-project.org/web/packages/reinforcelearn/vignettes/agents.html .这是我使用的代码:

I am trying to use a keras(version 2.2.50) neural network / sequential model to create a simple agent in a reinforcement learning setting using the reinforcelearn package (version 0.2.1) according to this vignette: https://cran.r-project.org/web/packages/reinforcelearn/vignettes/agents.html . This is the code I use:

library('reinforcelearn')
library('keras')

model = keras_model_sequential() %>% 
  layer_dense(units = 10, input_shape = 4, activation = "linear") %>%
  compile(optimizer = optimizer_sgd(lr = 0.1), loss = "mae")

agent = makeAgent(policy = "softmax", val.fun = "neural.network", algorithm = "qlearning",
                  val.fun.args = list(model= model))

但是，当我尝试运行makeAgent函数时，出现以下错误消息:

However, when I try to run the makeAgent function I get the following error message:

Error in .subset2(public_bind_env, "initialize")(...) : 
  Assertion on 'model' failed: Must inherit from class 'keras.models.Sequential', but has classes 'keras.engine.sequential.Sequential','keras.engine.training.Model','keras.engine.network.Network','keras.engine.base_layer.Layer','tensorflow.python.module.module.Module','tensorflow.python.training.tracking.tracking.AutoTrackable','tensorflow.python.training.tracking.base.Trackable','python.builtin.object'.

问题似乎是模型的类别错误，但是我该怎么做才能解决此问题?

The problem seems to be the wrong class of the model, but what could I do to solve this problem?

推荐答案

我能够通过从CRAN(

I was able to solve the problem by downloading the source code from CRAN (https://cran.r-project.org/src/contrib/reinforcelearn_0.2.1.tar.gz) and commenting out the respective line in the definition of the ValueNetwork R6 class / initialise function:

ValueNetwork = R6::R6Class("ValueNetwork",
  public = list(
    model = NULL,

    # keras model # fixme: add support for mxnet
    initialize = function(model) {
      # checkmate::assertClass(model, "keras.models.Sequential")
      self$model = model
    },
...

然后，我通过以下方式从源代码重新安装了该软件包: install.packages([file path], repos = NULL, type="source")

Then I just reinstalled the package from source via: install.packages([file path], repos = NULL, type="source")

这篇关于使用Keras顺序模型进行“强化学习"的问题会出现. R中的包装的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！