Tensorflow：FailedPreconditionError：从容器：localhost读取资源变量时出错。在自定义损失函数上运行sess.run（）时 [英] Tensorflow: FailedPreconditionError: Error while reading resource variable from Container: localhost. When running sess.run() on custom loss function

问题描述

我有一个使用TensorFlow 1运行Keras的代码。该代码修改了loss函数以便进行深度强化学习：

I have a code running Keras with TensorFlow 1. The code modifies the loss function in order to do deep reinforcement learning:

import os
import gym
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
env = gym.make("CartPole-v0").env
env.reset()
n_actions = env.action_space.n
state_dim = env.observation_space.shape


from tensorflow import keras
import random
from tensorflow.keras import layers as L
import tensorflow as tf
from tensorflow.python.keras.backend import set_session

sess = tf.compat.v1.Session()
graph = tf.compat.v1.get_default_graph()
init = tf.global_variables_initializer()
sess.run(init)

network = keras.models.Sequential()
network.add(L.InputLayer(state_dim))

# let's create a network for approximate q-learning following guidelines above
network.add(L.Dense(5, activation='elu'))
network.add(L.Dense(5, activation='relu'))
network.add(L.Dense(n_actions, activation='linear'))

s = env.reset()

# Create placeholders for the <s, a, r, s'> tuple and a special indicator for game end (is_done = True)
states_ph = keras.backend.placeholder(dtype='float32', shape=(None,) + state_dim)
actions_ph = keras.backend.placeholder(dtype='int32', shape=[None])
rewards_ph = keras.backend.placeholder(dtype='float32', shape=[None])
next_states_ph = keras.backend.placeholder(dtype='float32', shape=(None,) + state_dim)
is_done_ph = keras.backend.placeholder(dtype='bool', shape=[None])

#get q-values for all actions in current states
predicted_qvalues = network(states_ph)

#select q-values for chosen actions
predicted_qvalues_for_actions = tf.reduce_sum(predicted_qvalues * tf.one_hot(actions_ph, n_actions),
                                              axis=1)

gamma = 0.99

# compute q-values for all actions in next states
predicted_next_qvalues = network(next_states_ph)

# compute V*(next_states) using predicted next q-values
next_state_values = tf.math.reduce_max(predicted_next_qvalues, axis=1)

# compute "target q-values" for loss - it's what's inside square parentheses in the above formula.
target_qvalues_for_actions = rewards_ph + tf.constant(gamma) * next_state_values

# at the last state we shall use simplified formula: Q(s,a) = r(s,a) since s' doesn't exist
target_qvalues_for_actions = tf.where(is_done_ph, rewards_ph, target_qvalues_for_actions)

#mean squared error loss to minimize
loss = (predicted_qvalues_for_actions - tf.stop_gradient(target_qvalues_for_actions)) ** 2
loss = tf.reduce_mean(loss)

# training function that resembles agent.update(state, action, reward, next_state) from tabular agent
train_step = tf.compat.v1.train.AdamOptimizer(1e-4).minimize(loss)

a = 0
next_s, r, done, _ = env.step(a)

sess.run(train_step, {
            states_ph: [s], actions_ph: [a], rewards_ph: [r],
            next_states_ph: [next_s], is_done_ph: [done]
        })

当我运行 sess.run（）培训步骤，我得到以下信息ror：

When I run a sess.run() training step, I get the following error:

tensorflow.python.framework.errors_impl.FailedPreconditionError: Error while reading resource variable beta1_power from Container: localhost. This could mean that the variable was uninitialized. Not found: Container localhost does not exist. (Could not find resource: localhost/beta1_power)

有什么问题的想法吗？

Any ideas on what might be the problem?

推荐答案

应该获取初始化操作，并在变量（即模型）后之后（仅一次）运行已创建或已定义计算图。因此，应该在执行训练步骤之前将它们放好：

The initialization operation should be fetched and run (only one time) after the variables (i.e. model) have been created or the computation graph has been defined. Therefore, they should be put right before running the training step:

# Define and create the computation graph/model
# ...

# Initialize variables in the graph/model
init = tf.global_variables_initializer()
sess.run(init)

# Start training
sess.run(train_step, ...)

这篇关于Tensorflow：FailedPreconditionError：从容器：localhost读取资源变量时出错。在自定义损失函数上运行sess.run（）时的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！