如何使用tf.contrib.estimator.forward_features [英] How to use tf.contrib.estimator.forward_features

问题描述

我正在尝试使用forward_features获取cloudml的实例密钥，但是我总是会收到不确定该如何解决的错误.使用tf.Transform的预处理部分是对 https://github的修改. com/GoogleCloudPlatform/cloudml-samples/tree/master/reddit_tft ，其中实例键是一个字符串，其他所有内容都是一堆浮点数.

I'm trying to use forward_features to get instance keys for cloudml, but I always get errors that I'm not sure how to fix. The preprocessing section that uses tf.Transform is a modification of https://github.com/GoogleCloudPlatform/cloudml-samples/tree/master/reddit_tft where the instance key is a string and everything else is a bunch of floats.

def gzip_reader_fn():
      return tf.TFRecordReader(options=tf.python_io.TFRecordOptions(
          compression_type=tf.python_io.TFRecordCompressionType.GZIP))


def get_transformed_reader_input_fn(transformed_metadata,
                                    transformed_data_paths,
                                    batch_size,
                                    mode):
  """Wrap the get input features function to provide the runtime arguments."""
  return input_fn_maker.build_training_input_fn(
      metadata=transformed_metadata,
      file_pattern=(
          transformed_data_paths[0] if len(transformed_data_paths) == 1
          else transformed_data_paths),
      training_batch_size=batch_size,
      label_keys=[],
      #feature_keys=FEATURE_COLUMNS,
      #key_feature_name='example_id',
      reader=gzip_reader_fn,
      reader_num_threads=4,
      queue_capacity=batch_size * 2,
      randomize_input=(mode != tf.contrib.learn.ModeKeys.EVAL),
      num_epochs=(1 if mode == tf.contrib.learn.ModeKeys.EVAL else None))

estimator = KMeansClustering(num_clusters=8, 
      initial_clusters=KMeansClustering.KMEANS_PLUS_PLUS_INIT, 
      kmeans_plus_plus_num_retries=32,
      relative_tolerance=0.0001)

estimator = tf.contrib.estimator.forward_features(
      estimator,
      'example_id')

train_input_fn = get_transformed_reader_input_fn(
      transformed_metadata, args.train_data_paths, args.batch_size,
      tf.contrib.learn.ModeKeys.TRAIN)

estimator.train(input_fn=train_input_fn)

如果我将训练功能旁边的键列传递给我，那么我会得到错误Tensors in list passed to 'values' of 'ConcatV2' Op have types [float32, float32, string, float32, float32, float32, float32, float32, float32, f loat32, float32, float32, float32, float32, float32, float32, float32, float32, float32, float32, float32, float32, float32, float32] that don't all match..但是，如果我在训练过程中不传递实例键，那么我会得到一个值错误，说功能中不存在密钥.另外，如果我将forward_features部分中的键列名称从'example_id'更改为不是列的某些随机名称，那么我仍然会得到前者错误而不是后者.谁能帮助我理解这一点?

If I were to pass in the keys column along side the training features, then I get the error Tensors in list passed to 'values' of 'ConcatV2' Op have types [float32, float32, string, float32, float32, float32, float32, float32, float32, f loat32, float32, float32, float32, float32, float32, float32, float32, float32, float32, float32, float32, float32, float32, float32] that don't all match. However, if I were to not pass in the instance keys during training, then I get the value error saying that the key doesn't exist in the features. Also, if I were to change the key column name in the forward_features section from 'example_id' to some random name that isn't a column, then I still get the former error instead of the latter. Can anyone help me make sense of this?

推荐答案

请检查以下内容:

(1)转发功能仅适用于TF.estimator.确保您没有使用contrib.learn.estimator. (更新:您正在使用从tf.estimator继承的类)

(1) Forward features only works with TF.estimator. Ensure that you are not using contrib.learn.estimator. (update: you are using a class that inherits from tf.estimator)

(2)确保输入功能读入键列.因此，键列必须是输入数据集的一部分.

(2) Make sure your input function reads in the key-column. So, the key column has to be part of your input dataset.

(3)在tf.transform的情况下，#2表示转换元数据必须反映键的架构.您看到的错误消息似乎表明该架构将其指定为浮点型，并且实际上是一个字符串.或类似的东西.

(3) In the case of tf.transform, #2 means that the transform metadata has to reflect the schema of the key. The error message you are seeing seems to indicate that the schema specified it as a float and it's actually a string. Or something like that.

(4)确保模型未使用键列.因此，您不应使用它创建FeatureColumn.换句话说，模型将简单地将input_fn读取的键传递给预测变量.

(4) Make sure the key column is NOT used by your model. So, you should not create a FeatureColumn with it. In other words, the model will simply pass through the key that is read by the input_fn to the predictor.

(5)如果在输出中看不到密钥，请查看此替代方法是否对您有帮助:

(5) If you don't see the key in the output, see if this workaround helps you:

本质上，forward_features更改内存中的图形，但不更改导出的签名.我的解决方法可以解决此问题.

Essentially, forward_features changes the graph in memory but not the exported signature. My workaround fixes this.

这篇关于如何使用tf.contrib.estimator.forward_features的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！