如何用张量流计算AUC? [英] How to calculate AUC with tensorflow?

问题描述

我已经使用 Tensorflow 构建了一个二元分类器，现在我想使用 AUC 和准确性来评估分类器.

就准确性而言，我可以很容易地这样做:

X = tf.placeholder('float', [None, n_input])y = tf.placeholder('float', [None, n_classes])pred = mlp(X，权重，偏差，dropout_keep_prob)正确预测 = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))精度 = tf.reduce_mean(tf.cast(correct_prediction, "float"))

在计算 AUC 时，我使用以下内容:

print(tf.argmax(pred, 1).dtype.name)打印(tf.argmax(pred, 1).dtype.name)a = tf.cast(tf.argmax(pred, 1),tf.float32)b = tf.cast(tf.argmax(y,1),tf.float32)auc = tf.contrib.metrics.streaming_auc(a, b)

在训练循环中:

train_acc = sess.run(accuracy, feed_dict={X: batch_xs, y: batch_ys, dropout_keep_prob:1.})train_auc = sess.run(auc, feed_dict={X: batch_xs, y: batch_ys, dropout_keep_prob:1.})

这给了我以下输出(和错误)错误:

int64int64/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/array_ops.py:1197: VisibleDeprecationWarning: 使用 ndim > 转换数组0 到一个索引将导致将来出错result_shape.insert(dim, 1)网络搭建成功...开始训练...时代:000/300 成本:0.618990561回溯(最近一次调用最后一次):文件/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py"，第 715 行，在 _do_call返回 fn(*args)文件/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py"，第 697 行，在 _run_fn状态，运行元数据)文件/usr/lib/python3.5/contextlib.py"，第 66 行，在 __exit__ 中下一个(self.gen)文件/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/errors.py"，第450行，raise_exception_on_not_ok_statuspywrap_tensorflow.TF_GetCode(status))tensorflow.python.framework.errors.FailedPreconditionError:试图使用未初始化的值 auc/false_positives[[节点:auc/false_positives/read = Identity[T=DT_FLOAT, _class=["loc:@auc/false_positives"], _device="/job:localhost/replica:0/task:0/cpu:0"](auc/false_positives)]]在处理上述异常的过程中，又发生了一个异常:回溯(最近一次调用最后一次):文件./mlp_.py"，第 152 行，位于 <module>train_auc = sess.run(auc, feed_dict={X: batch_xs, y: batch_ys, dropout_keep_prob:1.})运行中的文件/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py"，第 372 行run_metadata_ptr)文件/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py"，第 636 行，在 _runfeed_dict_string、选项、run_metadata)文件/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py"，第 708 行，在 _do_run目标列表、选项、运行元数据)文件/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py"，第 728 行，在 _do_call引发类型(e)(节点定义，操作，消息)tensorflow.python.framework.errors.FailedPreconditionError:试图使用未初始化的值 auc/false_positives[[节点:auc/false_positives/read = Identity[T=DT_FLOAT, _class=["loc:@auc/false_positives"], _device="/job:localhost/replica:0/task:0/cpu:0"](auc/false_positives)]]由 op 'auc/false_positives/read' 引起，定义于:文件./mlp_.py"，第 121 行，位于 <module>auc = tf.contrib.metrics.streaming_auc(a, b)文件/usr/local/lib/python3.5/dist-packages/tensorflow/contrib/metrics/python/ops/metric_ops.py"，第718行，streaming_auc预测、标签、阈值、ignore_mask)文件/usr/local/lib/python3.5/dist-packages/tensorflow/contrib/metrics/python/ops/metric_ops.py"，第603行，_tp_fn_tn_fpfalse_positives = _create_local('false_positives', shape=[num_thresholds])文件/usr/local/lib/python3.5/dist-packages/tensorflow/contrib/metrics/python/ops/metric_ops.py"，第 75 行，在 _create_local集合=集合)文件/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variables.py"，第211行，在__init__数据类型=数据类型)文件/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variables.py"，第 319 行，在 _init_from_argsself._snapshot = array_ops.identity(self._variable, name="read")文件/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/gen_array_ops.py"，第831行，身份result = _op_def_lib.apply_op("身份", input=input, name=name)文件/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/op_def_library.py"，第704行，在apply_opop_def=op_def)文件/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py"，第 2260 行，在 create_op 中original_op=self._default_original_op, op_def=op_def)文件/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py"，第 1230 行，在 __init__ 中self._traceback = _extract_stack()

我不明白我做错了什么，为什么在使用准确性时只有代码运行良好，但在使用 AUC 时会引发此错误.您能否提示我正确的方向以了解如何解决此问题?

我的目标是计算 AUC 和 ROC，以便更好地评估二元分类器的性能.

解决方案

我在 github 上发现了同样的问题

一>.目前，您似乎还需要运行 sess.run(tf.initialize_local_variables()) 才能使 tf.contrib.metrics.streaming_auc() 工作.他们正在努力.

这里有一个示例演示如何解决此问题:

 将 tensorflow 导入为 tfa = tf.Variable([0.1, 0.5])b = tf.Variable([0.2, 0.6])auc = tf.contrib.metrics.streaming_auc(a, b)sess = tf.Session()sess.run(tf.initialize_all_variables())sess.run(tf.initialize_local_variables()) # 尝试注释这一行，你会得到错误train_auc = sess.run(auc)打印(train_auc)

I've built a binary classifier using Tensorflow and now I would like to evaluate the classifier using AUC and accuracy.

As far as accuracy is concerned, I can easily do like this:

X = tf.placeholder('float', [None, n_input])
y = tf.placeholder('float', [None, n_classes])
pred = mlp(X, weights, biases, dropout_keep_prob)
correct_prediction = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))

When calculating AUC I use the following:

print(tf.argmax(pred, 1).dtype.name)
print(tf.argmax(pred, 1).dtype.name)

a = tf.cast(tf.argmax(pred, 1),tf.float32)
b = tf.cast(tf.argmax(y,1),tf.float32)

auc = tf.contrib.metrics.streaming_auc(a, b)

and in the training loop:

train_acc = sess.run(accuracy, feed_dict={X: batch_xs, y: batch_ys, dropout_keep_prob:1.})
train_auc = sess.run(auc, feed_dict={X: batch_xs, y: batch_ys, dropout_keep_prob:1.})

which gives me the following output (and error) error:

int64
int64
/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/array_ops.py:1197: VisibleDeprecationWarning: converting an array with ndim > 0 to an index will result in an error in the future
  result_shape.insert(dim, 1)
Net built successfully...

Starting training...

Epoch: 000/300 cost: 0.618990561
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 715, in _do_call
    return fn(*args)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 697, in _run_fn
    status, run_metadata)
  File "/usr/lib/python3.5/contextlib.py", line 66, in __exit__
    next(self.gen)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/errors.py", line 450, in raise_exception_on_not_ok_status
    pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors.FailedPreconditionError: Attempting to use uninitialized value auc/false_positives
     [[Node: auc/false_positives/read = Identity[T=DT_FLOAT, _class=["loc:@auc/false_positives"], _device="/job:localhost/replica:0/task:0/cpu:0"](auc/false_positives)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./mlp_.py", line 152, in <module>
    train_auc = sess.run(auc, feed_dict={X: batch_xs, y: batch_ys, dropout_keep_prob:1.})
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 372, in run
    run_metadata_ptr)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 636, in _run
    feed_dict_string, options, run_metadata)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 708, in _do_run
    target_list, options, run_metadata)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 728, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors.FailedPreconditionError: Attempting to use uninitialized value auc/false_positives
     [[Node: auc/false_positives/read = Identity[T=DT_FLOAT, _class=["loc:@auc/false_positives"], _device="/job:localhost/replica:0/task:0/cpu:0"](auc/false_positives)]]
Caused by op 'auc/false_positives/read', defined at:
  File "./mlp_.py", line 121, in <module>
    auc = tf.contrib.metrics.streaming_auc(a, b)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/contrib/metrics/python/ops/metric_ops.py", line 718, in streaming_auc
    predictions, labels, thresholds, ignore_mask)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/contrib/metrics/python/ops/metric_ops.py", line 603, in _tp_fn_tn_fp
    false_positives = _create_local('false_positives', shape=[num_thresholds])
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/contrib/metrics/python/ops/metric_ops.py", line 75, in _create_local
    collections=collections)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variables.py", line 211, in __init__
    dtype=dtype)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variables.py", line 319, in _init_from_args
    self._snapshot = array_ops.identity(self._variable, name="read")
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/gen_array_ops.py", line 831, in identity
    result = _op_def_lib.apply_op("Identity", input=input, name=name)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/op_def_library.py", line 704, in apply_op
    op_def=op_def)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 2260, in create_op
    original_op=self._default_original_op, op_def=op_def)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 1230, in __init__
    self._traceback = _extract_stack()

I don't understand what I am doing wrong and why when using accuracy only the code runs fine but when using AUC it throws this error. Could you please hint me in the right direction to understand how to fix this?

My objective is to calculate AUC and ROC for better evaluating the binary classifier performances.

解决方案

I've found the same issue on github. At the moment, it seems that you also need to run sess.run(tf.initialize_local_variables()) in order to make tf.contrib.metrics.streaming_auc() work. They're working on it.

Here you have an example demonstrating how you can solve this issue:

import tensorflow as tf

a = tf.Variable([0.1, 0.5])
b = tf.Variable([0.2, 0.6])

auc = tf.contrib.metrics.streaming_auc(a, b)

sess = tf.Session()
sess.run(tf.initialize_all_variables())
sess.run(tf.initialize_local_variables()) # try commenting this line and you'll get the error
train_auc = sess.run(auc)

print(train_auc)

这篇关于如何用张量流计算AUC?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！