如何在 Keras 中计算 Mobilenet FLOPs [英] how to calculate a Mobilenet FLOPs in Keras

问题描述

run_meta = tf.RunMetadata()
enter codwith tf.Session(graph=tf.Graph()) as sess:
K.set_session(sess)


with tf.device('/cpu:0'):
    base_model = MobileNet(alpha=1, weights=None, input_tensor=tf.placeholder('float32', shape=(1,224,224,3)))




opts = tf.profiler.ProfileOptionBuilder.float_operation()    
flops = tf.profiler.profile(sess.graph, run_meta=run_meta, cmd='op', options=opts)

opts = tf.profiler.ProfileOptionBuilder.trainable_variables_parameter()    
params = tf.profiler.profile(sess.graph, run_meta=run_meta, cmd='op', options=opts)

print("{:,} --- {:,}".format(flops.total_float_ops, params.total_parameters))

当我运行上面的代码时，我得到了下面的结果

When I run above code, I got a below result

1,137,481,704 --- 4,253,864

这与论文中描述的触发器不同.

This is different from the flops described in the paper.

mobilenet:https://arxiv.org/pdf/1704.04861.pdf

ShuffleNet:https://arxiv.org/pdf/1707.01083.pdf

ShuffleNet: https://arxiv.org/pdf/1707.01083.pdf

如何计算论文中描述的精确触发器?

How to calculate exact flops described in the paper?

推荐答案

tl;dr 您实际上得到了正确的答案！您只是将触发器与乘法累加(来自论文)进行比较，因此需要除以二.

tl;dr You've actually got the right answer! You are simply comparing flops with multiply accumulates (from the paper) and therefore need to divide by two.

如果您使用的是 Keras，那么您列出的代码有点过于复杂了...

If you're using Keras, then the code you listed is slightly over-complicating things...

让 model 成为任何已编译的 Keras 模型.我们可以通过以下代码得出模型的失败.

Let model be any compiled Keras model. We can arrive at the flops of the model with the following code.

import tensorflow as tf
import keras.backend as K


def get_flops():
    run_meta = tf.RunMetadata()
    opts = tf.profiler.ProfileOptionBuilder.float_operation()

    # We use the Keras session graph in the call to the profiler.
    flops = tf.profiler.profile(graph=K.get_session().graph,
                                run_meta=run_meta, cmd='op', options=opts)

    return flops.total_float_ops  # Prints the "flops" of the model.


# .... Define your model here ....
# You need to have compiled your model before calling this.
print(get_flops())

但是，当我查看自己在计算机上执行的示例(不是 Mobilenet) 时，打印出的 total_float_ops 为 2115，并且当我得到以下结果时我只是打印了 flops 变量:

However, when I look at my own example (not Mobilenet) that I did on my computer, the printed out total_float_ops was 2115 and I had the following results when I simply printed the flops variable:

[...]
Mul                      1.06k float_ops (100.00%, 49.98%)
Add                      1.06k float_ops (50.02%, 49.93%)
Sub                          2 float_ops (0.09%, 0.09%)

很明显，total_float_ops 属性考虑了乘法、加法和减法.

It's pretty clear that the total_float_ops property takes into consideration multiplication, addition and subtraction.

然后我回顾了 MobileNets 的例子，简单地翻阅了论文，我发现 MobileNet 的实现是基于参数数量的默认 Keras 实现:

I then looked back at the MobileNets example, looking through the paper briefly, I found the implementation of MobileNet that is the default Keras implementation based on the number of parameters:

表中的第一个模型与您的结果 (4,253,864) 相匹配，并且乘法运算大约是您拥有的 flops 结果的一半.因此，您有正确的答案，只是您将翻牌误认为是乘法加法(又名乘法累加或 MAC).

The first model in the table matches the result you have (4,253,864) and the Mult-Adds are approximately half of the flops result that you have. Therefore you have the correct answer, it's just you were mistaking flops for Mult-Adds (aka multiply accumulates or MACs).

如果您想计算 MAC 的数量，您只需将上述代码的结果除以二即可.

If you want to compute the number of MACs you simply have to divide the result from the above code by two.

重要说明

如果您尝试运行代码示例，请记住以下几点:

Keep the following in mind if you are trying to run the code sample:

1. 代码示例于 2018 年编写，不适用于 tensorflow 版本 2.有关 tensorflow 版本 2 兼容性的完整示例，请参阅 @driedler 的回答.
2. 代码示例原本打算在编译模型上运行一次...作为一个更好的例子，以没有副作用的方式使用它(因此可以运行多次次在同一模型上)，请参阅 @ch271828n 的答案.

1. The code sample was written in 2018 and doesn't work with tensorflow version 2. See @driedler 's answer for a complete example of tensorflow version 2 compatibility.
2. The code sample was originally meant to be run once on a compiled model... For a better example of using this in a way that does not have side effects (and can therefore be run multiple times on the same model), see @ch271828n 's answer.

这篇关于如何在 Keras 中计算 Mobilenet FLOPs的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！