适用于 Android 的 Tensorflow 量化图 [英] Tensorflow Quantized Graph for Android

问题描述

我正在尝试将量化图加载到 Android 应用中.我的 BUILD 文件包含

deps = ["//tensorflow/core:android_tensorflow_lib","//tensorflow/contrib/quantization:cc_array_ops","//tensorflow/contrib/quantization:cc_math_ops","//tensorflow/contrib/quantization:cc_nn_ops","//tensorflow/contrib/quantization/kernels:quantized_ops"]

额外的量化依赖项适用于独立的 C++ 构建.

我无法使用 Bazel 进行编译，因为 GEMMLOWP 中存在大量错误.在 Android 中包含 gemmlowp 和量化操作的正确方法是什么?

这是一个示例错误:

external/gemmlowp/eight_bit_int_gemm/eight_bit_int_gemm.cc:125:13: 错误:'int32_t' 不是 'std' 的成员MatrixMap结果(c，m，n，ldc)；

这是在带有 Bazel 0.3.0 的 Ubuntu 16.04 上.

这是一个要点，它具有两次连续尝试构建包的输出 - 第一次在高速公路哈希上失败，第二次在 gemmlowp 上失败.https://gist.github.com/ericdanz/81b799f2e0bbb3cc462aa3c9046

最终让它编译并运行，并在 gemmlowp 和 Highwayhash 的 BUILD 文件中自由添加-std=c++11"，并替换了 android 框架以替换量化操作中的框架依赖项.但是它产生了完全不同的结果，运行速度慢了大约 4 倍(26-3200 毫秒与 6-800 毫秒).我会尝试做更深入的调查.

解决方案

这对我有用 - 它基本上是上面 Eric D 的所有评论的组合，但我想把它放在一个地方给新人谁遇到过这个问题:

将 deps 中的 quantized_ops 添加到 Android 应用程序的 BUILD 文件中的 libtensorflow_demo.so:

deps = ["//tensorflow/core:android_tensorflow_lib","//tensorflow/contrib/quantization/kernels:quantized_ops",]

修改tensorflow/contrib/quantization/kernels/BUILD中quantized_ops的deps:

deps = ["//tensorflow/contrib/quantization:cc_array_ops","//tensorflow/contrib/quantization:cc_math_ops","//tensorflow/contrib/quantization:cc_nn_ops","//tensorflow/core:android_tensorflow_lib",#"//张量流/核心",#"//tensorflow/core:framework",#"//tensorflow/core:lib",#"//tensorflow/core/kernels:concat_lib_hdrs",#"//tensorflow/core/kernels:conv_ops",#"//tensorflow/core/kernels:ops_util_hdrs",#"//tensorflow/core/kernels:pooling_ops_hdrs",#"//tensorflow/core/kernels:eigen_helpers",#"//tensorflow/core/kernels:ops_util",#"//tensorflow/core/kernels:pooling_ops","//third_party/eigen3","@gemmlowp//:eight_bit_int_gemm",],

删除/注释掉 tensorflow/contrib/quantization/ops/array_ops.cc、math_ops.cc 和 nn_ops.cc 中的 .Doc() 部分

修改tensorflow/contrib/quantization/BUILD中cc_array_ops、cc_math_ops和cc_nn_ops的deps:

deps = [#"//tensorflow/core:framework","//tensorflow/core:android_tensorflow_lib",],

使用 --cxxopt="-std=c++11" 标志为 Android 应用运行 bazel 构建命令.

I'm trying to load a quantized graph into an Android app. My BUILD file contains

deps = ["//tensorflow/core:android_tensorflow_lib",
      "//tensorflow/contrib/quantization:cc_array_ops",
      "//tensorflow/contrib/quantization:cc_math_ops",
      "//tensorflow/contrib/quantization:cc_nn_ops",
      "//tensorflow/contrib/quantization/kernels:quantized_ops"]

The additional quantization deps work for standalone C++ builds.

I can't compile with Bazel, due to a large number of errors in GEMMLOWP. What's the proper way to include gemmlowp and the quantization ops in Android?

Here is an example error:

external/gemmlowp/eight_bit_int_gemm/eight_bit_int_gemm.cc:125:13: error: 'int32_t' is not a member of 'std'
   MatrixMap<std::int32_t, ResultOrder> result(c, m, n, ldc);

This is on Ubuntu 16.04 with Bazel 0.3.0.

Here's a gist that has the output of two sequential attempts to build the package - it fails on highwayhash the first time and gemmlowp the second. https://gist.github.com/ericdanz/81b799f2e0bbb3cc462aa3c90468c71b

Ultimately got it to compile and run with liberal addition of "-std=c++11" in BUILD files for gemmlowp and highwayhash, and substitution of the android framework for the framework dependencies in the quantized ops. It produces fairly different results though, and runs about 4x slower (26-3200ms vs 6-800 ms). I'll try to do a little deeper investigation.

解决方案

Here's what worked for me - it is basically a combination of all the comments from Eric D above, but I wanted to put it all in one place for someone new who comes across this problem:

Add quantized_ops in deps to libtensorflow_demo.so in the BUILD file for the Android app:

deps = ["//tensorflow/core:android_tensorflow_lib",
        "//tensorflow/contrib/quantization/kernels:quantized_ops",]

Modify the deps for quantized_ops in tensorflow/contrib/quantization/kernels/BUILD:

deps = [
        "//tensorflow/contrib/quantization:cc_array_ops",
        "//tensorflow/contrib/quantization:cc_math_ops",
        "//tensorflow/contrib/quantization:cc_nn_ops",
        "//tensorflow/core:android_tensorflow_lib",
        #"//tensorflow/core",
        #"//tensorflow/core:framework",
        #"//tensorflow/core:lib",
        #"//tensorflow/core/kernels:concat_lib_hdrs",
        #"//tensorflow/core/kernels:conv_ops",
        #"//tensorflow/core/kernels:ops_util_hdrs",
        #"//tensorflow/core/kernels:pooling_ops_hdrs",
        #"//tensorflow/core/kernels:eigen_helpers",
        #"//tensorflow/core/kernels:ops_util",
        #"//tensorflow/core/kernels:pooling_ops",
        "//third_party/eigen3",
        "@gemmlowp//:eight_bit_int_gemm",
    ],

Remove/comment out the .Doc() parts in tensorflow/contrib/quantization/ops/array_ops.cc, math_ops.cc and nn_ops.cc

Modify the deps for cc_array_ops, cc_math_ops and cc_nn_ops in tensorflow/contrib/quantization/BUILD:

deps = [
        #"//tensorflow/core:framework",
        "//tensorflow/core:android_tensorflow_lib",
    ],

Run bazel build command for Android app with --cxxopt="-std=c++11" flag.

这篇关于适用于 Android 的 Tensorflow 量化图的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！