提高对犰狳矩阵的序列化 [英] Boost serializing armadillo matrices

问题描述

我正在尝试确保序列化适用于矩阵，例如Armadillo，但出现一些错误.

I'm trying to make sure serializing works for matrices i.e. Armadillo and I'm getting some errors.

我做了一个简单的随机矩阵示例，并尝试使用boost将内容保存在二进制文件中，然后使用boost加载它:

I made a simple example of a random matrix and am trying to save the content using boost in a binary file, then load it using boost:

#include <iostream>
#include <fstream>
#include <boost/archive/tmpdir.hpp>
#include <boost/archive/binary_oarchive.hpp>
#include <boost/archive/binary_iarchive.hpp>
#include <armadillo>


int main() {

arma::mat A = arma::randu<arma::mat>(4,5);


std::ofstream outputStream;
outputStream.open("bin.dat");
std::ostringstream oss;
boost::archive::binary_oarchive oa(outputStream);
oa & A;
outputStream.close(); getting error

arma::mat B;
std::ifstream inputStream;
inputStream.open("bin.dat", std::ifstream::in);
boost::archive::binary_iarchive ia(inputStream);
ia & B;
return 0;


}

我收到以下错误(运行命令时:g++ -std=c++11 arma_boost.cpp -larmadillo -lboost_serialization ):

I'm getting the following errors (when running the command: g++ -std=c++11 arma_boost.cpp -larmadillo -lboost_serialization ):

In file included from /usr/include/boost/serialization/split_member.hpp:23:0,
                 from /usr/include/boost/serialization/nvp.hpp:33,
                 from /usr/include/boost/serialization/array.hpp:19,
                 from /usr/include/boost/archive/basic_binary_oprimitive.hpp:50,
                 from /usr/include/boost/archive/binary_oarchive_impl.hpp:22,
                 from /usr/include/boost/archive/binary_oarchive.hpp:21,
                 from arma_boost.cpp:4:
/usr/include/boost/serialization/access.hpp: In instantiation of ‘static void boost::serialization::access::serialize(Archive&, T&, unsigned int) [with Archive = boost::archive::binary_oarchive; T = arma::Mat<double>]’:
/usr/include/boost/serialization/serialization.hpp:69:69:   required from ‘void boost::serialization::serialize(Archive&, T&, unsigned int) [with Archive = boost::archive::binary_oarchive; T = arma::Mat<double>]’
/usr/include/boost/serialization/serialization.hpp:128:27:   required from ‘void boost::serialization::serialize_adl(Archive&, T&, unsigned int) [with Archive = boost::archive::binary_oarchive; T = arma::Mat<double>]’
/usr/include/boost/archive/detail/oserializer.hpp:152:5:   required from ‘void boost::archive::detail::oserializer<Archive, T>::save_object_data(boost::archive::detail::basic_oarchive&, const void*) const [with Archive = boost::archive::binary_oarchive; T = arma::Mat<double>]’
/usr/include/boost/archive/detail/oserializer.hpp:101:1:   required from ‘class boost::archive::detail::oserializer<boost::archive::binary_oarchive, arma::Mat<double> >’
/usr/include/boost/archive/detail/oserializer.hpp:253:13:   required from ‘static void boost::archive::detail::save_non_pointer_type<Archive>::save_standard::invoke(Archive&, const T&) [with T = arma::Mat<double>; Archive = boost::archive::binary_oarchive]’
/usr/include/boost/archive/detail/oserializer.hpp:308:28:   [ skipping 2 instantiation contexts, use -ftemplate-backtrace-limit=0 to disable ]
/usr/include/boost/archive/detail/common_oarchive.hpp:69:40:   required from ‘void boost::archive::detail::common_oarchive<Archive>::save_override(T&, int) [with T = const arma::Mat<double>; Archive = boost::archive::binary_oarchive]’
/usr/include/boost/archive/basic_binary_oarchive.hpp:75:7:   required from ‘void boost::archive::basic_binary_oarchive<Archive>::save_override(const T&, int) [with T = arma::Mat<double>; Archive = boost::archive::binary_oarchive]’
/usr/include/boost/archive/binary_oarchive_impl.hpp:51:9:   required from ‘void boost::archive::binary_oarchive_impl<Archive, Elem, Tr>::save_override(T&, int) [with T = const arma::Mat<double>; Archive = boost::archive::binary_oarchive; Elem = char; Tr = std::char_traits<char>]’
/usr/include/boost/archive/detail/interface_oarchive.hpp:63:9:   required from ‘Archive& boost::archive::detail::interface_oarchive<Archive>::operator<<(T&) [with T = const arma::Mat<double>; Archive = boost::archive::binary_oarchive]’
/usr/include/boost/archive/detail/interface_oarchive.hpp:71:35:   required from ‘Archive& boost::archive::detail::interface_oarchive<Archive>::operator&(T&) [with T = arma::Mat<double>; Archive = boost::archive::binary_oarchive]’
arma_boost.cpp:19:6:   required from here
/usr/include/boost/serialization/access.hpp:118:9: error: ‘class arma::Mat<double>’ has no member named ‘serialize’
         t.serialize(ar, file_version);
         ^
/usr/include/boost/serialization/access.hpp: In instantiation of ‘static void boost::serialization::access::serialize(Archive&, T&, unsigned int) [with Archive = boost::archive::binary_iarchive; T = arma::Mat<double>]’:
/usr/include/boost/serialization/serialization.hpp:69:69:   required from ‘void boost::serialization::serialize(Archive&, T&, unsigned int) [with Archive = boost::archive::binary_iarchive; T = arma::Mat<double>]’
/usr/include/boost/serialization/serialization.hpp:128:27:   required from ‘void boost::serialization::serialize_adl(Archive&, T&, unsigned int) [with Archive = boost::archive::binary_iarchive; T = arma::Mat<double>]’
/usr/include/boost/archive/detail/iserializer.hpp:192:5:   required from ‘void boost::archive::detail::iserializer<Archive, T>::load_object_data(boost::archive::detail::basic_iarchive&, void*, unsigned int) const [with Archive = boost::archive::binary_iarchive; T = arma::Mat<double>]’
/usr/include/boost/archive/detail/iserializer.hpp:120:1:   required from ‘class boost::archive::detail::iserializer<boost::archive::binary_iarchive, arma::Mat<double> >’
/usr/include/boost/archive/detail/iserializer.hpp:387:13:   required from ‘static void boost::archive::detail::load_non_pointer_type<Archive>::load_standard::invoke(Archive&, const T&) [with T = arma::Mat<double>; Archive = boost::archive::binary_iarchive]’
/usr/include/boost/archive/detail/iserializer.hpp:439:28:   [ skipping 2 instantiation contexts, use -ftemplate-backtrace-limit=0 to disable ]
/usr/include/boost/archive/detail/common_iarchive.hpp:66:40:   required from ‘void boost::archive::detail::common_iarchive<Archive>::load_override(T&, int) [with T = arma::Mat<double>; Archive = boost::archive::binary_iarchive]’
/usr/include/boost/archive/basic_binary_iarchive.hpp:70:7:   required from ‘void boost::archive::basic_binary_iarchive<Archive>::load_override(T&, int) [with T = arma::Mat<double>; Archive = boost::archive::binary_iarchive]’
/usr/include/boost/archive/binary_iarchive_impl.hpp:50:9:   required from ‘void boost::archive::binary_iarchive_impl<Archive, Elem, Tr>::load_override(T&, int) [with T = arma::Mat<double>; Archive = boost::archive::binary_iarchive; Elem = char; Tr = std::char_traits<char>]’
/usr/include/boost/archive/detail/interface_iarchive.hpp:60:9:   required from ‘Archive& boost::archive::detail::interface_iarchive<Archive>::operator>>(T&) [with T = arma::Mat<double>; Archive = boost::archive::binary_iarchive]’
/usr/include/boost/archive/detail/interface_iarchive.hpp:67:32:   required from ‘Archive& boost::archive::detail::interface_iarchive<Archive>::operator&(T&) [with T = arma::Mat<double>; Archive = boost::archive::binary_iarchive]’
arma_boost.cpp:26:6:   required from here
/usr/include/boost/serialization/access.hpp:118:9: error: ‘class arma::Mat<double>’ has no member named ‘serialize’

    #include <iostream>
    #include <fstream>
    #include <boost/archive/tmpdir.hpp>
    #include <boost/archive/binary_oarchive.hpp>
    #include <boost/archive/binary_iarchive.hpp>
    #include <armadillo>

    namespace boost {
    namespace serialization {

    template<class Archive>
    void serialize(Archive & ar, arma::mat &m, const unsigned int version)
    {

        ar & m.n_cols;          
        ar & m.n_rows;          
        ar & m.n_elem;                      


    }

      }
      }


    int main() {


  arma::mat A = arma::randu<arma::mat>(4,5);

  std::ofstream outputStream;
  outputStream.open("bin.dat");
  std::ostringstream oss;
  boost::archive::binary_oarchive oa(outputStream);
  oa & A;
  outputStream.close();

  arma::mat b; //tried also arma::mat b(4,5);
  std::ifstream inputStream;
  inputStream.open("bin.dat", std::ifstream::in);
  boost::archive::binary_iarchive ia(inputStream);
  ia & b;


      return 0;
    }

这些是我收到的错误:

In file included from /usr/include/boost/archive/basic_text_oprimitive.hpp:34:0,
                 from /usr/include/boost/archive/text_oarchive.hpp:30,
                 from arma_boost.cpp:3:
/usr/include/boost/archive/detail/check.hpp: In instantiation of ‘void boost::archive::detail::check_const_loading() [with T = const unsigned int]’:
/usr/include/boost/archive/detail/iserializer.hpp:577:38:   required from ‘void boost::archive::load(Archive&, T&) [with Archive = boost::archive::binary_iarchive; T = const unsigned int]’
/usr/include/boost/archive/detail/common_iarchive.hpp:66:40:   required from ‘void boost::archive::detail::common_iarchive<Archive>::load_override(T&, int) [with T = const unsigned int; Archive = boost::archive::binary_iarchive]’
/usr/include/boost/archive/basic_binary_iarchive.hpp:70:7:   required from ‘void boost::archive::basic_binary_iarchive<Archive>::load_override(T&, int) [with T = const unsigned int; Archive = boost::archive::binary_iarchive]’
/usr/include/boost/archive/binary_iarchive_impl.hpp:50:9:   required from ‘void boost::archive::binary_iarchive_impl<Archive, Elem, Tr>::load_override(T&, int) [with T = const unsigned int; Archive = boost::archive::binary_iarchive; Elem = char; Tr = std::char_traits<char>]’
/usr/include/boost/archive/detail/interface_iarchive.hpp:60:9:   required from ‘Archive& boost::archive::detail::interface_iarchive<Archive>::operator>>(T&) [with T = const unsigned int; Archive = boost::archive::binary_iarchive]’
/usr/include/boost/archive/detail/interface_iarchive.hpp:67:32:   [ skipping 8 instantiation contexts, use -ftemplate-backtrace-limit=0 to disable ]
/usr/include/boost/archive/detail/common_iarchive.hpp:66:40:   required from ‘void boost::archive::detail::common_iarchive<Archive>::load_override(T&, int) [with T = arma::Mat<double>; Archive = boost::archive::binary_iarchive]’
/usr/include/boost/archive/basic_binary_iarchive.hpp:70:7:   required from ‘void boost::archive::basic_binary_iarchive<Archive>::load_override(T&, int) [with T = arma::Mat<double>; Archive = boost::archive::binary_iarchive]’
/usr/include/boost/archive/binary_iarchive_impl.hpp:50:9:   required from ‘void boost::archive::binary_iarchive_impl<Archive, Elem, Tr>::load_override(T&, int) [with T = arma::Mat<double>; Archive = boost::archive::binary_iarchive; Elem = char; Tr = std::char_traits<char>]’
/usr/include/boost/archive/detail/interface_iarchive.hpp:60:9:   required from ‘Archive& boost::archive::detail::interface_iarchive<Archive>::operator>>(T&) [with T = arma::Mat<double>; Archive = boost::archive::binary_iarchive]’
/usr/include/boost/archive/detail/interface_iarchive.hpp:67:32:   required from ‘Archive& boost::archive::detail::interface_iarchive<Archive>::operator&(T&) [with T = arma::Mat<double>; Archive = boost::archive::binary_iarchive]’
arma_boost.cpp:54:8:   required from here
/usr/include/boost/archive/detail/check.hpp:162:5: error: static assertion failed: typex::value
     BOOST_STATIC_ASSERT(typex::value);
     ^
In file included from /usr/include/boost/archive/binary_iarchive_impl.hpp:21:0,
                 from /usr/include/boost/archive/binary_iarchive.hpp:20,
                 from arma_boost.cpp:11:
/usr/include/boost/archive/basic_binary_iprimitive.hpp: In instantiation of ‘void boost::archive::basic_binary_iprimitive<Archive, Elem, Tr>::load(T&) [with T = const unsigned int; Archive = boost::archive::binary_iarchive; Elem = char; Tr = std::char_traits<char>]’:
/usr/include/boost/archive/detail/iserializer.hpp:107:9:   required from ‘static void boost::archive::load_access::load_primitive(Archive&, T&) [with Archive = boost::archive::binary_iarchive; T = const unsigned int]’
/usr/include/boost/archive/detail/iserializer.hpp:362:46:   required from ‘static void boost::archive::detail::load_non_pointer_type<Archive>::load_primitive::invoke(Archive&, T&) [with T = const unsigned int; Archive = boost::archive::binary_iarchive]’
/usr/include/boost/archive/detail/iserializer.hpp:439:28:   required from ‘static void boost::archive::detail::load_non_pointer_type<Archive>::invoke(Archive&, T&) [with T = const unsigned int; Archive = boost::archive::binary_iarchive]’
/usr/include/boost/archive/detail/iserializer.hpp:592:24:   required from ‘void boost::archive::load(Archive&, T&) [with Archive = boost::archive::binary_iarchive; T = const unsigned int]’
/usr/include/boost/archive/detail/common_iarchive.hpp:66:40:   required from ‘void boost::archive::detail::common_iarchive<Archive>::load_override(T&, int) [with T = const unsigned int; Archive = boost::archive::binary_iarchive]’
/usr/include/boost/archive/basic_binary_iarchive.hpp:70:7:   [ skipping 11 instantiation contexts, use -ftemplate-backtrace-limit=0 to disable ]
/usr/include/boost/archive/detail/common_iarchive.hpp:66:40:   required from ‘void boost::archive::detail::common_iarchive<Archive>::load_override(T&, int) [with T = arma::Mat<double>; Archive = boost::archive::binary_iarchive]’
/usr/include/boost/archive/basic_binary_iarchive.hpp:70:7:   required from ‘void boost::archive::basic_binary_iarchive<Archive>::load_override(T&, int) [with T = arma::Mat<double>; Archive = boost::archive::binary_iarchive]’
/usr/include/boost/archive/binary_iarchive_impl.hpp:50:9:   required from ‘void boost::archive::binary_iarchive_impl<Archive, Elem, Tr>::load_override(T&, int) [with T = arma::Mat<double>; Archive = boost::archive::binary_iarchive; Elem = char; Tr = std::char_traits<char>]’
/usr/include/boost/archive/detail/interface_iarchive.hpp:60:9:   required from ‘Archive& boost::archive::detail::interface_iarchive<Archive>::operator>>(T&) [with T = arma::Mat<double>; Archive = boost::archive::binary_iarchive]’
/usr/include/boost/archive/detail/interface_iarchive.hpp:67:32:   required from ‘Archive& boost::archive::detail::interface_iarchive<Archive>::operator&(T&) [with T = arma::Mat<double>; Archive = boost::archive::binary_iarchive]’
arma_boost.cpp:54:8:   required from here
/usr/include/boost/archive/basic_binary_iprimitive.hpp:88:35: error: invalid conversion from ‘const void*’ to ‘void*’ [-fpermissive]
         load_binary(& t, sizeof(T));
                                   ^
/usr/include/boost/archive/basic_binary_iprimitive.hpp:149:1: note: initializing argument 1 of ‘void boost::archive::basic_binary_iprimitive<Archive, Elem, Tr>::load_binary(void*, std::size_t) [with Archive = boost::archive::binary_iarchive; Elem = char; Tr = std::char_traits<char>; std::size_t = long unsigned int]’
 basic_binary_iprimitive<Archive, Elem, Tr>::load_binary(
 ^

推荐答案

Armadillo提供了一种将其类型保存到HDF5文件的方法，该文件效果很好，可能对您来说足够了.

Armadillo provides a way to save its type to an HDF5 file that works well and might be enough for you.

Armadillo不支持增强序列化.从 boost序列化教程中，我们可以看到一个类必须具有serialize方法，才能使用boost进行序列化.犰狳没有这种方法，这就是您得到的错误.

Armadillo has no support to boost serialization. From boost serialization tutorial we can see that a class must have a serialize method to be serializable with boost. Armadillo has no such method and that is the error you get.

或者，boost也允许一种非侵入性的方法来使它与类一起工作，这涉及实现serialize自由函数.由于您需要的只是矩阵中的元素及其维度，应该在提升

Alternatively, boost also allows a non-intrusive way to make a class work with it, which involves implementing a serialize free function. Since all you need is the elements in a matrix and its dimension that should be easy to implement following the boost serialization tutorial.

在问题编辑后进行编辑.

Edit after the question was edited.

开始时确实很难确定错误的原因，但是如果用加载部分注释该行(返回之前的最后一行)，您会发现它可以很好地编译.如果运行可执行文件，将创建二进制文件，从而使序列化部分正常工作(但请记住，问题中的序列化功能尚未对元素进行序列化.)

The error you are getting is indeed hard to pinpoint the reason at first, but if you comment the line with the loading part (last line before the return) you will see that it compiles just fine. If you ran the executable the binary file will be created and thus the serialization part is working (but remember the serialize function in your question is not serializing the elements yet).

现在让我们回到错误

/usr/include/boost/archive/basic_binary_iprimitive.hpp:88:35: error: invalid conversion from ‘const void*’ to ‘void*’ [-fpermissive]
         load_binary(& t, sizeof(T));

有一些不应该是const的const类型(被boost擦除，因此您得到void *).如果我们再看看序列化 template 函数和本教程，我们可以看到它既用于序列化又用于加载，尽管在每种情况下Archive类型都是不同的.这意味着在下面的代码中

There is some const type (erased by boost and thus you get void *) that should not be const. If we have another look into the serialize template function and the tutorial we can see that it is used for both serialization and loading, although in each case Archive type is different. That means that in the code below

ar & m.n_cols;          
ar & m.n_rows;          
ar & m.n_elem;  

在序列化期间，从m读取值(确定).但是，在加载过程中，会将值写入m，这是不正确的，因为它们是常量属性.那就是问题所在.为了解决此问题，您需要将保存和加载拆分为单独的功能.然后，在load实现中，可以使用犰狳构造函数之一从加载的数据创建新矩阵.最简单的方法是删除模板并为序列化函数创建两个重载

during the serialization, the values are read from m (OK). However, during the loading the values are written to m, which is not OK since these are constant attributes. That is the problem. In order to fix this you need to split the save and load into separate functions. Then, in the load implementation, you can use one of the armadillo constructors to create a new matrix from the loaded data. The easiest way to do this is to remove the template and create two overloads for the serialize function

// This is the saving part
void serialize(boost::archive::binary_oarchive &ar, arma::mat &m,
               const unsigned int version) {

  ar &m.n_cols;
  ar &m.n_rows;
  ar &m.n_elem; // You don't really need this one

  // Note: You must also serialize the elements, which is currently missing
}

// This is the loading part
void serialize(boost::archive::binary_iarchive &ar, arma::mat &m,
               const unsigned int version) {

    // use one of the suitable constructors here
    m = arma::mat( ... )
}

两个序列化重载的可能完整实现如下所示(我已经通过打印A和B矩阵对其进行了测试，并且它们是相同的).要记住的最重要的一点是，必须按照保存它们的顺序读取这些值.

A possible full implementation of both serialize overloads is shown below (I have tested them by printing both A and B matrices and they are the same). The most important thing to remember is that you must read the values in the same order you save them.

void serialize(boost::archive::binary_oarchive &ar, arma::mat &m,
               const unsigned int version) {
  // Since this is used only for reading I prefer the "<<" symbol instead of
  // "&", but "&" would work just fine
  ar << m.n_cols;
  ar << m.n_rows;
  // Loop to save each element in order
  for (unsigned i = 0; i < m.n_elem; i++) {
    // memptr gives us a pointer to the raw memory used to store the elements
    ar << m.memptr()[i];
  }
}

void serialize(boost::archive::binary_iarchive &ar, arma::mat &m,
               const unsigned int version) {

  arma::uword n_cols;
  arma::uword n_rows;
  ar >> n_cols;
  ar >> n_rows;
  m = arma::mat(n_rows, n_cols);

  // Loop to read each element in order
  for (unsigned i = 0; i < m.n_elem; i++) {
    ar >> m.memptr()[i];
  }
}

免责声明:实际上，我之前从未使用过boost序列化，因此可能会有更好的方法来实现这一点.

Disclaimer: I actually had never used boost serialization before and thus there might be a better way to do this.

这篇关于提高对犰狳矩阵的序列化的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！