什么是最快的读取和写入Float64矩阵以在julia中归档的方法 [英] What is the fastest method(s) for reading and writing a matrix of Float64 to file in julia

问题描述

让x = randn(100, 2).我想将x写入其自己的文件.该文件将包含x，并且仅包含x，而x的类型将永远为Matrix{Float64}.过去，我一直使用HDF5来完成此操作，但是我觉得这太过分了，因为在此设置中，每个文件只有一个数组.请注意，JLD使用HDF5，因此也是过分杀伤.

Let x = randn(100, 2). I want to write x to its own file. This file will contain x, and only x, and x will only ever be of type Matrix{Float64}. In the past, I have always used HDF5 for this, but it occurs to me that this is over-kill, since in this setup I will only have one array per file. Note that JLD uses HDF5, and so is also over-kill.

1)假设我只想读取整个矩阵，什么是最快的读写x方法?

1) What is the fastest method for reading and writing x assuming I will only ever want to read the entire matrix?

2)假设我可能想读取矩阵的一部分，那么最快的读写x方法是什么?

2) What is the fastest method for reading and writing x assuming I might want to read a slice of the matrix?

3)假设我可能想读取矩阵的一个切片或覆盖矩阵的一个切片(但不是更改矩阵大小)?

3) What is the fastest method for reading and writing x assuming I might want to read a slice of the matrix, or over-write a slice of the matrix (but not change the matrix size)?

推荐答案

您可以使用serialize函数，只要您注意文档中有关版本之间无保证的警告.

You could use the serialize function, provided you heed the warnings in the documentation about non-guarantees between versions etc.

serialize(stream :: IO，value)

serialize(stream::IO, value)

以不透明格式将任意值写入流，以便可以通过反序列化来读取它.回读值将与原始值尽可能相同.通常，如果阅读和写作是由不同的人完成的，则此过程将不起作用 版本的Julia，或具有不同系统映像的Julia实例. Ptr值被序列化为全零位模式(NULL).

Write an arbitrary value to a stream in an opaque format, such that it can be read back by deserialize. The read-back value will be as identical as possible to the original. In general, this process will not work if the reading and writing are done by different versions of Julia, or an instance of Julia with a different system image. Ptr values are serialized as all-zero bit patterns (NULL).

首先将8字节的标识标头写入流.为避免编写标头，请构造一个SerializationState并将其用作要序列化的第一个参数.另请参见Serializer.writeheader.

An 8-byte identifying header is written to the stream first. To avoid writing the header, construct a SerializationState and use it as the first argument to serialize instead. See also Serializer.writeheader.

实际上，JLD(或者实际上是其后继者 JLD2 )通常是推荐方式*.

Really though, JLD (or in fact, its successor, JLD2) is generally the recommended way*.

_{*您可能特别感兴趣的是以下语句:"JLD2以包含HDF5子集的格式保存和加载Julia数据结构，而不依赖于HDF5 C库" ，并且它通常优于以前的JLD软件包(有时要高出多个数量级)，并且通常要优于Julia的内置序列化器".}

这篇关于什么是最快的读取和写入Float64矩阵以在julia中归档的方法的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！