在Julia中读取和写入Float64矩阵到文件的最快方法是什么 [英] What is the fastest method(s) for reading and writing a matrix of Float64 to file in julia

问题描述

令 x = randn(100, 2).我想将 x 写入它自己的文件.该文件将包含 x，并且只有 x，并且 x 将永远是 Matrix{Float64} 类型.过去，我一直为此使用 HDF5，但我觉得这太过分了，因为在这个设置中，每个文件只有一个数组.请注意，JLD 使用 HDF5，因此也是过度杀戮.

Let x = randn(100, 2). I want to write x to its own file. This file will contain x, and only x, and x will only ever be of type Matrix{Float64}. In the past, I have always used HDF5 for this, but it occurs to me that this is over-kill, since in this setup I will only have one array per file. Note that JLD uses HDF5, and so is also over-kill.

1) 假设我只想读取整个矩阵，读取和写入 x 的最快方法是什么?

1) What is the fastest method for reading and writing x assuming I will only ever want to read the entire matrix?

2) 假设我可能想读取矩阵的一部分，读取和写入 x 的最快方法是什么?

2) What is the fastest method for reading and writing x assuming I might want to read a slice of the matrix?

3) 什么是读取和写入最快的方法 x 假设我可能想要读取矩阵的一部分，或者覆盖矩阵的一部分(但不改变矩阵大小)?

3) What is the fastest method for reading and writing x assuming I might want to read a slice of the matrix, or over-write a slice of the matrix (but not change the matrix size)?

推荐答案

您可以使用 serialize 功能，前提是您注意文档中关于版本之间无保证等的警告.

You could use the serialize function, provided you heed the warnings in the documentation about non-guarantees between versions etc.

序列化(流::IO，值)

serialize(stream::IO, value)

以不透明格式将任意值写入流，以便可以通过反序列化读取.回读值将尽可能与原始值相同.一般来说，如果读写是不同的，这个过程是行不通的Julia 的版本，或具有不同系统映像的 Julia 实例.Ptr 值被序列化为全零位模式 (NULL).

Write an arbitrary value to a stream in an opaque format, such that it can be read back by deserialize. The read-back value will be as identical as possible to the original. In general, this process will not work if the reading and writing are done by different versions of Julia, or an instance of Julia with a different system image. Ptr values are serialized as all-zero bit patterns (NULL).

首先将一个 8 字节的标识标头写入流.为避免编写标头，请构造一个 SerializationState 并将其用作序列化的第一个参数.另请参阅 Serializer.writeheader.

An 8-byte identifying header is written to the stream first. To avoid writing the header, construct a SerializationState and use it as the first argument to serialize instead. See also Serializer.writeheader.

真的，JLD(或者事实上，它的继任者，JLD2)通常是推荐方式*.

Really though, JLD (or in fact, its successor, JLD2) is generally the recommended way*.

_{*您可能特别感兴趣的是以下语句:JLD2 以包含 HDF5 子集的格式保存和加载 Julia 数据结构，而不依赖于 HDF5 C 库" 并且 它通常优于以前的 JLD 包(有时高出多个数量级)，并且通常优于 Julia 的内置序列化程序".}

_{*Of particular interest to you might be the statements that: "JLD2 saves and loads Julia data structures in a format comprising a subset of HDF5, without any dependency on the HDF5 C library" and that "it typically outperforms the previous JLD package (sometimes by multiple orders of magnitude) and often outperforms Julia's built-in serializer".}

这篇关于在Julia中读取和写入Float64矩阵到文件的最快方法是什么的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！