在循环中从 Python pickle 文件加载数据? [英] Load data from Python pickle file in a loop?

问题描述

在一个小型数据采集项目中，我们使用 Python 的 pickle 来存储记录的数据，即对于每个事件"，我们将其添加到输出文件 f 中，使用

In a small data-acquisition project we use the Python's pickle to store recorded data, i.e. for each "event" we add it to the output file f with

pkl.dump(event, f, pkl.HIGHEST_PROTOCOL)

where import cPickle as pkl.

在数据分析中，我们读取每个事件，但与普通文件相比，可以以一种相当优雅的方式进行处理:

In the analysis of the data we read each event, but in contrast to a normal file where processing can be one in a rather elegant way:

with open(filename) as f:
    for line in f:
        do_something()

遍历 pickle 文件中的所有数据，这变得有点尴尬:

looping over all the data in a pickle file this becomes a bit more awkward:

with open(filename) as f:
    try:
        while True:
            event = pkl.load(f)
            do_something()
    except (EOFError, UnpicklingError):
        pass

是否有可能让泡菜阅读更像上面的常规文件示例?

Is it possible to make pickle reading more like the example for regular files above?

推荐答案

是的，确实如此.使用下面的这个生成器使事件在循环中可读:

Yes, indeed. Use this generator below to make the events readable in a loop:

def pickleLoader(pklFile):
    try:
        while True:
            yield pkl.load(pklFile)
    except EOFError:
        pass

现在你可以简单地写:

with open(filename) as f:
    for event in pickleLoader(f):
        do_something()

这篇关于在循环中从 Python pickle 文件加载数据?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！