使用莳萝或泡菜后无法检索数据 [英] Unable to retrieve data after using dill or pickle

问题描述

我使用dill.dump_session(filename)转储了Jupyter Notebook会话，有一次它告诉我磁盘存储已满.但是，我在磁盘上留出一些空间，然后重试.现在，我无法使用dill.load_session(filename)重新加载会话.

I dumped a Jupyter Notebook session using dill.dump_session(filename), and at one point it told me that the disk storage was full. However, I made some space on the disk and tried again. Now, I am unable to load back the session using, dill.load_session(filename).

我收到以下错误:

~/.local/lib/python3.6/site-packages/dill/_dill.py in load_session(filename, main)
    408         unpickler._main = main
    409         unpickler._session = True
--> 410         module = unpickler.load()
    411         unpickler._session = False
    412         main.__dict__.update(module.__dict__)

 EOFError: Ran out of input

文件(即文件名)的数据大小约为30 gigs.

And the file (i.e. filename) is about 30 gigs in size of data.

如何从文件中检索数据?

How can I retrieve my data from the file?

顺便说一句，我正在Google Cloud上运行所有这些程序，这使我不得不花很多钱来保持实例的正常运行.

BTW, I’m running all this on Google Cloud, and it’s costing me a fortune to keep the instance up and running.

我尝试使用undill和其他unpickle方法.

I have tried using undill, and other unpickle methods.

例如，我尝试过:

 open(file, 'a').close()
      try:
     with open(file, "rb") as Score_file:
            unpickler = pickle.Unpickler(Score_file)
            scores = unpickler.load()
            return scores

但是出现了这个错误:

      `6         with open(file, "rb") as Score_file:
       7             unpickler = pickle.Unpickler(Score_file);
 ----> 8             scores = unpickler.load();
       9 
      10             return scores

   ModuleNotFoundError: No module named '__builtin__'`

推荐答案

我知道这可能不是您想听到的答案，但是...听起来您的泡菜文件已损坏.在这种情况下，只有手动编辑数据，您才能取回数据，并且可以了解腌制后的字符串是什么以及它们的结构.请注意，在某些极少数情况下，对象将dump而不是load-但是，很有可能文件已损坏.无论哪种方式，分辨率都是相同的...手动编辑是可能保存您腌制的内容的唯一方法.

I know this probably isn't the answer you want to hear, but... it sounds like you may have a corrupt pickle file. If that's the case, you can get the data back only if you edit it by hand, and can understand what the pickled strings are and how they are structured. Note that there are some very rare cases that an object will dump, but not load -- however, it's much more likely you have a corrupt file. Either way, the resolution is the same... a hand edit is the only way to potentially save what you have pickled.

此外，请注意，如果您使用dump_session，则实际上应该使用load_session(因为它在标准load之上执行了一系列步骤，与dump_session中的操作相反)-与该问题无关紧要，但是问题可能是泡菜文件不完整或损坏.

Also, note that if you use dump_session, you really should use load_session (as it does a sequence of steps on top of a standard load, reversing what is done in dump_session) -- that's really irrelevant for the issue however, the issue likely is having an incomplete or corrupt pickle file.

这篇关于使用莳萝或泡菜后无法检索数据的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！