使用Dill在Load Object上执行Python TypeError [英] Python TypeError on Load Object using Dill

问题描述

试图将一个大的(可能非常)无法拾取的对象渲染到文件中以供以后使用.

Trying to render a large and (possibly very) unpicklable object to a file for later use.

dill.dump(file)方面没有投诉:

In [1]: import echonest.remix.audio as audio

In [2]: import dill

In [3]: audiofile = audio.LocalAudioFile("/Users/path/Track01.mp3")
en-ffmpeg -i "/Users/path/audio/Track01.mp3" -y -ac 2 -ar 44100 "/var/folders/X2/X2KGhecyG0aQhzRDohJqtU+++TI/-Tmp-/tmpWbonbH.wav"
Computed MD5 of file is b3820c166a014b7fb8abe15f42bbf26e
Probing for existing analysis

In [4]: with open('audio_object_dill.pkl', 'wb') as f:
   ...:     dill.dump(audiofile, f)
   ...:  

In [5]: 

但是尝试加载.pkl文件:

In [1]: import dill

In [2]: with open('audio_object_dill.pkl', 'rb') as f:
   ...:     audio_object = dill.load(f)
   ...:  

返回以下错误:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-2-203b696a7d73> in <module>()
      1 with open('audio_object_dill.pkl', 'rb') as f:
----> 2     audio_object = dill.load(f)
      3 

/Users/mikekilmer/Envs/GLITCH/lib/python2.7/site-packages/dill-0.2.2.dev-py2.7.egg/dill/dill.pyc in load(file)
    185     pik = Unpickler(file)
    186     pik._main_module = _main_module
--> 187     obj = pik.load()
    188     if type(obj).__module__ == _main_module.__name__: # point obj class to main
    189         try: obj.__class__ == getattr(pik._main_module, type(obj).__name__)

/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.pyc in load(self)
    856             while 1:
    857                 key = read(1)
--> 858                 dispatch[key](self)
    859         except _Stop, stopinst:
    860             return stopinst.value

/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.pyc in load_newobj(self)
   1081         args = self.stack.pop()
   1082         cls = self.stack[-1]
-> 1083         obj = cls.__new__(cls, *args)
   1084         self.stack[-1] = obj
   1085     dispatch[NEWOBJ] = load_newobj

TypeError: __new__() takes at least 2 arguments (1 given)

AudioObject比进行上述调用的class object要复杂得多(而且更大)(来自

The AudioObject is much more complex (and large) than the class object the above calls are made on (from SO answer), and I'm unclear as to whether I need to send a second argument via dill, and if so, what that argument would be or how to tell if any approach to pickling is viable for this specific object.

稍微检查一下对象本身:

Examining the object itself a bit:

In [4]: for k, v in vars(audiofile).items():
...:     print k, v
...: 

返回:

is_local False
defer False
numChannels 2
verbose True
endindex 13627008
analysis <echonest.remix.audio.AudioAnalysis object at 0x103c61bd0>
filename /Users/mikekilmer/Envs/GLITCH/glitcher/audio/Track01.mp3
convertedfile /var/folders/X2/X2KGhecyG0aQhzRDohJqtU+++TI/-Tmp-/tmp9ADD_Z.wav
sampleRate 44100
data [[0 0]
 [0 0]
 [0 0]
 ..., 
 [0 0]
 [0 0]
 [0 0]]

audiofile.analysis似乎包含一个名为audiofile.analysis.source的属性，该属性包含(或显然指向)audiofile.analysis.source.analysis

And audiofile.analysis seems to contain an attribute called audiofile.analysis.source which contains (or apparently points back to) audiofile.analysis.source.analysis

推荐答案

在这种情况下，答案位于模块本身内.

In this case, the answer lay within the module itself.

LocalAudioFile类提供了(并且每个实例都可以利用)它自己的save方法，该方法通过LocalAudioFile.save或更可能是the_audio_object_instance.save调用.

The LocalAudioFile class provides (and each of it's instances can therefor utilize) it's own save method, called via LocalAudioFile.save or more likely the_audio_object_instance.save.

对于.mp3文件，LocalAudioFile实例由指向临时.wav文件(其为.mp3的解压缩版本)的指针以及一堆完整的分析数据组成.与基于Internet的Echonest API接口连接后，从初始音频文件返回.

In the case of an .mp3 file, the LocalAudioFile instance consists of a pointer to a temporary .wav file which is the decompressed version of the .mp3, along with a whole bunch of analysis data which is returned from the initial audiofile, after it's been interfaced with the (internet-based) Echonest API.

LocalAudioFile.save 调用shutil.copyfile(path_to_wave, wav_path)以与链接到音频对象的原始文件相同的名称和路径保存.wav文件，如果该文件已经存在，则返回错误.它会调用pickle.dump(self, f)来将分析数据保存到一个文件中，该文件也位于调用原始音频对象文件的目录中.

LocalAudioFile.save calls shutil.copyfile(path_to_wave, wav_path) to save the .wav file with same name and path as original file linked to audio object and returns an error if the file already exists. It calls pickle.dump(self, f) to save the analysis data to a file also in the directory the initial audio object file was called from.

可以简单地通过pickle.load()重新引入LocalAudioFile对象.

The LocalAudioFile object can be reintroduced simply via pickle.load().

这是一个iPython会话，在其中我使用了dill，这是一个非常有用的包装器或接口，它提供了大多数标准的pickle方法以及更多的方法:

Here's an iPython session in which I used the dill, which is a very useful wrapper or interface that offers most of the standard pickle methods plus a bunch more:

audiofile = audio.LocalAudioFile("/Users/mikekilmer/Envs/GLITCH/glitcher/audio/Track01.mp3")

In [1]: import echonest.remix.audio as audio

In [2]: import dill
# create the audio_file object
In [3]: audiofile = audio.LocalAudioFile("/Users/mikekilmer/Envs/GLITCH/glitcher/audio/Track01.mp3")
en-ffmpeg -i "/Users/path/audio/Track01.mp3" -y -ac 2 -ar 44100 "/var/folders/X2/X2KGhecyG0aQhzRDohJqtU+++TI/-Tmp-/tmp_3Ei0_.wav"
Computed MD5 of file is b3820c166a014b7fb8abe15f42bbf26e
Probing for existing analysis
#call the LocalAudioFile save method
In [4]: audiofile.save()
Saving analysis to local file /Users/path/audio/Track01.mp3.analysis.en
#confirm the object is valid by calling it's duration method
In [5]: audiofile.duration
Out[5]: 308.96
#delete the object - there's probably a "correct" way to do this
in [6]: audiofile = 0
#confirm it's no longer an audio_object
In [7]: audiofile.duration
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-12-04baaeda53a4> in <module>()
----> 1 audiofile2.duration

AttributeError: 'int' object has no attribute 'duration'


#open the pickled version (using dill)
In [8]: with open('/Users/path/audio/Track01.mp3.analysis.en') as f:
   ....:     audiofile = dill.load(f)
   ....:     
#confirm it's a valid LocalAudioFile object
In [8]: audiofile.duration
Out[8]: 308.96

Echonest 是一个非常强大的API，并且remix包提供了大量功能.在此处处，列出了一小部分相关链接.

Echonest is a very robust API and the remix package provides a ton of functionality. There's a small list of relevant links assembled here.

这篇关于使用Dill在Load Object上执行Python TypeError的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！