BeautifulSoup对象不会味酸，导致国米preTER为默认崩溃 [英] BeautifulSoup Object Will Not Pickle, Causes Interpreter to Silently Crash

问题描述

我从 BeautifulSoup 汤，我不能咸菜。当我尝试酸洗蟒间preTER默默崩溃的对象（例如，它不能作为例外处理）。我已经使用多包（其中泡菜对象，通过他们的进程之间）能够以酸洗对象，以返回对象。我该如何解决/解决问题？不幸的是，我不能张贴页面的HTML（这是不公开的），我一直无法找到问题的重复的例子。我曾尝试通过循环在汤和酸洗各个组件找出问题，产生误差最小的是＆LT;类的BeautifulSoup.NavigableString'＆GT; 。当我打印对象它打印出 U'\\ n'。

I have a soup from BeautifulSoup that I cannot pickle. When I try to pickle the object the python interpreter silently crashes (such that it cannot be handled as an exception). I have to be able to pickle the object in order to return the object using the multiprocessing package (which pickles objects to pass them between processes). How can I troubleshoot/work around the problem? Unfortunately, I cannot post the html for the page (it is not publicly available), and I have been unable to find a reproducible example of the problem. I have tried to isolate the problem by looping over the soup and pickling individual components, the smallest thing that produces the error is <class 'BeautifulSoup.NavigableString'>. When I print the object it prints out u'\n'.

推荐答案

类 NavigableString 与不可序列泡菜或的cPickle ，<其中code>多使用。您应该能够序列化这个类萝，但是。 萝的酱菜接口的超集，并可以序列最蟒蛇。 多处理还是会失败，除非使用的一个分支，多处理，它使用萝，名为 pathos.multiprocessing 。

The class NavigableString is not serializable with pickle or cPickle, which multiprocessing uses. You should be able to serialize this class with dill, however. dill has a superset of the pickle interface, and can serialize most of python. multiprocessing will still fail, unless you use a fork of multiprocessing which uses dill, called pathos.multiprocessing.

在这里获得code： https://github.com/uqfoundation

Get the code here: https://github.com/uqfoundation.

有关更多信息，请参阅：
<一href=\"http://stackoverflow.com/questions/19984152/what-can-multiprocessing-and-dill-do-together\">What可以多处理和莳萝一起做？

For more information see: What can multiprocessing and dill do together?

http://matthewrocklin.com/blog/work /十二分之二千零十三/ 05 /并行和串行化/

http://nbviewer.ipython.org/gist/minrk/5241793

这篇关于BeautifulSoup对象不会味酸，导致国米preTER为默认崩溃的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！