多线程环境和模块，例如pickle或json [英] mulithreading environment and modules like pickle or json

问题描述

我正在使用导入线程"和python 3.4. 简单的情况下，我有一个主父线程和一个子线程.我需要将我的字典保存到子线程文件中.在线程函数中，我有一个变量:

I am using "import threading" and python 3.4. Simple case, I have one main parent thread and one child thread. I need to save my dict to file from child thread. In thread function I have variable:

def thread_function(...)
    def save_to_file():
        this_thread_data.my_dict or nonlocal this_thread_data.my_dict 
        ... json or pickle


    this_thread_data = local()
    this_thread_data.my_dict = {...}
    ...

当我用泡菜时会出错

_pickle.PicklingError: Can't pickle <class '_thread.lock'>: attribute lookup lock on _thread failed

当我使用json时出现错误

When I use json I get error

TypeError: <threading.Event object at 0x7f49115a9588> is not JSON serializable

pickle或json将在多线程环境中工作还是我需要使用其他东西代替?

Will pickle or json work in multithreading environment or I need to use something else instead?

谢谢.

推荐答案

在多线程环境中使用pickle和json可以正常工作(但可能不是线程安全的，因此请确保您要腌制的数据不能(例如通过使用锁进行更改).问题在于，您将只能使用实际可以保存到磁盘的数据类型.

Using pickle and json will work fine in a multi-threaded environment (but probably is not thread-safe so make sure the data you're pickling can't changing at the time, for example by using a lock). The catch is that you will be restricted to the kind of data you can actually save to disk.

正如您所发现的，并非所有对象都是可序列化的.最简单的方法是确保字典仅具有与pickle或json序列化程序兼容的值.例如，您似乎在字典中存储了一个锁定对象，这使pickle失败了.您可能想创建一个仅包含可以腌制的值的新字典，然后腌制该值.

Not all objects are serialisable, as you have found. The simplest approach is to make sure your dictionary only has values that are compatible with pickle or the json serialiser. For example, you seem to have stored a lock object in your dictionary that is making pickle fail. You might want to create a new dictionary with only the values that can be pickled, and then pickle that.

或者，如果您想创建一个自定义对象来存储数据，则可以确切地告诉pickle如何腌制它.这是更高级的方法，在您的情况下可能是不必要的，但是您可以在此处找到更多文档: https://docs.python.org/3.4/library/pickle.html#pickling-class-instances

Alternatively, if you want to create a custom object to store your data, you can tell pickle exactly how to pickle it. This is more advanced and probably unnecessary in your case, but you can find more documentation here: https://docs.python.org/3.4/library/pickle.html#pickling-class-instances

这篇关于多线程环境和模块，例如pickle或json的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！