如何使一个2.7 python上下文管理器线程安全 [英] how do I make a 2.7 python context manager threadsafe

问题描述

我有一个运行在Django服务上的大型python应用程序。我需要关闭某些操作的权限测试，所以我创建了这个上下文管理器：

  class OverrideTests（object）：
 
 def __init __（self）：
 self.override = 0 
 
 def __enter __（self）：
 self.override + = 1 
 
＃noinspection PyUnusedLocal 
 def __exit __（self，exc_type，exc_val，exc_tb）：
 self.override  -  = 1 
 assert not self.override< 0 
 
 @property 
 def overriding（self）：
 return self.override> 0 
 
 override_tests = OverrideTests（）
  

应用程序的各个部分可以使用上下文管理器覆盖测试：

  with override_tests：
 do stuff 
 ... 
  

上述上下文管理器可以在不同的功能中多次使用。计数器的使用使得它受到控制，它似乎工作正常...直到线程被卷入。

一旦涉及到线程，全局上下文管理器就会获得因此，测试可能不正确地覆盖。

这是一个简单的测试用例 - 如果线程可以正常工作。 start_new_thread（do_id，（））行被替换为一个简单的 do_it ，但是如图所示非常惊人：

  def stat（k，expected）：
x ='。'如果override_tests.overriding == expected else'*'
 sys.stdout.write （'{0} {1}'。format（k，x））
 
 
 def do_it_inner（）：
 with override_tests：
 stat（2，True ）
 stat（3，True）＃outer with context make true true 
 
 
 def do_it（）：
 with override_tests：
 stat（1，真的）
 do_it_inner（）
 stat（4，False）
 
 
 def do_it_lots（ntimes = 10）：
 for i in range（ntimes） ： 
 thread.start_new_thread（do_it，（））
  

我如何使这个上下文管理器线程安全的，所以在每个Python线程中，即使它是重新进入的，它一直使用

解决方案

似乎工作：使您的OverrideTests类成为 threading.local 的子类。为了安全起见，您应该调用 __ init __ 中的超类 __ init __ （虽然它似乎工作，即使你不t）：

  class OverrideTests（threading.local）：
 
 def __init __（self）：
 super（OverrideTests，self）.__ init __（）
 self.override = 0 
 
＃其余的类与之前的
 
 override_tests = OverrideTests（） 
  

然后：

 code>>>> do_it_lots（）
 1.1.1.2.2.1.1.1.1.1.1.3.3.2.2.2.2.2.2.4.4.3.1.3.3.3.3.4.3.2.4.4.2.4.3.4.4.4.3.4。 
  

但是，我不会把钱放在某种角落的情况下，您的真实应用程序比您在此处显示的示例更为复杂。最终你真的应该重新思考你的设计。在您的问题中，您正在关注如何使上下文管理器线程安全。但真正的问题不仅在于您的上下文管理器，而且在您的示例中使用函数（ stat ）。 stat 依赖于全局状态（全局 override_tests ），这在线程环境中本身就是脆弱的。 >

I have a large python application which is running on a Django service. I need to turn off permission tests for certain operations so I created this context manager:

class OverrideTests(object):

    def __init__(self):
        self.override = 0

    def __enter__(self):
        self.override += 1

    # noinspection PyUnusedLocal
    def __exit__(self, exc_type, exc_val, exc_tb):
        self.override -= 1
        assert not self.override < 0

    @property
    def overriding(self):
        return self.override > 0

override_tests = OverrideTests()

Various parts of the application can then overide the tests using the context manager:

with override_tests:
    do stuff
    ...

Within the do stuff, the above context manager may be used multiple times in different functions. The use of the counter keeps this under control and it seems to work fine... until threads get involved.

Once there are threads involved, the global context manager gets re-used and as a result, tests may be incorrectly over-ridden.

Here is a simple test case - this works fine if the thread.start_new_thread(do_id, ()) line is replaced with a simple do_it but fails spectacularly as shown:

def stat(k, expected):
    x = '.' if override_tests.overriding == expected else '*'
    sys.stdout.write('{0}{1}'.format(k, x))


def do_it_inner():
    with override_tests:
        stat(2, True)
    stat(3, True)  # outer with context makes this true


def do_it():
    with override_tests:
        stat(1, True)
        do_it_inner()
    stat(4, False)


def do_it_lots(ntimes=10):
    for i in range(ntimes):
        thread.start_new_thread(do_it, ())

How can I make this context manager thread safe so that in each Python thread, it is consistently used even though it is re-entrant?

解决方案

Here is a way that seems to work: make your OverrideTests class a subclass of threading.local. For safety, you should then call the superclass __init__ in your __init__ (although it seems to work even if you don't):

class OverrideTests(threading.local):

    def __init__(self):
        super(OverrideTests, self).__init__()
        self.override = 0

    # rest of class same as before

override_tests = OverrideTests()

Then:

>>> do_it_lots()
1.1.1.2.2.1.1.1.1.1.1.3.3.2.2.2.2.2.2.4.4.3.1.3.3.3.3.4.3.2.4.4.2.4.3.4.4.4.3.4.

However, I wouldn't put money on this not failing in some kind of corner case, especially if your real application is more complex than the example you showed here. Ultimately, you really should rethink your design. In your question, you are focusing on how to "make the context-manager threadsafe". But the real problem is not just with your context manager but with your function (stat in your example). stat is relying on global state (the global override_tests), which is inherently fragile in a threaded environment.

这篇关于如何使一个2.7 python上下文管理器线程安全的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！