织物2.3并行执行sudo命令 [英] fabric 2.3 parallel execution of sudo commands

问题描述

我正在尝试使用Fabric 2.3在一些服务器上运行一些需要sudo的任务.我的主要目的是使操作并行化，因此我想到了使用Fabric API的ThreadingGroup类，但它不支持sudo.

I am trying to use fabric 2.3 to run few tasks that require sudo on some servers . My main goal here was to parallelize the operation so i thought of using ThreadingGroup class of fabric api however it does not support sudo.

为了清楚起见，下面是我的代码

Below is my code for sake of clarity

#!/usr/bin/env python

from fabric import ThreadingGroup, Config
from getpass import getpass


sudo_pass = getpass("Enter your sudo password: ")
sudo_config = Config(overrides={'sudo': {'password': sudo_pass}})
server_pool = ThreadingGroup("test1", "test2",
                             config=sudo_config)

result = server_pool.sudo('cat /etc/shadow', hide='stderr')
print(result)

现在这不能像上面提到的那样工作，因为ThreadingGroup不支持Connection类支持的所有方法.

Now this does not work as mentioned above because ThreadingGroup does not support all of the methods that Connection class support.

我可以在多个服务器上运行sudo，方法是遍历各个主机，然后为每个主机创建连接，但这效率不高.

I can run sudo on multiple servers by iterating over the individual hosts and then creating connection for each but that isn't efficient.

那么有没有办法使它与Fabric 2.3并行?我也浏览了官方文档，但没有找到任何东西.

So is there a way to make this parallel with fabric 2.3 ? I have gone through the official documentation as well but did not find anything.

进一步，我根据官方文档对其进行了更多测试，并且似乎ThreadingGroup仅当您按照以下方式运行时，才能实现并行性

Further i did some more testing on it following the official documentation and it seems like ThreadingGroup achieves parallelism only if you run it like below

fabric.ThreadingGroup('test1', 'test2').run('uname -s')

但是，如果您像下面那样运行它，则它不会并行运行

however if you run it like below it does not run in parallel

def run_task(c):
    c.run('uname -s')


for cxn in fabric.ThreadingGroup('test1', 'test2'):
    run_task(cxn)

因此，到目前为止，似乎在Fabric 2.3中没有对并行性的灵活支持，我可能只需要切换回Fabric版本1.

So it looks like there isn't much flexible support for parallelism in fabric 2.3 as of now and i might have to switch back to fabric version 1 only.

推荐答案

似乎某些功能未在当前版本(到目前为止为2.4)中实现. 一个可选的解决方案是在其源文件中添加一些代码. 您可以找到fabric的安装路径，然后编辑group.py.

It seems some functions are not implemented in current version (2.4 so far). An optional solution is to add some code in its source file. You can find the install path of fabric, and edit group.py.

首先，将此功能添加到group.py中:

First, add this function in group.py:

def thread_worker_sudo(cxn, queue, args, kwargs):
    result = cxn.sudo(*args, **kwargs)
    # TODO: namedtuple or attrs object?
    queue.put((cxn, result))

，然后在ThreadingGroup类中添加sudo函数:

and then add sudo function in class ThreadingGroup:

class ThreadingGroup(Group):
    .... original ThreadingGroup


    def sudo(self, *args, **kwargs):
        results = GroupResult()
        queue = Queue()
        threads = []
        for cxn in self:
            my_kwargs = dict(cxn=cxn, queue=queue, args=args, kwargs=kwargs)
            thread = ExceptionHandlingThread(
                target=thread_worker_sudo, kwargs=my_kwargs
            )
            threads.append(thread)
        for thread in threads:
            thread.start()
        for thread in threads:
            # TODO: configurable join timeout
            # TODO: (in sudo's version) configurability around interactive
            # prompting resulting in an exception instead, as in v1
            thread.join()
        # Get non-exception results from queue
        while not queue.empty():
            # TODO: io-sleep? shouldn't matter if all threads are now joined
            cxn, result = queue.get(block=False)
            # TODO: outstanding musings about how exactly aggregate results
            # ought to ideally operate...heterogenous obj like this, multiple
            # objs, ??
            results[cxn] = result
        # Get exceptions from the threads themselves.
        # TODO: in a non-thread setup, this would differ, e.g.:
        # - a queue if using multiprocessing
        # - some other state-passing mechanism if using e.g. coroutines
        # - ???
        excepted = False
        for thread in threads:
            wrapper = thread.exception()
            if wrapper is not None:
                # Outer kwargs is Thread instantiation kwargs, inner is kwargs
                # passed to thread target/body.
                cxn = wrapper.kwargs["kwargs"]["cxn"]
                results[cxn] = wrapper.value
                excepted = True
        if excepted:
            raise GroupException(results)
        return results

我只是复制功能 run 的代码并替换此行

I just copy the code of function run and replace this line

thread = ExceptionHandlingThread(
    target=thread_worker, kwargs=my_kwargs
)

与

thread = ExceptionHandlingThread(
    target=thread_worker_sudo, kwargs=my_kwargs
)

像这样对我有用:

def test_sudo(group):
    group.sudo('whoami', user='root')

$ python fabfile.py
root
root
root
root
root

但是，我不确定，它在所有情况下都能正常工作.

However, I am not sure, it will work well for all situations.

这篇关于织物2.3并行执行sudo命令的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！