当从网络接收到数据时，如何运行程序并执行代码? [英] How to run a program and also execute code when data is received from the network?

问题描述

我用 Python 编写了一个小程序，它基本上执行以下操作:

I've written a little program in Python that basically does the following:

1. 从用户那里获取一个启动指令作为输入.如果它与 set 关键字匹配，它将继续.
2. 输入正确的启动指令后，系统会要求用户输入命令.
3. 读取命令后，程序会检查命令文件以查看是否有与该输入匹配的命令
4. 如果找到匹配的命令，则执行该命令所说的任何内容.

我想添加通过网络执行命令的功能，如下所示(并在途中学习使用 Twisted):

I'd like to add the ability to execute commands over a network as follows (and learn to use Twisted on the way):

1. 客户端 #1 输入针对客户端 #2 的命令.
2. 命令被发送到服务器，该服务器将其路由到客户端 #2.
3. 客户端 #2 接收命令并在其有效时执行.

注意:本地(使用下面的代码)和远程输入命令应该是可能的.

Note: Entering commands locally (using the code below) and remotely should be possible.

经过一番思考，我想不出任何其他方法来实现这一点，除了:

After some thinking I couldn't come up with any other way to implement this other than:

1. 让上面的程序作为进程 #1 运行(我在开始时写的本地运行的程序).
2. Twisted 客户端将作为进程 #2 运行并接收来自远程客户端的命令.每当收到命令时，Twisted 客户端都会初始化一个线程，该线程将解析该命令，检查其有效性并在有效时执行.

由于我没有太多线程经验，也没有网络编程经验，所以我想不出任何其他对我有意义的方案.

Since I don't have that much experience with threads and none with network programming, I couldn't think of any other scheme that makes sense to me.

上述方案是否过于复杂?在尝试以这种方式实施之前，我将不胜感激.

python程序(不含客户端)的代码为:

The code for the python program (without the client) is:

main(也就是start()方法):

The main (which is the start() method):

class Controller:
    def __init__(self,listener, executor):
        self.listener = listener
        self.executor = executor


    def start(self):
        while True:
            text = self.listener.listen_for_hotword()

            if self.executor.is_hotword(text):
                text = self.listener.listen_for_command()
                if self.executor.has_matching_command(text):
                    self.executor.execute_command(text)
                else:
                    tts.say("No command found. Please try again")

监听器(从用户那里获取输入):

The Listener (gets input from the user):

class TextListener(Listener):
    def listen_for_hotword(self):
        text = raw_input("Hotword: ")
        text =' '.join(text.split()).lower()
        return text

    def listen_for_command(self):
        text = raw_input("What would you like me to do: ")
        text = ' '.join(text.split()).lower()
        return text

执行者(执行给定命令的类):

The executor (the class that executes the given command):

class Executor:
    #TODO: Define default path
    def __init__(self,parser, audio_path='../Misc/audio.wav'):
        self.command_parser = parser
        self.audio_path = audio_path

    def is_hotword(self,hotword):
        return self.command_parser.is_hotword(hotword)

    def has_matching_command(self,command):
        return self.command_parser.has_matching_command(command)

    def execute_command(self,command):
        val = self.command_parser.getCommand(command)
        print val
        val = os.system(val) #So we don't see the return value of the command

命令文件解析器:

class KeyNotFoundException(Exception):
    pass


class YAMLParser:
    THRESHOLD = 0.6

    def __init__(self,path='Configurations/commands.yaml'):
        with open(path,'r') as f:
            self.parsed_yaml = yaml.load(f)

    def getCommand(self,key):
        try:
            matching_command = self.find_matching_command(key)
            return self.parsed_yaml["Commands"][matching_command]
        except KeyError:
            raise KeyNotFoundException("No key matching {}".format(key))

    def has_matching_command(self,key):
        try:
            for command in self.parsed_yaml["Commands"]:
                if jellyfish.jaro_distance(command,key) >=self.THRESHOLD:
                    return True
        except KeyError:
            return False

    def find_matching_command(self,key):
        for command in self.parsed_yaml["Commands"]:
            if jellyfish.jaro_distance(command,key) >=0.5:
                return command

    def is_hotword(self,hotword):
        return jellyfish.jaro_distance(self.parsed_yaml["Hotword"],hotword)>=self.THRESHOLD

示例配置文件:

Commands:
  echo : echo hello


Hotword: start

推荐答案

我发现很难理解你的问题的背景，但我会自己解决问题.

I'm finding it extremely difficult to follow the background in your questions, but I'll take a stab at the questions themselves.

正如您在问题中指出的那样，写走路和嚼口香糖"的典型方式是样式应用程序是以线程或事件循环样式设计您的代码.

As you noted in your question, the typical way to write a "walk and chew-gum" style app is to design your code in a threaded or an event-loop style.

鉴于您谈论线程和 Twisted(这是事件循环样式)，我担心您可能正在考虑将两者混合使用.

Given you talking about threading and Twisted (which is event-loop style) I'm worried that you may be thinking about mixing the two.

我认为它们是完全不同的编程风格(每个都有自己擅长的地方)，混合它们通常是一条通往地狱的道路.

I view them as fundamentally different styles of programming (each with places they excel) and that mixing them is generally a path to hell.

让我给你一些背景解释

• 如何思考这个概念:

• How to think of the concept:

我需要同时做多项事情，并且我希望我的操作系统能够确定如何以及何时运行这些单独的任务.

I have multiple things I need to do at the same time, and I want my operating system to figure how and when to run those separate tasks.

优点:

Pluses:

• '' 让一个程序同时使用多个处理器内核的方法

• 'The' way to let one program use multiple processor cores at the same time

在 posix 世界中，唯一让一个进程同时在多个 CPU 内核上运行的方法是通过线程(典型的理想线程数不超过给定的内核数)机器)

In the posix world the only way to let one process run on multiple CPU cores at the same time is via threads (with the typical ideal number of threads being no more then the cores in a given machine)

更容易上手

通常可以将您内联运行的相同代码扔到线程中，而无需重新设计(如果没有 GIL，则需要一些锁定，但稍后会更多)

The same code that you were running inline can be tossed off into a thread usually without needing a redesign (without GIL some locking would be required but more on that later)

更容易用于那些会占用你可以给它们的所有 CPU 的任务

Much easier to use with tasks that will eat all the CPU you can give at them

I.E.在大多数情况下，使用线程解决方案比使用事件/异步框架更容易处理数学方式

I.E. in most cases math is way easier to deal with using threading solutions then using event/async frameworks

缺点:

• Python 在线程方面有一个特殊的问题

• Python has a special problem with threads

在 CPython 中，全局解释器锁(GIL) 可以取消线程执行多任务的能力(使线程几乎没用).避免 GIL 很乱，可以取消所有在线程

In CPython the global interpreter lock(GIL) can negate threads ability to multitask (making threads nearly useless). Avoiding the GIL is messy and can undo all the ease of use of working in threads

当您添加线程(和锁)时，事情会变得越来越复杂，请参阅以下内容:线程最佳实践

As you add threads (and locks) things get complicated fast, see this SO: Threading Best Practices

很少适合 IO/用户交互任务

Rarely optimal for IO/user-interacting tasks

虽然线程非常擅长处理需要使用大量 CPU 的少量任务(理想情况下每个内核一个线程)，但它们在处理大部分时间都在等待的大量任务方面却远没有那么理想.

While threads are very good at dealing with small numbers of tasks that want to use lots of CPU (Ideally one thread per core), they are far less optimal at dealing with large counts of tasks that spend most of their time waiting.

最佳用途:

计算成本很高的东西.

如果您想同时运行大量数学运算，那么您不太可能比操作系统更智能地调度 CPU 利用率.

If you have big chucks of math that you want to run concurrently, its very unlikely that your going to be able to schedule the CPU utilization more intelligently then the operation system.

(鉴于 CPythons GIL 问题，不应手动将线程用于数学运算，而应使用内部线程(如 NumPy)的库)

( Given CPythons GIL problem, threading shouldn't manually be used for math, instead a library that internally threads (like NumPy) should be used )

• 如何思考这个概念:

• How to think of the concept:

我有很多事情需要同时做，但我(程序员)想要直接控制/直接实现我的子任务的运行方式和时间

I have multiple things I need to do at the same time, but I (the programer) want direct control/direct-implimentation over how and when my sub-tasks are run

你应该如何看待你的代码:

How you should be thinking about your code:

把你所有的子任务想象成一个相互交织的大整体，你的头脑应该总是想到这段代码运行得足够快，不会弄乱我正在管理的其他子任务"

Think of all your sub-tasks in one big intertwined whole, your mind should always have the thought of "will this code run fast enough that it doesn't goof up the other sub-tasks I'm managing"

优点:

Pluses:

• 使用网络/IO/UI 连接非常高效，包括大量连接

• Extraordinarily efficient with network/IO/UI connections, including large counts of connections

事件循环式程序是解决c10k 的关键技术之一href="http://www.kegel.com/c10k.html" rel="nofollow noreferrer">问题.像 Twisted 这样的框架实际上可以在一台小型机器上运行的一个 python 进程中处理数万个连接.

Event-loop style programs are one of the key technologies that solved the c10k problem. Frameworks like Twisted can literally handle tens-of-thousands of connections in one python process running on a small machine.

随着其他连接/任务的添加，复杂性的可预测(少量)增加

Predictable (small) increase in complexity as other connections/tasks are added

虽然学习曲线相当陡峭(尤其是在扭曲中)，但一旦了解了基础知识，就可以将新的连接类型添加到项目中，而总体复杂性的增加最小.从提供键盘接口的程序转移到提供键盘/telnet/web/ssl/ssh 连接的程序可能只是每个接口的几行胶水代码(...这因框架而异，但无论随着连接数的增加，框架事件循环的复杂性比线程更容易预测)

While there is a fairly steep learning curve (particularly in twisted), once the basics are understood new connection types can be added to projects with a minimal increase in the overall complexity. Moving from a program that offers a keyboard interface to one that offers keyboard/telnet/web/ssl/ssh connections may just be a few lines of glue code per-interface (... this is highly variable by framework, but regardless of the framework event-loops complexity is more predictable then threads as connection counts increase)

缺点:

• 更难上手.

• Harder to get started.

事件/异步编程需要与第一行代码不同的设计风格(尽管您会发现这种设计风格在某种语言之间是可移植的)

Event/async programming requires a different style of design from the first line of code (though you'll find that style of design is somewhat portable from language to language)

一个事件循环，一个核心

One event-loop, one core

虽然事件循环可以让您同时处理数量惊人的 IO 连接，但它们本身并不能同时在多个内核上运行.处理这个问题的常规方法是编写程序，使程序的多个实例可以同时运行，每个内核一个(这种技术绕过了 GIL 问题)

While event-loops can let you handle a spectacular number of IO connections at the same time, they in-and-of-themselves can't run on multiple cores at the same time. The conventional way to deal with this is to write programs in such a way that multiple instances of the program can be run at the same time, one for each core (this technique bypasses the GIL problem)

多路复用高 CPU 任务可能很困难

Multiplexing high CPU tasks can be difficult

事件编程需要将高 CPU 任务切割成小块，这样每块都占用(理想情况下可预测的)少量 CPU，否则只要运行高 CPU 任务，事件系统就会停止多任务处理.

Event programing requires cutting high CPU tasks into pieces such that each piece takes an (ideally predictably) small amount of CPU, otherwise the event system ceases to multitask whenever the high CPU task is run.

最佳用途:

基于 IO 的事物

虽然您的应用程序似乎并非完全基于 IO，但似乎都不是基于 CPU 的(看起来您当前通过 system 调用播放音频，system 每次被调用时都会分离出一个独立的进程，因此它的工作不会消耗你的进程 CPU - 尽管 system 阻塞，所以它在扭曲时是禁忌 - 你必须使用不同的调用扭曲).

While your application doesn't seem to be exclusively IO based, none of it seems to be CPU based (it looks like your currently playing audio via a system call, system spins off an independent process every time its called, so its work doesn't burn your processes CPU - though system blocks, so its a no-no in twisted - you have to use different calls in twisted).

您的问题也不表明您担心最大化多个内核.

Your question also doesn't suggest your concerned about maxing out multiple cores.

因此，鉴于您专门谈到了 Twisted，并且事件循环解决方案似乎最适合您的应用程序，我建议您查看 Twisted 和 -not-threads.

Therefor, given you specifically talked about Twisted, and an event-loop solution seems to be the best match for your application, I would recommend looking at Twisted and -not- threads.

鉴于上面列出的最佳用途"，您可能会认为混合扭曲和线程是一种方法，但是当您这样做时，如果您做任何事情，即使是轻微错误，您也会禁用事件循环(你会阻塞)和线程(GIL 不会让线程执行多任务)的优势，并且有一些超级复杂的东西给你没有优势.

Given the 'Best use' listed above you might be tempted to think that mixing twisted and threads is the way-to-got, but when doing that if you do anything even slightly wrong you will disable the advantages of both the event-loop (you'll block) and threading (GIL won't let the threads multitask) and have something super complex that provides you no advantage.

你给出的方案"是:

经过一番思考，我想不出任何其他方法来实现这一点，除了:

After some thinking I couldn't come up with any other way to implement this other than:

1. 让上面的程序作为进程 #1 运行(我在开始时写的本地运行的程序).
2. Twisted 客户端将作为进程 #2 运行并接收来自远程客户端的命令.每当收到命令时，Twisted 客户端都会初始化一个线程，该线程将解析该命令，检查其有效性并在有效时执行.

由于我没有太多线程方面的经验，也没有网络编程方面的经验，所以我想不出任何其他对我有意义的方案.

Since I don't have that much experience with threads and none with network programming, I couldn't think of any other scheme that makes sense to me.

在回答方案是否......过于复杂"时，我几乎可以肯定地说是的，因为您谈论的是扭曲的 和 线程.(见上面的 tr; dr)

In answer to "Is the scheme ... overly complicated", I would say almost certainly yes because your talking about twisted and threads. (see tr; dr above)

鉴于我对您要构建的内容的理解肯定不完整(并且很困惑)，我想您的一个扭曲的解决方案如下所示:

Given my certainly incomplete (and confused) understanding of what you want to build, I would imagine a twisted solution for you would look like:

1. 仔细研究krondo Twisted Introduction(它确实需要跟踪示例代码行-for-行，但如果你做这项工作，它是一个很棒的学习工具，用于扭曲和事件编程)
2. 从头开始，您可以使用您在 krondo 指南中学到的东西，以扭曲的方式重写您的启动词" - 开始时只需提供您当前拥有的任何界面(键盘?)
3. 向该代码添加其他通信接口(telnet、Web 等)，以便您访问您为键盘(?)接口编写的处理代码.

1. Carefully study the krondo Twisted Introduction (it really requires tracing the example code line-for-line, but if you do the work its an AWESOME learning tool for twisted - and event programing in general)
2. From the ground-up you rewrite your 'hotword' thingy in twisted using what you learned in the krondo guide - starting out just providing whichever interface you currently have (keyboard?)
3. Add other communication interfaces to that code (telnet, web, etc) which would let you access the processing code you wrote for the keyboard(?) interface.

如果如您在问题中所述，您确实需要一台服务器，您可以编写第二个扭曲程序来提供该服务器(您将在 krondo 指南中看到所有这些示例).虽然我猜当您了解了 twisted 的库支持时，您会意识到您不必构建任何额外的服务器，您可以在基本代码中包含您需要的任何协议.

If, as you state in your question, you really need a server, you could write a second twisted program to provide that (you'll see examples of all that in the krondo guide). Though I'm guessing when you understand twisted's library support you'll realize you don't have to build any extra servers, that you can just include whichever protocols you need in your base code.

这篇关于当从网络接收到数据时，如何运行程序并执行代码?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！