从网络接收数据时如何运行程序并执行代码? [英] How to run a program and also execute code when data is received from the network?

问题描述

我已经用Python编写了一个小程序，基本上可以完成以下任务:

I've written a little program in Python that basically does the following:

1. 从用户那里获得一个热门词作为输入.如果它与set关键字匹配，则继续.
2. 输入正确的关键词后，要求用户输入命令.
3. 读取命令后，程序会检查命令文件以查看是否有与输入匹配的命令
4. 如果找到匹配的命令，请执行该命令所说的内容.

我想增加以下通过网络执行命令的能力(并学习使用Twisted的方式):

I'd like to add the ability to execute commands over a network as follows (and learn to use Twisted on the way):

1. 客户端#1输入针对客户端#2的命令.
2. 该命令被发送到服务器，该服务器将其路由到客户端#2.
3. 2号客户端接收命令并在有效时执行该命令.

注意:应该可以在本地(使用下面的代码)和远程输入命令.

Note: Entering commands locally (using the code below) and remotely should be possible.

经过一番思考，除了:

1. 使上面的程序作为进程#1运行(该程序在开始时就在本地运行).
2. Twisted客户端将作为进程#2运行，并从远程客户端接收命令.每当收到命令时，Twisted客户端都会初始化一个线程，该线程将解析该命令，检查其有效性并执行该命令是否有效.

由于我没有线程方面的丰富经验，也没有网络编程方面的经验，因此我无法想到其他对我有意义的方案.

Since I don't have that much experience with threads and none with network programming, I couldn't think of any other scheme that makes sense to me.

上述方案是否过于复杂? 在尝试以这种方式实现之前，我将不胜感激.

Is the scheme stated above overly complicated? I would appreciate some insight before trying to implement it this way.

(没有客户端)python程序的代码为:

The code for the python program (without the client) is:

主要(这是start()方法):

The main (which is the start() method):

class Controller:
    def __init__(self,listener, executor):
        self.listener = listener
        self.executor = executor


    def start(self):
        while True:
            text = self.listener.listen_for_hotword()

            if self.executor.is_hotword(text):
                text = self.listener.listen_for_command()
                if self.executor.has_matching_command(text):
                    self.executor.execute_command(text)
                else:
                    tts.say("No command found. Please try again")

侦听器(从用户那里获取输入):

The Listener (gets input from the user):

class TextListener(Listener):
    def listen_for_hotword(self):
        text = raw_input("Hotword: ")
        text =' '.join(text.split()).lower()
        return text

    def listen_for_command(self):
        text = raw_input("What would you like me to do: ")
        text = ' '.join(text.split()).lower()
        return text

执行程序(执行给定命令的类):

The executor (the class that executes the given command):

class Executor:
    #TODO: Define default path
    def __init__(self,parser, audio_path='../Misc/audio.wav'):
        self.command_parser = parser
        self.audio_path = audio_path

    def is_hotword(self,hotword):
        return self.command_parser.is_hotword(hotword)

    def has_matching_command(self,command):
        return self.command_parser.has_matching_command(command)

    def execute_command(self,command):
        val = self.command_parser.getCommand(command)
        print val
        val = os.system(val) #So we don't see the return value of the command

命令文件解析器:

class KeyNotFoundException(Exception):
    pass


class YAMLParser:
    THRESHOLD = 0.6

    def __init__(self,path='Configurations/commands.yaml'):
        with open(path,'r') as f:
            self.parsed_yaml = yaml.load(f)

    def getCommand(self,key):
        try:
            matching_command = self.find_matching_command(key)
            return self.parsed_yaml["Commands"][matching_command]
        except KeyError:
            raise KeyNotFoundException("No key matching {}".format(key))

    def has_matching_command(self,key):
        try:
            for command in self.parsed_yaml["Commands"]:
                if jellyfish.jaro_distance(command,key) >=self.THRESHOLD:
                    return True
        except KeyError:
            return False

    def find_matching_command(self,key):
        for command in self.parsed_yaml["Commands"]:
            if jellyfish.jaro_distance(command,key) >=0.5:
                return command

    def is_hotword(self,hotword):
        return jellyfish.jaro_distance(self.parsed_yaml["Hotword"],hotword)>=self.THRESHOLD

示例配置文件:

Commands:
  echo : echo hello


Hotword: start

推荐答案

我很难跟踪您提出问题的背景，但是我会对问题本身进行调查.

I'm finding it extremely difficult to follow the background in your questions, but I'll take a stab at the questions themselves.

正如您在问题中指出的那样，编写行走和口香糖"样式的应用程序的典型方法是以线程或事件循环样式设计代码.

As you noted in your question, the typical way to write a "walk and chew-gum" style app is to design your code in a threaded or an event-loop style.

考虑到您谈论线程和扭曲(这是事件循环样式)，我担心您可能正在考虑将两者混在一起.

Given you talking about threading and Twisted (which is event-loop style) I'm worried that you may be thinking about mixing the two.

我认为它们是根本不同的编程风格(每种语言都有它们擅长的地方)，并且将它们混合通常是通往地狱的途径.

I view them as fundamentally different styles of programming (each with places they excel) and that mixing them is generally a path to hell.

让我给你一些背景来解释

Let me give you some background to explain

• 如何思考这个概念:

• How to think of the concept:

我需要同时完成多项工作，并且我希望操作系统确定如何以及何时运行这些单独的任务.

加号:

Pluses:

• ' '让一个程序同时使用多个处理器内核的方式

• 'The' way to let one program use multiple processor cores at the same time

在posix世界中，让一个进程同时在多个CPU内核上运行的 only 方法是通过线程(典型的理想线程数不再是给定内核的数量)机器)

In the posix world the only way to let one process run on multiple CPU cores at the same time is via threads (with the typical ideal number of threads being no more then the cores in a given machine)

更容易上手

内联运行的相同代码通常可以扔入线程中，而无需重新设计(无需GIL，则需要一些锁定，但稍后需要更多锁定)

The same code that you were running inline can be tossed off into a thread usually without needing a redesign (without GIL some locking would be required but more on that later)

使用任务将更容易使用，这些任务将耗尽您在这些任务上可以提供的所有CPU

Much easier to use with tasks that will eat all the CPU you can give at them

I.E.在大多数情况下，使用线程解决方案然后使用事件/异步框架对数学进行处理是更容易的方法.

I.E. in most cases math is way easier to deal with using threading solutions then using event/async frameworks

缺点:

• Python的线程有特殊问题

• Python has a special problem with threads

在CPython中，全局解释器锁定(GIL)可以使线程无法执行多任务(线程几乎没有用). 避免

In CPython the global interpreter lock(GIL) can negate threads ability to multitask (making threads nearly useless). Avoiding the GIL is messy and can undo all the ease of use of working in threads

在添加线程(和锁)时，事情变得很快复杂，请参见以下内容:线程化最佳实践

As you add threads (and locks) things get complicated fast, see this SO: Threading Best Practices

对于IO/用户交互任务而言极不理想

Rarely optimal for IO/user-interacting tasks

尽管线程非常擅长处理想要使用大量CPU的少量任务(理想情况下，每个内核一个线程)，但对于处理大量花费大量时间等待的任务而言，它们却不是最佳选择.

While threads are very good at dealing with small numbers of tasks that want to use lots of CPU (Ideally one thread per core), they are far less optimal at dealing with large counts of tasks that spend most of their time waiting.

最佳用途:

计算上昂贵的东西.

如果要同时运行大量数学运算，那么与操作系统相比，您不太可能能够更智能地调度CPU利用率.

If you have big chucks of math that you want to run concurrently, its very unlikely that your going to be able to schedule the CPU utilization more intelligently then the operation system.

(鉴于CPython的GIL问题，不应手动将线程用于数学运算，而应使用内部使用线程的库(如NumPy))

( Given CPythons GIL problem, threading shouldn't manually be used for math, instead a library that internally threads (like NumPy) should be used )

• 如何思考这个概念:

• How to think of the concept:

我需要同时执行多项操作，但是我(程序员)想直接控制/直接执行子任务的运行方式和时间

您应该如何看待代码:

How you should be thinking about your code:

考虑所有子任务，将它们交织在一起，就应该始终想到:此代码能否以足够快的速度运行，以至于不会使我正在管理的其他子任务变得无用".

Think of all your sub-tasks in one big intertwined whole, your mind should always have the thought of "will this code run fast enough that it doesn't goof up the other sub-tasks I'm managing"

加号:

Pluses:

• 网络/IO/UI连接非常高效，包括大量连接

• Extraordinarily efficient with network/IO/UI connections, including large counts of connections

事件循环样式程序是解决 c10k 问题.像Twisted这样的框架可以在小型计算机上运行的一个python进程中处理数万个连接.

Event-loop style programs are one of the key technologies that solved the c10k problem. Frameworks like Twisted can literally handle tens-of-thousands of connections in one python process running on a small machine.

随着其他连接/任务的添加，复杂性的可预测(少量)增加

Predictable (small) increase in complexity as other connections/tasks are added

虽然学习曲线相当陡峭(特别是在扭曲状态)，但是一旦了解了基础知识，就可以将新的连接类型添加到项目中，而总的复杂性却只有最小的增加.从提供键盘接口的程序转移到提供键盘/telnet/web/ssl/ssh连接的程序，可能只是每个接口几行粘合代码(...这在框架上是高度可变的，但是无论框架事件循环的复杂度比连接数量增加时的线程更容易预测)

While there is a fairly steep learning curve (particularly in twisted), once the basics are understood new connection types can be added to projects with a minimal increase in the overall complexity. Moving from a program that offers a keyboard interface to one that offers keyboard/telnet/web/ssl/ssh connections may just be a few lines of glue code per-interface (... this is highly variable by framework, but regardless of the framework event-loops complexity is more predictable then threads as connection counts increase)

缺点:

• 更难入门.

• Harder to get started.

事件/异步编程要求与第一行代码不同的设计风格(尽管您会发现设计风格在某种语言之间是可移植的)

Event/async programming requires a different style of design from the first line of code (though you'll find that style of design is somewhat portable from language to language)

一个事件循环，一个核心

One event-loop, one core

尽管事件循环可以让您同时处理大量的IO连接，但它们本身并不能同时在多个内核上运行.解决此问题的常规方法是编写程序，以便可以同时运行该程序的多个实例，每个实例一个实例(此技术绕过了GIL问题)

While event-loops can let you handle a spectacular number of IO connections at the same time, they in-and-of-themselves can't run on multiple cores at the same time. The conventional way to deal with this is to write programs in such a way that multiple instances of the program can be run at the same time, one for each core (this technique bypasses the GIL problem)

复杂的高CPU任务可能很困难

Multiplexing high CPU tasks can be difficult

事件编程需要将高CPU任务切成小块，以便每个块占用(理想地是可预测的)少量CPU，否则，只要运行高CPU任务，事件系统就会停止执行多任务.

Event programing requires cutting high CPU tasks into pieces such that each piece takes an (ideally predictably) small amount of CPU, otherwise the event system ceases to multitask whenever the high CPU task is run.

最佳用途:

基于IO的东西

虽然您的应用程序似乎并不完全基于IO，但似乎都不基于CPU(看起来您当前正在通过system调用播放音频，system每次其应用程序都会独立运行一个进程)调用，因此它的工作不会消耗您的进程CPU-尽管system会阻塞，所以它在扭曲状态下是不行的-您必须在扭曲状态下使用不同的调用.

While your application doesn't seem to be exclusively IO based, none of it seems to be CPU based (it looks like your currently playing audio via a system call, system spins off an independent process every time its called, so its work doesn't burn your processes CPU - though system blocks, so its a no-no in twisted - you have to use different calls in twisted).

您的问题也并不表示您担心要最大化多个内核.

Your question also doesn't suggest your concerned about maxing out multiple cores.

因此，鉴于您专门谈论了Twisted，并且事件循环解决方案似乎是您的应用程序的最佳选择，所以我建议您查看Twisted和-not-线程.

Therefor, given you specifically talked about Twisted, and an event-loop solution seems to be the best match for your application, I would recommend looking at Twisted and -not- threads.

鉴于上面列出的最佳使用"，您可能会想当然地认为混合使用扭曲和线程是很容易的方法，但是如果您这样做，甚至稍作，都会出错禁用事件循环(您将阻塞)和线程(GIL不会让线程执行多任务)的优势，并且具有一些超级复杂的特性，这些特性将无济于事.

Given the 'Best use' listed above you might be tempted to think that mixing twisted and threads is the way-to-got, but when doing that if you do anything even slightly wrong you will disable the advantages of both the event-loop (you'll block) and threading (GIL won't let the threads multitask) and have something super complex that provides you no advantage.

您给出的方案"是:

经过一番思考，除了:

After some thinking I couldn't come up with any other way to implement this other than:

1. 使上面的程序作为进程#1运行(该程序在开始时就在本地运行).
2. Twisted客户端将作为进程#2运行，并从远程客户端接收命令.每当收到命令时，Twisted客户端都会初始化一个线程，该线程将解析该命令，检查其有效性并执行该命令是否有效.

由于我没有太多线程方面的经验，也没有网络编程方面的经验，所以我想不出任何其他对我有意义的方案.

Since I don't have that much experience with threads and none with network programming, I couldn't think of any other scheme that makes sense to me.

在回答方案是否过于复杂"时，我可以肯定地说是肯定的，因为您谈论的是扭曲的和线程. (请参阅tr；上面的博士)

In answer to "Is the scheme ... overly complicated", I would say almost certainly yes because your talking about twisted and threads. (see tr; dr above)

鉴于我对要构建的内容的理解是不完全的(并且很困惑)，我可以想象一个扭曲的解决方案看起来像:

Given my certainly incomplete (and confused) understanding of what you want to build, I would imagine a twisted solution for you would look like:

1. 仔细研究 krondo Twisted Introduction (这确实需要对示例代码进行逐行追踪，行，但如果您进行这项工作，则可以使用它的AWESOME学习工具进行扭曲-以及一般的事件编程)
2. 从头开始，您使用在krondo指南中学到的东西来重新编写"hotword"，这只是提供当前使用的任何界面(键盘?)
3. 在该代码中添加其他通信接口(telnet，web等)，使您可以访问为键盘(?)接口编写的处理代码.

1. Carefully study the krondo Twisted Introduction (it really requires tracing the example code line-for-line, but if you do the work its an AWESOME learning tool for twisted - and event programing in general)
2. From the ground-up you rewrite your 'hotword' thingy in twisted using what you learned in the krondo guide - starting out just providing whichever interface you currently have (keyboard?)
3. Add other communication interfaces to that code (telnet, web, etc) which would let you access the processing code you wrote for the keyboard(?) interface.

如果您在问题中陈述确实需要服务器，则可以编写第二个扭曲程序来提供该服务器(在krondo指南中将看到所有示例).尽管我猜您了解了Twisted的库支持，但您会意识到您不必构建任何额外的服务器，但是您可以在基本代码中仅包含所需的任何协议.

If, as you state in your question, you really need a server, you could write a second twisted program to provide that (you'll see examples of all that in the krondo guide). Though I'm guessing when you understand twisted's library support you'll realize you don't have to build any extra servers, that you can just include whichever protocols you need in your base code.

这篇关于从网络接收数据时如何运行程序并执行代码?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！