在python和c ++之间管道二进制数据 [英] piping binary data between python and c++
问题描述
我一直在使用Python 2.7的插件QGIS，它工作正常，直到我实际上去做一些图像处理的映射层。即使是简单的任务，如收集栅格图层的RGB值（比如说一个5k x 1k的部分）需要一点（约2分钟）。我可以活着，如果我不得不，但是当我开始计算简单的指标数据（如熵）处理所需的时间爆炸（我停止处理后〜40没有反应分钟）。
I have been working on a plugin to QGIS using Python 2.7 which works fine until I actually go to do some image processing on the mapped layers. Even simple tasks like collecting the RGB values of the raster layer (say a 5k x 1k section) take a bit (~2 minutes). I could live with that if I had to, but when I start calculating simple metrics on the data (like Entropy) the amount of time needed for processing explodes (I stopped the processing after ~40 unresponsive minutes).
我在C ++中编写了熵代码，并且希望能够以这种方式处理数据。在做了一些研究后，我发现Python可以使用stdin和stdout管道数据，并遇到下面的例子淹没在论坛 http://ubuntuforums.org/archive/index.php/t-524072.html
I've written the Entropy code before in C++ and would love to be able to just process the data that way. After doing some research I found that Python can use stdin and stdout to pipe data and came across the following example buried in a forum at http://ubuntuforums.org/archive/index.php/t-524072.html
使用此Python代码作为驱动程序
using this Python code as a driver
import subprocess
proc = subprocess.Popen("C:\Python27\PythonPipes.exe",
stdin=subprocess.PIPE,
stdout=subprocess.PIPE)
state = "run"
while state == "run":
input = raw_input("Message to CPP>> ")
if input == "quit":
state = "terminate" # any string other than "run" will do
proc.stdin.write(input + "\n")
cppMessage = proc.stdout.readline().rstrip("\n")
print "cppreturn message ->" + cppMessage + " written by python \n"
和此c ++代码作为数据处理程序
and this c++ code as a data processer
#include <iostream>
#include <string>
using namespace std;
int main(int argc, char* args[]){
string python_message = "";
bool quit = false;
while (!quit){
cin >> python_message;
if (python_message == "quit"){
quit = true;
}
else if (python_message == "first"){
cout << "First Hello!" << endl;
}
else if (python_message == "second"){
cout << "Second Hello!" << endl;
}
else if (python_message == "third"){
cout << "Third Hello!" << endl;
}
else {
cout << "Huh?" << endl;
}
}
return 0;
}
此代码适用于文本数据。我可以管理文本来回整天与它。但我想要做的是传递二进制数据来回，以便我可以将整数光栅图层数据从python发送到c ++进行处理，然后返回结果。
This code works great for text data. I can pipe text back and forth all day long with it. but what I want to do is pass binary data back and forth so that I can send the integer raster layer data from python to c++ for processing, and then return the results.
我已经环顾四周，尝试了将字节缓冲区放入
I have looked around and tried various combinations of putting a buffer of bytes into
proc.stdin.write(bufferdata)
并使用
fread(Arraypointer, 4,1,stdin)
程序端以及
fwrite(outArraypointer,4,1,stdout)
将管道数据回到python，但nopt已成功。我认为部分问题可能是文本版本使用cin并等待EOL指示器。我不清楚如何在二进制数据的情况下做类似的事情。
to pipe data back to python, but have nopt been successful. I think that part of the problem may be that the text version uses cin and waits for the EOL indicator. I am unclear how to do something similar in the case of Binary data.
我想知道如何修改上面的示例代码从python程序发送一个int到c ++程序。在c ++程序中增加int，然后将该int发送回python程序。请记住，我将要做到这一点与数百万ints一次，以防止警告你的提议的解决方案。
I would like to know how to modify the example code above to send an int from a python program to a c++ program. increment the int in the c++ program, and then send that int back to the python program. please bear in mind that I am going to have to do this with millions of ints at a time in case that caveat ways on your proposed solution.
如果这不工作，我会移动到只有python写出一个二进制文件，c ++代码读入，但我真的想使用
If this doesn't work out I'll move to just having python write out a binary file that the c++ code reads in, but I'd really like to use this approach if it's possible.
感谢您的提前帮助。
strong> Roland的解决方案是我需要的起点。对于那些后来，我上面描述的代码的工作原型如下。这是Roland的
Update Roland's solution was the starting point I needed. For those who come later, a working prototype of the code I described above is below. It is a slight modification of Roland's
Python驱动程序的一个细微修改：
Python Driver:
import subprocess
proc = subprocess.Popen("C:\Python27\PythonPipes.exe",
stdin=subprocess.PIPE,stdout=subprocess.PIPE)
s1 = bytearray(10)
s1[0] = 65 #A
s1[1] = 66 #B
s1[2] = 67 #C
s1[3] = 68 #D
s1[4] = 69 #E
s1[5] = 70 #F
s1[6] = 71 #G
s1[7] = 72 #H
s1[8] = 73 #I
t = buffer(s1)
proc.stdin.write(t)
value = [0,0,0,0,0,0,0,0]
for i in range(8):
value[i] = ord(proc.stdout.read(1))
print "value i -> " + str(value[i])
proc.stdin.write('q')
proc.wait()
C ++处理器
#include <stdio.h>
char increase;
int main(int argc, char **argv) {
for (;;) {
char buf;
fread(&buf, 1, 1, stdin);
if ('q' == buf)
break;
increase = buf + 1;
fwrite(&increase, 1, 1, stdout);
fflush(stdout);
}
return 0;
}
推荐答案
默认情况下，C stdio缓冲写入stdout的所有内容，并且只有在写入换行符（行缓冲）时才将缓冲区刷回到管道中。写入后调用 fflush（stdout）时，问题消失。您还可以通过< stdio.h> 中定义的 setvbuf 函数禁用（或控制）使用 setvbuf（stdout，NULL，_IONBF，0）可完全禁用缓冲。
The problem probably is buffering: by default, the C stdio buffers everything written to stdout and flushes the buffer back into the pipe only when a newline is written (line buffering). The problem disappears when you call fflush(stdout) after writing. You can also disable (or control) buffering via the setvbuf function defined in <stdio.h>, e.g. use setvbuf(stdout, NULL, _IONBF, 0) to disable buffering completely.
具有以下两个程序：
import subprocess
proc = subprocess.Popen("./echotest",
stdin=subprocess.PIPE,
stdout=subprocess.PIPE)
proc.stdin.write('abc')
message = proc.stdout.read(3)
print "return message ->" + message + " written by python \n"
proc.stdin.write('q')
proc.wait()
和一个小C程式：
#include <stdio.h>
int main (int argc, char **argv) {
for (;;) {
char buf;
fread(&buf, 1, 1, stdin);
if ('q' == buf)
break;
fwrite(&buf, 1, 1, stdout);
fflush(stdout);
}
return 0;
}
请注意，您必须指定要从子进程或程序会阻塞，等待更多的输出。如果这让您感到困扰，请尝试其中一个解决方案这个问题。
Note that you have to specify how many bytes you want to read back from the subprocess or the program will block, waiting for more output. If this bothers you, try one of the solutions to this question.
这篇关于在python和c ++之间管道二进制数据的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
