c在插座效率提高的方式＃画面传输 [英] c# screen transfer over socket efficient improve ways
问题描述
这就是我如何写你的美丽的代码（我一些简单的修改更容易理解）
thats how i wrote your beautiful code(some simple changes for me for easier understanding)
private void Form1_Load(object sender, EventArgs e)
{
prev = GetDesktopImage();//get a screenshot of the desktop;
cur = GetDesktopImage();//get a screenshot of the desktop;
var locked1 = cur.LockBits(new Rectangle(0, 0, cur.Width, cur.Height),
ImageLockMode.ReadWrite, PixelFormat.Format32bppArgb);
var locked2 = prev.LockBits(new Rectangle(0, 0, prev.Width, prev.Height),
ImageLockMode.ReadWrite, PixelFormat.Format32bppArgb);
ApplyXor(locked1, locked2);
compressionBuffer = new byte[1920* 1080 * 4];
// Compressed buffer -- where the data goes that we'll send.
int backbufSize = LZ4.LZ4Codec.MaximumOutputLength(this.compressionBuffer.Length) + 4;
backbuf = new CompressedCaptureScreen(backbufSize);
MessageBox.Show(compressionBuffer.Length.ToString());
int length = Compress();
MessageBox.Show(backbuf.Data.Length.ToString());//prints the new buffer size
}
压缩缓冲区长度例如 8294400
和backbuff.Data.length为 8326947
the compression buffer length is for example 8294400
and the backbuff.Data.length is 8326947
推荐答案
我不喜欢压缩的建议，所以。这里就是我会做
I didn't like the compression suggestions, so here's what I would do.
您不想压缩的视频流（所以MPEG，AVI等都是出了问题 - 这些没有要实时），你不希望压缩单个图片（因为这是愚蠢的）。
You don't want to compress a video stream (so MPEG, AVI, etc are out of the question -- these don't have to be real-time) and you don't want to compress individual pictures (since that's just stupid).
基本上你想要做的是什么，检测是否事物的变化送的差异。你在正确的轨道上;大多数视频压缩做到这一点。您还需要一个快速的压缩/解压缩算法;特别是如果你去更多的FPS，这将变得更加重要。
Basically what you want to do is detect if things change and send the differences. You're on the right track with that; most video compressors do that. You also want a fast compression/decompression algorithm; especially if you go to more FPS that will become more relevant.
差异。首先，消除所有分支机构在你的代码，并确保内存访问是连续的（例如迭代中X内环）。后者会给你缓存定位。至于分歧，我可能会使用一个64位的XOR; 。它很容易，网点和快速
Differences. First off, eliminate all branches in your code, and make sure memory access is sequential (e.g. iterate x in the inner loop). The latter will give you cache locality. As for the differences, I'd probably use a 64-bit XOR; it's easy, branchless and fast.
如果你想表现，它可能更好地做到这在C ++：目前C＃实现不向量化你的代码，这将。帮你很大此处
If you want performance, it's probably better to do this in C++: The current C# implementation doesn't vectorize your code, and that will help you a great deal here.
做这样的事情（我假设32位的像素格式）：
Do something like this (I'm assuming 32bit pixel format):
for (int y=0; y<height; ++y) // change to PFor if you like
{
ulong* row1 = (ulong*)(image1BasePtr + image1Stride * y);
ulong* row2 = (ulong*)(image2BasePtr + image2Stride * y);
for (int x=0; x<width; x += 2)
row2[x] ^= row1[x];
}
快速压缩和解压缩通常意味着简单的压缩算法。 https://code.google.com/p/lz4/ 是这样的算法，和有可用于合适的.NET端口，以及。您可能希望它如何工作太阅读;有在LZ4流功能，如果你可以把它处理2图像，而不是1，这将可能给你一个很好的压缩的推动作用。
Fast compression and decompression usually means simpler compression algorithms. https://code.google.com/p/lz4/ is such an algorithm, and there's a proper .NET port available for that as well. You might want to read on how it works too; there is a streaming feature in LZ4 and if you can make it handle 2 images instead of 1 that will probably give you a nice compression boost.
总而言之，如果你正在尝试压缩白噪声，这是行不通的，你的帧速率将下降。解决这个的一种方法是降低颜色如果在一个帧太多随机性。一种随机性措施是熵，而且有几种方法可以得到一个图片的熵的度量（的 https://en.wikipedia.org/wiki/Entropy_（information_theory））。我用一个非常简单的棒：检查所述压缩图像的大小 - 如果它是超过一定的限制，减少比特数;如果下面，增加的比特数。
All in all, if you're trying to compress white noise, it simply won't work and your frame rate will drop. One way to solve this is to reduce the colors if you have too much 'randomness' in a frame. A measure for randomness is entropy, and there are several ways to get a measure of the entropy of a picture ( https://en.wikipedia.org/wiki/Entropy_(information_theory) ). I'd stick with a very simple one: check the size of the compressed picture -- if it's above a certain limit, reduce the number of bits; if below, increase the number of bits.
注意，增加和减少的位不与在这种情况下，变速完成的;您不必删除你的位，你只需要你的压缩，以更好地工作。这可能只是因为不好用一个简单的和有位掩码。例如，如果要删除2位，你可以做这样的：
Note that increasing and decreasing bits is not done with shifting in this case; you don't need your bits to be removed, you simply need your compression to work better. It's probably just as good to use a simple 'AND' with a bitmask. For example, if you want to drop 2 bits, you can do it like this:
for (int y=0; y<height; ++y) // change to PFor if you like
{
ulong* row1 = (ulong*)(image1BasePtr + image1Stride * y);
ulong* row2 = (ulong*)(image2BasePtr + image2Stride * y);
ulong mask = 0xFFFCFCFCFFFCFCFC;
for (int x=0; x<width; x += 2)
row2[x] = (row2[x] ^ row1[x]) & mask;
}
PS：我不知道我会与alpha分量呢，我会离开，你的实验。
PS: I'm not sure what I would do with the alpha component, I'll leave that up to your experimentation.
祝你好运！
长的答案
我有一些空闲时间，所以我只是测试这种方法。下面是一些代码来支持这一切。
I had some time to spare, so I just tested this approach. Here's some code to support it all.
这代码通常与我的笔记本电脑一个漂亮的常量内存压力下运行超过130 FPS，所以瓶颈不应该在这里了。请注意，您需要LZ4得到这个工作，而且LZ4是针对的高速的，不是的高压缩比的的。有点稍后更多。
This code normally run over 130 FPS with a nice constant memory pressure on my laptop, so the bottleneck shouldn't be here anymore. Note that you need LZ4 to get this working and that LZ4 is aimed at high speed, not high compression ratio's. A bit more on that later.
首先，我们需要的东西，我们可以用它来容纳所有我们要发送的数据。我不执行插座本身的东西在这里（虽然这应该是以此作为一个起点很简单），我主要集中在得到你需要送东西过来的数据。
First we need something that we can use to hold all the data we're going to send. I'm not implementing the sockets stuff itself here (although that should be pretty simple using this as a start), I mainly focused on getting the data you need to send something over.
// The thing you send over a socket
public class CompressedCaptureScreen
{
public CompressedCaptureScreen(int size)
{
this.Data = new byte[size];
this.Size = 4;
}
public int Size;
public byte[] Data;
}
我们还需要一个类，将容纳所有的法宝：
We also need a class that will hold all the magic:
public class CompressScreenCapture
{
接下来，如果我运行高性能的代码，我让一个习惯，先预分配所有的缓冲区。这将实际的算法东西时节省您的时间。 1080的4缓冲区约为33 MB，这是很好的 - 所以让我们分配了一个
Next, if I'm running high performance code, I make it a habit to preallocate all the buffers first. That'll save you time during the actual algorithmic stuff. 4 buffers of 1080p is about 33 MB, which is fine - so let's allocate that.
public CompressScreenCapture()
{
// Initialize with black screen; get bounds from screen.
this.screenBounds = Screen.PrimaryScreen.Bounds;
// Initialize 2 buffers - 1 for the current and 1 for the previous image
prev = new Bitmap(screenBounds.Width, screenBounds.Height, PixelFormat.Format32bppArgb);
cur = new Bitmap(screenBounds.Width, screenBounds.Height, PixelFormat.Format32bppArgb);
// Clear the 'prev' buffer - this is the initial state
using (Graphics g = Graphics.FromImage(prev))
{
g.Clear(Color.Black);
}
// Compression buffer -- we don't really need this but I'm lazy today.
compressionBuffer = new byte[screenBounds.Width * screenBounds.Height * 4];
// Compressed buffer -- where the data goes that we'll send.
int backbufSize = LZ4.LZ4Codec.MaximumOutputLength(this.compressionBuffer.Length) + 4;
backbuf = new CompressedCaptureScreen(backbufSize);
}
private Rectangle screenBounds;
private Bitmap prev;
private Bitmap cur;
private byte[] compressionBuffer;
private int backbufSize;
private CompressedCaptureScreen backbuf;
private int n = 0;
首先要做的就是捕捉的屏幕。这是比较容易的部分：只需填写当前屏幕的位图：
First thing to do is capture the screen. This is the easy part: simply fill the bitmap of the current screen:
private void Capture()
{
// Fill 'cur' with a screenshot
using (var gfxScreenshot = Graphics.FromImage(cur))
{
gfxScreenshot.CopyFromScreen(screenBounds.X, screenBounds.Y, 0, 0, screenBounds.Size, CopyPixelOperation.SourceCopy);
}
}
正如我所说，我不希望压缩原始像素。相反，我宁愿压缩之前的异或口罩和当前图像。大多数时候这会给你一大堆的0，这是很容易压缩：
As I said, I don't want to compress 'raw' pixels. Instead, I'd much rather compress XOR masks of previous and the current image. Most of the times this will give you a whole lot of 0's, which is easy to compress:
private unsafe void ApplyXor(BitmapData previous, BitmapData current)
{
byte* prev0 = (byte*)previous.Scan0.ToPointer();
byte* cur0 = (byte*)current.Scan0.ToPointer();
int height = previous.Height;
int width = previous.Width;
int halfwidth = width / 2;
fixed (byte* target = this.compressionBuffer)
{
ulong* dst = (ulong*)target;
for (int y = 0; y < height; ++y)
{
ulong* prevRow = (ulong*)(prev0 + previous.Stride * y);
ulong* curRow = (ulong*)(cur0 + current.Stride * y);
for (int x = 0; x < halfwidth; ++x)
{
*(dst++) = curRow[x] ^ prevRow[x];
}
}
}
}
有关压缩算法，我只是缓冲传递给LZ4并让它做它的魔力。
For the compression algorithm I simply pass the buffer to LZ4 and let it do its magic.
private int Compress()
{
// Grab the backbuf in an attempt to update it with new data
var backbuf = this.backbuf;
backbuf.Size = LZ4.LZ4Codec.Encode(
this.compressionBuffer, 0, this.compressionBuffer.Length,
backbuf.Data, 4, backbuf.Data.Length-4);
Buffer.BlockCopy(BitConverter.GetBytes(backbuf.Size), 0, backbuf.Data, 0, 4);
return backbuf.Size;
}
这里有一点要注意的是，我让一个习惯，把一切的在我的缓冲，我需要通过TCP / IP套接字发送。我不希望移动的数据，如果我可以很容易地避免它，所以我干脆把我需要对对方的任何东西。
One thing to note here is that I make it a habit to put everything in my buffer that I need to send over the TCP/IP socket. I don't want to move data around if I can easily avoid it, so I'm simply putting everything that I need on the other side there.
对于插座本身，你可以在这里使用一个同步TCP套接字（我会），但如果你这样做，你将需要添加额外的缓冲。
As for the sockets itself, you can use a-sync TCP sockets here (I would), but if you do, you will need to add an extra buffer.
唯一令仍然是粘合在一起的一切，并把屏幕上的一些统计数据：
The only thing that remains is to glue everything together and put some statistics on the screen:
public void Iterate()
{
Stopwatch sw = Stopwatch.StartNew();
// Capture a screen:
Capture();
TimeSpan timeToCapture = sw.Elapsed;
// Lock both images:
var locked1 = cur.LockBits(new Rectangle(0, 0, cur.Width, cur.Height),
ImageLockMode.ReadWrite, PixelFormat.Format32bppArgb);
var locked2 = prev.LockBits(new Rectangle(0, 0, prev.Width, prev.Height),
ImageLockMode.ReadWrite, PixelFormat.Format32bppArgb);
try
{
// Xor screen:
ApplyXor(locked2, locked1);
TimeSpan timeToXor = sw.Elapsed;
// Compress screen:
int length = Compress();
TimeSpan timeToCompress = sw.Elapsed;
if ((++n) % 50 == 0)
{
Console.Write("Iteration: {0:0.00}s, {1:0.00}s, {2:0.00}s " +
"{3} Kb => {4:0.0} FPS \r",
timeToCapture.TotalSeconds, timeToXor.TotalSeconds,
timeToCompress.TotalSeconds, length / 1024,
1.0 / sw.Elapsed.TotalSeconds);
}
// Swap buffers:
var tmp = cur;
cur = prev;
prev = tmp;
}
finally
{
cur.UnlockBits(locked1);
prev.UnlockBits(locked2);
}
}
请注意，我减少控制台输出，保证这不是瓶颈。 ： - ）
Note that I reduce Console output to ensure that's not the bottleneck. :-)
简单的改进
这是一个有点浪费压缩所有这些0的，对不对？这是很容易跟踪分钟，并具有使用简单的布尔数据最大y位置。
It's a bit wasteful to compress all those 0's, right? It's pretty easy to track the min and max y position that has data using a simple boolean.
ulong tmp = curRow[x] ^ prevRow[x];
*(dst++) = tmp;
hasdata |= tmp != 0;
您也可能不想叫压缩如果你不就得了。
You also probably don't want to call Compress if you don't have to.
添加此功能后，您会得到这样的事情在屏幕上：
After adding this feature you'll get something like this on your screen:
迭代：0.00S，为0.01s，0.01秒1 KB => 152.0 FPS
Iteration: 0.00s, 0.01s, 0.01s 1 Kb => 152.0 FPS
使用另一种压缩算法也可能有帮助。我坚持LZ4因为它简单易用，它的速度极快，并压缩相当不错 - 不过，也有可能会更好地工作的其他选项。请参见 http://fastcompression.blogspot.nl/ 了比较。
Using another compression algorithm might also help. I stuck to LZ4 because it's simple to use, it's blazing fast and compresses pretty well -- still, there are other options that might work better. See http://fastcompression.blogspot.nl/ for a comparison.
如果你有一个坏的连接，或者如果你的视频流通过远程连接，这一切都将无法工作。最好在这里减少的像素值。这是相当简单：异或双方以前和当前的画面中采用一个简单的64位掩码......你也可以尝试使用索引颜色 - 无论如何，有一吨的不同的东西，你可以尝试在这里;我只是不停地简单，因为这是不够可能是好的。
If you have a bad connection or if you're streaming video over a remote connection, all this won't work. Best to reduce the pixel values here. That's quite simple: apply a simple 64-bit mask during the xor to both the previous and current picture... You can also try using indexed colors - anyhow, there's a ton of different things you can try here; I just kept it simple because that's probably good enough.
您也可以使用的Parallel.For 的XOR循环;个人而言，我真的不关心这个。
You can also use Parallel.For for the xor loop; personally I didn't really care about that.
有点更具挑战性
如果您有服务于多个客户端1服务器，事情会变得更具挑战性的一点，因为他们会以不同的速度刷新。我们希望最快清爽客户机来确定服务器的速度 - 而不是最慢的。 ： - ）
If you have 1 server that is serving multiple clients, things will get a bit more challenging, as they will refresh at different rates. We want the fastest refreshing client to determine the server speed - not slowest. :-)
要实现这一点，之间的关系的上一个和 CUR 必须改变。如果我们只是'异'走喜欢这里，我们将与较慢的客户完全乱码的画面结束。
To implement this, the relation between the prev and cur has to change. If we simply 'xor' away like here, we'll end up with a completely garbled picture at the slower clients.
要解决这个问题，我们不希望交换上一个了，因为它应该持有关键帧（即当压缩数据变得太大，你会刷新）和 CUR 将从'异'的结果持有增量数据。这意味着你基本上可以抓住一个任意'xor'red帧发送过线了 - 只要上一个位图是最近的事。
To solve that, we don't want to swap prev anymore, as it should hold key frames (that you'll refresh when the compressed data becomes too big) and cur will hold incremental data from the 'xor' results. This means you can basically grab an arbitrary 'xor'red frame and send it over the line - as long as the prev bitmap is recent.
这篇关于c在插座效率提高的方式＃画面传输的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
