写文件需要针对繁重的流量第2部分进行优化 [英] Write file need to optimised for heavy traffic part 2
问题描述
如果有兴趣看到我来自哪里，你可以参考第1部分，但没有必要。
For anyone interest to see where I come from you can refer to part 1, but it is not necessary.
以下是我编写的代码片段，用于从代理API中捕获一些财务分析数据。代码将无错误地运行。我需要优化代码，因为在高峰时段，zf_TickEvent方法每秒调用的次数超过10000次。我使用内存流来保存数据，直到达到一定的大小，然后将其输出到文本文件中。
Below is a snippet of code I have written to capture some financial tick data from the broker API. The code will run without error. I need to optimize the code, because in peak hours the zf_TickEvent method will be call more than 10000 times a second. I use a memorystream to hold the data until it reaches a certain size, then I output it into a text file.
代理API只是单线程的。
The broker API is only single threaded.
void zf_TickEvent(object sender, ZenFire.TickEventArgs e)
{
outputString = string.Format("{0},{1},{2},{3},{4}\r\n",
e.TimeStamp.ToString(timeFmt),
e.Product.ToString(),
Enum.GetName(typeof(ZenFire.TickType), e.Type),
e.Price,
e.Volume);
fillBuffer(outputString);
}
public class memoryStreamClass
{
public static MemoryStream ms = new MemoryStream();
}
void fillBuffer(string outputString)
{
byte[] outputByte = Encoding.ASCII.GetBytes(outputString);
memoryStreamClass.ms.Write(outputByte, 0, outputByte.Length);
if (memoryStreamClass.ms.Length > 8192)
{
emptyBuffer(memoryStreamClass.ms);
memoryStreamClass.ms.SetLength(0);
memoryStreamClass.ms.Position = 0;
}
}
void emptyBuffer(MemoryStream ms)
{
FileStream outStream = new FileStream("c:\\test.txt", FileMode.Append);
ms.WriteTo(outStream);
outStream.Flush();
outStream.Close();
}
问题：
-
有什么建议可以让它更快吗？我会尝试改变缓冲区长度但是在代码结构方面，这（几乎）是最快的吗？
Any suggestion to make this even faster? I will try to vary the buffer length but in terms of code structure, is this (almost) the fastest?
当内存流被填满并且我正在清空时它到文件中，新数据会发生什么？我在清空第一个缓冲区时是否需要实现第二个缓冲区来保存数据？或者c＃足够智能搞清楚吗？
When memorystream is filled up and I am emptying it to the file, what would happen to the new data coming in? Do I need to implement a second buffer to hold that data while I am emptying my first buffer? Or is c# smart enough to figure it out?
感谢您的任何建议
推荐答案
最快的方法是让一个（或多个）线程将 byte [] 放入 BlockingCollection 并让一个线程以尽可能快的速度取出项目并将它们写入文件。这样你的制作人和文件写作消费者完全脱钩了。这样你就能承受很高的负荷。
The fastest way to do this is to have one (or more) threads put byte[]'s into a BlockingCollection and have one thread take the items out as fast as it possibly can and write them into a file. That way your producers and the file-writing consumer are totally decoupled. You will be able to sustain very high load doing that.
这篇关于写文件需要针对繁重的流量第2部分进行优化的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
