[c＃] outofmemory io exception @ streamwriter / binarywriter / strings [英] [ c# ] outofmemory io exception @ streamwriter/binarywriter/strings
问题描述
你好，
我想知道为什么我在下面得到一个OutOfMemory Exception：
基本上我有一个带有~90.000.000个字符的文本框/字符串（或者更多）
而我想做的就是将它们写入文件..
Hello,
I wanted to know why i get a OutOfMemory Exception in the following:
basicly I have a textbox/string with ~ 90.000.000 characters (or more)
And what I want do is writing them to a file..
BinaryWriter writer = new BinaryWriter(new FileStream(AppDomain.CurrentDomain.BaseDirectory + "test.txt", FileMode.Create, FileAccess.Write));
writer.Write(textBox1.Text);
writer.Close();
using (StreamWriter outfile = new StreamWriter(AppDomain.CurrentDomain.BaseDirectory + "test.txt"))
{
outfile.Write(textBox1.Text);
}
所以这样我得到了内存异常。
如果我尝试通过将主字符串分成
subs来减小尺寸..
So on this way I get the memory exceptions.
If I try to reduce the size by splitting the main string into
subs ..
if (textBox1.TextLength >= 28000000)
{
int step_size = 28000000;
int step_count = textBox1.TextLength / step_size;
int cur_begin = 0;
int cur_end = step_size;
textBox4.AppendText("TextBoxCharacterCount: " + textBox1.TextLength.ToString() + Environment.NewLine);
for (int i = 0; i < step_count; i++)
{
string sub = textBox1.Text.Substring(cur_begin, cur_end);
if (i == 0)
{
BinaryWriter writer = new BinaryWriter(new FileStream(AppDomain.CurrentDomain.BaseDirectory + textBox2.Text + ".h", FileMode.Create, FileAccess.Write));
writer.Write(sub);
writer.Close();
}
else
{
BinaryWriter writer = new BinaryWriter(new FileStream(AppDomain.CurrentDomain.BaseDirectory + textBox2.Text + ".h", FileMode.Open, FileAccess.Write));
writer.Write(sub);
writer.Close();
}
cur_begin = cur_end + 1;
cur_end = cur_end + step_size - 1;
}
string end = textBox1.Text.Substring(cur_begin, textBox1.TextLength);
BinaryWriter end_writer = new BinaryWriter(new FileStream(AppDomain.CurrentDomain.BaseDirectory + textBox2.Text + ".h", FileMode.Open, FileAccess.Write));
end_writer.Write(end);
end_writer.Close();
}
else
{
BinaryWriter writer = new BinaryWriter(new FileStream(AppDomain.CurrentDomain.BaseDirectory + textBox2.Text + ".h", FileMode.Create, FileAccess.Write));
writer.Write(textBox1.Text);
writer.Close();
}
这次我仍然得到子记忆异常。
但为什么呢？一个字符串可以在c＃
中包含2.147.483.647个字符，当前大小将远远低于，意味着~28,000.000
和最重要的事实是，如果我想用第一种方法用
~29.000.000写一个字符串，它可以工作，但是如果我试图分开
90.000。 000到28.000.000零件它根本不起作用...
，问候
我尝试过的事情：
在相关问题中排除上述内容：）
I still get the memory exception at sub this time.
But why ? a string can contain 2.147.483.647 characters in c#
and the current size would be far below, means ~ 28.000.000
And the most important fact is, that if i want write a string with
~ 29.000.000 with the very first method, it works , but if i try to split
90.000.000 into 28.000.000 parts it doesnt work at all ...
,greetings
What I have tried:
destribes above in the related question :)
推荐答案
首先，为什么要使用BinaryWriter来编写文本文件？
这样做的代码要少得多，并最终将文件视为文本文件：
First, why are you using BinaryWriter to write a text file?
It's a lot less code to just do this and end up treating the files as text files:
File.WriteAllText("someFilePath.h", TextBox1.Text);
根据上述评论澄清了这个的最终目的：
无需一次读取整个文件。你可以阅读并转换成块。
这应该做你想要的;我用100MB文件测试了它：
After clarifying the ultimate purpose of this as per the comments above:
There's no need to read the whole file at once. You can read and convert it in chunks.
This should do what you want; I tested it with a 100MB file:
string inputFile = @"...";
string outputFile = inputFile + ".txt";
int bufferSize = 50000;
using (var reader = new BinaryReader(new FileStream(inputFile, FileMode.Open, FileAccess.Read)))
{
if (reader.BaseStream.Length > 0)
{
using (var writer = new StreamWriter(outputFile, append: false, encoding: Encoding.ASCII))
{
writer.Write("0x");
writer.Write(reader.ReadByte().ToString("X2"));
byte[] buffer = new byte[bufferSize];
while (true)
{
int remainingBytes = (int)(reader.BaseStream.Length - reader.BaseStream.Position);
int bytesToRead = Math.Min(bufferSize, remainingBytes);
if (bytesToRead == 0)
break;
if (bytesToRead == bufferSize)
reader.Read(buffer, 0, bytesToRead);
else
buffer = reader.ReadBytes(bytesToRead);
string hex = BitConverter.ToString(buffer).Replace("-", ", 0x");
writer.Write(", 0x");
writer.Write(hex);
}
}
}
}
你应该循环使用比它小得多的块。 8KB到64KB之间的东西可能就足够了。
另外，你不应该在每次迭代中流和写。这是非常基本的代码优化。这不是过早的优化，而只是编写高效代码的好习惯。
另外，你应该使用使用创建一次性对象时的语句。你的代码也不例外！
[ ^ ]
然后你有一个错误。在某些情况下，你的代码不会写最后一个部分块。
还有代码重复。你有没有听说过：不要重复自己 - 维基百科，免费的百科全书 [ ^ ]
所以你不应该重复写入数据的代码（3个地方），也不要重复计算目标文件名的代码。此代码很难重复使用。剪切和粘贴编程是一个非常糟糕的习惯。
最后，你没有给出适当的参数string.Substring（Int32，Int32）。如果在编写代码时阅读文档或至少是Intellisense信息是个好主意。
String.Substring方法（Int32，Int32）（系统） [ ^ ]
解决方案2代码将是结构的一个很好的起点，虽然我不确定他是否按照您的预期写入数据。
You should loop with much smaller block than that. Something between 8KB and 64 KB might be adequate.
Also, you should not stream and writer in each iteration. This is very basic code optimization. This is not premature optimization but just a good habit to write efficient code.
Also, you should useusingstatement when creating an object that is disposable. Your code is not exception safe!
Exception safety in C# – more than just trying and catching[^]
Then you have off-by one errors. In some case, your code won't write last partial block.
Also also code duplication. Have you never heard of: Don't repeat yourself - Wikipedia, the free encyclopedia[^]
So you should not repeat code that write data (3 places) and also not repeat code that compute target file name. This code is poorly reusable. Cut and paste programming is a very bad habit.
Finally, you are not given proper argument tostring.Substring(Int32, Int32). If would be a good idea to read document or at least Intellisense information while writing code.
String.Substring Method (Int32, Int32) (System)[^]
Solution 2 code would be a good starting point for the structure although I'm not sure that he write data as you expect it.
这篇关于[c＃] outofmemory io exception @ streamwriter / binarywriter / strings的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
