在C中处理100kb数据的有效方法 [英] efficient way to process 100kb of data in C

问题描述

大家好，


我目前正在嵌入式环境中工作，因此资源如同内存一样受限于b $ b。而且我需要将整个文本

文件传递给解析器（在lex / yacc中实现）来解析它。文本文件的最大尺寸是100kb，我不能分配一个缓冲区来保存内存中完整的100 kb数据，以供解析器读取。


我意识到另一个解决方案是将数据存储在temp

文件中并将文件传递给解析器。但是，在我的情况下，我想

避免使用文件方法，因为它性能成本很高。


我知道这比编程问题更多到C，

i决定在这里发布，因为我在C中实现逻辑。任何人都可以帮助建议其他解决方案吗？


提前致谢!!!

解决方案

sa ***** @ yahoo.co.in 写道：

大家好，


我目前正在嵌入式环境中工作，因此资源如同内存一样受限于b $ b。而且我需要将整个文本

文件传递给解析器（在lex / yacc中实现）来解析它。文本文件的最大尺寸是100kb，我不能分配一个缓冲区来保存内存中完整的100 kb数据，以供解析器读取。

为什么要将整个文件存储在内存中？ lex在缓冲模式下读取输入

文件，这意味着文件中只有一小部分是内存中的
。如果您的yacc语法具有递归规则，请确保它们递归递归以最小化内存使用。

我意识到另一个解决方案是数据存储在temp

文件中，并将文件传递给解析器。但是，在我的情况下，我想

避免使用文件方法，因为它性能成本很高。


我知道这比编程问题更多到C，

i决定在这里发布，因为我在C中实现逻辑。任何人都可以帮助建议其他解决方案吗？

使用类似函数式语言的解析器组合器。你可以阅读

http://www.math.chalmers.se/~koen/Pa...r-combo-c.html


介绍此主题在C.它的效率可能会低一些（取决于你的语法），但在内存使用方面效率更高。


a +，ld。

在文章< 11 ********************** @ y80g2000hsf.googlegroups .com> ;,

< sa ***** @ yahoo.co.inwrote：

我目前正在嵌入式环境中工作，因此资源

我意识到另一个解决方案是将数据存储在temp
文件中并将文件传递给解析器。但是，在我的情况下，我想避免使用文件方法，因为它的性能成本很高。

如果数据已存在于文件中，为什么需要将它放入

临时文件中？为什么你不能从原始文件中解析它？


- Richard

-

考虑应在一些字母表中需要多达32个字符

" - X3.4,1963。

>

如果数据已经在一个文件，你为什么要把它放进

a临时文件？为什么你不能从原始文件解析它？

对不起每个人都没有给出完整的图片。

实际上是解析器库提供的是第三方代码，它们的当前实现从缓冲区读取输入，而不是从

文件读取。 （他们重新定义了输入函数以从缓冲区读取

而不是文件）。


这就是为什么我想避免使用临时文件的方法。

了解了这一点，你能否提出任何其他想法或方法。


提前致谢!!!

Hi Everyone,

I''m currently working in embedded environment and hence resources
like memory are constrained. And i''m in a need to pass a entire text
file to a parser (implemented in lex/yacc) to parse it. The maximum
size of the text file is 100kb and i can''t allocate a buffer to hold
the complete 100 kb of data in memory for the parser to read from.

I realise that another solution is to have the data stored in a temp
file and pass the file to the parser. However, in my case, i want to
avoid the file approach as it is performance costly.

I know that this is more of programming question than related to C,
i decided to post here as i''m implementing the logic in C. Can anyone
help suggesting other solutions?

Thanks in advance!!!

解决方案

sa*****@yahoo.co.in wrote:

Hi Everyone,

I''m currently working in embedded environment and hence resources
like memory are constrained. And i''m in a need to pass a entire text
file to a parser (implemented in lex/yacc) to parse it. The maximum
size of the text file is 100kb and i can''t allocate a buffer to hold
the complete 100 kb of data in memory for the parser to read from.

why do you want to store the entire file in memory? lex reads the input
files in buffered mode which means that only a small part of the file is
in memory. if your yacc grammar has recursive rules, ensure that they
left recursive to minimize memory usage.

I realise that another solution is to have the data stored in a temp
file and pass the file to the parser. However, in my case, i want to
avoid the file approach as it is performance costly.

I know that this is more of programming question than related to C,
i decided to post here as i''m implementing the logic in C. Can anyone
help suggesting other solutions?

Using a parser combinators like in functional languages. You can read

http://www.math.chalmers.se/~koen/Pa...r-combo-c.html

for an introduction to this topic in C. It can be less efficient in
speed (depending on your grammar) but more efficient in memory usage.

a+, ld.

In article <11**********************@y80g2000hsf.googlegroups .com>,
<sa*****@yahoo.co.inwrote:

I''m currently working in embedded environment and hence resources
like memory are constrained. And i''m in a need to pass a entire text
file to a parser (implemented in lex/yacc) to parse it. The maximum
size of the text file is 100kb and i can''t allocate a buffer to hold
the complete 100 kb of data in memory for the parser to read from.

I realise that another solution is to have the data stored in a temp
file and pass the file to the parser. However, in my case, i want to
avoid the file approach as it is performance costly.

If the data is already in a file, why would you need to put it into
a temporary file? Why can''t you parse it from the original file?

-- Richard
--
"Consideration shall be given to the need for as many as 32 characters
in some alphabets" - X3.4, 1963.

>
If the data is already in a file, why would you need to put it into
a temporary file? Why can''t you parse it from the original file?

Sorry everybody for not giving a complete picture.
Actually the parser library provided is a third party code and their
current implementation reads the input from a buffer and not from a
file. (they have redefined the input function to read from a buffer
instead of a file).

which is why i want to avoid the temp file approach.
Having understood this, can you suggest any other idea or approach.

Thanks in advance!!!

这篇关于在C中处理100kb数据的有效方法的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！