如何以比此代码更快的速度读取文件，通过更改可以使用类似的缓冲区技术或其他比此更快的速度. [英] how to a read a file faster than this code,by alter can using like buffer technique or anyother as faster than this.

问题描述

FILE* fp;
fp = fopen(strFilePath, "r");
char s1[651];
int iCount = 0;
bool stx_found = false;
while(!feof(fp))
{
  CASSImportRow structCASSImport;
  if(fgets(s1, 651, fp) != NULL)
  {
    if(s1[0] == ''\n'')
      continue;
    CString tempRecord = s1;

    if (address_seq_perfix.Compare("STX") != 0)
    {
      continue;
    }
    stx_found = true;
    structCASSImport.strRegNo   = CORBA::string_dup(tempRecord.Mid(3  , 12)); 
    structCASSImport.strADDR  = CORBA::string_dup((tempRecord.Mid(92 , 35)).TrimRight());
    structCASSImport.strCity    = CORBA::string_dup((tempRecord.Mid(127, 20)).TrimRight());
    structCASSImport.strCountry    = CORBA::string_dup((tempRecord.Mid(161, 20)).TrimRight());
    structCASSImport.strState = CORBA::string_dup((tempRecord.Mid(445, 2 )).TrimRight());
    structCASSImport.strReturnedZIP    = CORBA::string_dup(tempRecord.Mid(447, 5 ));
    structCASSImport.strReturnedZIPExt = CORBA::string_dup(tempRecord.Mid(452, 4 ));
    structCASSImport.strReturnedDPC    = CORBA::string_dup(tempRecord.Mid(456, 2 ));
    structCASSImport.strRCHK       = CORBA::string_dup(tempRecord.Mid(458, 1 ));
    iCount++;
    (*ptrCASSImportIn).length(iCount);
    (*ptrCASSImportIn)[iCount-1] = structCASSImport;
  }
}

fclose(fp);

推荐答案

fgets 可能会非常慢.假设文件读取速度变慢，而不是其他功能，则可以通过使用读取大大提高速度.

使用读取，您将读取大量数据，例如1Mb，

fgets can be a very slow function if you have a very large file. Assuming the file reading is the slowdown and not your other functions, you can improve speed a lot by using fread.

With fread you''d read a chunk of data, for example 1Mb,

FILE* fp = fopen(strFilePath, "r");
char *buffer = malloc(1048576); // Allocate a 1Mb buffer
int read = fread(buffer,1,1048576,fp);

然后扫描内存缓冲区中的换行符以逐行处理它.

and then scan the memory buffer for newline characters to process it line by line.

char sl[651];
int  slcounter = 0;
for(int i=0;i<read;i++){
    if(buffer[i] != ''\n''){
        sl[slcounter] = buffer[i];
        slcounter++;
    }
    else{
        sl[slcounter] = 0;
        slcounter = 0; // reset the character counter for the next string
        // do your regular processing of sl here
    }
}

如果您的文件大于1Mb，请调整缓冲区大小或进行更多次读取，直到到达文件末尾.

If your file is bigger than 1Mb, adjust the buffer size or do more freads until the end of file is reached.

根据我的信息，CORBA :: string_dup用于保留以下值:因此，由于此代码是我工作的一个非常大的项目的一部分，因此，因此在这里需要CORBA :: String_dup.

除了CORBA :: string_dup不会在堆上分配内存的方法之外，您能否建议这样做的其他方法.

我要提供的另一个信息是上面的代码片段正在读取的文件非常大..近10MB的行将近1,50,000行.现在要花10个小时才能读取整个文件.我想减少读取时间.

由于我们正在运行一个非常大的应用程序，因此在这种情况下就需要CORBA.由于变化很大，现在不能更改它并且我们不能转向任何其他技术.因此在这种情况下CORBA是必不可少的.
address_seq_perfix是在本地进行了decalerd的变量，并且在同一函数(在代码中)中进行了设置.使用字符串"STX"进行检查是通过从文件中读取来完成的.并且在651字节副本之前无法进行检查. br/>
在此先感谢
Muthu

As per my information, CORBA::string_dup is used for retaining values of variables within components.So, since this code is a part of a very big project where i work; so CORBA::String_dup is needed here.

Can you suggest any method of doing so other than CORBA::string_dup which won''t allocate memory on the heap.

Another informaton that i want to provide is the file from which the above code snippet is reading is of a very large size..nearly 10MB having nearly 1,50,000 rows.Now it''s taking 10 hours to read the entire file. I want to reduce this read time.

Since we are running a very big application, so CORBA is needed in such cases.It cant be changed now and we cant shift to any other technology since changes would be huge.So CORBA is essential in this case.

The address_seq_perfix is a variable decalerd locally and is set within the same function(that in the code).The checking with the sting "STX" is done by reading from the file.And this cant be checked before the 651 byte copy..

Thanks in Advance
Muthu

这篇关于如何以比此代码更快的速度读取文件，通过更改可以使用类似的缓冲区技术或其他比此更快的速度.的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！