如何通过C行处理C中的文本文件? [英] How do I process a text file in C by chunks of lines?

问题描述

我正在用C编写一个程序，该程序处理文本文件并跟踪每个唯一单词(通过使用具有单词字符数组和出现次数的结构)并将该结构存储到数据结构.但是，分配包括以下内容:整个txt文件可能非常大，无法保存在主内存中.请在您的程序中对此进行说明."

I'm writing a program in C that processes a text file and keeps track of each unique word (by using a struct that has a char array for the word and a count for its number of occurrences) and stores this struct into a data structure. However, the assignment has this included: "The entire txt file may be very large and not able to be held in the main memory. Account for this in your program."

我下课后问他，他说一次一次用X行读取文本文件(我认为是他的建议20,000?)，分析它们并更新结构，直到到达终点为止.文件.

I asked him after class, and he said to read the text file by X lines at a time (I think 20,000 was his suggestion?) at a time, analyze them and update the structs, until you've reached the end of the file.

任何人都可以帮助解释执行此操作的最佳方法，并告诉我要使用哪些功能吗?我对C非常非常陌生.

Can anyone help explain the best way to do this and tell me what functions to use? I'm very, very new to C.

(我当前的程序对于小文件是正确且正确的，我只需要使其容纳大文件即可.)

(my current program is accurate and correct for small files, I just need to make it accommodate enormous files).

非常感谢！！

        fp = fopen(argv[w], "r");
        if ((fp) == NULL){
           fprintf( stderr, "Input file %s cannot be opened.\n", argv[w] );
         return 2;
        }

        /* other parts of my program here */

        char s[MaxWordSize];

        while (fscanf(fp,"%s",s) != EOF){   
            nonAlphabeticDelete(s); // removes non letter characters

            toLowerCase(s); //converts the string to lowercase

            //attempts to add to data structure 
            pthread_mutex_lock(&lock);
            add(words, &q, s);
            pthread_mutex_unlock(&lock);
        }

这行得通，我只需要将其调整为一次遍历文本文件X行即可.

This works, I just need to adjust it to go X lines at a time through the text file.

推荐答案

getline()怎么样? 以下是手册页中的示例 http://man7.org/linux/man -pages/man3/getline.3.html

How about getline() ? Here an example from the manpage http://man7.org/linux/man-pages/man3/getline.3.html

   #define _GNU_SOURCE
   #include <stdio.h>
   #include <stdlib.h>

   int
   main(void)
   {
       FILE *stream;
       char *line = NULL;
       size_t len = 0;
       ssize_t read;

       stream = fopen("/etc/motd", "r");
       if (stream == NULL)
           exit(EXIT_FAILURE);

       while ((read = getline(&line, &len, stream)) != -1) {
           printf("Retrieved line of length %zu :\n", read);
           printf("%s", line);
       }

       free(line);
       fclose(stream);
       exit(EXIT_SUCCESS);
   }

这篇关于如何通过C行处理C中的文本文件?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！