文件大小对于Java太大 [英] file size too big for java

问题描述

我的程序读取各种大小的文本文件.然后，它从文本文件中获取数字，并根据这些数字创建数组列表.我计划使用的最大文件是286,040 KB.当我运行程序并读取文件时，程序停止运行.

My program reads in text files of various sizes. It then takes the numbers from the text file and creates array lists based on the numbers. The largest file i plan on using is 286,040 KB. When I run my program and it reads the file, my program stops working.

我怎么知道我的Java程序可以处理的最大大小?有没有一种方法可以计算我的Java程序可以处理的文件大小?

How do I know what size is the maximum my java program can handle? Is there a way of computing how big a size a file my java program can handle?

另外，关于使我的程序能够容纳如此大的数组列表的最佳建议是什么?但是，我听说过哈希表.我还不能完全理解这个概念.

Also, what are the best suggestions for making my program be able to hold array lists of such a large size? I have heard of hash tables, however; I have not been able to full understand the concept.

根据请求，我要添加文件的上传方式:

Per Request, I'm adding how I upload the file:

String name = getFileName();
Scanner scanDaily = new Scanner(new File(name));

public static String getFileName()
{ //getFileName
    Scanner getName = new Scanner (System.in);
    System.out.println("Please input File Name");
    String fileName = getName.nextLine();
    return fileName;    
}  //getFileName

更新:谢谢那些回答，它非常有帮助

Update : Thank you to those who responded, its been very helpful

新问题

我现在想将文件中的数字读入arraylist

I now want to read the numbers from the file into an arraylist

          String name = getFileName();
    FileReader f= new FileReader(new File(name));
        BufferedReader bf=new BufferedReader(f);
        Scanner sc=new Scanner(bf);

    ArrayList<Double> ID = new ArrayList<Double>();
    ArrayList<Double> Contract = new ArrayList<Double>();
    ArrayList<Double> Date = new ArrayList<Double>();
    ArrayList<Double> Open = new ArrayList<Double>();
    ArrayList<Double> High = new ArrayList<Double>();
    ArrayList<Double> Low = new ArrayList<Double>();
    ArrayList<Double> Close = new ArrayList<Double>();
    ArrayList<Double> Volume = new ArrayList<Double>();

    int rows = 8;
    int counter1 = 0;

    //Update code to prompt user for file
    ArrayList<Double> list = new ArrayList<Double>();

    while (scanDaily.hasNext())
    { //while
        double value = scanDaily.nextDouble();
        DecimalFormat df = new DecimalFormat("#.#####");
        df.format(value);
        list.add(value);
    }  //while

在我使用扫描仪读取文件之前，该扫描仪被命名为scandaily.现在我有了一个文件读取器和一个缓冲读取器，我该使用哪一个读取txt文件?

before I used a scanner to read my file, and that scanner was named scandaily. Now that I have a filereader and a buffered reader, which one do i use to go through my txt file?

推荐答案

当我运行程序并读取文件时，程序停止 工作.

When I run my program and it reads the file, my program stops working.

我以为问题是这样，并在添加代码后得到确认.我以前也遇到过类似的问题.

I thought the problem would be this and confirmed after you have added the code. I have faced the similar problem before.

直接将Scanner与File对象一起使用会引起问题.因为没有缓冲.使用BufferedReader代替.直接将扫描仪与大文件对象一起使用被证明是失败的.因为，我猜这并没有被缓冲.

Use of Scanner directly with File object causing the problem. Because that's not buffered. Use BufferedReader instead. Using scanner with big file object directly proved to be failed. Because, that's not buffered I guess.

Scanner scanDaily = new Scanner(new File(name));  //problematic for big files.

使用BufferedReader代替FileReader.它会根据需要但不立即缓冲文件中的数据.

Use BufferedReader with using FileReader instead of that. It buffers the data from file as is needed but not at once.

示例:

     import java.io.BufferedReader;
     import java.io.FileReader;
     import java.util.Scanner;
     import java.io.File;
     ...............
     FileReader f=new FileReader(new File(fileName));
     BufferedReader bf=new BufferedReader(f);
     Scanner sc=new Scanner(bf);

所以您的代码现在变为:

So your code now becomes:

     String name = getFileName();
     FileReader f= new FileReader(new File(name));
     BufferedReader bf=new BufferedReader(f);
     Scanner sc=new Scanner(bf);

您的程序与扫描仪代码一起挂起，因为它会将您的大文件立即全部加载到内存中，因此会花费一些时间.

Your program hangs with your scanner code because, it is loading your big file all at once into memory and hence taking time.

此外，使我的程序能够做到的最佳建议是什么? 持有这么大的数组列表?我听说过哈希表 然而;我还不能完全理解这个概念.

Also, what are the best suggestions for making my program be able to hold array lists of such a large size? I have heard of hash tables, however; I have not been able to full understand the concept.

在这种情况下，由于文件很大.我建议您使用内存映射文件.因此，您可以将文件映射到内存中，并像访问数组一样使用它来访问文件. 请参阅此链接，了解有关Java中内存映射的信息. /a>

In this case, since the file size is large. I would suggest you using memory mapped file. So, that you can map the file in memory and use it access it like an array. See this link about memory mapping in java.

似乎您已经了解ArrayLists.

我将简要介绍HashMap: HashMap使用键值对存储数据，您具有基于其存储值的键.您将使用密钥存储数据并获取数据.

I will brief about HashMap: HashMap uses key value pair to store the data, you have key based on which the value is stored. You will use key to store the data and get the data.

示例:

          HashMap<KeyType,ValueType> hm=new HashMap<KeyType,ValueType>

因此，您可以将任何类型用作键，并将任何类型用作值.

So this way you can use any type as key and any type as value.

          HashMap<Integer,String> hm = new HashMap<Integer,String>
          hm.set(0,"hello");
          hm.set(5,"bello");

          HashMap<String,String> sm=new HashMap<String,String>
          sm.set("USA","United States of America");
          sm.set("UK","United Kingdom");
          sm.set("IND","India");
          sm.set("AUS","Australia");              

          so, you can query `sm.get("AUS")` to get `"Australia"`,

决定使用哪种数据结构:何时在LinkedList或ArrayList上使用HashMap，反之亦然

To decide which data structure to use: When to use HashMap over LinkedList or ArrayList and vice-versa

我希望这能解决问题.

这篇关于文件大小对于Java太大的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！