在增加核心数时降级性能 [英] Degrading performance when increasing number of cores

问题描述

  System.out.println（Runtime.getRuntime（）。availableProcessors（） ）; // 16 
  

我运行下面的代码看看利用我的内核的效率。线程'CountFileLineThread'只是计算一个文件中的行数（一个文件夹中有133个文件）

我在这行记录笔记：

  ExecutorService es = Executors.newFixedThreadPool（NUM_CORES）; 
  

其中NUM_CORES介于1到16之间。

从下面的结果中你会注意到，5核以上的性能开始下降。我不会指望6核心及以上的递减收益的产品（btw，对于7核心，需要超过22分钟，你好！）！我的问题是为什么？

>

  public class TestCores 
 {
 public static void main（String args []）throws Exception 
 {
 long start = System.currentTimeMillis（）; 
 System.out.println（START）; 
 
 int NUM_CORES = 1; 
 
 List< File> files = Util.getFiles（/ Users / adhg / Desktop / DEST /）; 
 System.out.println（total files：+ files.size（））; 
 ExecutorService es = Executors.newFixedThreadPool（NUM_CORES）; 
 List< Future< Integer>> futures = new ArrayList< Future< Integer>>（）; 
 for（文件file：files）
 {
 Future< Integer> future = es.submit（new CountFileLineThread（file））; 
 futures.add（future）; 
} 
 
整数total = 0; 
 
 for（Future< Integer> future：futures）
 {
 Integer result = future.get（）; 
 total + = result; 
 System.out.println（result：+ result）; 
 
} 
 
 System.out.println（----->+ total）; 
 
 long end = System.currentTimeMillis（）; 
 System.out.println（END。+（end-start）/1000.0）; 
} 
} 
  

解决方案

I添加这个作为注释，但我要把它作为回答在那里。因为你的测试是做文件I / O，你可能已经在第6个线程，你现在做了太多的I / O，从而减缓了一切。如果你真的想看到你有16个核心的好处，你应该重写你的文件读取线程使用非阻塞I / O。

My mac is armed with 16 cores.

System.out.println(Runtime.getRuntime().availableProcessors());  //16

I'm running the code below to see the effectiveness of utilizing my cores. The thread 'CountFileLineThread' simply count the number of lines in a file (There are 133 files in a folder)

I'm taking notes on this line:

ExecutorService es = Executors.newFixedThreadPool(NUM_CORES);

Where NUM_CORES is between 1 to 16.

You will note from the result below that above 5 cores the performance starts to degrade. I wouldn't expect a 'product of diminishing return' for 6 cores and above (btw, for 7 cores it takes over 22 minutes, hello?!?!) my question is why?

public class TestCores
{   
  public static void main(String args[]) throws Exception
  {
    long start = System.currentTimeMillis();
    System.out.println("START");

    int NUM_CORES = 1;

    List<File> files = Util.getFiles("/Users/adhg/Desktop/DEST/");
    System.out.println("total files: "+files.size());
    ExecutorService es = Executors.newFixedThreadPool(NUM_CORES);
    List<Future<Integer>> futures = new ArrayList<Future<Integer>>();
    for (File file : files)
    {
        Future<Integer> future = es.submit(new CountFileLineThread(file));
        futures.add(future);
    }

    Integer total = 0;

    for (Future<Integer> future : futures)
    {
        Integer result = future.get();
        total+=result;
        System.out.println("result :"+result);

    }

    System.out.println("----->"+total);

    long end = System.currentTimeMillis();
    System.out.println("END. "+(end-start)/1000.0);
}
}

解决方案

I added this as a comment, but I'm going to throw it in there as answer too. Because your test is doing file I/O, you have probably hit a point with that 6th thread where you are now doing too much I/O and thus slowing everything down. If you really want to see the benefit of the 16 cores you have, you should re-write your file reading thread to use non-blocking I/O.

这篇关于在增加核心数时降级性能的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！