Java ExecutorService-扩展 [英] Java ExecutorService - scaling
问题描述
我正在尝试使用ExecutorService及其功能invokeAll用Java编写程序.我的问题是:invokeAll函数是否可以同时解决任务?我的意思是，如果我有两个处理器，那么会同时有两个工人吗?因为aI无法使其正确缩放.如果我给出newFixedThreadPool(2)或1，则需要花费相同的时间来解决问题.
I am trying to write a program in Java using ExecutorService and its function invokeAll. My question is: does the invokeAll function solve the tasks simultaneously? I mean, if I have two processors, there will be two workers at the same time? Because aI can't make it scale correctly. It takes the same time to complete the problem if I give newFixedThreadPool(2) or 1.
List<Future<PartialSolution>> list = new ArrayList<Future<PartialSolution>>();
Collection<Callable<PartialSolution>> tasks = new ArrayList<Callable<PartialSolution>>();
for(PartialSolution ps : wp)
{
tasks.add(new Map(ps, keyWords));
}
list = executor.invokeAll(tasks);
Map是实现Callable的类，而wp是Partial Solutions的向量，Partial Solutions是在不同时间保存一些信息的类.
Map is a class that implements Callable and wp is a vector of Partial Solutions, a class that holds some information in different times.
为什么不缩放?可能是什么问题?
Why doesn't it scale? What could be the problem?
这是PartialSolution的代码:
This is the code for PartialSolution:
import java.util.HashMap;
import java.util.Vector;
public class PartialSolution
{
public String fileName;//the name of a file
public int b, e;//the index of begin and end of the fragment from the file
public String info;//the fragment
public HashMap<String, Word> hm;//here i retain the informations
public HashMap<String, Vector<Word>> hmt;//this i use for the final reduce
public PartialSolution(String name, int b, int e, String i, boolean ok)
{
this.fileName = name;
this.b = b;
this.e = e;
this.info = i;
hm = new HashMap<String, Word>();
if(ok == true)
{
hmt = new HashMap<String, Vector<Word>>();
}
else
{
hmt = null;
}
}
}
这是Map的代码:
public class Map implements Callable<PartialSolution>
{
private PartialSolution ps;
private Vector<String> keyWords;
public Map(PartialSolution p, Vector<String> kw)
{
this.ps = p;
this.keyWords = kw;
}
@Override
public PartialSolution call() throws Exception
{
String[] st = this.ps.info.split("\\n");
for(int j = 0 ; j < st.length ; j++)
{
for(int i = 0 ; i < keyWords.size() ; i++)
{
if(keyWords.elementAt(i).charAt(0) != '\'')
{
int k = 0;
int index = 0;
int count = 0;
while((index = st[j].indexOf(keyWords.elementAt(i), k)) != -1)
{
k = index + keyWords.elementAt(i).length();
count++;
}
if(count != 0)
{
Word wr = this.ps.hm.get(keyWords.elementAt(i));
if(wr != null)
{
Word nw = new Word(ps.fileName);
nw.nrap = wr.nrap + count;
nw.lines = wr.lines;
int grep = count;
while(grep > 0)
{
nw.lines.addElement(ps.b + j);
grep--;
}
this.ps.hm.put(keyWords.elementAt(i), nw);
}
else
{
Word nw = new Word(ps.fileName);
nw.nrap = count;
int grep = count;
while(grep > 0)
{
nw.lines.addElement(ps.b + j);
grep--;
}
this.ps.hm.put(keyWords.elementAt(i), nw);
}
}
}
else
{
String regex = keyWords.elementAt(i).substring(1, keyWords.elementAt(i).length() - 1);
StringBuffer sb = new StringBuffer(regex);
regex = sb.toString();
Pattern pt = Pattern.compile(regex);
Matcher m = pt.matcher(st[j]);
int count = 0;
while(m.find())
{
count++;
}
if(count != 0)
{
Word wr = this.ps.hm.get(keyWords.elementAt(i));
if(wr != null)
{
Word nw = new Word(this.ps.fileName);
nw.nrap = wr.nrap + count;
nw.lines = wr.lines;
int grep = count;
while(grep > 0)
{
nw.lines.addElement(ps.b + j);
grep--;
}
this.ps.hm.put(keyWords.elementAt(i), nw);
}
else
{
Word nw = new Word(this.ps.fileName);
nw.nrap = count;
int grep = count;
while(grep > 0)
{
nw.lines.addElement(ps.b + j);
grep--;
}
this.ps.hm.put(keyWords.elementAt(i), nw);
}
}
}
}
}
this.ps.info = null;
return this.ps;
}
}
因此，在Map中，我从片段中提取每一行，并搜索每个表达式的出现次数，并且我还保存了行数.在处理完所有片段之后，在同一PartialSolution中，我将信息保存在哈希图中，并返回新的PartialSolution.在下一步中，我将PartialSolutions与相同的fileName合并，并将它们引入与地图相同的Callable类Reduce中，不同之处在于它执行其他操作，但还会返回PartialSolution.
So in Map i take every line from the fragment and search for every expression the number of appearances and i save also the number of line. After i process all the fragment, in the same PartialSolution i save the informations in a hash map and return the new PartialSolution. In the next step i combine the PartialSolutions with the same fileName and introduce them in a Callable class Reduce, who is the same as map, the difference is that it makes other operations, but returns also a PartialSolution.
这是运行Map任务的代码:
This is the code to run the Map tasks:
List<Future<PartialSolution>> list = new ArrayList<Future<PartialSolution>>();
Collection<Callable<PartialSolution>> tasks = new ArrayList<Callable<PartialSolution>>();
for(PartialSolution ps : wp)
{
tasks.add(new Map(ps, keyWords));
}
list = executor.invokeAll(tasks);
在任务中，我创建了Map类型的任务，并在列表中获得了它们.我不知道如何读取JVM线程转储.我希望我给您的信息足够好.如果有帮助，我可以在NetBeans 7.0.1中工作.
In task i create task of type Map and in list i obtain them. I don't know how to read the JVM thread dump. I hope it's good enough what informations i gave you. I work in NetBeans 7.0.1 if that helps.
谢谢你， 亚历克斯
推荐答案
我想知道的是，如果我使用10个线程创建了ExcutorService方法，那么invokeAll方法将同时解决10个任务还是一次解决一个任务?
What i want to know is if the method invokeAll, if i created the ExcutorService with 10 threads, will solve 10 tasks at the same time or will solve one at a time?
如果使用十个线程向ExecutorService提交十个任务，它将同时运行所有任务.他们是否可以完全平行且彼此独立进行取决于他们在做什么.但是它们每个都有各自的线程.
If you submit ten tasks to an ExecutorService with ten threads, it will run them all concurrently. Whether they can proceed completely parallel and independent from each-other depends on what they are doing. But they will each have their own thread.
还有另一个问题，如果我说list.get(i).get()，这将在解决后返回PartialSolution吗?
And another question, if i say list.get(i).get() this will return the PartialSolution after it was solved?
是的，它将阻塞直到计算完成(如果尚未完成)并返回其结果.
Yes, it will block until the computation is done (if not done already) and return its result.
我真的不明白，如果我使用2个线程而不是1个线程，为什么时间没有缩短?
I really don't understand why doesn't the time improves if i use 2 threads instead of 1.
我们需要查看更多代码.它们是否在某些共享数据上同步?这些任务需要多长时间?如果它们很短，您可能不会注意到任何区别.如果它们花费的时间更长，请查看JVM线程转储以验证所有这些线程都在运行.
We need to see more code. Do they synchronize on some shared data? How long do these tasks take? If they are very short, you may not notice any difference. If they take longer, look at the JVM thread dump to verify that all of them are running.
这篇关于Java ExecutorService-扩展的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
