在使用Streams API时使用Chronicle Map产生垃圾 [英] Using Chronicle Map producing garbage while using Streams API
问题描述
今天，我正在尝试编年史地图.这是一个代码示例:
Today I was experimenting with Chronicle Map. Here is a code sample:
package experimental;
import net.openhft.chronicle.core.values.IntValue;
import net.openhft.chronicle.map.ChronicleMap;
import net.openhft.chronicle.values.Values;
public class Tmp {
public static void main(String[] args) {
try (ChronicleMap<IntValue, User> users = ChronicleMap
.of(IntValue.class, User.class)
.name("users")
.entries(100_000_000)
.create();) {
User user = Values.newHeapInstance(User.class);
IntValue id = Values.newHeapInstance(IntValue.class);
for (int i = 1; i < 100_000_000; i++) {
user.setId(i);
user.setBalance(Math.random() * 1_000_000);
id.setValue(i);
users.put(id, user);
if (i % 100 == 0) {
System.out.println(i + ". " +
users.values()
.stream()
.max(User::compareTo)
.map(User::getBalance)
.get());
}
}
}
}
public interface User extends Comparable<User> {
int getId();
void setId(int id);
double getBalance();
void setBalance(double balance);
@Override
default int compareTo(User other) {
return Double.compare(getBalance(), other.getBalance());
}
}
}
如您在上面的代码中看到的，我只是创建User对象并将其放置在Chronicle Map中，在第100条记录之后，我仅以最大余额打印User.但不幸的是，它正在产生一些垃圾.当我用VisualVM监视它时，得到以下信息:
As you see in above code I am just creating User object and putting it in Chronicle Map, and after each 100th record I am just printing the User with max balance. But unfortunately it is producing some garbage. When I monitored it with VisualVM I got the following:
似乎在Chronicle Map中使用流仍然会产生垃圾.
It seems using streams in Chronicle Map will produce garbage anyway.
所以我的问题是:
*这是否表示我不应该将Chronicle Map与Streams API一起使用.
*还有其他解决方案/方法吗?
*如何以正确的方式过滤/搜索编年史地图，因为我有除以下情况以外的用例只是将数据放入其中.
So my questions are:
* Does this mean that I should not use Streams API with Chronicle Map.
* Are there any other solutions/ways of doing this?
* How to filter/search Chronicle Map in proper way because I have use cases other than
just putting/getting data in it.
推荐答案
ChronicleMap 的 entrySet().iterator()(以及上的迭代器)实现了keySet()和 values())，以便在迭代之前将编年史地图的段中的所有对象都转储到内存中.
ChronicleMap's entrySet().iterator() (as well as iterator on keySet() and values()) is implemented so that it dumps all objects in a Chronicle Map's segment into memory before iterating over them.
您可以通过调用 map.segments()检查您有多少段.您还可以在ChronicleMap构建阶段对其进行配置，请检出
You can inspect how much segments do you have by calling map.segments(). You could also configure it during the ChronicleMap construction phase, check out ChronicleMapBuilder javadoc.
因此，在迭代过程中，您应该定期将大约 numEntries/numSegments 个条目一次转储到内存 中，其中numEntries是编年史地图的大小.
So, during iteration, you should expect regularly, approximately numEntries / numSegments entries to be dumped into memory at once, where numEntries is the size of your Chronicle Map.
您可以通过分段上下文API通过重用对象，在编年史地图上实现流处理，从而避免创建大量垃圾:
You can implement streaming processing on a Chronicle Map avoiding creating a lot of garbage, by reusing objects, via Segment Context API:
User[] maxUser = new User[1];
for (int i = 0; i < users.segments(); i++) {
try (MapSegmentContext<IntValue, User, ?> c = map.segmentContext(i)) {
c.forEachSegmentEntry((MapEntry<IntValue, User> e) -> {
User user = e.value().get();
if (maxUser[0] == null || user.compareTo(maxUser[0]) > 0) {
// Note that you cannot just assign `maxUser[0] = user`:
// this object will be reused by the SegmentContext later
// in the iteration, and it's contents will be rewritten.
// Check out the doc for Data.get().
if (maxUser[0] == null) {
maxUser[0] = Values.newHeapInstance(User.class);
}
User newMaxUser = e.value().getUsing(maxUser[0]);
// assert the object is indeed reused
assert newMaxUser == maxUser[0];
}
});
}
}
链接到 Data.get() .
The code of the above example is adapted from here.
这篇关于在使用Streams API时使用Chronicle Map产生垃圾的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
