咖啡因缓存使用创建后策略延迟条目过期 [英] Caffeine cache delayed entries expiration with after-creation policy

问题描述

我将 Caffeine lib用于缓存目的，输入后的有效期为1秒创建.事实证明，条目过期会有所延迟，有时可能需要多达2倍的时间才能过期，而不是配置的1秒间隔.根据我的实验，执行程序和调度程序线程计数配置对该延迟没有影响.

I'm using Caffeine lib for caching purposes with 1-sec expiration interval after entry creation. It turned out that entries expire with some delay sometimes it could take up to 2x more time to expire instead of configured 1-sec interval. Based on my experiments executor and scheduler threads count config has no effect on that delay.

在我的测试代码段中，我测量了特定条目在缓存中花费的时间并打印出来:

There is my test snippet where I measure time spent in cache for the particular entry and print it out:

private final Cache<String, LinkedList<Pair<Long, Long>>> groupedEvents = Caffeine.newBuilder()
    .expireAfter(new Expiry<String, LinkedList<Pair<Long, Long>>>() {
        public long expireAfterCreate(String key, LinkedList<Pair<Long, Long>> val, long currentTime) {
            return TimeUnit.SECONDS.toNanos(1);
        }
        public long expireAfterUpdate(String key, LinkedList<Pair<Long, Long>> val, long currentTime, long currentDuration) {
            return currentDuration;
        }
        public long expireAfterRead(String key, LinkedList<Pair<Long, Long>> val, long currentTime, long currentDuration) {
            return currentDuration;
        }
    })
    .scheduler(Scheduler.forScheduledExecutorService(Executors.newScheduledThreadPool(4)))
    .executor(Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors() * 2))
    .removalListener((key, events, cause) -> {
        long timeInCache = System.currentTimeMillis() - events.get(0).getLeft();
        System.out.println(String.format("key %s, values count: %s, timeInCache: %s ms",
                key, events.size(), timeInCache));
    }).build(); 

@Test
public void test() throws InterruptedException {

    for (int i = 0; i < 1000; i++) {
        Thread.sleep(30);
        String key = String.valueOf(new Random().nextInt(30));

        groupedEvents.asMap().compute(key, (s, values) -> {
            if (values == null) {
                values = new LinkedList<>();
            }
            values.add(Pair.of(System.currentTimeMillis(), 123L));
            return values;
        });
    }

输出如下:

key 10, values count: 1, timeInCache: 1223 ms
key 0, values count: 3, timeInCache: 1470 ms
key 20, values count: 1, timeInCache: 2295 ms
key 15, values count: 2, timeInCache: 2325 ms
key 16, values count: 1, timeInCache: 2023 ms
key 23, values count: 1, timeInCache: 1566 ms
key 18, values count: 1, timeInCache: 2052 ms
key 14, values count: 2, timeInCache: 1079 ms
key 3, values count: 3, timeInCache: 1628 ms
key 4, values count: 2, timeInCache: 1109 ms
key 2, values count: 2, timeInCache: 1841 ms
key 17, values count: 1, timeInCache: 1535 ms
key 13, values count: 2, timeInCache: 1011 ms
key 7, values count: 1, timeInCache: 1471 ms
key 12, values count: 1, timeInCache: 1441 ms

如您所见，负载并不高，每秒大约增加33个条目(基于 Thread.sleep(30))，但是对于某些条目，它最多需要2300毫秒而不是希望创建后1秒钟过期，而这种延迟对于我的应用程序至关重要，因为最终用户将不会在1-1.3秒钟的间隔内获取我的数据.

As you see the load is not high and around 33 entry addition per sec (based on the Thread.sleep(30)) but for some entries, it takes up to ~2300ms instead of the desired 1sec to expire after creation and that delay is critical for my application since the end-user will not get my data within 1-1.3 sec interval.

是否有机会调整撤离时间以减少延迟?我正在使用最新的'com.github.ben-manes.caffeine:caffeine:2.8.5''版本

Is there any chance to tune that entry eviction time to reduce the delay? Im using latest 'com.github.ben-manes.caffeine:caffeine:2.8.5'' version

推荐答案

计划执行的时间间隔为〜1.07秒，如 Caffeine.scheduler 中所述:

The scheduled executions are paced at a minimum interval of ~1.07 seconds, as described in Caffeine.scheduler:

指定在基于到期事件调度例行维护时要使用的调度程序.这增加了在正常高速缓存操作期间发生的定期维护，以允许迅速删除过期的条目，而不管当时是否发生任何高速缓存活动.默认情况下，使用{@link Scheduler#disabledScheduler()}.

Specifies the scheduler to use when scheduling routine maintenance based on an expiration event. This augments the periodic maintenance that occurs during normal cache operations to allow for the prompt removal of expired entries regardless of whether any cache activity is occurring at that time. By default, {@link Scheduler#disabledScheduler()} is used.

调整到期事件之间的调度以利用批处理并在短期内最大程度地减少执行.计划执行之间的最小差异是特定于实现的，当前约为1秒(2 ^ 30 ns).此外，提供的调度程序可能不提供实时保证(包括{@link ScheduledThreadPoolExecutor}).该计划是尽力而为的，不能保证何时删除过期条目.

The scheduling between expiration events is paced to exploit batching and to minimize executions in short succession. This minimum difference between the scheduled executions is implementation-specific, currently at ~1 second (2^30 ns). In addition, the provided scheduler may not offer real-time guarantees (including {@link ScheduledThreadPoolExecutor}). The scheduling is best-effort and does not make any hard guarantees of when an expired entry will be removed.

之所以持续时间长，是因为该条目在上次维护运行之后的某个时间到期.这可以导致最大寿命是起搏间隔的2倍，例如当下一个要过期的条目在将来只有1ns时.由于基础系统不提供实时保证，因此扩展了更长的时间，因此在您的示例中又损失了十几毫秒.

The long durations is due to the entry expiring sometime after the last maintenance run. This can result in a maximum lifetime of 2x the pace interval, e.g. when the next entry to expire is just 1ns in the future. This is extended a bit longer because the underlying system doesn't provide real-time guarantees, so it lost another dozen milliseconds in your example.

要提供O(1)到期，高速缓存会将条目保存在

To offer O(1) expiration, the cache holds entries in a timing wheel where the smallest bucket duration is 1.07s. To avoid constantly re-running the maintenance it paces the executions by this minimum threshold. Even if this pacer was removed, the worst-case lifetime would remain because of the bucket size, as an entry ready to be expired must wait for the entire bucket to be evictable.

因此，减少这种最坏情况的唯一方法是使用较短的持续时间来增加车轮.该轮子可能在2 ^ 29ns(530ms)处有2个存储桶，在2 ^ 28ns(268ms)处有4个存储桶，等等.最坏的情况是延迟时间是存储桶的持续时间，因此我们必须确定可以接受的新值.如果您想探索此选项，请打开Github问题.

Therefore the only way to reduce this worst-case would be to add a wheel using a smaller duration. That wheel might have 2 buckets at 2^29ns (530ms), 4 at 2^28ns (268ms), etc. The worst case delay would be the bucket's duration, so we would have to determine what new value is acceptable. Please open a Github issue if you want to explore this option.

这说明了技术细节和可能的改进.但是，如果另一个用户想要更好的分辨率，则这是一种平衡的行为.在极端情况下，这会导致完美的分辨率，需要O(lg n)成本来维持排序顺序，这会降低超大型缓存的性能.当我们尝试平衡设计权衡时，可能会导致某些情况不太适合，最好选择更适合您权衡的方案.

That explains the technical details and a possible improvement. However it is a balancing act if another user wants an even finer resolution. At the extreme this leads a perfect resolution, requiring an O(lg n) cost to maintain a sorted order which degrades performance for very large caches. As we try to balance design tradeoffs, we might cause some scenarios not being a good fit and it is perfectly fine to consider alternatives better suited for your trade-offs.

例如，您可能更喜欢将每个条目插入到 ScheduledExecutorService 中，而不是依赖于Caffeine的到期时间.您的代码将安排在计算中的删除.在最坏的情况下，我的测试运行会延迟13毫秒.

You might, for example, prefer to insert every entry into an ScheduledExecutorService instead of relying on Caffeine's expiration. Your code would schedule the removal in the compute. That has a worst-case delay of an extra 13ms in my test runs.

groupedEvents.asMap().compute(key, (s, values) -> {
  if (values == null) {
    scheduledExecutor.schedule(
      () -> groupedEvents.invalidate(key), 1, TimeUnit.SECONDS);
    values = new LinkedList<>();
  }
  values.add(Pair.of(System.currentTimeMillis(), 123L));
  return values;
});

这篇关于咖啡因缓存使用创建后策略延迟条目过期的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！