Java缓存框架，用于维护海量数据 [英] Java Caching frameworks for maintaining huge data
问题描述
用于存储大量数据的Java缓存框架.
Java Caching frameworks for storing huge data.
上下文:我们正在使用Jersey 2.6开发Restful服务，并将其部署在WAS 8.5上.这项服务每天需要处理超过一千万个请求.
Context: We are developing a Restful service using Jersey 2.6 and will deploy it on WAS 8.5. This service need to serve more than 10 million requests per day.
我们需要实现一个缓存来存储超过300k的对象(数据将来自DB).我们需要某种方式每天更新缓存.
We need to implement a cache to store more than 300k object (data will come from DB). And we need some way to update the cache on a daily basis.
- 是否建议使用这种每天缓存300k对象并对其进行更新的方法?
- 有没有支持这种功能的Java框架?
推荐答案
您的问题过于笼统，无法获得明确的答案.您需要描述您要解决的问题.
Your question is too general to get a clear answer. You need to be describe what the problem you are trying to solve is.
- 您是否担心响应时间?
- 您是否要保护数据库免受繁重的负担?
- 是否期望扩大规模并希望确保您可以应付将来的负荷?
此外，一些更多的上下文信息将很有用，尤其是:
Additionally some more contextual information would be useful, especially:
- 与请求相比，您的数据有多动态?
- 平均每天将需要多少百分比的数据? (每天至少要查询一次30万个对象中的多少个?如果您不知道，请提供最佳猜测).
您提供的数据为30万(30万)个数据点和1000万个请求，这意味着您希望平均每天击中每个对象33次，这表明您更关注后端数据库负载，而不是正确的响应最新.
Your figures given as 3 lakh (300k) data points and 10M requests means that you are expecting to hit each object on average 33 times a day, which indicates that you are more concerned about back end DB load than your responses being right up to date.
根据我的经验，有许多相当原始的解决方案会比采用重量级分布式系统(如Mongo，Cassandra或Coherence)要好得多.
In my experience there are a lot of fairly primitive solutions which will work much better than going for a heavyweight distributed systems such as Mongo, Cassandra or Coherence.
我的第一个响应是:保持简单-将30万个对象存储在内部哈希表中并不过分，您每天刷新一次并在首次请求时填充该哈希表.
My first response would be: Keep it simple - 300k objects is not too much to store in an internal hash table which you flush once a day and populate on first request.
如果您需要水平扩展，建议使用1天缓存的Memcache Spymemcached 时间，当您找不到现有条目时填充.
If you need to scale horizontally, I would suggest Memcache Spymemcached with a 1 day cache time, which populate when you don't find an existing entry.
除非您有确凿的理由需要持久存储，否则我不会选择Cassandra或Mongo之类的东西.基本原理:清除可能会变得非常繁重，尤其是在您的数据快速移动的情况下.例如:Cassandra并不真正知道如何删除，而是逻辑删除"删除的条目，这意味着您的数据存储将不断增长，直到您创建清除策略为止.
I would NOT go for something like Cassandra or Mongo unless you have real compelling reasons to require a persistent store. Rationale: Purging can become really onerous, especially if your data is fast moving. For example: Cassandra does not really know how to delete, but instead "tombstones" deleted entries, which means that your data store will simply grow and grow until you create a strategy for purging.
这篇关于Java缓存框架，用于维护海量数据的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
