Web 服务架构:Redis(作为缓存)&用于持久性的 PostgreSQL [英] Web Service Architecture: Redis (as cache) & PostgreSQL for persistence
问题描述
我正在开发一个 Java REST API，它使用来自 postgreSQL 数据库的客户端数据.
数字:.初期约600个客户.他们中的一些人每隔几秒钟就会发出一次请求
因为客户按请求付费，我们需要控制他们的成功请求数量是否达到他们的限制，并且在每次请求后查询postgresql数据(更新'hitsCounter'字段的值)在性能方面很糟糕，我们考虑用redis实现一个缓存系统.
想法:在客户端完成他的第一个请求后，我们从 postgresql 中检索他的数据并将其存储到 redis 缓存中.然后使用这个缓存数据，例如增加 'hitsCounter' 键值，直到客户端停止执行请求.同时，每隔几分钟，后台进程就会将数据从 redis 缓存持久化到 db 表中，因此最后我们将更新后的数据返回到 postgresql，我们可以在未来处理它们.>
我认为它明显提高了性能，但我不确定这个后台进程".一个选项是检查缓存元素的 TTL，如果它小于某个值(这意味着客户端已完成请求)，则保留数据.
我很想听听关于这方面的一些意见.这是一个好主意吗?你知道一些更好的选择吗?
完全合理的想法，但您没有提到您所做的任何测量.具有目标事务级别的目标硬件的瓶颈是什么?不知道，你不能说.
也许您可以使用未记录的表.只需在每个查询中插入一行，然后每 5 分钟汇总一次，清除旧数据.再说一次，使用 HOT 更新，并说 75% 的填充因子也许更新更有效.我不知道(你也不知道)我们没有测量过它.
还不够?将其粘贴到 ssd 上自己的表空间.
还不够?将其粘贴在自己的虚拟机/机器上.
还不够?只需将这些该死的东西写入每个前端框上的平面文件，然后每分钟将数据批处理一次到数据库中.
此外 - 他们每次查询要支付多少费用?您是否在意电源故障并丢失五秒钟的查询日志?您是否需要能够使用原始详细信息和时间戳重现每个查询的收据?
I'm developing a Java REST API that uses client data from a postgreSQL database.
The numbers: . About 600 clients at the beginning . Some of them doing requests every few seconds
Because clients pay per request, we need to control if their number of successful requests reach their limit, and as querying postgresql data (update the value of 'hitsCounter' field) after every request is bad in terms of performance, we are thinking about implementing a cache system with redis.
The idea: After a client does his first request, we retrieve his data from postgresql and store it into redis cache. Then work with this cache-data, for example incrementing the 'hitsCounter' key value, till the client stops doing requests. In parallel, every few minutes a background process persist data from redis cache to db tables, so at the end we have the updated data back to postgresql, and we can deal with them in the future.
I think it obviously increase performance, but I'm not sure about this "background process". An option is to check the TTL of the cache elements and if it's minor than some value (it means client has finished doing requests), persist the data.
I would love to hear some opinions about this. Is this a good idea? Do you know some better alternatives?
Perfectly reasonable idea, but you've not mentioned any measurements you've made. What is the bottleneck in your target hardware with your target transaction levels? Without knowing that, you can't say.
You could use an unlogged table perhaps. Just insert a row with every query, then summarise every 5 minutes, clearing out old data. Then again, with HOT updates, and say 75% fill-factor maybe updates are more efficient. I don't know (and nor do you) we haven't measured it.
Not enough? Stick it on its own tablespace on ssd.
Not enough? Stick it on its own vm/machine.
Not enough? Just write the damn stuff to flat files on each front-end box and batch the data once a minute into the database.
Also - how much are they paying per query? Do you care if power fails and you lose five seconds of query logs? Do you need to be able to reproduce receipts for each query with originating details and a timestamp?
这篇关于Web 服务架构:Redis(作为缓存)&用于持久性的 PostgreSQL的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
