在服务器端缓存 JSON 对象 [英] Caching JSON objects on server side

问题描述

我有一个服务器，其中包含要根据来自移动客户端的 API 请求提供的数据.数据是持久的，更新频率非常低(比如一周一次).但是表的设计比较繁重，导致API请求的服务速度很慢

I have a server which contains the data to be served upon API requests from mobile clients. The data is kind of persistent and update frequency is very low (say once in a week). But the table design is pretty heavy which makes API requests to be served slowly

Web Service 使用 Yii + Postgre SQL 实现.

The Web Service is implemented with Yii + Postgre SQL.

• 使用memcached是解决这个问题的方法吗?如果是，如果缓存数据变脏，我该如何管理?
• 对此有任何替代解决方案吗?Postgre 有什么内置机制，比如 MySQL 中的 MEMORY?
• Redis 怎么样?

推荐答案

你可以使用 memcached，但同样每个人都会攻击你的数据库服务器.在您的情况下，您是说查询结果有点持久，因此缓存来自您的 Web 服务的 JSON 响应可能更有意义.

You could use memcached, but again everybody would hit you database server. In your case, you are saying the query results are kind of persistent so it might make more sense to cache the JSON responses from your Web Service.

这可以使用带有内置缓存的反向代理来完成.我想一个例子可能对你最有帮助，我们如何使用 Jetty (Java) 和 NGINX:

This could be done using a Reverse Proxy with a built in cache. I guess an example might help you the most how we do it with Jetty (Java) and NGINX:

在我们的设置中，我们有一个 Jetty (Java) 实例，为我们的移动客户端提供 API.API 正在侦听 localhost:8080/api 并返回从本地 Mysql 数据库上的某些查询中获取的 JSON 结果.

In our setup, we have a Jetty (Java) instance serving an API for our mobile clients. The API is listening on localhost:8080/api and returning JSON results fetched from some queries on a local Mysql database.

此时，我们可以直接向我们的客户提供 API，但是反向代理来了:

At this point, we could serve the API directly to our clients, but here comes the Reverse Proxy:

在 API 前面有一个 NGINX 网络服务器，从 0.0.0.0:80/(到处都是，端口 80)监听当移动客户端连接到 0.0.0.0:80/api 时，内置的反向代理会尝试从其缓存中获取准确的查询字符串.如果失败，它会从 localhost:8080/api 获取它，将它放入它的缓存中并提供在缓存中找到的新值.

In front of the API sits an NGINX webserver listening from 0.0.0.0:80/ (everywhere, port 80) When a mobile client connects to 0.0.0.0:80/api the built-in Reverse Proxy tries to fetch the exact query string from it's cache. If this fails, it fetches it from localhost:8080/api, puts it in it's cache and serves the new value found in the cache.

优点:

• 您可以使用其他 NGINX 优点:缓存 JSON 文件的自动 GZIP 压缩
• 在 NGINX 处终止 SSL 端点.
• NGINX Worker 可能会让您受益，当您有更多连接时，所有连接都从缓存中请求数据.
• 您可以整合您的服务端点

考虑缓存失效:

您必须考虑缓存失效.您可以告诉 NGINX 保留它的缓存，例如，对于 localhost:8080/api 的所有 HTTP 200 请求，保留一周，或为所有其他 HTTP 状态代码保留 1 分钟.但是如果有时间，你想在一周内更新 API，缓存是无效的，所以你必须以某种方式删除它或将缓存时间调低到一个小时或一天(这样大多数人会点击缓存).

You have to think about cache-invalidation. You can tell NGINX to hold on it's cache, say, for a week for all HTTP 200 request for localhost:8080/api, or 1 minute for all other HTTP status codes. But if there comes the time, where you want to update the API in under a week, the cache is invalid, so you have to delete it somehow or turn down the caching time to an hour or day (so that most people will hit the cache).

这就是我们所做的:我们选择在缓存脏时删除它.我们在服务器上运行另一个 JOB，监听通过 Puppet 触发的 Update-API 事件.JOB 将负责为我们清除 NGINX 缓存.

This is what we do: We chose to delete the cache, when it is dirty. We have another JOB running on the Server listening to an Update-API event triggered via Puppet. The JOB will take care of clearing the NGINX cache for us.

另一个想法是在您的 Web 服务中添加清除缓存功能.我们决定反对这个解决方案的原因是:Web 服务必须知道它在反向代理之后运行，这打破了关注点分离.但我想说，这取决于你的计划.

Another idea would be to add the clearing cache function inside your Web Service. The reason we decided against this solution is: The Web Service would have to know it runs behind a reverse proxy, which breaks separation of concerns. But I would say, it depends on what you are planning.

另一件让您的 Web 服务更正确的事情是为每个 JSON 文件提供正确的 ETAG 和缓存过期标头.同样，我们没有这样做，因为我们有一个大的更新事件，而不是每个文件的小更新事件.

Another thing, which would make your Web Service more right would be to serve correct ETAG and cache-expires headers with each JSON file. Again, we did not do that, because we have one big Update Event, instead of small ones for each file.

附注:

• 你不必使用 NGINX，但它真的很容易配置
• NGINX 和 Apache 支持 SSL
• 还有著名的反向代理(https://www.varnish-cache.org)，但据我所知它不支持 SSL(还没有?)

• You do not have to use NGINX, but it really easy to configure
• NGINX and Apache have SSL support
• There is also the famous Reverse Proxy (https://www.varnish-cache.org), but to my knowledge it does not do SSL (yet?)

因此，如果您要在 Web Service + SSL 前面使用 Varnish，您将使用如下配置:NGINX -> Varnish -> Web 服务.

So, if you were to use Varnish in front of your Web Service + SSL, you would use a configuration like: NGINX -> Varnish -> Web Service.

参考资料:- NGINX 服务器:http://nginx.com- 清漆反向代理:https://www.varnish-cache.org- Puppet IT 自动化:https://puppetlabs.com- NGINX反向代理教程:http://www.cyberciti.biz/faq/howto-linux-unix-setup-nginx-ssl-proxy/ http://www.cyberciti.biz/tips/using-nginx-as-reverse-proxy.html

References: - NGINX server: http://nginx.com - Varnish Reverse Proxy: https://www.varnish-cache.org - Puppet IT Automation: https://puppetlabs.com - NGINX reverse proxy tutorial: http://www.cyberciti.biz/faq/howto-linux-unix-setup-nginx-ssl-proxy/ http://www.cyberciti.biz/tips/using-nginx-as-reverse-proxy.html

这篇关于在服务器端缓存 JSON 对象的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！