使用 Elastic Search 5.5.0 时如何正确关闭 Raw RestClient 以获得最佳性能? [英] How to Properly Close Raw RestClient When Using Elastic Search 5.5.0 for Optimal Performance?
问题描述
我使用 Spring Boot 1.5.4.RELEASE 微服务通过 ElasticSearch 提供的低级 Rest Client 连接到 ElasticSearch 5.5.0 实例.
Am using a Spring Boot 1.5.4.RELEASE Microservice to connect to an ElasticSearch 5.5.0 instance using the low level Rest Client that ElasticSearch provides.
pom.xml
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.5.4.RELEASE</version>
</parent>
<dependencies>
<!-- Spring -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<!-- Elasticsearch -->
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>5.5.0</version>
</dependency>
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>transport</artifactId>
<version>5.5.0</version>
</dependency>
<!-- Apache Commons -->
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>3.6</version>
</dependency>
<!-- Jackson -->
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-core</artifactId>
<version>2.8.9</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.8.9</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-annotations</artifactId>
<version>2.8.9</version>
</dependency>
<!-- Log4j -->
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>
<!-- JUnit -->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
<scope>test</scope>
</dependency>
<!-- Swagger -->
<dependency>
<groupId>io.springfox</groupId>
<artifactId>springfox-swagger2</artifactId>
<version>2.6.1</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>io.springfox</groupId>
<artifactId>springfox-swagger-ui</artifactId>
<version>2.6.1</version>
<scope>compile</scope>
</dependency>
</dependencies>
一切都设置正确，但经过一系列点击后，客户端应用程序报告了 HTTP 500 错误，这就是日志文件中出现的内容:
Everything is setup correctly but after a bunch of hits, client apps were reporting an HTTP 500 error and this is what appeared in the log files:
java.io.IOException: Too many open files
at sun.nio.ch.IOUtil.makePipe(Native Method) ~[na:1.8.0_141]
at sun.nio.ch.EPollSelectorImpl.<init>(EPollSelectorImpl.java:65) ~[na:1.8.0_141]
at sun.nio.ch.EPollSelectorProvider.openSelector(EPollSelectorProvider.java:36) ~[na:1.8.0_141]
at java.nio.channels.Selector.open(Selector.java:227) ~[na:1.8.0_141]
at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.<init>(AbstractMultiworkerIOReactor.java:142) ~[httpcore-nio-4.4.5.jar!/:4.4.5]
at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.<init>(DefaultConnectingIOReactor.java:79) ~[httpcore-nio-4.4.5.jar!/:4.4.5]
at org.apache.http.impl.nio.client.IOReactorUtils.create(IOReactorUtils.java:43) ~[httpasyncclient-4.1.3.jar!/:4.1.3]
at org.apache.http.impl.nio.client.HttpAsyncClientBuilder.build(HttpAsyncClientBuilder.java:666) ~[httpasyncclient-4.1.3.jar!/:4.1.3]
at org.elasticsearch.client.RestClientBuilder.createHttpClient(RestClientBuilder.java:202) ~[rest-5.5.0.jar!/:5.5.0]
at org.elasticsearch.client.RestClientBuilder.build(RestClientBuilder.java:180) ~[rest-5.5.0.jar!/:5.5.0]
at com.myapp.controller.SearchController.getSearchQueryResults(SearchController.java:94) ~[classes!/:1.0]
在 SearchController 内部(//注释之后的第二行是第 94 行):
Inside SearchController (the second line after the // comment is line 94):
@RestController
@RequestMapping("/api/v1")
public class SearchController {
@RequestMapping(value = "/search", method = RequestMethod.GET, produces="application/json" )
public ResponseEntity<Object> getSearchQueryResults(@RequestParam(value = "criteria") String criteria) throws IOException {
// Setup HTTP Headers
HttpHeaders headers = new HttpHeaders();
headers.add("Content-Type", "application/json");
// Setup RestClient
RestClient restClient = RestClient.builder(new HttpHost("localhost", 9200))
.setRequestConfigCallback(new RestClientBuilder.RequestConfigCallback() {
@Override
public RequestConfig.Builder customizeRequestConfig(RequestConfig.Builder requestConfigBuilder) {
return requestConfigBuilder.setConnectTimeout(5000).setSocketTimeout(60000);
}
}).setMaxRetryTimeoutMillis(60000).build();
// Setup query and send and return ResponseEntity...
}
}
很明显，它在调用 restClient.performRequest() 方法后从未关闭...
Its obvious that what it was never closed after calling the restClient.performRequest() method...
所以，我把它放到我的代码中:
So, I put this into my code:
Response response = null;
try {
// Submit Query and Obtain Response
response = restClient.performRequest("POST", endPoint, Collections.singletonMap("pretty", "true"), entity);
}
catch (IOException e) {
LOG.error("
Exception: " + e + "
");
e.printStackTrace();
}
finally {
restClient.close();
}
阅读 Elastic Search 的文档，RestClient 类是线程安全的...
Read on Elastic Search's documentation that the RestClient class is thread-safe...
另外，请阅读 restClient.performRequestAsync() 方法，但对线程有些缺乏经验，并且文档中的描述含糊不清.
Also, read about the restClient.performRequestAsync() method but am somewhat inexperienced with threads and the description inside the documentation is vague.
问题:
我的解决方案是处理和关闭一堆套接字资源的最佳方式吗?
Is my solution the best way to handle and close a bunch of socket resources?
如果有人能告诉我一种更好的方法来使用低级别的 RestClient 和 Elastic Search，我会很感激，因为它不会导致与未释放套接字资源导致 HTTP 500 相同的问题.应该我正在使用 restClient.performRequestAsync?有人可以提供一个例子吗?
Would appreciate if someone could show me a better way to use the low level RestClient with Elastic Search in sense that it won't cause the same issue with the socket resources not being freed resulting in an HTTP 500. Should I be using restClient.performRequestAsync? Could someone please provide an example?
感谢您花时间阅读本文...
Thank you for taking the time to read this...
推荐答案
为每个请求都创建一个 RestClient 并不是一个好习惯.您应该通过如下所示的配置 bean 创建单个实例:
It's is not a good practice to create a RestClient on every single request. You should create a single instance via a configuration bean like the one below:
@Configuration
public class ElasticsearchConfig {
@Value("${elasticsearch.host}")
private String host;
@Value("${elasticsearch.port}")
private int port;
@Bean
public RestClient restClient() {
return RestClient.builder(new HttpHost(host, port))
.setRequestConfigCallback(new RestClientBuilder.RequestConfigCallback() {
@Override
public RequestConfig.Builder customizeRequestConfig(RequestConfig.Builder requestConfigBuilder) {
return requestConfigBuilder.setConnectTimeout(5000).setSocketTimeout(60000);
}
}).setMaxRetryTimeoutMillis(60000).build();
}
}
然后在您的 SearchController 类中，您可以像这样注入它(并添加一个清理方法以在您的容器出现故障时关闭 restClient 实例):
And then in your SearchController class you can inject it like this (and also add a cleanup method to close the restClient instance when your container goes down):
@RestController
@RequestMapping("/api/v1")
public class SearchController {
@Autowired
private RestClient restClient;
@RequestMapping(value = "/search", method = RequestMethod.GET, produces="application/json" )
public ResponseEntity<Object> getSearchQueryResults(@RequestParam(value = "criteria") String criteria) throws IOException {
// Setup HTTP Headers
HttpHeaders headers = new HttpHeaders();
headers.add("Content-Type", "application/json");
// Setup query and send and return ResponseEntity...
Response response = this.restClient.performRequest(...);
}
@PreDestroy
public void cleanup() {
try {
logger.info("Closing the ES REST client");
this.restClient.close();
} catch (IOException ioe) {
logger.error("Problem occurred when closing the ES REST client", ioe);
}
}
}
这篇关于使用 Elastic Search 5.5.0 时如何正确关闭 Raw RestClient 以获得最佳性能?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
