在多线程的情况下，httpurlconnection没有给出正确的响应代码 [英] httpurlconnection is not giving proper response code in case of multithreading

问题描述

我想从给定的网址中获取所有网址，并想检查每个网址的状态.

I want to get all urls from given url and I want to check status of each url.

为此，我正在使用ExecutorService进行多线程检查每个URL的状态.

For that I am using ExecutorService for multi-threading to check status of each and every url.

检查url响应代码的类代码如下

Code of class which check response code for url is given below

public class ConnectionTester  implements Callable<Object> {
    private URL url;
    private Map<String,Integer> map;
    private static final Log LOGGER =      
    LogFactoryUtil.getLog(ConnectionTester.class);

    public ConnectionTester(String url,Map<String,Integer> map) {
        try {
            this.url = new URL(url);
            this.map = map;
        } catch (MalformedURLException e) {
            LOGGER.error(e.getMessage(), e);
        }
    }

    /**
     * Get status code of url
     * @return
     */
    public void getStatuscode(URL url) {
        HttpURLConnection http = null;
        try {
            http = (HttpURLConnection)url.openConnection();
            http.setConnectTimeout(0);
            http.setReadTimeout(0);
            http.setRequestMethod("HEAD");
            http.setRequestProperty("User-Agent","Mozilla/5.0 (Windows NT 10.0;    
            Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko)   
            Chrome/56.0.2924.87 Safari/537.36");
            http.connect();
            map.put(url.toString(), http.getResponseCode());
        } catch (IOException e) {
            map.put(url.toString(), 500 );
        }finally{
            if(http !=null)http.disconnect();
        }
    }

    @Override
    public Object call() throws Exception {
        getStatuscode(url);
        return null;

    }
}

然后我使用以下代码检查每个URL的响应代码

And I use following code to check response code for each and every url

ExecutorService service = Executors.newFixedThreadPool(urls.size());
   List<ConnectionTester> connectionTesters = new ArrayList<ConnectionTester>(urls.size());
   Map<String,Integer> map  = new HashMap<String, Integer>();
   for (String string : urls) {
       if(Validator.isNotNull(string) && !string.contains("mailto"))
           connectionTesters.add(new ConnectionTester(string, map));
   }
   service.invokeAll(connectionTesters);

现在的问题是，当我不使用多线程时，我会获得每个URL的正确响应代码，但是当我使用多线程时，我会得到连接超时异常.

Now problem is when I don't use multi-threading I get proper response code of each and every URL but when I use multi-threading I am getting connection timeout exception.

到目前为止，我已经检查并尝试了以下操作.

So far I have checked and tried below things.

1. 我的互联网速度很高
2. 我已将http.setConnectTimeout(0); http.setReadTimeout(0);设置为无限超时.
3. 我正在检查的url响应时间也更少，并且工作正常.

1. My internet speed is high
2. I have set http.setConnectTimeout(0); http.setReadTimeout(0); to set infinite time out.
3. Response time of url I am checking is also less and working fine.

我在这里想念什么?

推荐答案

您正在使服务器超载.当Unix/Linux TCP服务器的 listen(2)积压队列填满时，它将开始忽略传入的连接请求，这会导致客户端的连接超时.

You are overloading the server. When a Unix/Linux TCP server's listen(2) backlog queue fills, it starts ignoring incoming connect requests, which causes connect timeouts at the clients.

解决方案:不.减少线程数，当线程获得连接超时时，不再创建.

Solution: don't. Cut the number of threads, and when a thread gets a connect timeout, don't create any more.

这篇关于在多线程的情况下，httpurlconnection没有给出正确的响应代码的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！