在短时间内打开大量连接时的ECONNRESET [英] ECONNRESET when opening a large number of connection in small time period

问题描述

我遇到要在Orion上创建大量实体的情况。我在此docker-compose上使用的是Orion和mongo的docker版本。

I have situation where I want to create large number of entities on orion. I am using docker version of Orion and mongo with this docker-compose.

version: "3"
services:
  mongo:
    image: mongo:3.4
    volumes:
      - /data/docker-mongo/db:/data/db
      - /data/docker-mongo/log/mongodb.log:/var/log/mongodb/mongod.log
    command: --nojournal
  orion:
    image: fiware/orion
    volumes:
      - /data/docker-mongo/log/contextBroker.log:/tmp/contextBroker.log
    links:
      - mongo
    ports:
      - "1026:1026"
    command: -dbhost mongo

现在当我要上传2000个实体时出现问题（打开新对于每个连接，我知道可以做不同的操作，但是现在这是请求的操作），我成功创建了不超过600个（或更少永远不精确的数字）的其余部分，创建失败并出现错误：

Now problems happens when I want to upload 2000 entities (opening new connection for each, I know it can be done different but for now this is request), I successfully create no more than 600 (or less never exact number) of them rest fail to create with error:

"error": {
            "errno": "ECONNRESET",
            "code": "ECONNRESET",
            "syscall": "read"
        },

所以我认为这个问题有待解决使用Orion中的maxConnections，reqPoolSize等设置。但是在docker中，我找不到Orion配置文件，我无法知道何时输入诸如 contextBroker -maxConnections 123456 之类的命令，该设置已被Orion和docker容器接受。

So I assume this issue has something to do with maxConnections, reqPoolSize etc settings in Orion. But in docker I failed to locate Orion config file, I have no way of knowing when I type commands like contextBroker -maxConnections 123456 that setting is being accepted by Orion and docker container.

Orion的日志也为空，当Orion在docker上运行时，我无法确定是什么导致了此问题。

Also log of Orion is empty, and i cannot determined what is causing this issue when Orion is running on docker.

所以主要问题：

• 在docker上运行的Orion是否可以在同一环境中使用Orion在VM上运行的方式（有一些后备功能）


• 当Orion在docker中运行时，如何检查此问题，因为我读了很多文档/问题，但没有运气（或我错过了一些东西。）

如果您有任何建议/建议，那将会很有帮助。
谢谢

If you have some advice/soultion it would really help. Thanks

{
"orion" : {
"version" : "1.13.0-next",
"uptime" : "2 d, 15 h, 46 m, 34 s",
"git_hash" : "ae72acf9e8eeaacaf4eb138f7de37bfee4514c6b",
"compile_time" : "Fri May 4 10:12:18 UTC 2018",
"compiled_by" : "root",
"compiled_in" : "1901fd6bb51a",
"release_date" : "Fri May 4 10:12:18 UTC 2018",
"doc" : "https://fiware-orion.readthedocs.org/en/master/"
}
}



{ Error: socket hang up
at createHangUpError (_http_client.js:313:15)
at Socket.socketOnEnd (_http_client.js:416:23)
at Socket.emit (events.js:187:15)
at endReadableNT (_stream_readable.js:1090:12)
at process._tickCallback (internal/process/next_tick.js:63:19) code: 'ECONNRESET' }


error:
{ Error: connect ECONNREFUSED ipofvirtualm:1026
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1174:14)
 errno: 'ECONNREFUSED',
 code: 'ECONNREFUSED',
 syscall: 'read',
 address: 'ipofvm',
 port: 1026 },

options:
{ method: 'POST',
 uri: 'http://ip:1026/v2/entities?options=keyValues',
 headers:
  { 'Fiware-Service': 'some service',
    'Fiware-ServicePath': 'some servicepath' },
 body:
  { id: 'F0B935',
    type: 'Transaction',
    refEmitter: 'F0B935',
    refReceiver: '7501JXG',
    refCapturer: 'testtdata',
    date: '12/12/2017 13:25',
    refTransferredResources: 'testtdata',
    transferredLoad: 92 },
 json: true,
 callback: [Function: RP$callback],
 transform: undefined,
 simple: true,
 resolveWithFullResponse: false,
 transform2xxOnly: false },

我正在使用请求承诺库进行呼叫，我尝试其他人也遇到了同样的问题。现在，由于我无法向您发送所有2000条回复，因此我将尝试描述一下。因此，当我开始发送此消息时，它就会表现出来。它创建了30个实体，然后再返回几个或更多个返回响应，提示ECONNRESET，然后再次创建，依此类推。

I am using request promise library for making calls, i try others they had same issue. Now since i cannot send u all 2000 responses i will try to describe. So it when i start to send this it behave. It create like 30 entities then next few or more return response saying ECONNRESET then it start creating again and so on.

让我感到困惑的是，它并没有完全失败有效，但不符合预期。同样，猎户座似乎关闭了插座或挂了一段时间，然后又重新打开并正常创建，依此类推。

What confuse me is that it is not failing totally meaning it works but not as intended. Also it seem that Orion close socket or hang it up some period then he is open again and create as normal and so on. If u need any more info ask, and thanks for quick answer.

推荐答案

而不是为每个实体打开新的连接，为什么呢？ t您使用

instead of opening a new connection per entity why don't you use

POST /v2/op/update 

并一次创建所有实体？或几批

and create all entities in just one batch? or a couple of batches

请参见

https://github.com/Fiware/dataModels/blob/master/Weather/WeatherObserved/harvest /spain_weather_observed_harvest.py#L235

这篇关于在短时间内打开大量连接时的ECONNRESET的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！