MiNiFi - NiFi 连接失败:未知主机异常:能够从运行 MiNiFi 的机器 telnet 主机 [英] MiNiFi - NiFi Connection Failure: Unknown Host Exception : Able to telnet host from the machine where MiNiFi is running

问题描述

我在我公司防火墙后面的 Linux Box(网关服务器)中运行 MiNiFi.我的 NiFi 在 AWS EC2 集群上运行(以独立模式运行).

I am running MiNiFi in a Linux Box (gateway server) which is behind my company's firewall. My NiFi is running on an AWS EC2 cluster (running in standalone mode).

我正在尝试将数据从网关发送到在 AWS EC2 中运行的 NiFi.

I am trying to send data from the Gateway to NiFi running in AWS EC2.

从网关，我可以使用公共 DNS 和我在 nifi.properties 文件中配置的远程端口 telnet 到 EC2 节点

From gateway, I am able to telnet to EC2 node with the public DNS and the remote port which I have configured in the nifi.properties file

nifi.properties

# Site to Site properties
nifi.remote.input.host=ec2-xxx.us-east-2.compute.amazonaws.com
nifi.remote.input.secure=false
nifi.remote.input.socket.port=1026
nifi.remote.input.http.enabled=true
nifi.remote.input.http.transaction.ttl=30 sec
nifi.remote.contents.cache.expiration=30 secs

从网关到 NiFi 的 Telnet 连接

iot1@iothdp02:~/minifi/minifi-0.5.0/conf$ telnet ec2-xxx.us-east-2.compute.amazonaws.com 1026
Trying xx.xx.xx.xxx...
Connected to ec2-xxx.us-east-2.compute.amazonaws.com.
Escape character is '^]'.

公共 DNS 正在解析为 EC2 节点的正确公共 IP.

The Public DNS is resolving to the correct Public IP of the EC2 node.

从 EC2 节点，当我在公共 DNS 上执行 nslookup 时，它会返回私有 IP.来自 AWS 文档:公网IP地址通过网络地址转换(NAT)映射到主私网IP地址."

From the EC2 node, when I do nslookup on the Public DNS, it gives back the private IP. From AWS Documentation: "The public IP address is mapped to the primary private IP address through network address translation (NAT). "

因此，我没有在 EC2 节点的/etc/host 文件中添加公共 DNS 和公共 IP.

Hence, I am not adding the Public DNS and the Public IP in /etc/host file in the EC2 node.

从 MiNiFi 方面，我收到以下错误:

From MiNiFi side, I am getting the below error:

minifi-app.log

iot1@iothdp02:~/minifi/minifi-0.5.0/logs$ cat minifi-app.log
2018-11-14 16:00:47,910 INFO [pool-31-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile Repository
2018-11-14 16:00:47,911 INFO [pool-31-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile Repository with 0 records in 0 milliseconds
2018-11-14 16:01:02,334 INFO [Write-Ahead Local State Provider Maintenance] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@67207d8a checkpointed with 0 Records and 0 Swap Files in 20 milliseconds (Stop-the-world time = 6 milliseconds, Clear Edit Logs time = 4 millis), max Transaction ID -1
2018-11-14 16:02:47,911 INFO [pool-31-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile Repository
2018-11-14 16:02:47,912 INFO [pool-31-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile Repository with 0 records in 0 milliseconds
2018-11-14 16:03:02,354 INFO [Write-Ahead Local State Provider Maintenance] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@67207d8a checkpointed with 0 Records and 0 Swap Files in 18 milliseconds (Stop-the-world time = 3 milliseconds, Clear Edit Logs time = 5 millis), max Transaction ID -1
2018-11-14 16:03:10,636 WARN [Timer-Driven Process Thread-8] o.a.n.r.util.SiteToSiteRestApiClient Failed to get controller from http://ec2-xxx.us-east-2.compute.amazonaws.com:9090/nifi-api due to java.net.UnknownHostException: ec2-xxx.us-east-2.compute.amazonaws.com: unknown error
2018-11-14 16:03:10,636 WARN [Timer-Driven Process Thread-8] o.apache.nifi.controller.FlowController Unable to communicate with remote instance RemoteProcessGroup[http://ec2-xxx.us-east-2.compute.amazonaws.com:9090/nifi] due to org.apache.nifi.controller.exception.CommunicationsException: org.apache.nifi.controller.exception.CommunicationsException: Unable to communicate with Remote NiFi at URI http://ec2-xxx.us-east-2.compute.amazonaws.com:9090/nifi due to: ec2-xxx.us-east-2.compute.amazonaws.com: unknown error
2018-11-14 16:04:47,912 INFO [pool-31-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile Repository
2018-11-14 16:04:47,912 INFO [pool-31-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile Repository with 0 records in 0 milliseconds
2018-11-14 16:05:02,380 INFO [Write-Ahead Local State Provider Maintenance] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@67207d8a checkpointed with 0 Records and 0 Swap Files in 25 milliseconds (Stop-the-world time = 8 milliseconds, Clear Edit Logs time = 6 millis), max Transaction ID -1
2018-11-14 16:06:47,912 INFO [pool-31-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile Repository
2018-11-14 16:06:47,912 INFO [pool-31-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile Repository with 0 records in 0 milliseconds
2018-11-14 16:07:02,399 INFO [Write-Ahead Local State Provider Maintenance] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@67207d8a checkpointed with

MiNiFi config.yml

MiNiFi Config Version: 3
Flow Controller:
  name: Gateway-IDS_v0.1
  comment: "1. ConsumeMQTT - MiNiFi will consume mqtt messages in gateway\n2. Remote\
    \ Process Group will send messages to NiFi "
Core Properties:
  flow controller graceful shutdown period: 10 sec
  flow service write delay interval: 500 ms
  administrative yield duration: 30 sec
  bored yield duration: 10 millis
  max concurrent threads: 1
  variable registry properties: ''
FlowFile Repository:
  partitions: 256
  checkpoint interval: 2 mins
  always sync: false
  Swap:
    threshold: 20000
    in period: 5 sec
    in threads: 1
    out period: 5 sec
    out threads: 4
Content Repository:
  content claim max appendable size: 10 MB
  content claim max flow files: 100
  always sync: false
Provenance Repository:
  provenance rollover time: 1 min
  implementation: org.apache.nifi.provenance.MiNiFiPersistentProvenanceRepository
Component Status Repository:
  buffer size: 1440
  snapshot frequency: 1 min
Security Properties:
  keystore: ''
  keystore type: ''
  keystore password: ''
  key password: ''
  truststore: ''
  truststore type: ''
  truststore password: ''
  ssl protocol: ''
  Sensitive Props:
    key:
    algorithm: PBEWITHMD5AND256BITAES-CBC-OPENSSL
    provider: BC
Processors:
- id: 6396f40f-118f-33f4-0000-000000000000
  name: ConsumeMQTT
  class: org.apache.nifi.processors.mqtt.ConsumeMQTT
  max concurrent tasks: 1
  scheduling strategy: TIMER_DRIVEN
  scheduling period: 0 sec
  penalization period: 30 sec
  yield period: 1 sec
  run duration nanos: 0
  auto-terminated relationships list: []
  Properties:
    Broker URI: tcp://localhost:1883
    Client ID: nifi
    Connection Timeout (seconds): '30'
    Keep Alive Interval (seconds): '60'
    Last Will Message:
    Last Will QoS Level:
    Last Will Retain:
    Last Will Topic:
    MQTT Specification Version: '0'
    Max Queue Size: '10'
    Password:
    Quality of Service(QoS): '0'
    SSL Context Service:
    Session state: 'true'
    Topic Filter: MQTT
    Username:
Controller Services: []
Process Groups: []
Input Ports: []
Output Ports: []
Funnels: []
Connections:
- id: f0007aa3-cf32-3593-0000-000000000000
  name: ConsumeMQTT/Message/85ebf198-0166-1000-5592-476a7ba47d2e
  source id: 6396f40f-118f-33f4-0000-000000000000
  source relationship names:
  - Message
  destination id: 85ebf198-0166-1000-5592-476a7ba47d2e
  max work queue size: 10000
  max work queue data size: 1 GB
  flowfile expiration: 0 sec
  queue prioritizer class: ''
Remote Process Groups:
- id: c00d3132-375b-323f-0000-000000000000
  name: ''
  url: http://ec2-xxx.us-east-2.compute.amazonaws.com:9090
  comment: ''
  timeout: 30 sec
  yield period: 10 sec
  transport protocol: RAW
  proxy host: ''
  proxy port: ''
  proxy user: ''
  proxy password: ''
  local network interface: ''
  Input Ports:
  - id: 85ebf198-0166-1000-5592-476a7ba47d2e
    name: From MiNiFi
    comment: ''
    max concurrent tasks: 1
    use compression: false
    Properties:      
        Port: 1026      
        Host Name: ec2-xxx.us-east-2.compute.amazonaws.com
  Output Ports: []
NiFi Properties Overrides: {}

有关如何解决此问题的任何指示?

Any pointers on how to troubleshoot this issue?

推荐答案

在 MiNiFi config.yml 中，我将远程进程组下的 URL 从 http://ec2-xxx.us-east-2.compute.amazonaws.com:9090 到 http://ec2-xxx.us-east-2.compute.amazonaws.com:9090/不错.

In MiNiFi config.yml, I changed the URL under Remote Process Groups from http://ec2-xxx.us-east-2.compute.amazonaws.com:9090 to http://ec2-xxx.us-east-2.compute.amazonaws.com:9090/nifi.

这篇关于MiNiFi - NiFi 连接失败:未知主机异常:能够从运行 MiNiFi 的机器 telnet 主机的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！