Java中的非阻塞（异步）DNS解析 [英] Non-blocking (async) DNS resolving in Java

问题描述

是否有一种干净的方法来解决Java异步的DNS查询（以主机名获取IP），以非阻塞的方式（即状态机，不是1个查询= 1个线程） - 我想运行数万个查询同时，但不能运行数以万计的线程）？

Is there a clean way to resolve a DNS query (get IP by hostname) in Java asynchronously, in non-blocking way (i.e. state machine, not 1 query = 1 thread - I'd like to run tens of thousands queries simultaneously, but not run tens of thousands of threads)?

到目前为止我已经发现：

What I've found so far:

• 标准 InetAddress.getByName（）实现是阻止，看起来像标准的Java库缺少任何非阻塞的实现。


• 批量解析DNS 问题讨论类似的问题，但唯一找到的解决方案是多线程方法（即一个线程在每个给定时刻只处理1个查询），这不是真正可扩展的。


• dnsjava 库也只阻止。


• 有从2006年开始的dnsjava的古代非阻塞扩展，因此缺乏任何现代的Java并发内容，如 Future paradigm的用法，唉，非常有限的队列实现。


• dnsjnio 项目也是dnsjava的扩展，但它也适用于线程模型（即1个查询= 1个线程）。


• asyncorg 似乎是我迄今为止针对这个问题找到的最好的解决方案，但是：
  
  
  
  
  • 它也是从2007年，看起来被放弃了
  
  
  • 几乎没有任何文档/ javadoc
  
  
  • 使用许多非标准技术，例如 Fun class
  
  
  • Standard InetAddress.getByName() implementation is blocking and looks like standard Java libraries lack any non-blocking implementations.
  • Resolving DNS in bulk question discusses similar problem, but the only solution found is multi-threaded approach (i.e. one thread working on only 1 query in every given moment of a time), which is not really scalable.
  • dnsjava library is also blocking only.
  • There are ancient non-blocking extensions to dnsjava dating from 2006, thus lacking any modern Java concurrency stuff such as Future paradigm usage and, alas, very limited queue-only implementation.
  • dnsjnio project is also an extension to dnsjava, but it also works in threaded model (i.e. 1 query = 1 thread).
  • asyncorg seems to be the best available solution I've found so far targeting this issue, but:
    • it's also from 2007 and looks abandoned
    • lacks almost any documentation/javadoc
    • uses lots of non-standard techniques such as Fun class

    我错过了任何其他想法/实现？

    Any other ideas/implementations I've missed?

    澄清。我有一个相当大的（每天几TB）的日志量。每个日志行都有一个主机名，可以在互联网上几乎任何地方，我需要一个该主机名的IP地址进行进一步的统计计算。线的顺序并不重要，所以基本上我的想法是开始2个线程：首先迭代线：

    Clarification. I have a fairly large (several TB per day) amount of logs. Every log line has a host name that can be from pretty much anywhere around the internet and I need an IP address for that hostname for my further statistics calculations. Order of lines doesn't really matter, so, basically, my idea is to start 2 threads: first to iterate over lines:

    
    
    • 读取一行，解析它，获取主机名称
    
    
    • 向DNS服务器发送查询以解析给定的主机名，不要阻止回答
    
    
    • 将线路和DNS查询套接字句柄存储在内存中的某个缓冲区中
    
    
    • 转到下一行
    
    
    • Read a line, parse it, get the host name
    • Send a query to DNS server to resolve a given host name, don't block for answer
    • Store the line and DNS query socket handle in some buffer in memory
    • Go to the next line

    第二个线程将：

    
    
    • 等待DNS服务器回答任何查询（使用 epoll / kqueue 喜欢技术）
    
    
    • 阅读答案，找到缓冲区中的哪一行
    • 
      
    • 将已解析IP的行写入输出
    
    
    • 继续等待下一个答案
    
    
    • Wait for DNS server to answer any query (using epoll / kqueue like technique)
    • Read the answer, find which line it was for in a buffer
    • Write line with resolved IP to the output
    • Proceed to waiting for the next answer

    使用 AnyEvent 的Perl中的一个简单的模型实现告诉我，我的想法一般是正确的，我可以轻松实现速度，如每秒15-20K个查询这样（naive blocking imp麻烦每秒可以看到2-3次查询 - 只是为了比较 - 这就像4个数量级的差异）。现在我需要在Java中实现相同的方式 - 我想跳过自己的DNS实现;）

    A simple model implementation in Perl using AnyEvent shows me that my idea is generally correct and I can easily achieve speeds like 15-20K queries per second this way (naive blocking implementation gets like 2-3 queries per second - just the sake of comparison - so that's like 4 orders of magnitude difference). Now I need to implement the same in Java - and I'd like to skip rolling out my own DNS implementation ;)

    推荐答案

    可能的是，MINA之上的 DNS目录服务实施DNS 是您正在寻找的。 JavaDocs和其他有用的指南位于该页面的左侧栏中。

    It may be that the Apache Directory Services implementation of DNS on top of MINA is what you're looking for. The JavaDocs and other useful guides are on that page, in the left-hand side-bar.

    这篇关于Java中的非阻塞（异步）DNS解析的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！