非阻塞（异步）DNS在Java中解析 [英] Non-blocking (async) DNS resolving in Java

问题描述

有没有解决异步Java中的DNS查询（按主机名获取IP），非阻塞方式（即状态机的清洁方式，而不是1查询= 1线程 - 我想运行查询数万同时，但不运行的线程数万）？

Is there a clean way to resolve a DNS query (get IP by hostname) in Java asynchronously, in non-blocking way (i.e. state machine, not 1 query = 1 thread - I'd like to run tens of thousands queries simultaneously, but not run tens of thousands of threads)?

我已经发现迄今：

• 标准 InetAddress.getByName（）实施拦截，看起来像标准的Java库，没有任何非阻塞的实现。


• 解析DNS散装问题讨论了类似的问题，却发现唯一的办法就是多线程的方式（即工作的一个线程上的时间每个特定的时刻只有1个查询），这是不是真的可扩展性。


• dnsjava 库也只堵


• 有古无阻塞扩展dnsjava 从2006年拍拖，因此缺乏任何现代的Java并发性的东西，如未来模式的使用和，唉，很有限的队列中的实现。


• dnsjnio 项目也是一个扩展dnsjava，但它也适用于线程模型（即1查询= 1线）


• asyncorg 似乎是最好的解决方案，我发现到目前为止，针对这个问题，但：
  
  
  
  • 这也是从2007年，并期待被遗弃
  
  
  • 几乎没有任何文件/的Javadoc
  
  
  • 使用大量的非标准技术，如娱乐类
  
  
  • Standard InetAddress.getByName() implementation is blocking and looks like standard Java libraries lack any non-blocking implementations.
  • Resolving DNS in bulk question discusses similar problem, but the only solution found is multi-threaded approach (i.e. one thread working on only 1 query in every given moment of a time), which is not really scalable.
  • dnsjava library is also blocking only.
  • There are ancient non-blocking extensions to dnsjava dating from 2006, thus lacking any modern Java concurrency stuff such as Future paradigm usage and, alas, very limited queue-only implementation.
  • dnsjnio project is also an extension to dnsjava, but it also works in threaded model (i.e. 1 query = 1 thread).
  • asyncorg seems to be the best available solution I've found so far targeting this issue, but:
    • it's also from 2007 and looks abandoned
    • lacks almost any documentation/javadoc
    • uses lots of non-standard techniques such as Fun class

    任何其他的想法/实现我错过了？

    Any other ideas/implementations I've missed?

    澄清。我有日志的相当大的（几TB每天）量。每个日志行有可能是从pretty在互联网的任何地方很多主机名，我需要一个IP地址为主机名我进一步的统计计算。线的顺序并不重要，所以，基本上，我的想法是启动2个线程：首先遍历行：

    Clarification. I have a fairly large (several TB per day) amount of logs. Every log line has a host name that can be from pretty much anywhere around the internet and I need an IP address for that hostname for my further statistics calculations. Order of lines doesn't really matter, so, basically, my idea is to start 2 threads: first to iterate over lines:

    
    
    • 读取一行，解析它，获取主机的名称
    
    
    • 将查询发送到DNS服务器来解析指定的主机名，没有为阻止答案
    
    
    • 存放线和DNS查询的套接字句柄在一些缓冲内存
    
    
    • 转到下一行
    
    

    和第二个线程将：

    
    
    • 等待DNS服务器回答任何查询（使用的epoll / 的kqueue 类似技术）
    
    
    • 阅读答案，发现哪一行它在缓冲区是
    
    
    • 与解析的IP写行输出
    
    
    • 继续等待下一个答案
    
    
    • Wait for DNS server to answer any query (using epoll / kqueue like technique)
    • Read the answer, find which line it was for in a buffer
    • Write line with resolved IP to the output
    • Proceed to waiting for the next answer

    在Perl中使用 AnyEvent 的一个简单模型的实现表明我的想法通常是正确的，我可以很容易地实现速度像15-20K查询每秒这种方式（天真阻塞实施得到像每秒2-3查询 - 比较只是着想 - 因此，像4个数量级的差异）。现在我需要在Java中实现相同的 - 我想跳过推出我自己的DNS实施;）

    A simple model implementation in Perl using AnyEvent shows me that my idea is generally correct and I can easily achieve speeds like 15-20K queries per second this way (naive blocking implementation gets like 2-3 queries per second - just the sake of comparison - so that's like 4 orders of magnitude difference). Now I need to implement the same in Java - and I'd like to skip rolling out my own DNS implementation ;)

    推荐答案

    这可能是 Apache目录服务实现在MINA的顶级DNS 的是你在找什么。的Javadoc和其他有用的指南是在该网页上，在左侧栏。

    It may be that the Apache Directory Services implementation of DNS on top of MINA is what you're looking for. The JavaDocs and other useful guides are on that page, in the left-hand side-bar.

    这篇关于非阻塞（异步）DNS在Java中解析的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！