告诉urllib2使用自定义DNS [英] Tell urllib2 to use custom DNS
问题描述
我想告诉 urllib2.urlopen
(或定制开启者)使用 127.0.0.1
(或 :: 1
)来解析地址。我不会更改我的 /etc/resolv.conf
。
I'd like to tell urllib2.urlopen
(or a custom opener) to use 127.0.0.1
(or ::1
) to resolve addresses. I wouldn't change my /etc/resolv.conf
, however.
一个可能的解决方案是使用工具像 dnspython
查询地址和 httplib
来构建一个自定义url开启器。我喜欢告诉 urlopen
来使用自定义名称服务器。任何建议?
One possible solution is to use a tool like dnspython
to query addresses and httplib
to build a custom url opener. I'd prefer telling urlopen
to use a custom nameserver though. Any suggestions?
推荐答案
看起来像名称解析最终由 socket.create_connection
。
Looks like name resolution is ultimately handled by socket.create_connection
.
-> urllib2.urlopen
-> httplib.HTTPConnection
-> socket.create_connection
虽然一旦设置了Host:头,您可以解析主机和传递IP地址到开启者。
Though once the "Host:" header has been set, you can resolve the host and pass on the IP address through down to the opener.
我建议您将 httplib.HTTPConnection
并将 connect
方法包含在之前修改
self.host
,然后再传递给 socket.create_connection
。
I'd suggest that you subclass httplib.HTTPConnection
, and wrap the connect
method to modify self.host
before passing it to socket.create_connection
.
然后子类 HTTPHandler
(和 HTTPSHandler
)替换 http_open
方法,其中一个将 HTTPConnection
而不是httplib自己的方法传递给 do_open
。
Then subclass HTTPHandler
(and HTTPSHandler
) to replace the http_open
method with one that passes your HTTPConnection
instead of httplib's own to do_open
.
像这样:
import urllib2
import httplib
import socket
def MyResolver(host):
if host == 'news.bbc.co.uk':
return '66.102.9.104' # Google IP
else:
return host
class MyHTTPConnection(httplib.HTTPConnection):
def connect(self):
self.sock = socket.create_connection((MyResolver(self.host),self.port),self.timeout)
class MyHTTPSConnection(httplib.HTTPSConnection):
def connect(self):
sock = socket.create_connection((MyResolver(self.host), self.port), self.timeout)
self.sock = ssl.wrap_socket(sock, self.key_file, self.cert_file)
class MyHTTPHandler(urllib2.HTTPHandler):
def http_open(self,req):
return self.do_open(MyHTTPConnection,req)
class MyHTTPSHandler(urllib2.HTTPSHandler):
def https_open(self,req):
return self.do_open(MyHTTPSConnection,req)
opener = urllib2.build_opener(MyHTTPHandler,MyHTTPSHandler)
urllib2.install_opener(opener)
f = urllib2.urlopen('http://news.bbc.co.uk')
data = f.read()
from lxml import etree
doc = etree.HTML(data)
>>> print doc.xpath('//title/text()')
['Google']
如果您使用HTTPS,显然有证书问题,您需要填写MyResolver ...
Obviously there are certificate issues if you use the HTTPS, and you'll need to fill out MyResolver...
这篇关于告诉urllib2使用自定义DNS的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!