python - 求高手，使用urllib2请求网页内容时请求成功却返回空的值？

问题描述

问 题

这是出现类似问题的一个网页http://detail.zol.com.cn/inde...

测试代码
import urllib2
url = 'http://detail.zol.com.cn/inde...'
response = None
try:
response = urllib2.urlopen(url,timeout=5)
html = response.read()
print html
print "hehe"
except urllib2.URLError as e:
if hasattr(e, 'code'):

print 'Error code:',e.code

elif hasattr(e, 'reason'):

print 'Reason:',e.reason

finally:
if response:

response.close()

运行结果：C:Python27python.exe C:/Users/Administrator/PycharmProjects/untitled/data02
hehe

Process finished with exit code 0

这段代码运行后也是空值
page = urllib2.Request(url)
page.add_header('Referer', url)
page.add_header('User-Agent', "Mozilla/5.0 (Windows NT 6.2; rv:16.0) Gecko/20100101 Firefox/16.0")
r = urllib2.urlopen(page,timeout=5.0)
html = r.read()
soup = BeautifulSoup(html, 'lxml')

解决方案

暴力一点，cookie带上

import requests

url = 'http://detail.zol.com.cn/index.php?c=SearchList&keyword=coolpad_8297_w01'

headers = {
    'Cookie': 'userProvinceId=2; userCityId=0; userLocationId=26; proIp=123; ip_ck=4cKD5vP/j7QuNjUyMTk4LjE0Njk0Mzg5MzQ%3D; lv=1469438963; vn=1; Hm_lvt_ae5edc2bc4fc71370807f6187f0a2dd0=1469438964; Hm_lpvt_ae5edc2bc4fc71370807f6187f0a2dd0=1469438964; z_day=rdetail=1; z_pro_city=s_provice%3Dshanghai%26s_city%3Dxingqu',
    'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36'
}
r = requests.get(url, headers=headers)
print r.text

这篇关于python - 求高手，使用urllib2请求网页内容时请求成功却返回空的值？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！