是否可以使用 Python requests 库阅读维基百科? [英] Is it possible to read Wikipedia using Python requests library?

问题描述

要从给定的 URL 读取内容，我执行以下操作:

To read a content from a given URL I do the following:

import requests
proxies = {'http':'http://user:[e-mail]:7777'}
url = 'http://example.com/foo/bar'
r = requests.get(url, proxies = proxies)
print r.text.encode('utf-8')

而且效果很好！我知道内容了.

And it works fine! I get the content.

但是，如果我使用另一个网址:

However, if I use another URL:

url = 'https://en.wikipedia.org/wiki/Mestisko'

它不起作用.我收到一条以以下内容开头的错误消息:

It does not work. I get an error message that starts with:

requests.exceptions.ConnectionError: ('Connection aborted.', error(10060

维基百科是否会阻止自动请求?

Is Wikipedia blocking automatic requests?

添加

我尝试通过以下方式设置用户代理:

I tried to set a user agent in the following way:

headers = {'User-Agent':'Mozilla/5.0'}
r = requests.get(url, proxies = proxies, headers = headers)

不幸的是它没有帮助.我仍然遇到同样的错误.

Unfortunately it does not help. I still get the same error.

添加了 2 个

现在我很困惑.如果我尝试通过设置代理从 http://example.com/foo/bar 获取内容，我会得到它.如果我不设置代理，我会得到代理生成的内容.这种行为我能理解.现在，如果我尝试从 Wikipedia 获取内容，无论是否设置代理，我都会收到相同的错误消息.所以，我不明白这个错误信息来自维基百科或代理(这两个选项都不可能是真的).​​

Now I am confused. If I try to get content from http://example.com/foo/bar with setting proxy, I get it. If I do not set proxy, I get content generated by proxy. This behavior I can understand. Now, if I try to get content from Wikipedia, I get the same error message independently on whether I set or do not set proxy. So, I do not understand where this error message comes from Wikipedia or proxy (both options cannot be true).

推荐答案

问题已通过替换解决:

proxies = {'http':'http://user:[e-mail]:7777'}

带有以下行:

proxies = {'http':'http://user:[e-mail]:7777', 'https':'http://user:[e-mail]:7777'}

这篇关于是否可以使用 Python requests 库阅读维基百科?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！