python tor的一般SOCKS服务器故障但在tor浏览器中工作 [英] General SOCKS server failure with python tor but working from tor browser

问题描述

我想使用 python 中的 tor 来自动化请求.我用一个页面进行了测试以检查 IP 并且它有效.

I would like to use tor from python to automate requests. I made a test with a page to check IP and it works.

然后我指向了我想要访问的站点，显然他们避免使用 Tor 端点，因为(请参阅下面的堆栈跟踪) - 但它可以在 Tor 浏览器中运行.

I then pointed to the site I want to, and apparently they avoid a tor endpoint because (see stack trace below) - but it works from tor browser.

有什么更好的方法来调试浏览器的响应?(例如连接被拒绝)

Any better way to debug response of browser? (e.g. connection refused)

从 python 查询而不是从浏览器查询我缺少哪些内容?

Which things I am missing to query from python instead than from browser?

我正在尝试类似的东西:

I am trying with something like:

socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5 , "127.0.0.1", 9150, True)
socket.socket = socks.socksocket

socks.wrapmodule(requests)
url = "myexample.com"
# r = requests.Session()
s = requests.get(url)
print s


    Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 127, in urlopen
    return _opener.open(url, data, timeout)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 404, in open
    response = self._open(req, data)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 422, in _open
    '_open', req)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 382, in _call_chain
    result = func(*args)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1222, in https_open
    return self.do_open(httplib.HTTPSConnection, req)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1181, in do_open
    h.request(req.get_method(), req.get_selector(), req.data, headers)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 973, in request
    self._send_request(method, url, body, headers)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1007, in _send_request
    self.endheaders(body)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 969, in endheaders
    self._send_output(message_body)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 829, in _send_output
    self.send(msg)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 791, in send
    self.connect()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1172, in connect
    self.timeout, self.source_address)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 562, in create_connection
    sock.connect(sa)
  File "/Library/Python/2.7/site-packages/socks.py", line 747, in connect
    negotiate(self, dest_addr, dest_port)
  File "/Library/Python/2.7/site-packages/socks.py", line 419, in _negotiate_SOCKS5
    CONNECT, dest_addr)
  File "/Library/Python/2.7/site-packages/socks.py", line 494, in _SOCKS5_request
    raise SOCKS5Error("{0:#04x}: {1}".format(status, error))
socks.SOCKS5Error: 0x01: General SOCKS server failure

推荐答案

即使您使用某些网站也在寻找用户代理.尝试将用户代理标头发送到您的请求.我遇到了同样的问题.

Even though you use tor some website are looking for User Agent. Try to had User Agent header to your request. I had the same problem.

这对我有用

from urllib.request import Request, urlopen
from bs4 import BeautifulSoup
import random
from fake_useragent import UserAgent
from torrequest import TorRequest
import time, socks, socket
from stem import Signal
from stem.control import Controller


ua = UserAgent()


with Controller.from_port(port = 9051) as controller:
    controller.authenticate(password = 'YourPasswordHere')
    socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, "127.0.0.1", 9050)
    socket.socket = socks.socksocket   

    controller.signal(Signal.NEWNYM)
    if controller.is_newnym_available() == False:
        print("Waitting time for Tor to change IP: "+ str(controller.get_newnym_wait()) +" seconds")
        time.sleep(controller.get_newnym_wait())
    req = Request(url)
    req.add_header('User-Agent', ua.random)
    req_doc= urlopen(req)#.read().decode('utf8')
    print(req_doc)

这篇关于python tor的一般SOCKS服务器故障但在tor浏览器中工作的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！