PySFTP / Paramiko异常泄漏到stderr [英] PySFTP/Paramiko exceptions leaking into stderr
问题描述
我正在尝试捕获 paramiko 异常，但它们仍被写入stderr。
有没有办法在那儿停止写作？
编辑：甚至在paramiko介入之前就发生了： / p>
导入pysftp
尝试：
pysftp.Connection（host = localhost）
除外，例如e：
print（e）
结果为：
带有适当SFTP参数的示例：
更新：
$ pipenv图
...
pysftp == 0.2.9
-paramiko [必需：> = 1.17，已安装： 2.6.0]
...
$ pipenv运行python
Python 3.7.3（默认值，2019年7月19日，11：21：39）
[Clang 11.0.0（clang-11 00.0.28.3）]在darwin
上键入帮助，版权，信用或许可证以获取更多信息。
>>导入pysftp
>>尝试：
... pysftp.Connection（host = localhost）
...例外，例如e：
... print（e）
...
找不到主机本地主机的主机密钥。
在< function Connection .__ del__ at 0x10f7e8268>中忽略的异常；
追溯（最近一次通话）：
文件 /Users/andrei/Work/try/.venv/lib/python3.7/site-packages/pysftp/__init__.py，第1013行，在__del__
self.close（）
文件 /Users/andrei/Work/try/.venv/lib/python3.7/site-packages/pysftp/__init__.py中，第784行如果self._sftp_live：
，则关闭
。AttributeError：连接对象没有属性 _sftp_live。
>>
我想首先指出 PySFTP （
I am trying to catch paramiko exceptions but they are still written to stderr.
Is there a way to stop writing there?
EDIT: It happens even before paramiko gets involved:
import pysftp
try:
pysftp.Connection(host="localhost")
except Exception as e:
print(e)
Results in:
Example with proper SFTP params:
UPDATE:
$ pipenv graph
...
pysftp==0.2.9
- paramiko [required: >=1.17, installed: 2.6.0]
...
$ pipenv run python
Python 3.7.3 (default, Jul 19 2019, 11:21:39)
[Clang 11.0.0 (clang-1100.0.28.3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pysftp
>>> try:
... pysftp.Connection(host="localhost")
... except Exception as e:
... print(e)
...
No hostkey for host localhost found.
Exception ignored in: <function Connection.__del__ at 0x10f7e8268>
Traceback (most recent call last):
File "/Users/andrei/Work/try/.venv/lib/python3.7/site-packages/pysftp/__init__.py", line 1013, in __del__
self.close()
File "/Users/andrei/Work/try/.venv/lib/python3.7/site-packages/pysftp/__init__.py", line 784, in close
if self._sftp_live:
AttributeError: 'Connection' object has no attribute '_sftp_live'
>>>
I want to start by pointing out that PySFTP ([PyPI]: PySFTP) has not been maintain for 3+ years (or has been moved to a different location - which so far is kept secret :) ).
I reproduced the problem. Below is a more verbose version of your code.
code00.py:
#!/usr/bin/env python3
import sys
import pysftp
import traceback
def main(argv):
hostname = argv[0] if argv else "localhost"
print("Attempting to connect to {0:s} ...".format(hostname))
try:
print("----------Before conn----------")
conn = pysftp.Connection(host=hostname)
print("----------After conn----------")
except:
print("----------Before exc print----------")
traceback.print_exc()
print("----------After exc print----------")
finally:
print("----------Finally----------")
print("----------After try / except / finally----------")
if __name__ == "__main__":
print("Python {0:s} {1:d}bit on {2:s}".format(" ".join(item.strip() for item in sys.version.split("\n")), 64 if sys.maxsize > 0x100000000 else 32, sys.platform))
print("pysftp version: {0:s}\n".format(pysftp.__version__))
main(sys.argv[1:])
print("\nDone.")
Output:
[cfati@CFATI-5510-0:e:\Work\Dev\StackOverflow\q058110732]> "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\Scripts\python.exe" code00.py Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD64)] 64bit on win32 pysftp version: 0.2.9 Attempting to connect to localhost ... ----------Before conn---------- e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\pysftp\__init__.py:61: UserWarning: Failed to load HostKeys from C:\Users\cfati\.ssh\known_hosts. You will need to explicitly load HostKeys (cnopts.hostkeys.load(filename)) or disableHostKey checking (cnopts.hostkeys = None). warnings.warn(wmsg, UserWarning) ----------Before exc print---------- Traceback (most recent call last): File "code00.py", line 13, in main conn = pysftp.Connection(host=hostname) File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\pysftp\__init__.py", line 132, in __init__ self._tconnect['hostkey'] = self._cnopts.get_hostkey(host) File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\pysftp\__init__.py", line 71, in get_hostkey raise SSHException("No hostkey for host %s found." % host) paramiko.ssh_exception.SSHException: No hostkey for host localhost found. ----------After exc print---------- Exception ignored in: <function Connection.__del__ at 0x000001CC720C80D0> Traceback (most recent call last): File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\pysftp\__init__.py", line 1013, in __del__ self.close() File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\pysftp\__init__.py", line 784, in close if self._sftp_live: AttributeError: 'Connection' object has no attribute '_sftp_live' ----------Finally---------- ----------After try / except / finally---------- Done.
It's a PySFTP bug:
- The Connection object is constructed (__new__)
- The initializer (__init__) is called
- Somewhere in the initializer an exception occurs
- The lines after the exception are not executed
- When the object is (automatically) garbage collected (when it goes out of scope, at the end of the except block), in its close method (called by the destructor (__del__)), some attributes are referenced
- But since those attributes initialization occurs after the line (raising the exception) from #2.2., they were never initialized, so their reference raises AttributeError
The fix is straightforward: initialize the attributes to some default values, at the initializer's beginning, so their reference doesn't represent a problem if the above scenario happens.
I noticed that you've already submitted an issue on BitBucket.
Considering that:
- I'm not a BitBucket expert
- I don't have permissions on [BitBucket]: dundeemt/pysftp - pysftp is an easy to use sftp module that utilizes paramiko and pycrypto. (I can neither push nor submit pull requests)
I created my own repo (out of the above one), and pushed the changes at: [BitBucket]: CristiFati0/pysftp - [Issue #144]: Exceptions leaking into stderr (one commit, so far).
Output (after manually applying the fix to the file installed by pip):
[cfati@CFATI-5510-0:e:\Work\Dev\StackOverflow\q058110732]> "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\Scripts\python.exe" code00.py Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD64)] 64bit on win32 pysftp version: 0.2.9 Attempting to connect to localhost ... ----------Before conn---------- e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\pysftp\__init__.py:61: UserWarning: Failed to load HostKeys from C:\Users\cfati\.ssh\known_hosts. You will need to explicitly load HostKeys (cnopts.hostkeys.load(filename)) or disableHostKey checking (cnopts.hostkeys = None). warnings.warn(wmsg, UserWarning) ----------Before exc print---------- Traceback (most recent call last): File "code00.py", line 13, in main conn = pysftp.Connection(host=hostname) File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\pysftp\__init__.py", line 135, in __init__ self._tconnect['hostkey'] = self._cnopts.get_hostkey(host) File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\pysftp\__init__.py", line 71, in get_hostkey raise SSHException("No hostkey for host %s found." % host) paramiko.ssh_exception.SSHException: No hostkey for host localhost found. ----------After exc print---------- ----------Finally---------- ----------After try / except / finally---------- Done.
Needless to say, other unhandled exceptions might be raised for different scenarios.
@EDIT0
Apparently, ther's more to this than meets the eye. Besides the PySFTP bug described above, there are 2 more things that pollute stderr.
1. The warning
In my case (as I am on Win and don't have any native SSH tool installed) it pops up every time (unless I create / copy some valid known_hosts file in my home dir), but on Nix systems it most likely won't.
Anyway, the fix for this one (if wanted) is easy, simply suppress the UserWarning for example by setting the %PYTHONWARNINGS% env var to ignore::UserWarning (can also be achieved from code - like in next item's case).
2. Paramiko exceptions
I was able to reproduce this by manually modifying transport.py (and raising socket.timeout).
paramiko.Transport which is initialized by pysftp.Connection._start_transport (called by the initializer) does its work in a thread (by subclassing threading.Thread). Any exception raised in that thread, can't be caught by the calling thread (ours). This is a Python limitation scheduled to be addressed in v3.8 ([Python.Bugs]: threading.Thread should have way to catch an exception thrown within).
For this one, there is a (lame) workaround (gainarie): redirecting stderr. Of course there are other workarounds, but they would imply modifying Paramiko, so I'd advice against them.
Below is an example that redirects stderr to stdout (but you can choose any other file - including /dev/null (or nul on Win)). It does it from code (but it can also be done from interpreter command line) so that it only affects the desired (hot) area(s).
code01.py:
#!/usr/bin/env python3
import sys
import pysftp
import paramiko
import traceback
import threading
_sys_stderr = sys.stderr # For restoring purposes
def main(argv):
hostname = argv[0] if argv else "localhost"
print("Attempting to connect to {0:s} ...".format(hostname))
try:
cnopts = pysftp.CnOpts()
cnopts.hostkeys = None
print("---------- STATS: {0:s} {1:d} ----------".format(__file__, threading.get_ident()))
print("---------- Before conn ----------")
sys.stderr.write("DUMMY TEXT before sent to stderr\n")
sys.stderr = sys.stdout # @TODO - cfati: decomment so that everything from stderr is redirected to stdout
conn = pysftp.Connection(host=hostname, port=22001, username="usr", password="pwd", cnopts=cnopts,)
print("---------- After conn ----------")
except:
sys.stderr = _sys_stderr
print("---------- Before exc tb ----------")
traceback.print_exc(file=sys.stdout)
print("---------- After exc tb ----------")
finally:
sys.stderr = _sys_stderr
print("---------- Finally ----------")
sys.stderr.write("DUMMY TEXT after sent to stderr\n")
print("---------- After try / except / finally ----------")
if __name__ == "__main__":
print("Python {0:s} {1:d}bit on {2:s}".format(" ".join(item.strip() for item in sys.version.split("\n")), 64 if sys.maxsize > 0x100000000 else 32, sys.platform))
print("pysftp version: {0:s}\nparamiko version: {1:s}".format(pysftp.__version__, paramiko.__version__))
main(sys.argv[1:])
print("\nDone.")
Output:
[cfati@CFATI-5510-0:e:\Work\Dev\StackOverflow\q058110732]> sopr.bat *** Set shorter prompt to better fit when pasted in StackOverflow (or other) pages *** [prompt]> dir /b code00.py code01.py [prompt]> :: Suppress warning [prompt]> set PYTHONWARNINGS=ignore::UserWarning [prompt]> :: Redirect stdout and stderr to different files, so it is obvious which is which [prompt]> "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\Scripts\python.exe" code01.py 1>1.out 2>1.err [prompt]> type 1.err DUMMY TEXT before sent to stderr DUMMY TEXT after sent to stderr [prompt]> type 1.out Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD64)] 64bit on win32 pysftp version: 0.2.9 paramiko version: 2.6.0 Attempting to connect to localhost ... ---------- STATS: code01.py 23016 ---------- ---------- Before conn ---------- ---------- STATS: e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\paramiko\transport.py 45616 ---------- Exception: Error reading SSH protocol banner Traceback (most recent call last): File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\paramiko\transport.py", line 2212, in _check_banner raise socket.timeout() socket.timeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\paramiko\transport.py", line 2039, in run self._check_banner() File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\paramiko\transport.py", line 2218, in _check_banner "Error reading SSH protocol banner" + str(e) paramiko.ssh_exception.SSHException: Error reading SSH protocol banner ---------- Before exc tb ---------- Traceback (most recent call last): File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\paramiko\transport.py", line 2212, in _check_banner raise socket.timeout() socket.timeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "code01.py", line 23, in main conn = pysftp.Connection(host=hostname, port=22001, username="usr", password="pwd", cnopts=cnopts,) File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\pysftp\__init__.py", line 144, in __init__ self._transport.connect(**self._tconnect) File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\paramiko\transport.py", line 1291, in connect self.start_client() File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\paramiko\transport.py", line 660, in start_client raise e File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\paramiko\transport.py", line 2039, in run self._check_banner() File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\paramiko\transport.py", line 2218, in _check_banner "Error reading SSH protocol banner" + str(e) paramiko.ssh_exception.SSHException: Error reading SSH protocol banner ---------- After exc tb ---------- ---------- Finally ---------- ---------- After try / except / finally ---------- Done.
And how it looks from PyCharm (had to stretch it to the max to fit the whole thing):
这篇关于PySFTP / Paramiko异常泄漏到stderr的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
