Python Requests/urllib — 监控带宽使用情况 [英] Python Requests/urllib — monitoring bandwidth usage
问题描述
我想记录我的 Python 脚本下载和上传的总字节数.
I want to log the total bytes downloaded and uploaded by my Python script.
total_downloaded_bytes = 0
def bandwidth_hook(r, *args, **kwargs):
global total_downloaded_bytes
total_downloaded_bytes += len(r.content)
req = requests.session()
req.hooks = {'response': bandwidth_hook}
上面的代码没有考虑 HTTP 压缩(如果我是对的)和标头的大小.
The above code doesn't take into account HTTP compression (if I'm right) and the size of headers.
有没有办法计算 requests.session 中上传和下载的总字节数?如果不是,那么脚本范围内的计数呢?
Is there a way to count total uploaded and downloaded bytes from a requests.session? If not, what about a script-wide count?
推荐答案
您可以访问 r.request
对象来计算传出字节,并且可以通过查看来确定传入字节(压缩与否)在传入请求的 content-length
标头处.这应该足以满足您通常会提出的所有请求的 99%.
You can access the r.request
object to calculate outgoing bytes, and you can determine incoming bytes (compressed or not) by looking at the content-length
header for the incoming request. This should suffice for 99% of all requests you normally would make.
计算头的字节大小很容易;只需将键和值的长度相加,为冒号和空格添加 4 个字节,为空行添加 2 个字节:
Calculating the byte size of headers is easy enough; just add up key and value lenghts, add 4 bytes for the colon and whitespace, plus 2 more for the blank line:
def header_size(headers):
return sum(len(key) + len(value) + 4 for key, value in headers.items()) + 2
还有初始行;这是请求的 {method} {path_url} HTTP/1.1{CRLF}
和响应的 HTTP/1.x {status_code} {reason}{CRLF}
.您也可以使用这些长度.
There is also the initial line; that's {method} {path_url} HTTP/1.1{CRLF}
for requests, and HTTP/1.x {status_code} {reason}{CRLF}
for the response. Those lengths are all also available to you.
总大小为:
request_line_size = len(r.request.method) + len(r.request.path_url) + 12
request_size = request_line_size + header_size(r.request.headers) + int(r.request.headers.get('content-length', 0))
response_line_size = len(r.response.reason) + 15
response_size = response_line_size + header_size(r.headers) + int(r.headers.get('content-length', 0))
total_size = request_size + response_size
这篇关于Python Requests/urllib — 监控带宽使用情况的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!