pycurl只得到响应的一部分 [英] pycurl only geting part of the response

查看:92
本文介绍了pycurl只得到响应的一部分的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

我正在使用pycurl在python中请求一个URL,该URL返回一个相当大的json格式的响应.当我在浏览器中转到URL时,我会看到全部内容,但是如果我使用pycurl并打印接收到的数据,则只能看到浏览URL时看到的内容的一半,并且在使用URL解析数据时遇到错误. json库说明:

I'm making a request in python using pycurl to a URL which returns a reasonably large json formatted response. When I goto the URL in a browser I see the entire contents, but if I use pycurl and print the received data, I only see about half of what I see when I browse to the URL, and I get an error parsing the data using the json library stating :

ValueError:从第1行第16078列(字符16078)开始的未终止字符串

ValueError: Unterminated string starting at: line 1 column 16078 (char 16078)

pycurl请求是这样的:

The pycurl request is this :

conn = pycurl.Curl()
conn.setopt(pycurl.URL, myUrl)
conn.setopt(pycurl.WRITEFUNCTION, on_receive)
conn.setopt(pycurl.CONNECTTIMEOUT, 30)
conn.setopt(pycurl.TIMEOUT, 30)
conn.setopt(pycurl.NOSIGNAL, 10)
conn.perform()

使用on_receive函数当前仅打印数据.

with the on_receive function currently just printing the data.

有人知道我为什么只得到回应的一部分吗?我已经使用大量超时来尝试解决此问题,最初我没有指定任何超时,但仍然收到此错误.

Does anybody know why I am only getting part of the response? I have used massive timeouts just for trying to solve this, I had initially not specified any timeouts but was still getting this error.

推荐答案

在pycurl中,您可以设置它,

in pycurl, you could set this,

import pycurl
pycurl.CONTENT_LENGTH_DOWNLOAD 

尝试使用

import Curl, pycurl
con = Curl()
con.set_option(pycurl.CONTENT_LENGTH_DOWNLOAD, 9999999999)
con.get('url' ....

还请尝试以下操作,直到起作用:

also try following until it works:

pycurl.SIZE_DOWNLOAD
pycurl.REQUEST_SIZE 

这篇关于pycurl只得到响应的一部分的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!

查看全文
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆