为什么cURL返回“附加的东西不好”？ [英] Why is cURL returning "additional stuff not fine"?

问题描述

我正在编写一个Python应用程序，通过cURL查询社交媒体API。我查询的大多数不同的服务器（Google+，Reddit，Twitter，Facebook，其他人）都有cURL抱怨：

c：1037：0 0

不寻常的是，当应用程序首次启动时，每个服务的响应将抛出此行一次或两次。几分钟后，线路将出现几次。显然cURL正在识别它不喜欢的东西。大约半小时后，服务器开始超时，这条线重复了几十次，所以它显示了一个真正的问题。

我该如何诊断？我尝试使用Wireshark捕获请求和响应头，以搜索异常，可能会导致cURL抱怨，但对于所有Wireshark的复杂性，似乎没有一种方法来隔离和显示只有头。

下面是代码的相关部分：

  output = cStringIO.StringIO $ bc = pycurl.Curl（）
 c.setopt（c.URL，url）
 c.setopt（c.USERAGENT，'Mozilla / 5.0（X11; Ubuntu; Linux x86_64; rv：17.0） Gecko / 20100101 Firefox / 17.0'）
 c.setopt（c.WRITEFUNCTION，output.write）
 c.setopt（c.CONNECTTIMEOUT，10）
 c.setopt（c.TIMEOUT， 15）
 c.setopt（c.FAILONERROR，True）
 c.setopt（c.NOSIGNAL，1）
 
 try：
 c.perform b $ b toReturn = output.getvalue（）
 output.close（）
 return toReturn 
 
 except pycurl.error，error：
 errno，errstr = error 
 print'发生以下cURL错误：'，errstr 
  

解决方案

我是99.99％确定这不是在任何HTTP头，但实际上是打印到 stderr 通过 libcurl 。

无论如何，快速搜索其他的东西不是finecurl transfer.c 已打开源代码的最近更改< a>其中的描述是：

Curl_readwrite：remove debug output

之前添加了一个附加的东西不好的文本为调试目的a
，但它并不真正帮助任何人，由于某种原因，一些
Linux发行版提供他们的libcurls建立的调试信息仍然
现在，因此（太多）用户得到这个信息。

基本上是无害的，你看到它的唯一原因是你有一个构建 libcurl （可能从您的linux发行版），启用完整的调试日志记录（尽管 curl 作者认为这是一个坏主意）。因此，您有三个选项：

1. 忽略它。


2. 升级到< c $ c> libcurl 。


3. 重新构建 libcurl b

您可以查看 libcurl 来源 transfer.c （如上面链接的）尝试理解 curl 是什么，并可能在同一时间查找邮件列表中的线程，或者只是通过电子邮件发送列表和问。

但是，我怀疑实际上可能与真实问题无关，因为你从一开始就看到这一点。 p>

这里有三个明显的错误：

2. 您的网络设置出现问题（例如，您的ISP会在30分钟内断开太多传出连接或使用太多字节，导致您断开连接） ）。


3. 您正在做的是让服务器认为您是垃圾邮件发送者/ DoS攻击者/他们阻止您。






第一个实际上似乎最不可能。如果你想排除它，只需捕获所有的请求，然后写一个琐碎的脚本，使用一些其他库来重放完全相同的请求，看看你是否得到相同的行为。





您可以根据时间来区分案例2和案例3 。如果所有的服务立即超时 - 特别是如果他们都这样做，即使你开始击中他们在不同的时间（例如，你开始打击Google+ 15分钟后Facebook，但他们都在你打了Facebook 30分钟后超时） ，它肯定是case 2.如果不是，可能是case 3。





如果你排除所有这三个，那么你可以开始寻找其他的东西错误，但我会从这里开始。





或者，如果你告诉我们更多关于你的应用程序是什么（例如，你试图击中服务器尽可能快，你试图连接代表一大堆不同的用户？你使用开发键或最终用户应用程序键？等等），可能有些人有更多的经验与那些服务猜测。




I am writing a Python application that queries social media APIs via cURL. Most of the different servers I query (Google+, Reddit, Twitter, Facebook, others) have cURL complaining:

additional stuff not fine transfer.c:1037: 0 0

The unusual thing is that when the application first starts, each service's response will throw this line once or twice. After a few minutes, the line will appear several several times. Obviously cURL is identifying something that it doesn't like. After about half an hour, the servers begin to time out and this line is repeated many tens of times, so it is showing a real problem.

How might I diagnose this? I tried using Wireshark to capture the request and response headers to search for anomalies that might cause cURL to complain, but for all Wireshark's complexity there does not seem to be a way to isolate and display only the headers.

Here is the relevant part of the code:

output = cStringIO.StringIO()
c = pycurl.Curl()
c.setopt(c.URL, url)
c.setopt(c.USERAGENT, 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:17.0) Gecko/20100101 Firefox/17.0')
c.setopt(c.WRITEFUNCTION, output.write)
c.setopt(c.CONNECTTIMEOUT, 10) 
c.setopt(c.TIMEOUT, 15) 
c.setopt(c.FAILONERROR, True)
c.setopt(c.NOSIGNAL, 1)

try:
    c.perform()
    toReturn = output.getvalue()
    output.close()
    return toReturn

except pycurl.error, error:
    errno, errstr = error
    print 'The following cURL error occurred: ', errstr

解决方案

I'm 99.99% sure this is not actually in any HTTP headers, but is rather being printed to stderr by libcurl. Possibly this happens in the middle of you logging the headers, which is why you were confused.

Anyway, a quick search for "additional stuff not fine" curl transfer.c turned up a recent change in the source where the description is:

Curl_readwrite: remove debug output

The text "additional stuff not fine" text was added for debug purposes a while ago, but it isn't really helping anyone and for some reason some Linux distributions provide their libcurls built with debug info still present and thus (far too many) users get to read this info.

So, this is basically harmless, and the only reason you're seeing it is that you got a build of libcurl (probably from your linux distro) that had full debug logging enabled (despite the curl author thinking that's a bad idea). So you have three options:

1. Ignore it.
2. Upgrade to a later version of libcurl.
3. Rebuild libcurl without debug info.

You can look at the libcurl source for transfer.c (as linked above) to try to understand what curl is complaining about, and possibly look for threads on the mailing list for around the same time—or just email the list and ask.

However, I suspect that actually may not relevant to the real problem at all, given that you're seeing this even right from the start.

There are three obvious things that could be going wrong here:

1. A bug in curl, or the way you're using it.
2. Something wrong with your network setup (e.g., your ISP cuts you off for making too many outgoing connections or using too many bytes in 30 minutes).
3. Something you're doing is making the servers think you're a spammer/DoS attacker/whatever and they're blocking you.

The first one actually seems the least likely. If you want to rule it out, just capture all of the requests you make, and then write a trivial script that uses some other library to replay the exact same requests, and see if you get the same behavior. If so, the problem obviously can't be in the implementation of how you make your requests.

You may be able to distinguish between cases 2 and 3 based on the timing. If all of the services time out at once—especially if they all do so even when you start hitting them at different times (e.g., you start hitting Google+ 15 minutes after Facebook, and yet they both time out 30 minutes after you hit Facebook), it's definitely case 2. If not, it could be case 3.

If you rule out all three of these, then you can start looking for other things that could be wrong, but I'd start here.

Or, if you tell us more about exactly what your app does (e.g., do you try to hit the servers over and over as fast as you can? do you try to connect on behalf of a slew of different users? are you using a dev key or an end-user app key? etc.), it might be possible for someone else with more experience with those services to guess.

这篇关于为什么cURL返回“附加的东西不好”？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！