App Engine：替代urlfetch？看起来很不可靠 [英] App Engine: Alternatives to urlfetch? Seems very unreliable

问题描述

我在我的应用程序中使用了urlfetch，虽然在开发环境中一切正常，但我发现urlfetch在实际部署时非常不可靠。有时它应该像检索数据一样工作，但几分钟后它可能不会返回任何内容，然后在几分钟后再次正常工作。这是非常不可接受的。我已经检查过，以确保它不是问题的源URL（YQL），并且再次，所有内容都可以在开发环境中正常使用。

有没有第三方库我可以试试吗？

示例代码：

  URL = http://query.yahooapis.com/v1/public/yql?q=%s&format=json ％urllib.quote_plus（查询）
结果= urlfetch.fetch（URL，截止= 10 ）
 
 if result.status_code == 200：
r = json.loads（result.content）
 else：
 return 
 
a = r ['query'] ['results'] 
＃做'a'的元素
  

有时它会像它应该那样工作，但是其他时候 - 完全随机地没有代码变化 - 我会得到这个错误：

  TypeError：'NoneType'对象不可订阅
  

解决方案

有时它会像我一样工作t应该是
，但是其他时间是完全随机的，没有代码变化。

这是您的应用程序请求超出的常见症状雅虎API调用速率限制。

引用雅虎开发者文档限额：

基于IP的限制

我们的服务费率限制是针对每个IP地址在特定
时间窗口期间API调用的数量
限制的
a限制。如果您的IP地址
在该时间段内发生变化，您
可能会发现自己拥有更多可用的信用
。但是，如果其他
已使用该地址并达到
限制，则需要等到该时间段的
结束才允许
做出更多API调用。

Google App Engine使用一组IP地址来发送urlfetch请求，您的应用程序将与其他应用程序共享这些IP地址即调用相同的雅虎端点;当超出速率限制时，端点会回复一个超出限制的错误，导致UrlFetch失败。

这里使用Twitter搜索API的另一个案例。

当您混合使用Google App Engine +第三方网络API时，您需要确保API提供经认证的调用，以便您的应用程序拥有自己的配额（ StackApps API ）。

I'm using urlfetch in my app and while everything works perfectly fine in the development environment, i'm finding urlfetch to be VERY unreliable when it's actually deployed. Sometimes it works as it should (retrieving data), but then a few minutes later it might return nothing, then it'll be working fine again a few minutes after that. This is very unacceptable. I've checked to make sure it's NOT the source URL that's the problem (YQL) and, again, everything works as it should in the development environment.

Are there any third-party libraries I could try?

Example code:

url = "http://query.yahooapis.com/v1/public/yql?q=%s&format=json" % urllib.quote_plus(query)
result = urlfetch.fetch(url, deadline=10)

if result.status_code == 200:
    r = json.loads(result.content)
else:
    return

a = r['query']['results']
# Do stuff with 'a'

Sometimes it'll work as it should, but other times - completely randomly with no code changes - i'll get this this error:

a = r['query']['results']
TypeError: 'NoneType' object is unsubscriptable

解决方案

Sometimes it'll work as it should, but other times completely randomly with no code changes

This is a common symptom that your application's requests have exceeded the Yahoo API calls rate limit.

Quoting Yahoo developer documentations rate limit:

IP Based Limits

Our service rate limits are imposed as a limit on the number of API calls made per IP address during a specific time window. If your IP address changes during that time period, you may find yourself with more "credit" available. However, if someone else had been using the address and hit the limit, you'll need to wait until the end of the time period to be allowed to make more API calls.

Google App Engine uses a pool of IP addresses for outgoing urlfetch requests and your application is sharing these IP addresses with other applications that are calling the same Yahoo endpoint; when the rate limit is exceeded, the endpoint replies with a limit exceeded error causing UrlFetch to fail.
Here another case using the Twitter search API.

When you mix Google App Engine+Third party web APIs, you need to be sure that the API provides authenticated calls allowing your application to have its own quota (StackApps API for example).

这篇关于App Engine：替代urlfetch？看起来很不可靠的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！