App引擎 - 难以访问Twitter数据 - Flask [英] App Engine - Difficulty Accessing Twitter Data - Flask
问题描述
使用Google App Engine从Twitter搜索API获取数据时遇到问题。在个人发展环境工作，我没有任何问题。我能够得到我想要的JSON。但是，一旦将代码部署到App Engine，就会遇到HTTP 500错误代码。
对于发生的事情，我唯一的想法是，我缺少和/或Twitter的开发和实时App Engine环境之间存在一些根本区别拒绝来自我运行的App Engine IP的请求。我写了一个小的@ app.route来绝对验证是否有问题 - 是在下面。
我很确定我没有达到一个速率限制，因为Twitter搜索API没有明确的。我看到他们要求一个用户代理，所以我一直在提供，但我仍然得到相同的500错误。
我的生活版本在这里工作： http://1-alpha-3.rich90usa.appspot.com/ twitter_test
任何有关错误的想法都将不胜感激。
Flask的相关部分：
@ app.route（'/ twitter_test'）
def twitter_test ）：
twitter_geo_url ='http://search.twitter.com/search.json?callback=?&rpp=100&geocode=29.6516344,-82.3248262,3mi'
twitter_result = urlfetch.fetch （twitter_geo_url，fetch_headers）
如果twitter_result.status_code == 200：
return twitter_result.content
其中'fetch_headers'之前被定义为：
fetch_headers = {'User-阿根t'：APPNAME-1-alpha-3}
file：
从flask导入Flask，request，render_template，session，redirect
将oauth导入为oauth
导入simplejson作为json
导入urlparse $ b $导入urllib
导入httplib
导入时间
from google.appengine.api导入urlfetch $ b正如Calvin指出的那样，所有的App Engine应用程序都会向外发送来自HTTP请求的HTTP请求同一个IP池，所以这个问题几乎可以肯定是速度限制的：
从 http://dev.twitter.com/pages/rate-limiting ：
但是，
来自IP地址
的所有请求将应用于搜索速率限制。
搜索速率限制不是为了避免不必要的
搜索使用率和滥用而使
公开，而是比REST速率限制高出
。我们b $ b觉得搜索速度限制是
自由和充分的大多数
应用程序，并知道许多
应用程序供应商已经发现
适合他们的需求。 p>
他们还特别呼吁GAE没有资格列入白名单：
应用程序必须具有静态IP以符合白名单的条件。这意味着包括Google App Engine在内的大多数云平台都不能被列入白名单。
Twitter的建议是避开限速切换到 Streaming API 。
I'm having problems getting data from the Twitter search API with Google App Engine. Working in the personal development environment I have no problems. I'm able to get the JSON I want. However, once the code gets deployed to App Engine, I encounter a HTTP 500 Error Code.
The only ideas I have for whats going on is that there is some fundamental difference between the dev and live App Engine environments that I'm missing and/or that Twitter is refusing requests from the App Engine IP I'm running from. I wrote a small @app.route to absolutely verify that there are issues - is is below.
I'm pretty sure I'm not hitting a rate limit because the Twitter Search API doesn't explicitly have one. I saw that they requested a user-agent so I've been providing that but I'm still getting the same 500 Error.
The live version of what I'm working with is here: http://1-alpha-3.rich90usa.appspot.com/twitter_test
Any thoughts on what's going wrong here would be greatly appreciated.
Relevant Section from Flask:
@app.route('/twitter_test')
def twitter_test():
twitter_geo_url = 'http://search.twitter.com/search.json?callback=?&rpp=100&geocode=29.6516344,-82.3248262,3mi'
twitter_result = urlfetch.fetch(twitter_geo_url, fetch_headers)
if twitter_result.status_code == 200:
return twitter_result.content
Where 'fetch_headers' is previously defined as:
fetch_headers = {'User-Agent': "APPNAME-1-alpha-3"}
Top of my Flask py file:
from flask import Flask, request, render_template, session, redirect
import oauth2 as oauth
import simplejson as json
import urlparse
import urllib
import httplib
import time
from google.appengine.api import urlfetch
解决方案 As Calvin indicated, all App Engine apps source outbound HTTP requests from the same pool of IPs, so the issue is almost certainly rate limiting:
From http://dev.twitter.com/pages/rate-limiting:
However,
all requests coming from an IP address
are applied to a Search Rate Limit.
The Search Rate Limit isn't made
public to discourage unnecessary
search usage and abuse, but it is
higher than the REST Rate Limit. We
feel the Search Rate Limit is both
liberal and sufficient for most
applications and know that many
application vendors have found it
suitable for their needs.
They also call out GAE specifically as being ineligible for whitelisting:
An application must have a static IP to be eligible for whitelisting. This means the majority of cloud platforms, including Google App Engine, cannot be whitelisted.
Twitter's suggestion to get around rate-limiting is to switch to the Streaming API.
这篇关于App引擎 - 难以访问Twitter数据 - Flask的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
