Google 趋势配额限制 [英] Google Trends Quota Limit
问题描述
我正在尝试从 Google 趋势中提取数据，但仅尝试了 2 次后就收到了您已达到每日限额"的错误消息.
I'm trying to pull data from Google trends and got a "You have reached your daily limit" error after only 2 tries.
有什么办法可以解决这个问题吗?我知道 Google API 项目有特殊的配额限制，但 Google Trends 没有 API.我还看到我们可能需要向它传递一个 cookie 文件，这样我就好像已经登录了.以前有人遇到过这个问题吗?
Is there any way to go around this? I know Google API projects have special quota limits but Google Trends doesn't have an API. I also read that we may need to pass it a cookie file so that it seems like I'm logged in. Has anyone faced this issue before?
推荐答案
我也遇到了同样的问题！从你的问题我无法弄清楚你达到了哪个阶段......但这是我找到的解决方案:
I'm struggling with the same issue! From your question I can't figure out what stage have you achieved... But here is the solution that I've found:
- 您应该使用 cookie 模拟浏览器.我认为最好的方法是使用 Mechanize 库.
- 首先，您的程序应该使用 GET 请求登录"到https://accounts.google.com/Login?hl=en"
- 之后您可以立即访问其他一些个人资源，但不能访问谷歌趋势！
- 经过一些重要时间后，您可以成功获取 CSV 格式的谷歌趋势数据.
- 我还没有发现确切的时间段，但已经超过了 10 分钟，不到几个小时:).这就是为什么保存 cookie 以备后用是个好主意！
- You should emulate browser with cookies. I think the best way to do it is to use Mechanize library.
- At first your program should "login" using GET request to "https://accounts.google.com/Login?hl=en"
- Immediately after that you can access some other personal resources, but not google trends!
- After some significant time you can successfully get google trends data as CSV.
- I still have not discovered the exact time period, but it is more than 10 minutes and less than several hours :). That is why saving your cookies for latter use is a good idea!
更多提示:
如果您在 Windows 下使用 python/ruby 进行开发，请不要忘记为 OpenSSL 库设置 CA ROOT 证书包.否则HTTPS连接将失败，您将无法登录！请参阅获取`证书验证失败(OpenSSL::SSL::SSLError)` 机械化对象错误
我建议您在程序关闭时将 cookie 保存到外部文件.并在启动时恢复它们.
I recommend you to save cookies to external file at program shutdown. And restoring them at startup.
不要忘记允许重定向，因为 Google 一直在使用重定向.
Do not forget to allow redirects, because Google is using redirects all the time.
Ruby 代码示例:
require 'mechanize'
require 'logger'
begin
agent = Mechanize.new { |a|
a.user_agent = 'Opera/9.80 (Windows NT 5.1) Presto/2.12.388 Version/12.16'
cert_store = OpenSSL::X509::Store.new
cert_store.add_file 'cacert.pem'
a.cert_store = cert_store
a.log = Logger.new('mech.log')
if File.file?('mech.cookies')
cookies = Mechanize::CookieJar.new
cookies.load('mech.cookies')
a.cookie_jar = cookies
end
a.open_timeout = 5
a.read_timeout = 6
a.keep_alive = true
a.redirect_ok = true
}
LOGIN_URL = "https://accounts.google.com/Login?hl=en&continue=http://www.google.com/trends/"
login_page = agent.get(LOGIN_URL)
login_form = login_page.forms.first
login_form.Email = *
login_form.Passwd = *
login_response_page = agent.submit(login_form)
page = agent.get(url)
# DO SOME TRENDS REQUESTS AFTER SIGNIFICANT PERIOD OF TIME
ensure
if agent
agent.cookie_jar.save('mech.cookies')
end
end
这篇关于Google 趋势配额限制的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
