如何使用oauth2为网站构建Python搜寻器 [英] How to build a Python crawler for websites using oauth2

问题描述

我是网络编程的新手.我想构建一个用于通过Python在Foursquare中搜寻社交图的搜寻器. 通过使用apiv2库，我有一个手动"控制的搜寻器.主要方法是:

I'm new in web programming. I want to build a crawler for crawling the social graph in Foursquare by Python. I've got a "manually" controlled crawler by using the apiv2 library. The main method is like:

def main():
    CODE = "******"
    url = "https://foursquare.com/oauth2/authenticate?client_id=****&response_type=code&redirect_uri=****"
    key = "***"
    secret = "****"
    re_uri = "***"

    auth = apiv2.FSAuthenticator(key, secret, re_uri)
    auth.set_token(code)    
    finder = apiv2.UserFinder(auth)        

    #DO SOME REQUIRES By USING THE FINDER
    finder.finde(ANY_USER_ID).mayorships()
    bla bla bla

问题是，目前，我必须在浏览器中键入URL并从重定向URL中获取CODE，然后在程序中更新CODE，然后再次运行它.我认为可能会有一些方法可以对CODE进行编码，从而使当前程序有所进展并使其自动化.

The problem is that at present, I have to type the URL in my browser and pick up the CODE from the redirect URL, and then update the CODE in my program, and run it again. I think there might be some way that I can code the CODE taking progress into my current program and make it automatic.

任何指令或示例代码都是值得赞赏的.

Any instruction or sample code is appreciated.

推荐答案

您应该查看 python-oauth2 模块.这似乎是那里最稳定的东西.

You should check out the python-oauth2 module. It seems to be the most stable thing out there.

尤其是，此博客文章运行得非常好深入介绍如何使用Python轻松进行Oauth.示例代码使用Foursquare API，因此我将首先进行检查.

In particular, this blog post has a really good run down on how to do Oauth easily with Python. The example code uses the Foursquare API, so I would check that out first.

我最近不得不让oauth使用Dropbox，并编写了此模块包含进行oauth交换的必要步骤.

I recently had to get oauth working with Dropbox, and wrote this module containing the necessary steps to do oauth exchange.

对于我的系统，我能想到的最简单的方法是pickle Oauth客户端.我的博客程序包仅反序列化了腌制的客户端并使用以下功能请求端点:

For my system, the simplest thing I could think of was to pickle the Oauth client. My blog package just deserialized the pickled client and requested endpoints with the following function:

get = lambda x: client.request(x, 'GET')[1]

只需确保您的工作人员具有此客户端对象，您就应该做好了:-)

Just makes sure your workers have this client object and you should be good to go :-)

这篇关于如何使用oauth2为网站构建Python搜寻器的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！