beautifulsoup和机械化得到AJAX调用结果 [英] beautifulsoup and mechanize to get ajax call result

问题描述

您好即时通讯使用python 2.5和beautifulsoup建立一个刮板 但在一个问题即时通讯的Stüble...的网页的一部分正在产生 后一些按钮的用户点击，whitch通过调用特定javacsript功能，使用适当的参数启动一个Ajax请求

hi im building a scraper using python 2.5 and beautifulsoup but im stuble upon a problem ... part of the web page is generating after user click on some button, whitch start an ajax request by calling specific javacsript function using proper parameters

有没有办法来模拟用户交互，并得到这样的结果？我碰到过一个机械化模块，但在我看来，这主要是用来处理表单工作...

is there a way to simulate user interaction and get this result? i come across a mechanize module but it seems to me that this is mostly used to work with forms ...

我会AP preciate任何链接或者一些code样品 谢谢

i would appreciate any links or some code samples thanks

推荐答案

行，所以我想到这一点......这是后，我意识到，我可以使用的urllib，ulrlib2和beautifulsoup

ok so i have figured it out ... it was quite simple after i realised that i could use combination of urllib, ulrlib2 and beautifulsoup

import urllib, urllib2
from BeautifulSoup import BeautifulSoup as bs_parse

data = urllib.urlencode(values)
req  = urllib2.Request(url, data)
res  = urllib2.urlopen(req)
page = bs_parse(res.read())

这篇关于beautifulsoup和机械化得到AJAX调用结果的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！