如何在href中单击具有javascript：__ doPostBack的链接？ [英] How to click a link that has javascript:__doPostBack in href?

问题描述

我正在使用模块'mechanize'在python中编写一个屏幕抓取器脚本，我想在href中使用javascript：__ doPostBack的链接上使用mechanize.click_link（）方法。
我相信我试图解析的页面是使用AJAX。

I am writing a screen scraper script in python with module 'mechanize' and I would like to use the mechanize.click_link() method on a link that has javascript:__doPostBack in href. I believe the page I am trying to parse is using AJAX.

注意：mech是mechanize.Browser（）

Note: mech is the mechanize.Browser()

>>> next_link.__class__.__name__
'Link'
>>> next_link
Link(base_url='http://www.citius.mj.pt/Portal/consultas/ConsultasDistribuicao.aspx', url="javascript:__doPostBack('ctl00$ContentPlaceHolder1$Pager1$lnkNext','')", text='2', tag='a', attrs=[('id', 'ctl00_ContentPlaceHolder1_Pager1_lnkNext'), ('title', 'P\xc3\xa1gina seguinte: 2'), ('href', "javascript:__doPostBack('ctl00$ContentPlaceHolder1$Pager1$lnkNext','')")])
>>> req = mech.click_link(next_link)
>>> req
<urllib2.Request instance at 0x025BEE40>
>>> req.has_data()
False

我想在点击后检索页面源链接。

I would like to retrieve the page source after clicking the link.

推荐答案

我不使用机械化，但是我用python做了大量的网页抓取。

当我遇到类似__doPostBack的javascript函数时，我会执行以下操作：

我访问Firefox中的网站，并使用 HttpFox 扩展，用于查看浏览器在单击相关链接时发送到Web服务器的POST请求的参数。

然后我构建使用urllib.parse.urlencode在python中使用相同的请求来构建我需要的查询字符串和POST数据。

有时网站也使用cookie，所以我只使用python的http.cookiejar。

我已多次成功使用此技术。

I don't use mechanize, but I do a lot of web scraping myself with python.

When I run into a javascript function like __doPostBack, I do the following:
I access the web site in Firefox, and use the HttpFox extension to see the parameters of the POST request the browser sent to the web server when clicking the relevant link.
I then build the same request in python using urllib.parse.urlencode to build the query strings and POST data I need.
Sometimes the website uses cookies as well, so I just use python's http.cookiejar.

I have used this technique successfully several times.

这篇关于如何在href中单击具有javascript：__ doPostBack的链接？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！