如何将数据输入网页以使用 Python 抓取结果输出? [英] How can I input data into a webpage to scrape the resulting output using Python?

问题描述

我熟悉使用 BeautifulSoup 和 urllib2 从网页中抓取数据.但是，如果在返回我要抓取的结果之前需要在页面中输入参数怎么办?

I am familiar with BeautifulSoup and urllib2 to scrape data from a webpage. However, what if a parameter needs to be entered into the page before the result that I want to scrape is returned?

我正在尝试使用此网站获取两个地址之间的地理距离:http://www.freemaptools.com/how-far-is-it-between.htm

I'm trying to obtain the geographic distance between two addresses using this website: http://www.freemaptools.com/how-far-is-it-between.htm

我希望能够转到该页面，输入两个地址，单击显示"，然后提取乌鸦飞行的距离"和陆路运输的距离"值​​并将它们保存到字典中.

I want to be able to go to the page, enter two addresses, click "Show", and then extract the "Distance as the Crow Flies" and "Distance by Land Transport" values and save them to a dictionary.

有没有什么办法可以使用Python将数据输入到网页中?

Is there any way to input data into a webpage using Python?

推荐答案

看看机械化或刮削等工具:

Take a look at tools like mechanize or scrape:

• http://pypi.python.org/pypi/mechanize
• http://stockrt.github.com/p/emulating-a-browser-in-python-with-mechanize/

• http://www.ibm.com/developerworks/linux/library/l-python-mechanize-beautiful-soup/

http://zesty.ca/scrape/

Packt Publishing 也有一篇关于这个问题的文章:

Packt Publishing has an article on that matter, too:

• http://www.packtpub.com/article/web-scraping-with-python

这篇关于如何将数据输入网页以使用 Python 抓取结果输出?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！