使用Javascript支持筛选屏幕的好工具是什么？ [英] What's a good tool to screen-scrape with Javascript support?

问题描述

是否有一个好的测试套件或工具集可以自动化网站导航 - 支持Javascript - 并从页面中收集HTML？

Is there a good test suite or tool set that can automate website navigation -- with Javascript support -- and collect the HTML from the pages?

当然我可以使用BeautifulSoup直接HTML。但这对我需要Javascript的网站没有好处。 ：）

Of course I can scrape straight HTML with BeautifulSoup. But this does me no good for sites that require Javascript. :)

推荐答案

你可以使用 Selenium 或 Watir 来推动真正的浏览器。

You could use Selenium or Watir to drive a real browser.

Ther也是一些基于JavaScript的无头浏览器：

Ther are also some JavaScript-based headless browsers:

• PhantomJS 是一个无头的Webkit浏览器。
  
  
  
  
  • pjscrape 正在刮刮基于PhantomJS和jQuery的框架。
  
  
  • CasperJS 是一个导航脚本编写在PhantomJS上测试实用程序，如果你需要做的只是指点要刮掉的URL。
  
  
  • PhantomJS is a headless Webkit browser.
    • pjscrape is a scraping framework based on PhantomJS and jQuery.
    • CasperJS is a navigation scripting & testing utility bsaed on PhantomJS, if you need to do a little more than point at URLs to be scraped.

    就我个人而言，我最熟悉Selenium，它支持在众多语言中编写自动化脚本并拥有更成熟的工具，例如Firefox的优秀 Selenium IDE 扩展，可用于编写和运行测试用例，以及可以将测试脚本导出为多种语言。

    Personally, I'm most familiar with Selenium, which has support for writing automation scripts in a good number of languagues and has more mature tooling, such as the excellent Selenium IDE extension for Firefox, which can be used to write and run testcases, and can export test scripts to many languages.

    这篇关于使用Javascript支持筛选屏幕的好工具是什么？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！