如何在 python 中将 JavaScript HTML 渲染为 HTML? [英] How can I render JavaScript HTML to HTML in python?

问题描述

我环顾四周，只找到了将 URL 呈现为 HTML 的解决方案.但是，我需要一种方法来将网页(我已经拥有，并且具有 JavaScript)呈现为正确的 HTML.

I have looked around and only found solutions that render a URL to HTML. However I need a way to be able to render a webpage (That I already have, and that has JavaScript) to proper HTML.

想要:网页(使用 JavaScript)---> HTML

Want: Webpage (with JavaScript) ---> HTML

不是:URL --> 网页(使用 JavaScript)---> HTML

Not: URL --> Webpage (with JavaScript) ---> HTML

我不知道如何让其他代码按照我想要的方式工作.

I couldn't figure out how to make the other code work the way I wanted.

这是我使用的呈现 URL 的代码:http://webscraping.com/blog/Scraping-JavaScript-webpages-with-网络套件/

This is the code I was using that renders URLs: http://webscraping.com/blog/Scraping-JavaScript-webpages-with-webkit/

为了清楚起见，上面的代码采用了网页的 URL，该网页的某些部分由 JavaScript 呈现，因此如果我通常使用 urllib2 抓取页面，那么我将不会获得呈现为的所有链接等在 JavaScript 之后.

For clarity, the code above takes a URL of a webpage that has some parts of the page rendered by JavaScript, so if I scrape the page normally using say urllib2 then I won't get all the links etc that are rendered as after the JavaScript.

但是我希望能够抓取一个页面，用 urllib2 再说一遍，然后呈现该页面并获得结果 HTML.(与上面的代码不同，因为它需要一个 URL 作为它的参数.

However I want to be able to scrape a page, say again with urllib2, and then render that page and get the outcome HTML. (Different to the above code since it takes a URL as it's argument.

感谢任何帮助，谢谢你们:)

Any help is appreciated, thanks guys :)

推荐答案

您可以从命令行pip install selenium，然后运行类似:

You can pip install selenium from a command line, and then run something like:

from selenium import webdriver
from urllib2 import urlopen

url = 'http://www.google.com'
file_name = 'C:/Users/Desktop/test.txt'

conn = urlopen(url)
data = conn.read()
conn.close()

file = open(file_name,'wt')
file.write(data)
file.close()

browser = webdriver.Firefox()
browser.get('file:///'+file_name)
html = browser.page_source
browser.quit()

这篇关于如何在 python 中将 JavaScript HTML 渲染为 HTML?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！