使用 Python 从网站抓取图形数据 [英] Scraping graph data from a website using Python

问题描述

是否可以从网站捕获图形数据?例如，网站 here 有许多图.是否可以使用 Python 代码捕获这些数据?

Is it possible to capture the graph data from a website? For example the website here, has a number of plots. Is is possible to capture these data using Python code?

推荐答案

查看您提供的链接的页面源，图表数据通过链接直接以 JSON 格式提供.http://www.fbatoolkit.com/chart_data/1414978499.87

Looking at the page source of the link you provided, the chart data is available directly in JSON format through the link. http://www.fbatoolkit.com/chart_data/1414978499.87

所以你的抓取工具可能想做这样的事情:

So your scraper might want to do something like this:

import requests
import re

r = requests.get('http://www.fbatoolkit.com')
data_link = b'http://www.fbatoolkit.com/' + re.search(b'chart_data/[^"]*', r.content).group()
data_string = requests.get(data_link).content.decode('utf-8')
chart_data = eval(data_string.replace('window.chart_data =', '').replace(';\n',''))

(编辑以解释我查找链接的过程) 当我遇到这样的问题时，我做的第一件事就是查看页面源代码(Windows 版 Chrome 中的 ctrl-u).我四处寻找与绘制图表相关的东西，直到我找到以下 javascript

(Edit to explain my process for finding the link) When I approach a problem like this, the first thing I do is view the page source, (ctrl-u in Chrome for Windows). I searched around for something related to drawing the charts, until I found the following javascript

  function make_containers(i){
        var chart = chart_data[i];

然后我在源代码中搜索了他们定义变量 chart_data 的位置.我找不到这个，但我确实找到了这条线

I then did a search through the source for where they defined the variable chart_data. I couldn't find this, but I did find the line

<script type="text/javascript" src="/chart_data/1414978499.87"></script>

按照这个链接，(你可以在 Chrome 的查看源页面中点击它)我可以看到这是一个定义这个变量的单行 javascript.(请注意，在我的示例代码的最后一行中，我必须对该文件进行一些更改才能使其在 Python 中进行评估).

Following this link, (you can just click on it in the view souce page in Chrome) I could see that this was a one-line piece of javascript which defines this variable. (Notice that in the last line of my example code I had to make a little change to this file to get it to evaluate in Python).

这篇关于使用 Python 从网站抓取图形数据的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！