使用scrapy抓取动态数据 [英] Scrape dynamic data using scrapy

问题描述

我想使用scrapy(以及其他数据)从纳斯达克网站抓取股票期权链

I would like to scrape option chain of stock from nasdaq website using scrapy (along with other data)

纳斯达克最近更新了他们的网站.这里是我说的网址.

Nasdaq recently updated their website. Here is the url I am talking about.

数据不是用普通蜘蛛和scrapy shell加载的.从scrapy docs，我需要使用scrapy-splash 或scrapy-selenium.

The data is not loaded with plain spider and in scrapy shell. From the scrapy docs, I need to use scrapy-splash or scrapy-selenium.

在投入时间之前，我想知道哪种工具最适合这份工作.或者还有什么值得推荐的吗?

I would like to know which one is the right tool for the job before investing my time. Or is there something else that is recommended?

谢谢！

推荐答案

对于该任务，您不需要 selenium 或 splash.相反，您可以检查 Chrome 开发人员工具中的网络选项卡.

For that task you don't need selenium or splash. Instead you can inspect the network tab in chrome developer tools.

该站点发出额外请求以从以下端点获取数据:https://api.nasdaq.com/api/quote/MSFT/option-chain?assetclass=stocks&limit=0

The site makes additional request to get data from the following endpoint: https://api.nasdaq.com/api/quote/MSFT/option-chain?assetclass=stocks&limit=0

干杯

这篇关于使用scrapy抓取动态数据的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！