如何从带有加载屏幕的网页中检索数据? [英] How can I retrieve data from a web page with a loading screen?

问题描述

我正在使用 requests 库从 nitrotype.com/racer/insert_name_here 中检索有关使用以下代码的用户进度的数据:

I am using the requests library to retrieve data from nitrotype.com/racer/insert_name_here about a user's progress using the following code:

import requests

base_url = 'https://www.nitrotype.com/racer/'
name = 'test'
url = base_url + name

page = requests.get(url)
print(page.text)

但是我的问题是这从加载屏幕检索数据，我想要加载屏幕后的数据.是否有可能做到这一点以及如何做到这一点?

However my problem is that this retrieves data from the loading screen, I want the data after the loading screen. Is it possible to do this and how?

推荐答案

这可能是因为动态加载，可以使用 selenium 或 pyppeteer 轻松导航.

This is likely because of dynamic loading and can easily be navigated by using selenium or pyppeteer.

在我的示例中，我使用 pyppeteer 来生成浏览器并加载 javascript，以便我可以获得所需的信息.

In my example, I have used pyppeteer to spawn a browser and load the javascript so that I can attain the required information.

示例:

import pyppeteer
import asyncio

async def main():
    # launches a chromium browser, can use chrome instead of chromium as well.
    browser = await pyppeteer.launch(headless=False)
    # creates a blank page
    page = await browser.newPage()
    # follows to the requested page and runs the dynamic code on the site.
    await page.goto('https://www.nitrotype.com/racer/tupac')
    # provides the html content of the page
    cont = await page.content()
    return cont

# prints the html code of the user profiel: tupac
print(asyncio.get_event_loop().run_until_complete(main()))

这篇关于如何从带有加载屏幕的网页中检索数据?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！