如何抓取动态分配其内容的 Microsoft CVE 网页(最好使用 Python)? [英] How can I scrape through the Microsoft CVE Webpage that assigns its content dynamically (preferably using Python)?

问题描述

相关页面是https:///portal.msrc.microsoft.com/en-US/security-guidance/advisory/CVE-2018-8176

在查看页面源代码时，我没有得到任何关于呈现页面内容的相关信息.如何抓取描述中可用的内容?

On viewing the page source, I do not get any relevant info about the content of the rendered page. How can I scrape the content available in the description ?

推荐答案

我使用 Chrome 开发人员工具的网络"选项卡和XHR"过滤器检查了您链接的页面发出的请求，它看起来像页面通过API查询漏洞信息.您可以使用 curl(带有一些漂亮的功能)检查此 API 返回的内容:

I checked the requests made by the page you linked using the "Network" tab of the Chrome developer tools with the "XHR" filter, and it looks like the page queries the vulnerability information from an API. You can inspect what this API returns using curl (with some prettyfication):

curl https://portal.msrc.microsoft.com/api/security-guidance/en-US/CVE/CVE-2018-8176 | python -m json.tool

回到你的Python代码，你不需要抓取你链接的页面的内容，直接查询微软提供的API即可:

Coming back to your Python code, you don't need to scrape the content of the page you linked, and can just query the API provided by Microsoft directly:

import requests

cve_url = "https://portal.msrc.microsoft.com/api/security-guidance/en-US/CVE/CVE-2018-8176"

response = requests.get(cve_url)
cve_dict = response.json()

print(cve_dict["cveTitle"]) # prints: Microsoft PowerPoint Remote Code Execution Vulnerability
print(cve_dict["description"]) # prints: <p>A remote code execution vulnerability...

这篇关于如何抓取动态分配其内容的 Microsoft CVE 网页(最好使用 Python)?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！