如何执行刮板外壳"URL"带笔记本 [英] how to execute scrapy shell "URL" with notebook

问题描述

我正试图刮

有一个刮擦的外壳程序"URL"命令，使用来自URL的响应对象执行控制台ipython.

and there is scrapy shell "URL" command, executing console ipython with a response object from the URL.

但是我想用笔记本做那件事.

but i want to do that thing with notebook.

有什么方法可以用笔记本执行刮y的外壳，

is there any way to execute scrapy shell with notebook,

或者如何在ipython中手动获取URL的相同响应对象?

or how can i get the same response object of the URL in ipython manually?

推荐答案

在这个悬而未决的问题中偶然发现了相同的答案. 我现在能想到的最接近的东西是:

Stumbled across this unanswered question looking for the same answer. The closest thing I can come up right now is this:

import requests
from scrapy.http import TextResponse

r = requests.get('http://stackoverflow.com/')
response = TextResponse(r.url, body=r.text, encoding='utf-8')

然后，您可以使用xpath()之类的常用方法调试响应.

Then you can debug response using usual methods like xpath().

这篇关于如何执行刮板外壳"URL"带笔记本的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！