我如何使用scrapy shell与url上的参数 [英] How can i use scrapy shell to with parameters on url

问题描述

我要废除工作网站。我想在刮板上做一些测试。

I want to scrap the job website. i want to do some testing in scrapy shell.

因此，如果我键入这个

scrapy shell http://www.seek.com.au

然后如果我键入

从scrapy.contrib.linkextractors.sgml导入SgmlLinkExtractor

然后它工作正常

但是如果我这样做

scrapy shell http://www.seek.com.au/JobSearch?DateRange=31&SearchFrom=quick&Keywords=python&nation=3000

然后，如果我键入

from scrapy.contrib.linkextractors.sgml import SgmlLinkExtractor

然后它表示从中的无效bash命令，它退出了刮痕工作，并从屏幕上停止工作

Then it says invalid bash command from and it exits the scrapy job and come on screen as stopped job

>>> from scrapy.contrib.linkextractors.sgml import SgmlLinkExtractor
-bash: from: command not found

[5]+  Stopped                 scrapy shell http://www.seek.com.au/JobSearch?DateRange=31
[7]   Done                    Keywords=php

推荐答案

显然，你需要用双引号括起你的网址：

apparently, you need to enclose your url within double quotes:

scrapy shell "http://www.seek.com.au/JobSearch?DateRange=31&SearchFrom=quick&Keywords=python&nation=3000"
>>> from scrapy.contrib.linkextractors.sgml import SgmlLinkExtractor
>>> lx = SgmlLinkExtractor() 

那么一切顺利（以上是我的实际shell输出）

then everything works smoothly (this above is my actual shell output)

尝试没有双引号，不工作（提取线程保持运行，第一个按键退出bash而不改变我的视觉输出，从而给我同样的错误）

tried it without double quotes, doesn't work (the fetch thread keeps running and first key press exits to bash without changing my visual output, thus giving me the same error you have)

这篇关于我如何使用scrapy shell与url上的参数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！