刮削错误:exceptions.ValueError:请求 url 中缺少方案: [英] scrapy error :exceptions.ValueError: Missing scheme in request url:

问题描述

我使用 try except 来避免错误，但我的终端仍然显示错误而不是日志消息:

I use try except to avoid error,but My terminal still show error but not the log message :

raise ValueError('Missing scheme in request url: %s' % self._url)
exceptions.ValueError: Missing scheme in request url: 

scrapy 没有得到 image_urls 时，如何避免这个错误?
请指导我，非常感谢.

How can I avoid this error when scrapy didn't get image_urls?
Please guide me ,thank you very much.

    try:

        item['image_urls'] = ["".join(image.extract()) ]     
    except:
        log.msg("no image foung!. url={}".format(response.url),level=log.INFO)

推荐答案

image_urls 字段应该是一个列表，而不是一个字符串.

the image_urls field should be a list, not a str.

item['image_urls'] = image.extract()

如果您这样做了，但仍然引发异常，则您抓取的网址似乎是相对路径.

If you do so, and still raise the exception, it seems that the urls you scraped is the relative path.

ImagePipeline 不知道您的主机，因此无法生成抓取的绝对路径.

the ImagePipeline doesn't know your host, so it can not generate the absolute path to crawl.

import urlparse
item['image_urls'] = [ urlparse.urljoin(response.url, u) for u in image.extract() ]

这篇关于刮削错误:exceptions.ValueError:请求 url 中缺少方案:的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！