Scrapy csv 文件有统一的空行? [英] Scrapy csv file has uniform empty rows?
本文介绍了Scrapy csv 文件有统一的空行?的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
这是蜘蛛:
import scrapy
from danmurphys.items import DanmurphysItem
class MySpider(scrapy.Spider):
name = 'danmurphys'
allowed_domains = ['danmurphys.com.au']
start_urls = ['https://www.danmurphys.com.au/dm/navigation/navigation_results_gallery.jsp?params=fh_location%3D%2F%2Fcatalog01%2Fen_AU%2Fcategories%3C%7Bcatalog01_2534374302084767_2534374302027742%7D%26fh_view_size%3D120%26fh_sort%3D-sales_value_30_days%26fh_modification%3D&resetnav=false&storeExclusivePage=false']
def parse(self, response):
urls = response.xpath('//h2/a/@href').extract()
for url in urls:
request = scrapy.Request(url , callback=self.parse_page)
yield request
def parse_page(self , response):
item = DanmurphysItem()
item['brand'] = response.xpath('//span[@itemprop="brand"]/text()').extract_first().strip()
item['name'] = response.xpath('//span[@itemprop="name"]/text()').extract_first().strip()
item['url'] = response.url
return item
这里是项目:
import scrapy
class DanmurphysItem(scrapy.Item):
brand = scrapy.Field()
name = scrapy.Field()
url = scrapy.Field()
当我用这个命令运行蜘蛛时:
when I run the spider with this command :
scrapy crawl danmurphys -o output.csv
输出是这样的:
推荐答案
要在 Scrapy 1.3 中修复此问题,您可以通过将 newline=''
作为参数添加到 io.TextIOWrapper 来修补它
在 scrapy.exporters
中的 CsvItemExporter
类的 __init__
方法中.
To fix this in Scrapy 1.3, you can patch it by adding newline=''
as parameter to io.TextIOWrapper
in the __init__
method of the CsvItemExporter
class in scrapy.exporters
.
这篇关于Scrapy csv 文件有统一的空行?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
查看全文