如何在scrapy item导出中每次启用覆盖文件? [英] How to enable overwriting a file everytime in scrapy item export?

问题描述

我正在抓取一个返回urls列表的网站. 示例-scrapy crawl xyz_spider -o urls.csv

I am scraping a website which returns in a list of urls. Example - scrapy crawl xyz_spider -o urls.csv

现在可以正常工作了，我想制作一个新的urls.csv而不是将data追加到文件中.我可以通过任何参数使它启用吗?

It is working absolutely fine now I want is to make new urls.csv not append data into the file. Is there any parameter passing I can do to make it enable?

推荐答案

不幸的是，目前抓不住不能做到这一点.
不过，在github上有一个建议的增强功能: https://github.com/scrapy/scrapy/issues/547

Unfortunately scrapy can't do this at the moment.
There is a proposed enhancement on github though: https://github.com/scrapy/scrapy/issues/547

不过，您可以轻松地将输出重定向到stdout并将其重定向到文件:

However you can easily do redirect the output to stdout and redirect that to a file:

scrapy crawl myspider -t json --nolog -o - > output.json

-o -表示输出为减号，在这种情况下，减号表示stdout.
您还可以在运行scrapy之前添加一些别名以删除文件，例如:

-o - means output to minus and minus in this case means stdout.
You can also make some aliases to delete the file before running scrapy, something like:

alias sc='-rm output.csv && scrapy crawl myspider -o output.csv'

这篇关于如何在scrapy item导出中每次启用覆盖文件?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！