我应该怎么做才能启用Cookies并对该网址使用scrapy？ [英] What should i do to enable cookies and use scrapy for this url?

问题描述

我正在使用此URL https://www.walmart.ca/en/clothing-shoes-accessories/men/mens-tops/N-2566+11

I am using scrapy for a scrapying project with this url https://www.walmart.ca/en/clothing-shoes-accessories/men/mens-tops/N-2566+11

我尝试使用url并在外壳中打开它，但是出现430错误，因此我在标头中添加了一些设置，例如：

I tried to play with the url and open it in the shell, but it got 430 error, so i added some settings to the header like that:

scrapy shell -s COOKIES_ENABLED = 1 -s USER_AGENT ='Mozilla / 5.0（X11; Ubuntu; Linux x86_64; rv：46.0​​）Gecko / 20100101 Firefox / 46.0' https://www.walmart.ca/en/clothing-shoes-accessories/men/mens-tops / N-2566 + 11

scrapy shell -s COOKIES_ENABLED=1 -s USER_AGENT='Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:46.0) Gecko/20100101 Firefox/46.0' "https://www.walmart.ca/en/clothing-shoes-accessories/men/mens-tops/N-2566+11"

它获得了页面 200，但是一旦我使用view（response），它便将我定向到了一个页面说：
对不起！
您的Web浏览器不接受cookie。

it got the page "200", but once i use view(response), it directed me to a page that say: Sorry! Your web browser is not accepting cookies.

这是日志的屏幕截图：

here is a screenshot of the log:

推荐答案

您应该拥有

COOKIES_ENABLED = True

在您的 settings.py 文件。

另请参见

COOKIES_DEBUG = True

要调试Cookie，您将看到每个响应/请求分别要传入/传出的Cookie。

To debug cookies, you will see what cookies are coming/outgoing which each response/request respectively.

这篇关于我应该怎么做才能启用Cookies并对该网址使用scrapy？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！