使用curl接收和发送cookie，但没有得到预期的响应 [英] Using curl to receive and send cookies, but not getting expected response

问题描述

相关问题：将会话设置为抓取页面

作为一个个人项目，我试图刮擦我大学的课程目录以提供它作为RESTful API。

As a personal project, I'm trying to scrape my University's course catalog to provide it as a RESTful API.

URL1： https://duapp3.drexel.edu/webtms_du/

URL2： https：// duapp3.drexel.edu/webtms_du/Colleges.asp?Term=201125&univ=DREX

转到URL1设置了需要的Cookie URL2。我正在成功接收和发送Cookie，如下所示：

Going to URL1 sets a cookie which is needed for URL2. I'm successfully receiving and sending the cookie like this:


curl -c recd url1

curl -v -b recd url2

但是，我得到的响应与我在浏览器中看到的响应不同。基本上，我的浏览器显示我的专业列表， curl 获取一个页面，而不是主菜单，它有

However, the response I get differs from the one I see in my browser. Basically, my browser shows me a list of majors, while curl gets a page where instead of the list of majors, it has

< FONT COLOR = red size = 5> < / FONT>< FONT COLOR = red size = 5>请稍后再回来。< / FONT>

我想也许他们正在检查用户代理，

I figured maybe they were checking user-agent, so I tried this as well:

curl -b recdcookies --user-agentMozilla / 4.0（compatible; MSIE 5.01; Windows NT 5.0）https：// duapp3 .drexel.edu / webtms_du / Colleges.asp？Term = 201125& univ = DREX

但同样意外的反应。

浏览器和curl之间的不同输出是什么？只要 curl -v 显示它只设置一个cookie。是因为他们是 https 页面，我需要添加另一个参数到 curl ？

What is causing the different outputs between browser and curl? As far as curl -v shows it only sets one cookie. Is it because they are https pages, and I need to be adding another argument to the curl ?

推荐答案

在命令行中使用curl时，如果url包含&符号（& ），因为它在unix系统上有特殊的意义。

When using curl on the command line, you must enclose the url in quotes if it contains an ampersand (&) because it has special meaning on unix systems.

这篇关于使用curl接收和发送cookie，但没有得到预期的响应的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！