基于cURL的HTTP请求是否完全模仿基于浏览器的请求？ [英] Can a cURL based HTTP request imitate a browser based request completely?

问题描述

这是一个两部分的问题。

Q1：可以基于cURL的请求100％模仿基于浏览器的请求吗？

Q2：如果是，应设置所有选项。如果没有，浏览器做什么，不能由cURL模仿？

我有一个网站，我看到成千上万的请求是从一个单一的IP在一个很短的时间。这些请求收集我的所有数据。当查看日志以标识使用的代理时，它看起来像来自浏览器的请求。

提前感谢

解决方案

R1：我想，如果你设置所有正确的头，那是的，基于curl的请求可以模仿基于浏览器的请求：毕竟， HTTP请求，这是一个几行文本遵循特定的约定（即HTTP RFC）

R2： 回答这个问题的最佳方法是查看浏览器发送的内容;例如，使用Firefox，您可以使用 Firebug 或 LiveHTTPHeaders 。

，Firefox发送了这些请求标头：

  GET / questions / 1926876 / can-a-curl-based-http-request-imitate基于浏览器的请求完全HTTP / 1.1 
主机：stackoverflow.com 
用户代理：Mozilla / 5.0（Windows; U; Windows NT 5.1; fr; rv：1.9.2b4） Gecko / 20091124 Firefox / 3.6b4 
 Accept：text / html，application / xhtml + xml，application / xml; q = 0.9，* / *; q = 0.8 
 Accept-Language：fr，fr-接受编码：gzip，deflate 
 Accept-Charset：ISO-8859-1，utf-8; q = 0， 0.7，*; q = 0.7 
 Keep-Alive：115 
连接：keep-alive 
参考：http://stackoverflow.com/questions/1926876/can-a-curl-based -http-request-imitate-a-browser-based-request-completely / 1926889 
 Cookie：....... 
缓存控制：max-age = 0 
  

（我只是删除了几个信息 - 但你得到的想法;-)） / p>



使用curl，您可以使用 curl_setopt 设置HTTP标头;这里，您可能需要使用 CURLOPT_HTTPHEADER ， CURLOPT_COOKIE ， CURLOPT_USERAGENT ，...

This is a two part question.

Q1: Can cURL based request 100% imitate a browser based request?

Q2: If yes, what all options should be set. If not what extra does the browser do that cannot bee imitated by cURL?

I have a website and I see thousands of request being made from a single IP in a very short time. These requests harvest all my data. When looked at the log to identify the agent used, it looks like a request from browser. So was curious to know if its a bot and not a user.

Thanks in advance

解决方案

R1 : I suppose, if you set all the correct headers, that, yes, a curl-based request can imitate a browser-based one : after all, both send an HTTP request, which is just a couple of lines of text following a specific convention (namely, the HTTP RFC)

R2 : The best way to answer that question is to take a look at what your browser is sending ; with Firefox, for instance, you can use either Firebug or LiveHTTPHeaders to get that.

For instance, to get this page, Firefox sent those request headers :

GET /questions/1926876/can-a-curl-based-http-request-imitate-a-browser-based-request-completely HTTP/1.1
Host: stackoverflow.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.2b4) Gecko/20091124 Firefox/3.6b4
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: fr,fr-fr;q=0.8,en-us;q=0.5,en;q=0.3
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive
Referer: http://stackoverflow.com/questions/1926876/can-a-curl-based-http-request-imitate-a-browser-based-request-completely/1926889
Cookie: .......
Cache-Control: max-age=0

(I Just removed a couple of informations -- but you get the idea ;-) )

Using curl, you can work with curl_setopt to set the HTTP headers ; here, you'd probably have to use a combination of CURLOPT_HTTPHEADER, CURLOPT_COOKIE, CURLOPT_USERAGENT, ...

这篇关于基于cURL的HTTP请求是否完全模仿基于浏览器的请求？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！