通过PHP或Apache从服务器端上传HTTP文件 [英] Break HTTP file uploading from server side by PHP or Apache

问题描述

将大文件（> 100M）上传到服务器时，PHP首先从浏览器始终接受全部数据POST。我们无法注入上传过程。例如，在整个数据发送到服务器之前检查标记的值是 IMPOSSIBLE / strong>在我的PHP代码中：

 < form enctype =multipart / form-dataaction =upload.php ？token = XXXXXXmethod =POST> 
< input type =hiddenname =MAX_FILE_SIZEvalue =3000000/> 
发送此文件：< input name =userfiletype =file/> 
< input type =submitvalue =发送文件/> 
< / form> 
  

所以我尝试使用 mod_rewrite 像这样：
$ b $ pre $ code $ RewriteEngine on
RewriteMap mymap prg：/tmp/map.php
RewriteCond％{ QUERY_STRING} ^ token =（。*）$ [NC]
RewriteRule ^ / upload / fake.php $ $ {mymap：％1} [L]

map.php

 ＃！/ usr / bin / php 
 <？php 
 define（REAL_TARGET，/upload/real.php\\\
）; 
 define（FORBIDDEN，/upload/forbidden.html\\\
）; 
 
 $ handle = fopen（php：// stdin，r）; 
 while（$ token = trim（fgets（$ handle）））{
 file_put_contents（/ tmp / map.log，$ token。\\\
，FILE_APPEND）; 
 if（check_token（$ token））{
 echo REAL_TARGET; 
} else {
 echo FORBIDDEN; 
 
 
 $ b函数check_token（$ token）{//做你自己的安全检查
 return substr（$ token，0,4）==='阿利克斯; 
} 
  

但...再次失败 。在这种情况下， mod_rewrite 看起来太晚了。数据仍然完全传输。

然后我尝试 Node.js ，像这样（代码片段）：

  var stream = new multipart.Stream（req）; 
 stream.addListener（'part'，function（part）{
 sys.print（req.uri.params.token +\\\
）; 
 if（req.uri.params .token！=xxxx）{//检查令牌
 res.sendHeader（200，{'Content-Type'：'text / plain'}）; 
 res.sendBody '）; 
 res.finish（）; 
 sys.puts（\\\
 => Block）; 
 return false; 
 
  

结果是...再次失败

。

所以请大家帮我找到解决这个问题的正确途径或者告诉我没有办法。

相关问题：

在POST请求完成之前，PHP（使用Apache或Nginx）是否可以检查HTTP标头？

有人可以告诉我如何让这个脚本在开始上传过程之前检查密码，而不是在上传文件之后？ 解决方案

首先，你可以使用我为此创建的GitHub仓库来自己尝试这个代码。只需克隆存储库并运行 node header 即可。

（破坏者，如果你正在阅读这本书，并且有时间压力让某些东西去工作，而没有学习的心情（:(），那么有一个更简单的解决方案在最后）

总体思路

这是一个很好的问题，你要求的是非常可能和不需要客户端，只需要更深入地了解HTTP协议的工作方式，同时显示node.js如何操作:)

如果我们深入到底层的 TCP协议，并且处理HTTP要求我们自己处理这个具体案件Node.js让你可以使用内建的网络模块轻松完成。

HTTP协议

首先，我们来看看HTTP请求是如何工作的。 b

HTTP请求包含key：value对由CRLF（ \r\\\
）分隔的一般格式。我们知道，当我们到达一个双重CRLF（即 \r\\\
\r\\\
）时，标题部分结束了。

一个典型的HTTP GET请求可能如下所示：

  GET / resource HTTP / 1.1 
 Cache-Control：no-cache 
 User-Agent：Mozilla / 5.0 
 
 Hello = World& stuff = other 
  

空行之前的顶部是标题部分，底部是请求的主体。你的请求在body部分看起来有点不同，因为它是用 multipart / form-data 编码的，但是头部仍然是类似的。

nodejs中的TCP

我们可以听取TCP中的原始请求并读取我们得到的数据包，直到我们阅读我们所谈论的双crlf。然后，我们将检查我们已经有的短头部分，用于我们需要的任何验证。在我们这样做之后，如果验证没有通过，我们可以结束请求（例如通过简单地结束TCP连接），或者传递它。这允许我们不接收或读取请求体，而只是头小得多。

一个简单的方法嵌入到一个已经存在的应用程序是代理请求从它到实际的HTTP服务器的具体用例。

实现细节

这个解决方案是<就像它们得到的一样。这只是一个建议。

以下是工作流程：

1. 我们需要node.js中的 net 模块，它允许我们在node.js中创建tcp服务器。




2. 使用 net 模块创建一个TCP服务器，它将监听数据： var tcpServer = net.createServer（function（socket）{.. 。。不要忘记告诉它听到正确的端口。

   
   
   
   
   
   • 在回调中，数据事件 socket.on（data，function（data）{，它会在数据包到达的时候触发。
     
   • 数据事件中传递的缓冲区的数据，并将其存储在一个变量中
     
   • 检查双重CRLF，这可以确保请求HEADER部分已经结束根据HTTP协议
   
   
   • 假设验证是一个头文件（在您的文字中有令牌）在解析 t之后检查它他的头，（也就是我们得到了双CRLF）。这也适用于检查内容长度标题。
   
   
   • 如果您注意到标题不检出，请调用 socket.end（）这将关闭连接。
     
   
   

下面是一些我们'b

阅读标题的方法：

 函数readHeaders（headers）{
 var parsedHeaders = {}; 
 var previous =; 
 headers.forEach（function（val）{
 //检查下一行是否实际上继续前一行的标题
 if（isContinuation（val））{
 if（前一个！==）{
 parsedHeaders [previous] + = decodeURIComponent（val.trimLeft（））; 
 return; 
} else {
 throw new Exception（continuation ，但是没有以前的头文件）; 
} 
} 
 
 //解析一个类似name：SP value的头文件
 var index = val .indexOf（：）; 
 
 if（index === -1）{
 throw new Exception（bad header structure：）; 
} 
 
 var head = val.substr（0，index）.toLowerCase（）; 
 var value = val.substr（index + 1）.trimLeft（）; 
 
 = head; 
 if（value！==）{
 parsedHeaders [head] = decodeURIComponent（value）; 
} else {
 parsedHeaders [head] = null; 
} 
}）; 
返回parsedHeaders; 
}; 
  

在数据事件缓冲区中检查双重CRLF的方法，并返回它的位置if它存在于一个对象中：

 函数checkForCRLF（data）{
 if（！Buffer.isBuffer（data）） {
 data = new Buffer（data，utf-8）; 
 
 for（var i = 0; i< data.length  -  1; i ++）{
 if（data [i] === 13）{// \r $ （数据[i + 1] === 10）{// \ n 
 if（i + 3  return {loc：i，after：i + 4}; 
 
 
} else if（data [i] === 10）{// \\\
 
 
 if（data [i + 1] = == 10）{// \\\
 
 return {loc：i，after：i + 2}; 
} 
} 
} 
 return {loc：-1，after：-1337}; 
}; 
  

而这个小实用方法：

 函数isContinuation（str）{
返回str.charAt（0）===|| str.charAt（0）===\t; 
 
 
 
 
 
 
 
 
  var net = require（net）; //使用TCP服务器的节点网络模块。如果你想使用HTTPS 
 
 //创建服务器
 var server = net.createServer（function（socket）{//创建一个TCP服务器
 var req = []; //缓存到目前为止，以保存数据，以防头文件没有到达一个包
 socket.on（data，function（data）{
 req.push（data）; //添加新的缓冲区
 var check = checkForCRLF（data）; 
 if（check.loc！== -1）{//这意味着我们得到
 var dataUpToHeaders = req.map（function（x）{
 return x.toString（）; //获取缓冲区字符串
}）。join（ ）; 
 //获取数据到/ r / n 
 dataUpToHeaders = dataUpToHeaders.substring（0，check.after）; 
 //按行分割
 var headerList = dataUpToHeaders .trim（）。split（\r\\\
）; 
 headerList.shift（）; //删除请求行本身，例如GET / HTTP1.1 
 console.log（Got headers！）; 
 //读取头文件
 var headerObject = readHeaders（headerList）; 
 //使用您的令牌获取标题
 console.log（headerObject [your-header-name]）; 
 
 //现在执行所有你需要的检查
 / * 
 if（！yourHeaderValueValid）{
 socket.end（）; 
} else {
 //继续读取请求体，并将控制权交给你想要的逻辑！ 
} 
 * / 
 
 
} 
}）; 
}）。listen（8080）; //为了这个例子听8080端口
  
如果您有任何问题可以随意问：）
 
 
好吧，我撒谎，有一个更简单的方法！
 
 
但是有什么好玩的呢？如果你最初跳过这里，你不会学习HTTP是如何工作的：）
 
 
 Node.js内置 http  
 
这一次，让我们使用 http 模块创建一个http服务器 
 
 
  server = http.createServer （函数（req，res）{//创建一个HTTP服务器
 //参数是请求/响应对象
 //检查方法是否发布，并且头文件包含你的值
 //这个连接已经建立，但是主体还没有被发送，
 //关于这个工作原理的更多信息在上面的解决方案中
 var specialRequest =（req.method ==POST）& ;& req.headers [YourHeader] ===YourTokenValue; 
 if（specialRequest）{//检测特殊处理请求
 //与TCP直接解决方案相同添加块
 req.on（'data'，function（chunkOfBody）{
 //处理消息体
}）; 
 } else {
 res.end（）; //中止底层的TCP连接，因为请求和响应使用相同的TCP连接，这将工作
 //req.destroy（）//在非干净的事情中销毁请求，可能不是你想要的。 
} 
}）。listen（8080）; 
  
这是基于请求处理一个nodejs  http 模块实际上是在发送头文件后挂起的（但是没有其他东西被执行）。 （在服务器模块中，这个在解析器模块中） 
 
 
 
用户<使用  100继续 标题，假定您的目标浏览器支持它。 100继续是一个状态代码，旨在完成你想要做的事： 
 
 
 
 
 100（继续）状态的目的第10.1.1节）是
允许客户端发送一个请求消息与请求体
来确定源服务器是否愿意接受请求
（基于请求头）在客户端发送请求
正文之前。在某些情况下，如果服务器拒绝
消息，而不注意正文，则可能是不适当的，或者对于客户端来说，发送正文的效率非常低
。
 blockquote> 
 
 
这是： 
 
 
  var http = require（'http'） ; 
 
函数句柄（req，rep）{
 req.pipe（process.stdout）; //将请求传递给输出流以进一步处理
 req.on（'end'，function（）{
 rep.end（）; 
 console.log（''）; 
}）; 
} 
 
 var server = new http.Server（）; 
 $ b server.on（'checkContinue'，function（req，rep）{
 if（！req.headers ['x-foo']）{
 console.log '没有foo'）; 
 rep.writeHead（400）; 
 rep.end（）; 
 return; 
} 
 
 rep。 writeContinue（）; 
 handle（req，rep）; 
}）; 
 
 server.listen（8080）; 
  
您可以看到示例输入/输出此处。这将需要您的请求与相应的 Expect：标题。
 
When uploading big file (>100M) to server, PHP always accept entire data POST from browser first. We cannot inject into the process of uploading. 

For example, check the value of "token"  before entire data send to server is IMPOSSIBLE in my PHP code:
<form enctype="multipart/form-data" action="upload.php?token=XXXXXX" method="POST">
    <input type="hidden" name="MAX_FILE_SIZE" value="3000000" />
    Send this file: <input name="userfile" type="file" />
    <input type="submit" value="Send File" />
</form>
So I've try to use mod_rewrite like this:
RewriteEngine On
RewriteMap mymap prg:/tmp/map.php
RewriteCond %{QUERY_STRING} ^token=(.*)$ [NC]
RewriteRule ^/upload/fake.php$ ${mymap:%1} [L]
map.php
#!/usr/bin/php
<?php
define("REAL_TARGET", "/upload/real.php\n");
define("FORBIDDEN", "/upload/forbidden.html\n");

$handle = fopen ("php://stdin","r");
while($token = trim(fgets($handle))) {
file_put_contents("/tmp/map.log", $token."\n", FILE_APPEND);
    if (check_token($token)) {
        echo REAL_TARGET;
    } else {
        echo FORBIDDEN;
    }
}

function check_token ($token) {//do your own security check
    return substr($token,0,4) === 'alix';
}
But ... It fails again. mod_rewrite looks working too late in this situation. Data still transfer entirely.

Then I tried Node.js, like this (code snip):
var stream = new multipart.Stream(req);
stream.addListener('part', function(part) {
    sys.print(req.uri.params.token+"\n");
    if (req.uri.params.token != "xxxx") {//check token
      res.sendHeader(200, {'Content-Type': 'text/plain'});
      res.sendBody('Incorrect token!');
      res.finish();
      sys.puts("\n=> Block");
      return false;
    }
Result is ... fail again.

So please help me to find the correct path to resolve this issue or tell me there is no way.

Related questions:

Can PHP (with Apache or Nginx) check HTTP header before POST request finished?

Can some tell me how to make this script check for the password before it starts the upload process instead of after the file is uploaded?
解决方案
First of all, you can try this code yourself using the GitHub repo I created for this. Just clone the repository and run node header. 

(Spoiler, if you're reading this and are under time pressure to get something to work and not in the mood to learn ( :( ), there is a simpler solution at the end)

The general idea

This is a great question. What you are asking for is very possible and no clientside is needed, just a deeper understanding of how the HTTP protocol works while showing how node.js rocks  :) 

This can be made easy if we go one level deeper to the underlying TCP protocol and process the HTTP requests ourselves for this specific case. Node.js lets you do this easily using the built in net module. 

The HTTP Protocol

First, let's look at how HTTP requests work.

An HTTP request consists of a headers section in the general format of key:value pairs seperated by CRLF (\r\n). We know that the header section ended when we reach a double CRLF (that is \r\n\r\n). 

A typical HTTP GET request might look something like this:
GET /resource HTTP/1.1  
Cache-Control: no-cache  
User-Agent: Mozilla/5.0 

Hello=World&stuff=other
The top part before the 'empty line' is the headers section and the bottom part is the body of the request. Your request will look a bit differently in the body section since it is encoded with multipart/form-data but the header will remain similarLet's explore how this applies to us. 

TCP in nodejs

We can listen to the raw request in TCP and read the packets we get until we read that double crlf we talked about. Then we will check the short header section which we already have for whatever validation we need. After we do that, we can either end the request if validation did not pass (For example by simply ending the TCP connection), or pass it through. This allows us to not receive or read the request body, but just the headers which are much smaller.

One easy way to embed this into an already existing application is to proxy requests from it to the actual HTTP server for the specific use case.

Implementation details

This solution is as bare bones as it gets. It is just a suggestion.

Here is the work flow:

We require the net module in node.js which allows us to create tcp servers in node.js
Create a TCP server using the net module which will listen to data:  var tcpServer = net.createServer(function (socket) {... . Don't forget to tell it to listen to the correct port 


Inside that callback, listen to data events socket.on("data",function(data){ , which will trigger whenever a packet arrives.
read the data of the passed buffer from the 'data' event, and store that in a variable
check for double CRLF, this ensures that the request HEADER section has ended according to the HTTP protocol
Assuming that the validation is a header (token in your words) check it after parsing just the headers , (that is, we got the double CRLF). This also works when checking for the content-length header.
If you notice that the headers don't check out, call socket.end() which will close the connection.



Here are some things we'll use

A method for reading the headers:
function readHeaders(headers) {
    var parsedHeaders = {};
    var previous = "";    
    headers.forEach(function (val) {
        // check if the next line is actually continuing a header from previous line
        if (isContinuation(val)) {
            if (previous !== "") {
                parsedHeaders[previous] += decodeURIComponent(val.trimLeft());
                return;
            } else {
                throw new Exception("continuation, but no previous header");
            }
        }

        // parse a header that looks like : "name: SP value".
        var index = val.indexOf(":");

        if (index === -1) {
            throw new Exception("bad header structure: ");
        }

        var head = val.substr(0, index).toLowerCase();
        var value = val.substr(index + 1).trimLeft();

        previous = head;
        if (value !== "") {
            parsedHeaders[head] = decodeURIComponent(value);
        } else {
            parsedHeaders[head] = null;
        }
    });
    return parsedHeaders;
};
A method for checking double CRLF in a buffer you get on a data event, and return its location if it exists in an object:
function checkForCRLF(data) {
    if (!Buffer.isBuffer(data)) {
        data = new Buffer(data,"utf-8");
    }
    for (var i = 0; i < data.length - 1; i++) {
        if (data[i] === 13) { //\r
            if (data[i + 1] === 10) { //\n
                if (i + 3 < data.length && data[i + 2] === 13 && data[i + 3] === 10) {
                    return { loc: i, after: i + 4 };
                }
            }
        } else if (data[i] === 10) { //\n

            if (data[i + 1] === 10) { //\n
                return { loc: i, after: i + 2 };
            }
        }
    }    
    return { loc: -1, after: -1337 };
};
And this small utility method:
function isContinuation(str) {
    return str.charAt(0) === " " || str.charAt(0) === "\t";
}


Implementation

var net = require("net"); // To use the node net module for TCP server. Node has equivalent modules for secure communication if you'd like to use HTTPS

//Create the server
var server = net.createServer(function(socket){ // Create a TCP server
    var req = []; //buffers so far, to save the data in case the headers don't arrive in a single packet
    socket.on("data",function(data){
        req.push(data); // add the new buffer
        var check = checkForCRLF(data);
        if(check.loc !== -1){ // This means we got to the end of the headers!
            var dataUpToHeaders= req.map(function(x){
                return x.toString();//get buffer strings
            }).join("");
            //get data up to /r/n
            dataUpToHeaders = dataUpToHeaders.substring(0,check.after);
            //split by line
            var headerList = dataUpToHeaders.trim().split("\r\n");
            headerList.shift() ;// remove the request line itself, eg GET / HTTP1.1
            console.log("Got headers!");
            //Read the headers
            var headerObject = readHeaders(headerList);
            //Get the header with your token
            console.log(headerObject["your-header-name"]);

            // Now perform all checks you need for it
            /*
            if(!yourHeaderValueValid){
                socket.end();
            }else{
                         //continue reading request body, and pass control to whatever logic you want!
            }
            */


        }
    });
}).listen(8080); // listen to port 8080 for the sake of the example
If you have any questions feel free to ask :)

Ok, I lied, there is a simpler way!

But what's the fun in that? If you skipped here initially, you wouldn't learn how HTTP works :)

Node.js has a built in http module. Since requests are chunked by nature in node.js, especially long requests, you can implement the same thing without the more advanced understanding of the protocol.

This time, let's use the http module to create an http server
server = http.createServer( function(req, res) { //create an HTTP server
    // The parameters are request/response objects
    // check if method is post, and the headers contain your value.
    // The connection was established but the body wasn't sent yet,
    // More information on how this works is in the above solution
    var specialRequest = (req.method == "POST") && req.headers["YourHeader"] === "YourTokenValue";
    if(specialRequest ){ // detect requests for special treatment
      // same as TCP direct solution add chunks
      req.on('data',function(chunkOfBody){
              //handle a chunk of the message body
      });
    }else{
        res.end(); // abort the underlying TCP connection, since the request and response use the same TCP connection this will work
        //req.destroy() // destroy the request in a non-clean matter, probably not what you want.
    }
}).listen(8080);
This is based on the fact the request handle in a nodejs http module actually hooks on after the headers were sent (but nothing else was performed) by default. (this in the server module , this in the parser module)

User igorw suggested a somewhat cleaner solution using the 100 Continue header assuming browsers you're targeting supports it. 100 Continue is a status code designed to do exactly what you're attempting to:

  
The purpose of the 100 (Continue) status (see section 10.1.1) is to
  allow a client that is sending a request message with a request body
  to determine if the origin server is willing to accept the request
  (based on the request headers) before the client sends the request
  body. In some cases, it might either be inappropriate or highly
  inefficient for the client to send the body if the server will reject
  the message without looking at the body.
Here it is :
var http = require('http');

function handle(req, rep) {
    req.pipe(process.stdout); // pipe the request to the output stream for further handling
    req.on('end', function () {
        rep.end();
        console.log('');
    });
}

var server = new http.Server();

server.on('checkContinue', function (req, rep) {
    if (!req.headers['x-foo']) {
        console.log('did not have foo');
        rep.writeHead(400);
        rep.end();
        return;
    }

    rep.writeContinue();
    handle(req, rep);
});

server.listen(8080);
You can see sample input/output here. This would require your request to fire with the appropriate Expect: header.

                        
这篇关于通过PHP或Apache从服务器端上传HTTP文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！