使用Laravel从Web服务器流式传输Amazon S3对象 [英] Streaming Amazon S3 Objects From a Web Server Using Laravel
问题描述
在使用laravel 5.1构建的Web应用程序中，用户可以上传一些我存储在Amazon S3中的敏感文件.稍后，我希望用户具有权限下载此文件.由于我希望此auth检查到位，因此我无法使用传统方法通过将文件直接链接到S3中的文件来下载文件.
In my web-application built using laravel 5.1, users can upload some sensitive files that I store in Amazon S3. Later I want users WITH PERMISSION to download this file. Since I want this auth check in place, I cannot download the file using traditional methods by giving them direct link to file in S3.
我的方法:
-
当用户请求下载时，我的服务器在本地下载文件，然后流式传输给用户. 问题:需要很长时间，因为有时文件太大.
When user requests a download, my servers download the file locally and then stream to user. Issue: Takes long time because files are too large sometimes.
为用户提供一个预签名的URL，以直接从S3下载.网址仅有效5分钟. 问题:如果该URL是共享的，则任何人都可以在5分钟内下载它.
Give the user a pre-signed URL to download directly from S3. URL is only valid for 5 minutes. Issue: If that URL is shared, anyone can download it within 5 minutes.
根据本文，将数据直接从S3流传输到客户端.这看起来很有希望，但我不知道如何实现.
According to this article, stream the data directly from S3 to clients. This looks promising, but I don't know how to implement this.
根据本文，我需要:
- 注册流包装器-，这是我的第一个问题，因为我不知道如何获得
S3Client对象的保全，因为laravel使用flysystem，而且我也不知道该调用什么方法得到这个对象.也许我需要在composer.json中单独包含S3软件包? - 禁用输出缓冲-我需要在laravel中执行此操作还是laravel已经处理了?
- Register stream wrapper - which is my first problem because I don't know how to get the hold of
S3Clientobject because laravel uses flysystem, and I don't know what methods to call to get this object. Maybe I need to include S3 package separately in my composer.json ? - Disable output buffering - Do I need to do this in laravel or does laravel has already taken care of it ?
我确信其他开发人员以前也曾遇到过此类问题，并希望获得一些帮助指示.如果有人已经使用laravel Response::download($pathToFile, $name, $headers)直接从S3流式传输到客户端，那么我很想听听您的方法.
I am sure other developers have seen such an issue before and would like to get some help pointers. If anybody has already streamed directly from S3 to client using laravel Response::download($pathToFile, $name, $headers), then I would love to hear your methods.
推荐答案
从评论中的讨论中，我得出了一些我想分享的要点.
From discussion in comments, I have arrived at some key points that I would like to share.
Pre-Signed URLs
Pre-Signed URLs
正如@ceejayoz指出的那样，pre-signed URL并不是一个坏主意，因为:
As @ceejayoz pointed out, pre-signed URLs are not a bad idea because:
- 我可以将时间保持在10秒之内，这对于任何重定向和开始下载都是完美的选择，但不足以共享链接.
- 我以前的理解是下载必须在给定的时间内完成.因此，如果链接在10秒钟后过期，则必须在此之前进行下载.但是@ceejayoz指出并非如此.已开始的下载允许完成.
- 使用
cloud front，我还可以限制IP地址，以增加安全性.
- I can keep time as low as 10 seconds which is perfect for any redirects and to start download, but not enough for the link to be shared.
- My previous understanding was that the download has to finish in the given time. So if the link expires in 10 seconds, the download has to happen before that. But @ceejayoz pointed that is not the case. The download which have started is allowed to finish.
- With
cloud front, I can also restrict on the IP address, to add more security.
IAM Roles
IAM Roles
他还指出了另一种不太好用的方法-创建临时IAM用户.如果未正确完成，这将是维护的噩梦，因此只有在知道自己在做什么的情况下，才进行维护.
He also pointed out another not so great method - to create temporary IAM users. This is a maintenance nightmare if not done correctly, so only do if you know what you are doing.
Stream From S3
Stream From S3
这是我现在选择的方法.也许以后我将转到第一种方法.
This is the method that I have chosen for now. Maybe later I will move to the first method.
警告::如果进行流传输，则您的服务器仍然是中间人，所有数据将通过您的服务器传输.因此，如果失败或速度缓慢，则下载速度将很慢.
Warning: If you stream, then your server is still the middleman and all the data will go via your server. So if it fails, or is slow, your download will be slow.
我的第一个问题是 how to register stream wrapper:
由于我使用Laravel，而laravel使用flysystem进行S3管理，所以没有简单的方法来获取S3Client.因此，我在composer.json
Since I am using Laravel and laravel uses flysystem for S3 management, there was no easy way for me to get the S3Client. Hence I added additional package AWS SDK for Laravel in my composer.json
"aws/aws-sdk-php-laravel" : "~3.0"
然后，我将代码编写如下:
Then I wrote my code as follows:
class FileDelivery extends Command implements SelfHandling
{
private $client;
private $remoteFile;
private $bucket;
public function __construct($remoteFile)
{
$this->client = AWS::createClient('s3');
$this->client->registerStreamWrapper();
$this->bucket = 'mybucket';
$this->remoteFile = $remoteFile;
}
public function handle()
{
try
{
// First get the meta-data of the object.
$headers = $this->client->headObject(array(
'Bucket' => $this->bucket,
'Key' => $this->remoteFile
));
$headers = $headers['@metadata'];
if($headers['statusCode'] !== 200)
{
throw new S3Exception();
}
}
catch(S3Exception $e)
{
return 404;
}
// return appropriate headers before the stream starts.
http_response_code($headers['statusCode']);
header("Last-Modified: {$headers['headers']['last-modified']}");
header("ETag: {$headers['headers']['etag']}");
header("Content-Type: {$headers['headers']['content-type']}");
header("Content-Length: {$headers['headers']['content-length']}");
header("Content-Disposition: attachment; filename=\"{$this->filename}\"");
// Since file sizes can be too large,
// buffers can suffer because they cannot store huge amounts of data.
// Thus we disable buffering before stream starts.
// We also flush anything pending in buffer.
if(ob_get_level())
{
ob_end_flush();
}
flush();
// Start the stream.
readfile("s3://{$this->bucket}/{$this->remoteFile}");
}
}
我的第二个问题是在laravel中的Do I need to Disable output buffering吗?
My second question was Do I need to Disable output buffering in laravel?
恕我直言，答案是肯定的.缓冲使数据立即从缓冲区中清除，从而降低了内存消耗.由于我们没有使用任何laravel函数将数据分流到客户端，因此laravel不能做到这一点，因此需要我们自己完成.
The answer IMHO is yes. The buffering lets the data flushed immediately from the buffer, resulting in lower memory consumption. Since we are not using any laravel function to offload the data to client, this is not done by laravel and hence needs to be done by us.
这篇关于使用Laravel从Web服务器流式传输Amazon S3对象的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
