使用 rails、nginx 和 send_file 在 Chrome 中流式传输 mp4 [英] Streaming mp4 in Chrome with rails, nginx and send_file
问题描述
我终生无法使用 html5 标签将 mp4 流式传输到 Chrome.如果我将文件放在 public 中，那么一切都会如预期的那样工作.但是如果我尝试使用 send_file 来提供它，几乎所有可以想象的都会出错.我正在使用由 nginx 代理的 rails 应用程序，其 Video 模型具有 location 属性，该属性是磁盘上的绝对路径.
I can't for the life of me stream a mp4 to Chrome with a html5 <video> tag. If I drop the file in public then everything is gravy and works as expected. But if I try to serve it using send_file, pretty much everything imaginable goes wrong. I am using a rails app that is proxied by nginx, with a Video model that has a location attribute that is an absolute path on disk.
起初我尝试过:
def show
send_file Video.find(params[:id]).location
end
而且我确信我会沉浸在现代 Web 开发的荣耀中.哈.这会在 Chrome 和 Firefox 中播放，但既不搜索也不知道视频的长度.我戳了一下响应头，意识到 Content-Type 是作为 application/octet-stream 发送的，并且没有设置 Content-Length.嗯……什么?
And I was sure I would be basking in the glory that is modern web development. Ha. This plays in both Chrome and Firefox, but neither seek and neither have any idea how long the video is. I poked at the response headers and realized that Content-Type is being sent as application/octet-stream and there is no Content-Length set. Umm... wth?
好的，我想我可以在 rails 中设置它们:
Okay, I guess I can set those in rails:
def show
video = Video.find(params[:id])
response.headers['Content-Length'] = File.stat(video.location).size
send_file(video.location, type: 'video/mp4')
end
在这一点上，一切都在 Firefox 中按预期工作.它知道视频的长度并按预期进行搜索.Chrome 似乎知道视频的长度(不显示时间戳，但搜索栏看起来很合适)但搜索不起作用.
At this point everything works pretty much as expected in Firefox. It knows how long the video is and seeking works as expected. Chrome appears to know how long the video is (doesn't show timestamps, but seek bar looks appropriate) but seeking doesn't work.
显然 Chrome 比 Firefox 更挑剔.它要求服务器以 Accept-Ranges 标头响应，值为 bytes，并以 206 响应后续请求(在用户搜索时发生)> 和文件的适当部分.
Apparently Chrome is pickier than Firefox. It requires that the server respond with a Accept-Ranges header with value bytes and respond to subsequent requests (that happen when the users seeks) with 206 and the appropriate portion of the file.
好的，所以我从这里借用了一些代码，然后我有了这个:
Okay, so I borrowed some code from here and then I had this:
video = Video.find(params[:id])
file_begin = 0
file_size = File.stat(video.location).size
file_end = file_size - 1
if !request.headers["Range"]
status_code = :ok
else
status_code = :partial_content
match = request.headers['Range'].match(/bytes=(\d+)-(\d*)/)
if match
file_begin = match[1]
file_end = match[2] if match[2] && !match[2].empty?
end
response.header["Content-Range"] = "bytes " + file_begin.to_s + "-" + file_end.to_s + "/" + file_size.to_s
end
response.header["Content-Length"] = (file_end.to_i - file_begin.to_i + 1).to_s
response.header["Accept-Ranges"]= "bytes"
response.header["Content-Transfer-Encoding"] = "binary"
send_file(video.location,
:filename => File.basename(video.location),
:type => 'video/mp4',
:disposition => "inline",
:status => status_code,
:stream => 'true',
:buffer_size => 4096)
现在 Chrome 会尝试搜索，但是当您执行此操作时，视频会停止播放，并且在页面重新加载之前不会再次运行.啊.所以我决定玩转 curl 看看发生了什么，我发现了这一点:
Now Chrome attempts to seek, but when you do the video stops playing and never works again until the page reloads. Argh. So I decided to play around with curl to see what was happening and I discovered this:
$ curl --header "Range: bytes=200-400" http://localhost:8080/videos/1/001.mp4ftypisomisomiso2avc1mp41 moovlmvhd @ trak\tkh
$ curl --header "Range: bytes=200-400" http://localhost:8080/videos/1/001.mp4 ftypisomisomiso2avc1mp41 �moovlmvhd��@��trak\tkh��
$ curl --header "Range: bytes=1200-1400" http://localhost:8080/videos/1/001.mp4ftypisomisomiso2avc1mp41 moovlmvhd @ trak\tkh
$ curl --header "Range: bytes=1200-1400" http://localhost:8080/videos/1/001.mp4 ftypisomisomiso2avc1mp41 �moovlmvhd��@��trak\tkh��
无论字节范围请求如何，数据总是从文件的开头开始.返回适当数量的字节(在本例中为 201 个字节)，但它始终从文件的开头开始.显然 nginx 尊重 Content-Length 标头，但忽略 Content-Range 标头.
No matter the byte range request, the data always starts from the beginning of the file. The appropriate amount of bytes is returned (201 bytes in this case), but it's always from the beginning of the file. Apparently nginx respects the Content-Length header but ignores the Content-Range header.
我的 nginx.conf 默认未修改:
user www-data;
worker_processes 4;
pid /run/nginx.pid;
events {
worker_connections 768;
}
http {
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
include /etc/nginx/mime.types;
default_type application/octet-stream;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
ssl_prefer_server_ciphers on;
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
gzip on;
gzip_disable "msie6";
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
}
我的 app.conf 非常基础:
and my app.conf is pretty basic:
upstream unicorn {
server unix:/tmp/unicorn.app.sock fail_timeout=0;
}
server {
listen 80 default deferred;
root /vagrant/public;
try_files $uri/index.html $uri @unicorn;
location @unicorn {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header HOST $http_host;
proxy_redirect off;
proxy_pass http://unicorn;
}
error_page 500 502 503 504 /500.html;
client_max_body_size 4G;
keepalive_timeout 5;
}
首先我尝试了 Ubuntu 14.04 附带的 nginx 1.4.x，然后从 ppa 尝试了 1.7.x - 结果相同.我什至尝试了 apache2 并得到了完全相同的结果.
First I tried the nginx 1.4.x that comes with Ubuntu 14.04, then tried 1.7.x from a ppa - same results. I even tried apache2 and had exactly the same results.
我想重申，视频文件不是问题.如果我将它放在 public 中，那么 nginx 会为它提供适当的 MIME 类型、标题以及 Chrome 正常工作所需的一切.
I would like to reiterate that the video file is not the problem. If I drop it in public then nginx serves it with the appropriate mime types, headers and everything needed for Chrome to work properly.
所以我的问题分为两个部分:
So my question is a two-parter:
为什么 nginx/apache 不使用
send_file(X-Accel-Redirect/X-Sendfile>) 就像从public静态提供文件时那样?在 Rails 中处理这些东西太落后了.
Why doesn't nginx/apache handle all this stuff automagically with
send_file(X-Accel-Redirect/X-Sendfile) like it does when the file is served statically frompublic? Handling this stuff in rails is so backwards.
我到底如何才能将 send_file 与 nginx(或 apache)一起使用才能让 Chrome 满意并允许搜索?
How the heck can I actually use send_file with nginx (or apache) so that Chrome will be happy and allow seeking?
更新 1
好的，所以我想我会尝试将 rails 的复杂性从图片中去掉，看看我是否可以让 nginx 正确地代理文件.所以我启动了一个非常简单的 nodjs 服务器:
Okay, so I thought I'd try to take the complication of rails out of the picture and just see if I could get nginx to proxy the file correctly. So I spun up a dead-simple nodjs server:
var http = require('http');
http.createServer(function (req, res) {
res.writeHead(200, {
'X-Accel-Redirect': '/path/to/file.mp4'
});
res.end();
}).listen(3000, '127.0.0.1');
console.log('Server running at http://127.0.0.1:3000/');
chrome 高兴得像个蛤蜊.=/curl -I 甚至显示 Accept-Ranges: bytes 和 Content-Type: video/mp4 正在由 nginx 自动插入 - 如应该.rails 会做什么阻止 nginx 这样做?
And chrome is happy as a clam. =/ curl -I even shows that Accept-Ranges: bytes and Content-Type: video/mp4 is being inserted by nginx automagically - as it should be. What could rails be doing that's preventing nginx from doing this?
更新 2
我可能会越来越近...
I might be getting closer...
如果我有:
def show
video = Video.find(params[:id])
send_file video.location
end
然后我得到:
$ curl -I localhost:8080/videos/1/001.mp4
HTTP/1.1 200 OK
Server: nginx/1.7.9
Date: Sun, 18 Jan 2015 12:06:38 GMT
Content-Type: application/octet-stream
Connection: keep-alive
Status: 200 OK
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
Content-Disposition: attachment; filename="001.mp4"
Content-Transfer-Encoding: binary
Cache-Control: private
Set-Cookie: request_method=HEAD; path=/
X-Meta-Request-Version: 0.3.4
X-Request-Id: cd80b6e8-2eaa-4575-8241-d86067527094
X-Runtime: 0.041953
而且我遇到了上述所有问题.
And I have all the problems described above.
但如果我有:
def show
video = Video.find(params[:id])
response.headers['X-Accel-Redirect'] = video.location
head :ok
end
然后我得到:
$ curl -I localhost:8080/videos/1/001.mp4
HTTP/1.1 200 OK
Server: nginx/1.7.9
Date: Sun, 18 Jan 2015 12:06:02 GMT
Content-Type: text/html
Content-Length: 186884698
Last-Modified: Sun, 18 Jan 2015 03:49:30 GMT
Connection: keep-alive
Cache-Control: max-age=0, private, must-revalidate
Set-Cookie: request_method=HEAD; path=/
ETag: "54bb2d4a-b23a25a"
Accept-Ranges: bytes
而且一切正常.
但是为什么?那些应该做完全相同的事情.为什么 nginx 不像简单的 nodejs 示例那样在这里自动设置 Content-Type ?我有 config.action_dispatch.x_sendfile_header = 'X-Accel-Redirect' 设置.我在 application.rb 和 development.rb 之间来回移动它，结果相同.我想我从来没有提到过……这是 Rails 4.2.0.
But why? Those should do exactly the same thing. And why doesn't nginx set Content-Type automagically here like it does for the simple nodejs example? I have config.action_dispatch.x_sendfile_header = 'X-Accel-Redirect' set. I have moved it back and forth between application.rb and development.rb with the same results. I guess I never mentioned... this is rails 4.2.0.
更新 3
现在我已经将我的独角兽服务器更改为侦听端口 3000(因为对于 nodejs 示例，我已经将 nginx 更改为侦听 3000).现在我可以直接向 unicorn 发出请求(因为它正在侦听端口而不是套接字)所以我发现 curl -I 直接向 unicorn 显示没有 X-Accel-Redirect 标头被发送，只是 curling unicorn 直接实际发送文件.就像 send_file 没有做它应该做的一样.
Now I've changed my unicorn server to listen on port 3000 (since I already changed nginx to listen on 3000 for the nodejs example). Now I can make requests directly to unicorn (since it's listening on a port and not a socket) so I have found that curl -I directly to unicorn shows that no X-Accel-Redirect header is sent and just curling unicorn directly actually sends the file. It's like send_file isn't doing what it's supposed to.
推荐答案
我终于得到了我最初问题的答案.我没想到我会来到这里.我所有的研究都导致了死胡同、笨拙的非解决方案和它只是开箱即用"(好吧，不适合我).
I finally have the answers to my original questions. I didn't think I'd ever get here. All my research had lead to dead-ends, hacky non-solutions and "it just works out of the box" (well, not for me).
为什么 nginx/apache 不像从公共静态提供文件时那样使用 send_file (X-Accel-Redirect/X-Sendfile) 自动处理所有这些东西?在 Rails 中处理这些东西太落后了.
Why doesn't nginx/apache handle all this stuff automagically with send_file (X-Accel-Redirect/X-Sendfile) like it does when the file is served statically from public? Handling this stuff in rails is so backwards.
它们可以，但必须正确配置才能取悦 Rack::Sendfile(见下文).试图在 Rails 中处理这个问题是一个非常棘手的非解决方案.
They do, but they have to be configured properly to please Rack::Sendfile (see below). Trying to handle this in rails is a hacky non-solution.
我到底如何才能将 send_file 与 nginx(或 apache)一起使用，以便 Chrome 会高兴并允许搜索?
How the heck can I actually use send_file with nginx (or apache) so that Chrome will be happy and allow seeking?
我迫不及待地开始研究机架源代码，这就是我在Rack::Sendfile 的评论中找到答案的地方.它们采用文档结构，您可以在 rubydoc 中找到.
I got desperate enough to start poking around rack source code and that's where I found my answer, in the comments of Rack::Sendfile. They are structured as documentation that you can find at rubydoc.
无论出于何种原因，Rack::Sendfile 要求前端代理发送一个 X-Sendfile-Type 标头.在 nginx 的情况下，它还需要一个 X-Accel-Mapping 标头.该文档还包含 apache 和 lighttpd 的示例.
For whatever reason, Rack::Sendfile requires the front end proxy to send a X-Sendfile-Type header. In the case of nginx it also requires a X-Accel-Mapping header. The documentation also has examples for apache and lighttpd as well.
人们会认为 rails 文档可以链接到 Rack::Sendfile 文档，因为 send_file 在没有额外配置的情况下无法开箱即用.也许我会提交拉取请求.
One would think the rails documentation could link to the Rack::Sendfile documentation since send_file does not work out of the box without additional configuration. Perhaps I'll submit a pull request.
最后我只需要在 app.conf 中添加几行:
In the end I only needed to add a couple lines to my app.conf:
upstream unicorn {
server unix:/tmp/unicorn.app.sock fail_timeout=0;
}
server {
listen 80 default deferred;
root /vagrant/public;
try_files $uri/index.html $uri @unicorn;
location @unicorn {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header HOST $http_host;
proxy_set_header X-Sendfile-Type X-Accel-Redirect; # ADDITION
proxy_set_header X-Accel-Mapping /=/; # ADDITION
proxy_redirect off;
proxy_pass http://localhost:3000;
}
error_page 500 502 503 504 /500.html;
client_max_body_size 4G;
keepalive_timeout 5;
}
现在我的原始代码按预期工作:
Now my original code works as expected:
def show
send_file(Video.find(params[:id]).location)
end
虽然这最初有效，但在我重新启动流浪箱后它停止工作，我不得不进行进一步的更改:
Although this worked initially, it stopped working after I restarted my vagrant box and I had to make further changes:
upstream unicorn {
server unix:/tmp/unicorn.app.sock fail_timeout=0;
}
server {
listen 80 default deferred;
root /vagrant/public;
try_files $uri/index.html $uri @unicorn;
location ~ /files(.*) { # NEW
internal; # NEW
alias $1; # NEW
} # NEW
location @unicorn {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header HOST $http_host;
proxy_set_header X-Sendfile-Type X-Accel-Redirect;
proxy_set_header X-Accel-Mapping /=/files/; # CHANGED
proxy_redirect off;
proxy_pass http://localhost:3000;
}
error_page 500 502 503 504 /500.html;
client_max_body_size 4G;
keepalive_timeout 5;
}
我发现将一个 URI 映射到另一个，然后将该 URI 映射到磁盘上的某个位置的整个过程是完全没有必要的.它对我的用例毫无用处，我只是将一个映射到另一个然后再映射回来.Apache 和 lighttpd 不需要它.但至少它有效.
I find this whole thing of mapping one URI to another and then mapping that URI to a location on disk to be totally unnecessary. It's useless for my use case and I'm just mapping one to another and back again. Apache and lighttpd don't require it. But at least it works.
我还添加了 Mime::Type.register('video/mp4', :mp4) 到 config/initializers/mime_types.rb 以便提供文件正确的 mime 类型.
I also added Mime::Type.register('video/mp4', :mp4) to config/initializers/mime_types.rb so the file is served with the correct mime type.
这篇关于使用 rails、nginx 和 send_file 在 Chrome 中流式传输 mp4的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
