机架rack.input变量被截断了吗? [英] Rack rack.input variable getting truncated?
问题描述
我写了一段Rack Middleware来自动解压缩压缩的请求正文.该代码似乎工作正常，但是当我将其插入Rails应用程序时，我从ActionController :: ParamsParser中收到无效的JSON"错误.
作为一种调试机制，我正在将压缩的内容和未压缩的内容都写到文件中(以确保代码正常工作)，并且确实收到了原始的JSON文档(在客户端将其压缩之前)
我发布的数据是 JSON数据，并且从 http ://jsonlint.com .
有什么想法我在做什么错吗?
class CompressedRequests
def initialize(app)
@app = app
end
def call(env)
input = env['rack.input'].read
#output the zipped data we received
File.open('/Users/ben/Desktop/data.gz', 'w+') do |f|
f.write input
end
if env['REQUEST_METHOD'] =~ /(POST|PUT)/
if env.keys.include? 'HTTP_CONTENT_ENCODING'
new_input = decode(input, env['HTTP_CONTENT_ENCODING'])
env['rack.input'] = StringIO.new(new_input)
#output our decoded data (for debugging)
File.open('/Users/ben/Desktop/data.txt', 'w+') do |f|
f.write env['rack.input'].read
end
env.delete('HTTP_CONTENT_ENCODING')
end
end
env['rack.input'].rewind
status, headers, response = @app.call(env)
return [status, headers, response]
end
def decode(input, content_encoding)
if content_encoding == 'gzip'
Zlib::GzipReader.new(input).read
elsif content_encoding == 'deflate'
Zlib::Inflate.new.inflate new input
else
input
end
end
end
这是我从控制台收到的错误:
Contents::"2010-05-17T12:46:30Z","background":false},{"longitude":-95.38620785000001,"latitude":29.62815358333334,"estimated_speed":14.04305,"timestamp":"2010-05-17T12:46:36Z","background":false},{"longitude":-95.3862767,"latitude":29.62926725,"estimated_speed":39.87791,"timestamp":"2010-05-17T12:46:42Z","background":false},{"longitude":-95.38655023333334,"latitude":29.63051011666666,"estimated_speed":46.09239,"timestamp":"2010-05-17T12:46:49Z","background":false},{"longitude":-95.38676226666666,"latitude":29.63158775,"estimated_speed":47.34936,"timestamp":"2010-05-17T12:46:55Z","background":false},{"longitude":-95.38675346666666,"latitude":29.63219841666666,"estimated_speed":22.54016,"timestamp":"2010-05-17T12:47:03Z","background":false},{"longitude":-95.38675491666666,"latitude":29.63265714999999,"estimated_speed":14.03642,"timestamp":"2010-05-17T12:47:10Z","background":false},{"longitude":-95.38677551666666,"latitude":29.63358661666667,"estimated_speed":29.29489,"timestamp":"2010-05-17T12:47:17Z","background":false},{"longitude":-95.38679026666662,"latitude":29.63466445,"estimated_speed":38.34926,"timestamp":"2010-05-17T12:47:24Z","background":false},{"longitude":-95.38681656666668,"latitude":29.63590941666666,"estimated_speed":44.82093,"timestamp":"2010-05-17T12:47:31Z","background":false},{"longitude":-95.38683366666667,"latitude":29.63679638333334,"estimated_speed":40.21729,"timestamp":"2010-05-17T12:47:37Z","background":false},{"longitude":-95.38685133333333,"latitude":29.63815714999999,"estimated_speed":44.86543,"timestamp":"2010-05-17T12:47:44Z","background":false},{"longitude":-95.3868655
/!\ FAILSAFE /!\ Mon Oct 18 18:18:43 -0500 2010
Status: 500 Internal Server Error
Invalid JSON string
/Library/Ruby/Gems/1.8/gems/activesupport-2.3.5/lib/active_support/json/backends/yaml.rb:14:in `decode'
/Library/Ruby/Gems/1.8/gems/activesupport-2.3.5/lib/active_support/json/decoding.rb:11:in `__send__'
/Library/Ruby/Gems/1.8/gems/activesupport-2.3.5/lib/active_support/json/decoding.rb:11:in `decode'
/Library/Ruby/Gems/1.8/gems/actionpack-2.3.5/lib/action_controller/params_parser.rb:42:in `parse_formatted_parameters'
/Library/Ruby/Gems/1.8/gems/actionpack-2.3.5/lib/action_controller/params_parser.rb:11:in `call'
/Library/Ruby/Gems/1.8/gems/actionpack-2.3.5/lib/action_controller/session/cookie_store.rb:93:in `call'
/Library/Ruby/Gems/1.8/gems/actionpack-2.3.5/lib/action_controller/failsafe.rb:26:in `call'
/Users/ben/projects/safecell/safecellweb/lib/compressed_requests.rb:36:in `call'
/Library/Ruby/Gems/1.8/gems/rack-1.0.1/lib/rack/lock.rb:11:in `call'
/Library/Ruby/Gems/1.8/gems/rack-1.0.1/lib/rack/lock.rb:11:in `synchronize'
/Library/Ruby/Gems/1.8/gems/rack-1.0.1/lib/rack/lock.rb:11:in `call'
/Library/Ruby/Gems/1.8/gems/actionpack-2.3.5/lib/action_controller/dispatcher.rb:114:in `call'
/Library/Ruby/Gems/1.8/gems/actionpack-2.3.5/lib/action_controller/reloader.rb:34:in `run'
/Library/Ruby/Gems/1.8/gems/actionpack-2.3.5/lib/action_controller/dispatcher.rb:108:in `call'
/Library/Ruby/Gems/1.8/gems/rails-2.3.5/lib/rails/rack/static.rb:31:in `call'
/Library/Ruby/Gems/1.8/gems/rack-1.0.1/lib/rack/urlmap.rb:46:in `call'
/Library/Ruby/Gems/1.8/gems/rack-1.0.1/lib/rack/urlmap.rb:40:in `each'
/Library/Ruby/Gems/1.8/gems/rack-1.0.1/lib/rack/urlmap.rb:40:in `call'
/Library/Ruby/Gems/1.8/gems/rails-2.3.5/lib/rails/rack/log_tailer.rb:17:in `call'
/Library/Ruby/Gems/1.8/gems/rack-1.0.1/lib/rack/content_length.rb:13:in `call'
/Library/Ruby/Gems/1.8/gems/rack-1.0.1/lib/rack/handler/webrick.rb:50:in `service'
最后一条信息，我是将此中间件插入ActionController :: Failsafe之后.
看起来这不是截断问题
更多的挖掘之后，看来这根本不是一个截断问题.日志只是剪切输出，因此它看起来像截断问题.
在这一点上，我不确定为什么JSON无效.我需要进行任何手动转义吗?
无论如何我都不是红宝石专家.我也没有尝试过重现此问题以验证我的结果.但是，在仔细研究了机架和动作包代码之后，我可能会有所收获.
"rack.input"的文档说明: 输入流是一个类似IO的对象，其中包含原始HTTP POST数据."
看来您正确使用了它.
但是，actionpack尝试从主体中解析JSON(如果将内容类型指定为JSON)并像这样检索主体:
when :json
body = request.raw_post
其中"request"是actionpack自己的Request类，"raw_post"的定义如下:
def raw_post
unless @env.include? 'RAW_POST_DATA'
@env['RAW_POST_DATA'] = body.read(@env['CONTENT_LENGTH'].to_i)
body.rewind if body.respond_to?(:rewind)
end
@env['RAW_POST_DATA']
end
和"Request.body"是:
def body
if raw_post = @env['RAW_POST_DATA']
raw_post.force_encoding(Encoding::BINARY) if raw_post.respond_to?(:force_encoding)
StringIO.new(raw_post)
else
@env['rack.input']
end
end
一切看起来都不错，也很好(尽管弄清楚谁先缓存了值很困惑:)). 看起来问题出在如何读取帖子数据:
@env['RAW_POST_DATA'] = body.read(@env['CONTENT_LENGTH'].to_i)
所以我猜测问题是，既然您更改了"rack.input"但没有更新"CONTENT_LENGTH"，则由于压缩后的内容显然比未压缩的内容要短，因此actionpack会截断数据. /p>
尝试更新中间件代码中的"CONTENT_LENGTH"，看看是否可以解决.
I wrote a piece of Rack Middleware to automatically unzip compressed request bodies. The code seems to be working just fine, but when I plug it into my rails app, I get a failure "Invalid JSON" from ActionController::ParamsParser.
As a debugging mechanism, I'm writing both the zipped content, and the unzipped content to a file (to make sure that code is working properly) and I do receive my original JSON document (before the client zips it up).
The data I'm posting is JSON data, and the unzipped content is detected as valid JSON from http://jsonlint.com.
Any ideas what I'm doing wrong?
class CompressedRequests
def initialize(app)
@app = app
end
def call(env)
input = env['rack.input'].read
#output the zipped data we received
File.open('/Users/ben/Desktop/data.gz', 'w+') do |f|
f.write input
end
if env['REQUEST_METHOD'] =~ /(POST|PUT)/
if env.keys.include? 'HTTP_CONTENT_ENCODING'
new_input = decode(input, env['HTTP_CONTENT_ENCODING'])
env['rack.input'] = StringIO.new(new_input)
#output our decoded data (for debugging)
File.open('/Users/ben/Desktop/data.txt', 'w+') do |f|
f.write env['rack.input'].read
end
env.delete('HTTP_CONTENT_ENCODING')
end
end
env['rack.input'].rewind
status, headers, response = @app.call(env)
return [status, headers, response]
end
def decode(input, content_encoding)
if content_encoding == 'gzip'
Zlib::GzipReader.new(input).read
elsif content_encoding == 'deflate'
Zlib::Inflate.new.inflate new input
else
input
end
end
end
Here is the error that I'm getting from the console:
Contents::"2010-05-17T12:46:30Z","background":false},{"longitude":-95.38620785000001,"latitude":29.62815358333334,"estimated_speed":14.04305,"timestamp":"2010-05-17T12:46:36Z","background":false},{"longitude":-95.3862767,"latitude":29.62926725,"estimated_speed":39.87791,"timestamp":"2010-05-17T12:46:42Z","background":false},{"longitude":-95.38655023333334,"latitude":29.63051011666666,"estimated_speed":46.09239,"timestamp":"2010-05-17T12:46:49Z","background":false},{"longitude":-95.38676226666666,"latitude":29.63158775,"estimated_speed":47.34936,"timestamp":"2010-05-17T12:46:55Z","background":false},{"longitude":-95.38675346666666,"latitude":29.63219841666666,"estimated_speed":22.54016,"timestamp":"2010-05-17T12:47:03Z","background":false},{"longitude":-95.38675491666666,"latitude":29.63265714999999,"estimated_speed":14.03642,"timestamp":"2010-05-17T12:47:10Z","background":false},{"longitude":-95.38677551666666,"latitude":29.63358661666667,"estimated_speed":29.29489,"timestamp":"2010-05-17T12:47:17Z","background":false},{"longitude":-95.38679026666662,"latitude":29.63466445,"estimated_speed":38.34926,"timestamp":"2010-05-17T12:47:24Z","background":false},{"longitude":-95.38681656666668,"latitude":29.63590941666666,"estimated_speed":44.82093,"timestamp":"2010-05-17T12:47:31Z","background":false},{"longitude":-95.38683366666667,"latitude":29.63679638333334,"estimated_speed":40.21729,"timestamp":"2010-05-17T12:47:37Z","background":false},{"longitude":-95.38685133333333,"latitude":29.63815714999999,"estimated_speed":44.86543,"timestamp":"2010-05-17T12:47:44Z","background":false},{"longitude":-95.3868655
/!\ FAILSAFE /!\ Mon Oct 18 18:18:43 -0500 2010
Status: 500 Internal Server Error
Invalid JSON string
/Library/Ruby/Gems/1.8/gems/activesupport-2.3.5/lib/active_support/json/backends/yaml.rb:14:in `decode'
/Library/Ruby/Gems/1.8/gems/activesupport-2.3.5/lib/active_support/json/decoding.rb:11:in `__send__'
/Library/Ruby/Gems/1.8/gems/activesupport-2.3.5/lib/active_support/json/decoding.rb:11:in `decode'
/Library/Ruby/Gems/1.8/gems/actionpack-2.3.5/lib/action_controller/params_parser.rb:42:in `parse_formatted_parameters'
/Library/Ruby/Gems/1.8/gems/actionpack-2.3.5/lib/action_controller/params_parser.rb:11:in `call'
/Library/Ruby/Gems/1.8/gems/actionpack-2.3.5/lib/action_controller/session/cookie_store.rb:93:in `call'
/Library/Ruby/Gems/1.8/gems/actionpack-2.3.5/lib/action_controller/failsafe.rb:26:in `call'
/Users/ben/projects/safecell/safecellweb/lib/compressed_requests.rb:36:in `call'
/Library/Ruby/Gems/1.8/gems/rack-1.0.1/lib/rack/lock.rb:11:in `call'
/Library/Ruby/Gems/1.8/gems/rack-1.0.1/lib/rack/lock.rb:11:in `synchronize'
/Library/Ruby/Gems/1.8/gems/rack-1.0.1/lib/rack/lock.rb:11:in `call'
/Library/Ruby/Gems/1.8/gems/actionpack-2.3.5/lib/action_controller/dispatcher.rb:114:in `call'
/Library/Ruby/Gems/1.8/gems/actionpack-2.3.5/lib/action_controller/reloader.rb:34:in `run'
/Library/Ruby/Gems/1.8/gems/actionpack-2.3.5/lib/action_controller/dispatcher.rb:108:in `call'
/Library/Ruby/Gems/1.8/gems/rails-2.3.5/lib/rails/rack/static.rb:31:in `call'
/Library/Ruby/Gems/1.8/gems/rack-1.0.1/lib/rack/urlmap.rb:46:in `call'
/Library/Ruby/Gems/1.8/gems/rack-1.0.1/lib/rack/urlmap.rb:40:in `each'
/Library/Ruby/Gems/1.8/gems/rack-1.0.1/lib/rack/urlmap.rb:40:in `call'
/Library/Ruby/Gems/1.8/gems/rails-2.3.5/lib/rails/rack/log_tailer.rb:17:in `call'
/Library/Ruby/Gems/1.8/gems/rack-1.0.1/lib/rack/content_length.rb:13:in `call'
/Library/Ruby/Gems/1.8/gems/rack-1.0.1/lib/rack/handler/webrick.rb:50:in `service'
One last piece of information, I'm inserting this middleware after ActionController::Failsafe.
EDIT: Looks like it's not a truncation problem
After more digging, it looks like it's not a truncation problem after all. The logs are simply clipping the output so it looks like a truncation issue.
At this point I'm not sure why the JSON is coming in as invalid. Do I need to do any manual escaping?
I'm not a ruby expert by any stretch. I also haven't tried to repro this problem in order to verify my results. But after digging through the rack and actionpack code, I might have something.
The doc for "rack.input" states: "The input stream is an IO-like object which contains the raw HTTP POST data."
So you're using that correctly, it would seem.
However, actionpack tries to parse JSON out of the body (if the content type is specified as JSON) and retrieves the body like this:
when :json
body = request.raw_post
where "request" is actionpack's own Request class, and "raw_post" is defined like this:
def raw_post
unless @env.include? 'RAW_POST_DATA'
@env['RAW_POST_DATA'] = body.read(@env['CONTENT_LENGTH'].to_i)
body.rewind if body.respond_to?(:rewind)
end
@env['RAW_POST_DATA']
end
and "Request.body" is:
def body
if raw_post = @env['RAW_POST_DATA']
raw_post.force_encoding(Encoding::BINARY) if raw_post.respond_to?(:force_encoding)
StringIO.new(raw_post)
else
@env['rack.input']
end
end
That all looks fine and good (though it's confusing to figure out who caches the value first :) ). It looks like the problem is in how the post data is read:
@env['RAW_POST_DATA'] = body.read(@env['CONTENT_LENGTH'].to_i)
So I'm guessing the problem is that since you change "rack.input" but don't update "CONTENT_LENGTH", actionpack is truncating the data since obviously the zipped content would've been shorter than the unzipped content.
Try updating "CONTENT_LENGTH" in your middleware code and see if that fixes it.
这篇关于机架rack.input变量被截断了吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
