HTTPS GZIP NGINX WTF(HTTPS GZIP NGINX WTF)

其他开发 IT屋
百度翻译此文   有道翻译此文
问 题

I've been trying for days now to get nginx to serve gzip content mainly because googles speed test told me to do it and we are trying to increase our SEO. I can't for the life of me understand what is going wrong here:

We are behind a firewall and serve two web heads underneath a load balancer. No matter what I've tried I can not get the response headers to come back with content-encoding:gzip. However, when I make the request using curl I can. Also when I access the site via https I do get the response back with gzip however it has nothing to do with nginx as I've turned gzip off in nginx and I still get the same response. What else would be serving the content as gzip?

UPDATE

Ok sorry after posting this several times with no response I was a little frustrated. Here is some info to go off:

We are using nginx 1.8.0 with php-fpm. The site is a Magento framework. I am trying to serve the main html pages compressed with gzip as well as the included css/javascript files. These files currently show not compressed in the response headers and Google pagespeed also says they are not compressed. Here is an example response header I see

Cache-Control:max-age=31536000
Connection:keep-alive
Content-Type:application/x-javascript; charset=utf-8
Date:Tue, 15 Mar 2016 15:15:13 GMT
ETag:"pub1448944926;gz"
Expires:Wed, 15 Mar 2017 15:15:13 GMT
Keep-Alive:timeout=8
Last-Modified:Tue, 01 Dec 2015 04:42:06 GMT
Server:nginx
Transfer-Encoding:chunked
Vary:Accept-Encoding

When I request the page via curl I get the gzip content.

curl -I -H 'Accept-encoding:gzip' mysite.com 

We have removed the staging site from the load balancer and the issue persists; eliminating any issues (for now) that may come form the load balancer.

When I access the site via https I get gzip content and here is the response header

Cache-Control:max-age=31536000
Connection:keep-alive
Content-Encoding:gzip
Content-Length:67085
Content-Type:application/x-javascript; charset=utf-8
Date:Tue, 15 Mar 2016 15:51:31 GMT
ETag:"pub1448944926;gz"
Expires:Wed, 15 Mar 2017 15:51:31 GMT
Keep-Alive:timeout=8
Last-Modified:Tue, 01 Dec 2015 04:42:06 GMT
Server:nginx
Vary:Accept-Encoding

Here are the relative config files for nginx

nginx.conf

user                    nginx;
worker_processes        4;
pid                     /var/run/nginx.pid;
error_log               /var/log/nginx/error.log;

events {
    worker_connections  1024;
    multi_accept        on;
    use                 epoll;
}

http {
    include             /etc/nginx/mime.types;
    charset             utf-8;
    default_type        application/octet-stream;

    #access_log         /var/log/nginx/access.log main;
    access_log          off;

    log_format main     '$remote_addr - $remote_user [$time_local]     "$request" '
    '$status $body_bytes_sent "$http_referer" '
    '"$http_user_agent" "$http_x_forwarded_for"';

    # compression
    gzip                on;
    gzip_http_version   1.0;
    gzip_vary           on;
    gzip_comp_level     5;
    gzip_proxied        any;
    gzip_min_length    100;
    #   gzip_min_length     10240;
    gzip_buffers        16 8k;
    gzip_types          text/plain text/css application/x-javascript     text/comma-separated-values text/xml application/xml application/xml+rss     application/atom+xml text/javascript;
    #gzip_disable       "MSIE [1-6].(?!.*SV1)";

    # general options
    sendfile            on;
    tcp_nopush          on;
    tcp_nodelay         off;
    autoindex           off;
    server_tokens       off;
    merge_slashes       on;
    client_header_buffer_size           1k;
    client_body_buffer_size             32k;
    client_max_body_size                64m;
    server_names_hash_bucket_size       128;
    large_client_header_buffers         2 1k;

    # timeouts
    send_timeout                        10;
    keepalive_timeout                   2 8;
    keepalive_requests                  200;
    client_body_timeout                 12;
    client_header_timeout               12;
    reset_timedout_connection           on;

    # pass through from load balancer
    real_ip_header X-Forwarded-For;
    set_real_ip_from 0.0.0.0/0;

    # detect https
    map $scheme $fastcgi_https {
        default "";
        https on;
    }

    # PHP-FPM
    upstream phpfpm {
        server unix:/run/php-fpm/php-fpm.sock weight=1 max_fails=5     fail_timeout=10;
    }

    # include active sites
    include /etc/nginx/sites-enabled/*;
    server {
        listen 80 spdy default_server;
        root /var/www/mysite.com;
        location ^~ /app/                       { return 403; }
        location ^~ /includes/                  { return 403; }
        location ^~ /media/downloadable/        { return 403; }
        location ^~ /pkginfo/                   { return 403; }
        location ^~ /report/config.xml          { return 403; }
        location ^~ /var/                       { return 403; }
        location ^~ /lib/                       { return 403; }
        location ^~ /dev/                       { return 403; }
        location ^~ /RELEASE_NOTES.txt          { return 403; }
        location ^~ /downloader/pearlib         { return 403; }
        location ^~ /downloader/template        { return 403; }
        location ^~ /downloader/Maged           { return 403; }
        location ~* ^/errors/.+\.xml            { return 403; }
    }

}

Sites-enabled/mysite

server {

    #mysiteip is an actual ip that I've removed for security

    listen mysiteip:80;  
    server_name www.test.mysite.com;
    return 301 $scheme://test.mysite.com$request_uri;
}
server {
    # settings
    listen mysiteip:80;
    listen mysiteip:443 ssl;
    server_name test.mysite.com;
    root /var/www/mysite.com/testing/current/;
    index index.html index.htm index.php;

    # security
    ssl_protocols TLSv1.2;
    ssl_ciphers RC4:HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers on;

    # SSL Certificate Settings
    ssl_certificate     /etc/nginx/ssl/bundle.crt;
    ssl_certificate_key /etc/nginx/ssl/star_mysite_com.key;


    access_log /var/log/nginx/mysite.access.log;
    error_log /var/log/nginx/www-mysite-com_error.log;

    # routes
    include /etc/nginx/conf.d/security.conf;
    include /etc/nginx/conf.d/assets.conf;
    include /etc/nginx/conf.d/rewrites.conf;

    # Attempt to serve the request by trying direct file, directory, Magento front controller

    large_client_header_buffers 8 16k;

    location / {
        try_files $uri $uri/ /index.php?$args;
        expires max;
    }

    # The downloader has its own index.php that needs to be used
    location ~* ^(/downloader)(.*) {
        try_files $uri $uri/ /downloader/index.php$1;
    }

    # REST API endpoint
    location /api {
        rewrite ^/api/rest /api.php?type=rest last;
        rewrite ^/api/v2_soap /api.php?type=v2_soap last;
        rewrite ^/api/soap /api.php?type=soap last;
    }

    # Pass PHP scripts to PHP-FPM daemon
    location ~* \.php$ {
        # filter out problem conditions
        location ~ \..*/.*\.php$ { return 404; }

        # bring in parameters
        include /etc/nginx/conf.d/fastcgi.conf;
        fastcgi_param MAGE_RUN_CODE default;
        fastcgi_param MAGE_RUN_TYPE store;

        # DEVELOPER MODE
        #fastcgi_param MAGE_IS_DEVELOPER_MODE true;

        # send requests to upstream, but blacklist media location from fcgi
        if ($uri !~ "^/(media)/") {
            fastcgi_pass phpfpm;
        }
    }
}

conf.d/rewrites.conf

# I am using this rewrite for the fooman speedster extension

location /skin/m {
   rewrite ^/skin/m/([^/]+)(/.*.(js|css))$ /lib/minify/m.php?f=$2&d=$1;
}

Any insight as to what is going on or being done wrong would be greatly appreciated. I can provide more info if needed. Also the rewrite for Fooman is used for minification and even without this extension installed and the rewrite removed I still get no gzip.

解决方案

Cant add comment now because of Bounty. @iquiot @iquito Thanks I disabled expires max and no luck. I tracked down to this being set in the Fooman extension. and set max-age to zero. Here is the new response header ... but still no gzip

Cache-Control   max-age=0
Connection  keep-alive
Content-Type    text/css; charset=utf-8
Date     Thu, 17 Mar 2016 19:35:05 GMT
Etag    "pub1457625059;gz"
Last-Modified Thu, 10 Mar 2016 15:50:59 GMT
Server nginx/1.8.0
Transfer-Encoding chunked
Vary    Accept-Encoding

UPDATE:

I removed myself from the larger picture to try and debug this issue. I created a test.html file and am using this to try and see if I can get any sort of compression going. One strange thing that I am noticing is that if I set the gzip_min_length to say 100 I get content-length in the header when the content is under 100 however as soon as I exceed 100 the content-length goes away.

UPDATE 2: When using curl any sort of "Accept:" header that is set does not return gzip in the response. Could this be a possible culprit?

UPDATE 3: using this little tool for firefox https://addons.mozilla.org/en-US/firefox/addon/modify-headers/developers I was able to modify the headers sent. By completely disabling the user agent header I now get the proper response header with content-encoding gzip. However I do not have anywhere in my nginx config a disable parameter set for gzip which would give such behavior.

本文地址:IT屋 » HTTPS GZIP NGINX WTF

问 题

我一直在努力让nginx服务gzip内容主要因为googles速度测试告诉我这样做,我们正在努力增加我们的搜索引擎优化。我不能为我的生活理解这里出了什么问题:



我们在防火墙后面并在负载均衡器下面提供两个网络头。无论我尝试过什么,我都无法通过 content-encoding返回响应标头:gzip 。但是,当我使用curl发出请求时,我可以。此外,当我通过https访问该网站时,我确实得到了gzip的回复,但它与nginx无关,因为我在nginx中关闭了gzip并且仍然得到相同的响应。还有什么可以作为gzip提供内容?



更新



好抱歉在几次没有回复的情况下发帖后,我有点沮丧。以下是一些信息:



我们正在使用带有php-fpm的nginx 1.8.0。该网站是Magento框架。我正在尝试提供使用gzip压缩的主要html页面以及包含的css / javascript文件。这些文件当前显示未在响应标头中压缩,Google pagespeed也表示它们未压缩。以下是我看到的示例响应标题



  Cache-Control:max-age = 31536000 
连接:keep-alive
内容类型:application / x-javascript; charset = utf-8
日期:星期二,2016年3月15日15:15:13 GMT
ETag:“pub14​​48944926; gz”
到期日:2017年3月15日星期三15:15:13 GMT
Keep-Alive:timeout = 8
Last-Modified:Tue,01 Dec 2015 04:42:06 GMT
Server:nginx
Transfer-Encoding:chunked
变化:接受编码


当我通过curl请求页面时,我得到了gzip内容。



  curl -I -H'接受编码:gzip'mysite.com 


我们已从负载均衡器中删除了暂存站点,问题仍然存在;消除可能来自负载均衡器的任何问题(现在)。



当我通过https访问该站点时,我得到gzip内容,这里是响应头



  Cache-Control:max-age = 31536000 
连接:keep-alive
内容编码:gzip
内容长度:67085
内容类型:application / x-javascript; charset = utf-8
日期:星期二,2016年3月15日15:51:31 GMT
ETag:“pub14​​48944926; gz”
到期日:2017年3月15日星期三15:51:31 GMT
Keep-Alive:timeout = 8
Last-Modified:Tue,01 Dec 2015 04:42:06 GMT
Server:nginx
Vary:Accept-Encoding


以下是nginx的相对配置文件



nginx。 conf



 用户nginx; 
worker_processes 4;
pid /var/run/nginx.pid;
error_log /var/log/nginx/error.log;

events {
worker_connections 1024;
multi_accept on;
使用epoll;
}

http {
include /etc/nginx/mime.types;
charset utf-8;
default_type application / octet-stream;

#access_log /var/log/nginx/access.log main;
access_log off;

log_format main'$ remote_addr - $ remote_user [$ time_local]“$ request”'
'$ status $ body_bytes_sent“$ http_referer”'
'“$ http_user_agent”“$ HTTP_X_FORWARDED_FOR“”;

#compress
gzip on;
gzip_http_version 1.0;
gzip_vary on;
gzip_comp_level 5;
gzip_proxied any;
gzip_min_length 100;
#gzip_min_length 10240;
gzip_buffers 16 8k;
gzip_types text / plain text / css application / x-javascript text /逗号分隔值text / xml application / xml application / xml + rss application / atom + xml text / javascript;
#gzip_disable“MSIE [1-6]。(?!。* SV1)”;

#general options
sendfile on;
tcp_nopush on;
tcp_nodelay off;
autoindex off;
server_tokens off;
merge_slashes on;
client_header_buffer_size 1k;
client_body_buffer_size 32k;
client_max_body_size 64m;
server_names_hash_bucket_size 128;
large_client_header_buffers 2 1k;

#timeouts
send_timeout 10;
keepalive_timeout 2 8;
keepalive_requests 200;
client_body_timeout 12;
client_header_timeout 12;
reset_timedout_connection on;

#从负载均衡器传递
real_ip_header X-Forwarded-For;
set_real_ip_from 0.0.0.0/0;

#detect https
map $ scheme $ fastcgi_https {
default“”;
https on;
}

#PHP-FPM
上游phpfpm {
服务器unix:/run/php-fpm/php-fpm.sock权重= 1 max_fails = 5 fail_timeout = 10;
}

#包括活动网站
包括/ etc / nginx / sites-enabled / *;
服务器{
listen 80 spdy default_server;
root /var/www/mysite.com;
location ^〜/ app / {return 403; }
location ^〜/ includes / {return 403; }
location ^〜/ media / downloadable / {return 403; }
location ^〜/ pkginfo / {return 403; }
location ^〜/ report / config.xml {return 403; }
location ^〜/ var / {return 403; }
location ^〜/ lib / {return 403; }
location ^〜/ dev / {return 403; }
location ^〜/ RELEASE_NOTES.txt {return 403; }
location ^〜/ downloader / pearlib {return 403; }
location ^〜/ downloader / template {return 403; }
location ^〜/ downloader / Maged {return 403; }
location〜* ^ / errors /.+ \.xml {return 403; }
}

}


Sites-enabled / mysite



  server {

#mysiteip是我为了安全而删除的实际IP

听mysiteip:80;
server_name www.test.mysite.com;
return 301 $ scheme://test.mysite.com$request_uri;
}
服务器{
#settings
listen mysiteip:80;
听mysiteip:443 ssl;
server_name test.mysite.com;
root /var/www/mysite.com/testing/current/;
index index.html index.htm index.php;

#security
ssl_protocols TLSv1.2;
ssl_ciphers RC4:HIGH:!aNULL:!MD5;
ssl_prefer_server_ciphers on;

#SSL证书设置
ssl_certificate /etc/nginx/ssl/bundle.crt;
ssl_certificate_key /etc/nginx/ssl/star_mysite_com.key;


access_log /var/log/nginx/mysite.access.log;
error_log /var/log/nginx/www-mysite-com_error.log;

#routes
include /etc/nginx/conf.d/security.conf;
包括/etc/nginx/conf.d/assets.conf;
包括/etc/nginx/conf.d/rewrites.conf;

#尝试通过尝试直接文件,目录,Magento前端控制器来提供请求

large_client_header_buffers 8 16k;

location / {
try_files $ uri $ uri / /index.php?$args;
最后到期;
}

#下载器有自己的index.php,需要使用
location〜* ^(/ downloader)(。*){
try_files $ uri $ uri / /downloader/index.php$1;
}

#REST API端点
location / api {
rewrite ^ / api / rest /api.php?type=rest last;
重写^ / api / v2_soap /api.php?type=v2_soap last;
重写^ / api / soap /api.php?type=soap last;
}

#将PHP脚本传递给PHP-FPM守护进程
location~ * \.php $ {
#过滤掉问题条件
location~ \ .. * /。* \.php $ {返回404; }

#引入参数
include /etc/nginx/conf.d/fastcgi.conf;
fastcgi_param MAGE_RUN_CODE默认值;
fastcgi_param MAGE_RUN_TYPE商店;

#DEVELOPER MODE
#fastcgi_param MAGE_IS_DEVELOPER_MODE true;

#向上游发送请求,但黑名单媒体位置来自fcgi
if($ uri!〜“^ /(media)/”){
fastcgi_pass phpfpm;
}
}
}


conf.d / rewrites。 conf



 #我正在使用此重写为fooman speedster扩展

location / skin / m {
重写^ / skin / m /([^ /] +)(/。*。(js | css))$ / lib /minify/m.php?f=$2&d=$1;
}


任何有关正在发生或正在做错的见解都将非常感激。如果需要,我可以提供更多信息。另外Fooman的重写用于缩小,即使没有安装此扩展并且重写被删除,我仍然没有gzip。


解决方案

由于Bounty,现在无法添加评论。 @iquiot @iquito谢谢我禁用过期最大,没有运气。我追踪到了Fooman扩展中的这个设置。并将max-age设置为零。这是新的响应标题...但仍然没有gzip



  Cache-Control max-age = 0 
连接keep-alive
Content-Type text / css; charset = utf-8
日期星期四,2016年3月17日19:35:05 GMT
Etag“pub14​​57625059; gz”
Last-Modified星期四,2016年3月10日15:50:59 GMT
Server nginx / 1.8.0
Transfer-Encoding chunked
Vary Accept-Encoding


更新:



我从较大的图片中删除了自己以尝试调试此问题。我创建了一个test.html文件,并使用它来尝试查看是否可以进行任何类型的压缩。我注意到的一个奇怪的事情是,如果我将gzip_min_length设置为100,那么当内容小于100时,我会在标题中获得内容长度,但是一旦超过100,内容长度就会消失。



UPDATE 2:
使用curl时,任何设置的“Accept:”标头都不会在响应中返回gzip。这可能是罪魁祸首吗?



更新3:
使用这个小工具for firefox
https://addons.mozilla.org/en-US/firefox/addon/modify-headers/developers
我当时能够修改发送的标头。
通过完全禁用用户代理头,我现在可以使用内容编码gzip获得正确的响应头。但是我的nginx配置中没有任何地方可以为gzip设置禁用参数集,这会产生这样的行为。


本文地址:IT屋 » HTTPS GZIP NGINX WTF

官方微信
扫一扫关注IT屋
微信公众号搜索 “ IT屋 ” ,选择关注
与百万开发者在一起