HTTPS GZIP NGINX WTF(HTTPS GZIP NGINX WTF)

I've been trying for days now to get nginx to serve gzip content mainly because googles speed test told me to do it and we are trying to increase our SEO. I can't for the life of me understand what is going wrong here:

We are behind a firewall and serve two web heads underneath a load balancer. No matter what I've tried I can not get the response headers to come back with content-encoding:gzip. However, when I make the request using curl I can. Also when I access the site via https I do get the response back with gzip however it has nothing to do with nginx as I've turned gzip off in nginx and I still get the same response. What else would be serving the content as gzip?

UPDATE

Ok sorry after posting this several times with no response I was a little frustrated. Here is some info to go off:

We are using nginx 1.8.0 with php-fpm. The site is a Magento framework. I am trying to serve the main html pages compressed with gzip as well as the included css/javascript files. These files currently show not compressed in the response headers and Google pagespeed also says they are not compressed. Here is an example response header I see

Cache-Control:max-age=31536000
Connection:keep-alive
Content-Type:application/x-javascript; charset=utf-8
Date:Tue, 15 Mar 2016 15:15:13 GMT
ETag:"pub1448944926;gz"
Expires:Wed, 15 Mar 2017 15:15:13 GMT
Keep-Alive:timeout=8
Last-Modified:Tue, 01 Dec 2015 04:42:06 GMT
Server:nginx
Transfer-Encoding:chunked
Vary:Accept-Encoding

When I request the page via curl I get the gzip content.

curl -I -H 'Accept-encoding:gzip' mysite.com 

We have removed the staging site from the load balancer and the issue persists; eliminating any issues (for now) that may come form the load balancer.

When I access the site via https I get gzip content and here is the response header

Cache-Control:max-age=31536000
Connection:keep-alive
Content-Encoding:gzip
Content-Length:67085
Content-Type:application/x-javascript; charset=utf-8
Date:Tue, 15 Mar 2016 15:51:31 GMT
ETag:"pub1448944926;gz"
Expires:Wed, 15 Mar 2017 15:51:31 GMT
Keep-Alive:timeout=8
Last-Modified:Tue, 01 Dec 2015 04:42:06 GMT
Server:nginx
Vary:Accept-Encoding

Here are the relative config files for nginx

nginx.conf

user                    nginx;
worker_processes        4;
pid                     /var/run/nginx.pid;
error_log               /var/log/nginx/error.log;

events {
    worker_connections  1024;
    multi_accept        on;
    use                 epoll;
}

http {
    include             /etc/nginx/mime.types;
    charset             utf-8;
    default_type        application/octet-stream;

    #access_log         /var/log/nginx/access.log main;
    access_log          off;

    log_format main     '$remote_addr - $remote_user [$time_local]     "$request" '
    '$status $body_bytes_sent "$http_referer" '
    '"$http_user_agent" "$http_x_forwarded_for"';

    # compression
    gzip                on;
    gzip_http_version   1.0;
    gzip_vary           on;
    gzip_comp_level     5;
    gzip_proxied        any;
    gzip_min_length    100;
    #   gzip_min_length     10240;
    gzip_buffers        16 8k;
    gzip_types          text/plain text/css application/x-javascript     text/comma-separated-values text/xml application/xml application/xml+rss     application/atom+xml text/javascript;
    #gzip_disable       "MSIE [1-6].(?!.*SV1)";

    # general options
    sendfile            on;
    tcp_nopush          on;
    tcp_nodelay         off;
    autoindex           off;
    server_tokens       off;
    merge_slashes       on;
    client_header_buffer_size           1k;
    client_body_buffer_size             32k;
    client_max_body_size                64m;
    server_names_hash_bucket_size       128;
    large_client_header_buffers         2 1k;

    # timeouts
    send_timeout                        10;
    keepalive_timeout                   2 8;
    keepalive_requests                  200;
    client_body_timeout                 12;
    client_header_timeout               12;
    reset_timedout_connection           on;

    # pass through from load balancer
    real_ip_header X-Forwarded-For;
    set_real_ip_from 0.0.0.0/0;

    # detect https
    map $scheme $fastcgi_https {
        default "";
        https on;
    }

    # PHP-FPM
    upstream phpfpm {
        server unix:/run/php-fpm/php-fpm.sock weight=1 max_fails=5     fail_timeout=10;
    }

    # include active sites
    include /etc/nginx/sites-enabled/*;
    server {
        listen 80 spdy default_server;
        root /var/www/mysite.com;
        location ^~ /app/                       { return 403; }
        location ^~ /includes/                  { return 403; }
        location ^~ /media/downloadable/        { return 403; }
        location ^~ /pkginfo/                   { return 403; }
        location ^~ /report/config.xml          { return 403; }
        location ^~ /var/                       { return 403; }
        location ^~ /lib/                       { return 403; }
        location ^~ /dev/                       { return 403; }
        location ^~ /RELEASE_NOTES.txt          { return 403; }
        location ^~ /downloader/pearlib         { return 403; }
        location ^~ /downloader/template        { return 403; }
        location ^~ /downloader/Maged           { return 403; }
        location ~* ^/errors/.+\.xml            { return 403; }
    }

}

Sites-enabled/mysite

server {

    #mysiteip is an actual ip that I've removed for security

    listen mysiteip:80;  
    server_name www.test.mysite.com;
    return 301 $scheme://test.mysite.com$request_uri;
}
server {
    # settings
    listen mysiteip:80;
    listen mysiteip:443 ssl;
    server_name test.mysite.com;
    root /var/www/mysite.com/testing/current/;
    index index.html index.htm index.php;

    # security
    ssl_protocols TLSv1.2;
    ssl_ciphers RC4:HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers on;

    # SSL Certificate Settings
    ssl_certificate     /etc/nginx/ssl/bundle.crt;
    ssl_certificate_key /etc/nginx/ssl/star_mysite_com.key;


    access_log /var/log/nginx/mysite.access.log;
    error_log /var/log/nginx/www-mysite-com_error.log;

    # routes
    include /etc/nginx/conf.d/security.conf;
    include /etc/nginx/conf.d/assets.conf;
    include /etc/nginx/conf.d/rewrites.conf;

    # Attempt to serve the request by trying direct file, directory, Magento front controller

    large_client_header_buffers 8 16k;

    location / {
        try_files $uri $uri/ /index.php?$args;
        expires max;
    }

    # The downloader has its own index.php that needs to be used
    location ~* ^(/downloader)(.*) {
        try_files $uri $uri/ /downloader/index.php$1;
    }

    # REST API endpoint
    location /api {
        rewrite ^/api/rest /api.php?type=rest last;
        rewrite ^/api/v2_soap /api.php?type=v2_soap last;
        rewrite ^/api/soap /api.php?type=soap last;
    }

    # Pass PHP scripts to PHP-FPM daemon
    location ~* \.php$ {
        # filter out problem conditions
        location ~ \..*/.*\.php$ { return 404; }

        # bring in parameters
        include /etc/nginx/conf.d/fastcgi.conf;
        fastcgi_param MAGE_RUN_CODE default;
        fastcgi_param MAGE_RUN_TYPE store;

        # DEVELOPER MODE
        #fastcgi_param MAGE_IS_DEVELOPER_MODE true;

        # send requests to upstream, but blacklist media location from fcgi
        if ($uri !~ "^/(media)/") {
            fastcgi_pass phpfpm;
        }
    }
}

conf.d/rewrites.conf

# I am using this rewrite for the fooman speedster extension

location /skin/m {
   rewrite ^/skin/m/([^/]+)(/.*.(js|css))$ /lib/minify/m.php?f=$2&d=$1;
}

Any insight as to what is going on or being done wrong would be greatly appreciated. I can provide more info if needed. Also the rewrite for Fooman is used for minification and even without this extension installed and the rewrite removed I still get no gzip.

Cant add comment now because of Bounty. @iquiot @iquito Thanks I disabled expires max and no luck. I tracked down to this being set in the Fooman extension. and set max-age to zero. Here is the new response header ... but still no gzip

Cache-Control   max-age=0
Connection  keep-alive
Content-Type    text/css; charset=utf-8
Date     Thu, 17 Mar 2016 19:35:05 GMT
Etag    "pub1457625059;gz"
Last-Modified Thu, 10 Mar 2016 15:50:59 GMT
Server nginx/1.8.0
Transfer-Encoding chunked
Vary    Accept-Encoding

UPDATE:

I removed myself from the larger picture to try and debug this issue. I created a test.html file and am using this to try and see if I can get any sort of compression going. One strange thing that I am noticing is that if I set the gzip_min_length to say 100 I get content-length in the header when the content is under 100 however as soon as I exceed 100 the content-length goes away.

UPDATE 2: When using curl any sort of "Accept:" header that is set does not return gzip in the response. Could this be a possible culprit?

UPDATE 3: using this little tool for firefox https://addons.mozilla.org/en-US/firefox/addon/modify-headers/developers I was able to modify the headers sent. By completely disabling the user agent header I now get the proper response header with content-encoding gzip. However I do not have anywhere in my nginx config a disable parameter set for gzip which would give such behavior.

本文地址：IT屋 » HTTPS GZIP NGINX WTF

我一直在努力让nginx服务gzip内容主要因为googles速度测试告诉我这样做，我们正在努力增加我们的搜索引擎优化。我不能为我的生活理解这里出了什么问题：

我们在防火墙后面并在负载均衡器下面提供两个网络头。无论我尝试过什么，我都无法通过 content-encoding返回响应标头：gzip 。但是，当我使用curl发出请求时，我可以。此外，当我通过https访问该网站时，我确实得到了gzip的回复，但它与nginx无关，因为我在nginx中关闭了gzip并且仍然得到相同的响应。还有什么可以作为gzip提供内容？

更新

好抱歉在几次没有回复的情况下发帖后，我有点沮丧。以下是一些信息：

我们正在使用带有php-fpm的nginx 1.8.0。该网站是Magento框架。我正在尝试提供使用gzip压缩的主要html页面以及包含的css / javascript文件。这些文件当前显示未在响应标头中压缩，Google pagespeed也表示它们未压缩。以下是我看到的示例响应标题

  Cache-Control：max-age = 31536000 
连接：keep-alive 
内容类型：application / x-javascript; charset = utf-8 
日期：星期二，2016年3月15日15:15:13 GMT 
 ETag：“pub14​​48944926; gz”
到期日：2017年3月15日星期三15:15:13 GMT 
 Keep-Alive：timeout = 8 
 Last-Modified：Tue，01 Dec 2015 04:42:06 GMT 
 Server：nginx 
 Transfer-Encoding：chunked 
变化：接受编码
  

当我通过curl请求页面时，我得到了gzip内容。

  curl -I -H'接受编码：gzip'mysite.com 
  

我们已从负载均衡器中删除了暂存站点，问题仍然存在;消除可能来自负载均衡器的任何问题（现在）。

当我通过https访问该站点时，我得到gzip内容，这里是响应头

  Cache-Control：max-age = 31536000 
连接：keep-alive 
内容编码：gzip 
内容长度：67085 
内容类型：application / x-javascript; charset = utf-8 
日期：星期二，2016年3月15日15:51:31 GMT 
 ETag：“pub14​​48944926; gz”
到期日：2017年3月15日星期三15:51:31 GMT 
 Keep-Alive：timeout = 8 
 Last-Modified：Tue，01 Dec 2015 04:42:06 GMT 
 Server：nginx 
 Vary：Accept-Encoding 
  

以下是nginx的相对配置文件

nginx。 conf

 用户nginx; 
 worker_processes 4; 
 pid /var/run/nginx.pid; 
 error_log /var/log/nginx/error.log; 
 
 events {
 worker_connections 1024; 
 multi_accept on; 
使用epoll; 
} 
 
 http {
 include /etc/nginx/mime.types; 
 charset utf-8; 
 default_type application / octet-stream; 
 
 #access_log /var/log/nginx/access.log main; 
 access_log off; 
 
 log_format main'$ remote_addr  -  $ remote_user [$ time_local]“$ request”'
'$ status $ body_bytes_sent“$ http_referer”'
'“$ http_user_agent”“$ HTTP_X_FORWARDED_FOR“”; 
 
 #compress 
 gzip on; 
 gzip_http_version 1.0; 
 gzip_vary on; 
 gzip_comp_level 5; 
 gzip_proxied any; 
 gzip_min_length 100; 
＃gzip_min_length 10240; 
 gzip_buffers 16 8k; 
 gzip_types text / plain text / css application / x-javascript text /逗号分隔值text / xml application / xml application / xml + rss application / atom + xml text / javascript; 
 #gzip_disable“MSIE [1-6]。（？！。* SV1）”; 
 
＃general options 
 sendfile on; 
 tcp_nopush on; 
 tcp_nodelay off; 
 autoindex off; 
 server_tokens off; 
 merge_slashes on; 
 client_header_buffer_size 1k; 
 client_body_buffer_size 32k; 
 client_max_body_size 64m; 
 server_names_hash_bucket_size 128; 
 large_client_header_buffers 2 1k; 
 
＃timeouts 
 send_timeout 10; 
 keepalive_timeout 2 8; 
 keepalive_requests 200; 
 client_body_timeout 12; 
 client_header_timeout 12; 
 reset_timedout_connection on; 
 
＃从负载均衡器传递
 real_ip_header X-Forwarded-For; 
 set_real_ip_from 0.0.0.0/0; 
 
＃detect https 
 map $ scheme $ fastcgi_https {
 default“”; 
 https on; 
} 
 
＃PHP-FPM 
上游phpfpm {
服务器unix：/run/php-fpm/php-fpm.sock权重= 1 max_fails = 5 fail_timeout = 10; 
} 
 
＃包括活动网站
包括/ etc / nginx / sites-enabled / *; 
服务器{
 listen 80 spdy default_server; 
 root /var/www/mysite.com; 
 location ^〜/ app / {return 403; } 
 location ^〜/ includes / {return 403; } 
 location ^〜/ media / downloadable / {return 403; } 
 location ^〜/ pkginfo / {return 403; } 
 location ^〜/ report / config.xml {return 403; } 
 location ^〜/ var / {return 403; } 
 location ^〜/ lib / {return 403; } 
 location ^〜/ dev / {return 403; } 
 location ^〜/ RELEASE_NOTES.txt {return 403; } 
 location ^〜/ downloader / pearlib {return 403; } 
 location ^〜/ downloader / template {return 403; } 
 location ^〜/ downloader / Maged {return 403; } 
 location〜* ^ / errors /.+ \.xml {return 403; } 
} 
 
} 
  

Sites-enabled / mysite

  server {
 
 #mysiteip是我为了安全而删除的实际IP 
 
听mysiteip：80; 
 server_name www.test.mysite.com; 
 return 301 $ scheme：//test.mysite.com$request_uri; 
} 
服务器{
＃settings 
 listen mysiteip：80; 
听mysiteip：443 ssl; 
 server_name test.mysite.com; 
 root /var/www/mysite.com/testing/current/; 
 index index.html index.htm index.php; 
 
 #security 
 ssl_protocols TLSv1.2; 
 ssl_ciphers RC4：HIGH：！aNULL：！MD5; 
 ssl_prefer_server_ciphers on; 
 
＃SSL证书设置
 ssl_certificate /etc/nginx/ssl/bundle.crt; 
 ssl_certificate_key /etc/nginx/ssl/star_mysite_com.key; 
 
 
 access_log /var/log/nginx/mysite.access.log; 
 error_log /var/log/nginx/www-mysite-com_error.log; 
 
＃routes 
 include /etc/nginx/conf.d/security.conf; 
包括/etc/nginx/conf.d/assets.conf; 
包括/etc/nginx/conf.d/rewrites.conf; 
 
＃尝试通过尝试直接文件，目录，Magento前端控制器来提供请求
 
 large_client_header_buffers 8 16k; 
 
 location / {
 try_files $ uri $ uri / /index.php?$args; 
最后到期; 
} 
 
＃下载器有自己的index.php，需要使用
 location〜* ^（/ downloader）（。*）{
 try_files $ uri $ uri / /downloader/index.php$1; 
} 
 
＃REST API端点
 location / api {
 rewrite ^ / api / rest /api.php?type=rest last; 
重写^ / api / v2_soap /api.php?type=v2_soap last; 
重写^ / api / soap /api.php?type=soap last; 
} 
 
＃将PHP脚本传递给PHP-FPM守护进程
 location~ * \.php $ {
＃过滤掉问题条件
 location~ \ .. * /。* \.php $ {返回404; } 
 
＃引入参数
 include /etc/nginx/conf.d/fastcgi.conf; 
 fastcgi_param MAGE_RUN_CODE默认值; 
 fastcgi_param MAGE_RUN_TYPE商店; 
 
＃DEVELOPER MODE 
 #fastcgi_param MAGE_IS_DEVELOPER_MODE true; 
 
＃向上游发送请求，但黑名单媒体位置来自fcgi 
 if（$ uri！〜“^ /（media）/”）{
 fastcgi_pass phpfpm; 
} 
} 
} 
  

conf.d / rewrites。 conf

 ＃我正在使用此重写为fooman speedster扩展
 
 location / skin / m { 
重写^ / skin / m /（[^ /] +）（/。*。（js | css））$ / lib /minify/m.php?f=$2&d=$1; 
} 
  

任何有关正在发生或正在做错的见解都将非常感激。如果需要，我可以提供更多信息。另外Fooman的重写用于缩小，即使没有安装此扩展并且重写被删除，我仍然没有gzip。

由于Bounty，现在无法添加评论。 @iquiot @iquito谢谢我禁用过期最大，没有运气。我追踪到了Fooman扩展中的这个设置。并将max-age设置为零。这是新的响应标题...但仍然没有gzip

  Cache-Control max-age = 0 
连接keep-alive 
 Content-Type text / css; charset = utf-8 
日期星期四，2016年3月17日19:35:05 GMT 
 Etag“pub14​​57625059; gz”
 Last-Modified星期四，2016年3月10日15:50:59 GMT 
 Server nginx / 1.8.0 
 Transfer-Encoding chunked 
 Vary Accept-Encoding 
  

更新：

我从较大的图片中删除了自己以尝试调试此问题。我创建了一个test.html文件，并使用它来尝试查看是否可以进行任何类型的压缩。我注意到的一个奇怪的事情是，如果我将gzip_min_length设置为100，那么当内容小于100时，我会在标题中获得内容长度，但是一旦超过100，内容长度就会消失。

UPDATE 2：
使用curl时，任何设置的“Accept：”标头都不会在响应中返回gzip。这可能是罪魁祸首吗？

更新3：
使用这个小工具for firefox
https://addons.mozilla.org/en-US/firefox/addon/modify-headers/developers
我当时能够修改发送的标头。
通过完全禁用用户代理头，我现在可以使用内容编码gzip获得正确的响应头。但是我的nginx配置中没有任何地方可以为gzip设置禁用参数集，这会产生这样的行为。

本文地址：IT屋 » HTTPS GZIP NGINX WTF