网关中关于gzip处理

[iliuyt]

gzip 流程

浏览器发起请求，携带是否支持gzip的header Accept-Encoding: gzip, deflate ,nginx检查 Accept-Encoding: 如果值为gzip,deflate,compress,且nginx开启了gzip功能,那么读取完请求返回流后，进行压缩流，然后返回流，并且添加header,content-encoding: gzip,浏览器接受到返回流后，检查到content-encoding: gzip后，浏览器进行gzip解压缩，然后给程序进行处理。

网关代理gzip相关处理

根据上面gzip的原理，网关需进行模仿gzip处理，在网关接收到gzip请求后，应将相关header代理到后端，如果后端返回为gzip流，应通过流代理，进行返回。

postman-request 处理gzip

当前使用postman-request进行代理，如果请求为gzip，且后端返回为gzip数据，

• 使用callback模式返回gzip流，需要设置gzip为false,encoding为null(必须是null)，需要代理header: content-encoding
• 使用callback模式返回解压gzip流，返回解压后的数据，需要设置gzip为true,且返回时，应该删除header: content-encoding
• 使用pipe返回gzip流，直接pipe即可,会原封不动的把流返回，包括header
• 使用pipe返回解压gzip流，设置gzip为true,header:content-encoding 会自动删除

postman-request gzip处理源码解析

• encoding 参数调用代码

      response.body = Buffer.concat(buffers, bufferLength)
      if (self.encoding !== null) {
        response.body = response.body.toString(self.encoding)
      }

• gzip 参数调用代码

    if (self.gzip && contentEncoding === 'gzip') {
        responseContent = zlib.createGunzip(zlibOptions)
        // 这里pipe不会写入header
        response.pipe(responseContent)
    } else if (self.gzip && contentEncoding === 'deflate') {
        responseContent = inflate.createInflate(zlibOptions)
        response.pipe(responseContent)
    } else if (self.brotli && contentEncoding === 'br') {
        responseContent = brotli.createBrotliDecompress()
        response.pipe(responseContent)
    }
    ...

• gzip 解压后删除header

    Request.prototype.pipeDest = function (dest) {
    ...
    if (dest.setHeader && !dest.headersSent) {
        for (var i in response.headers) {
        // If the response content is being decoded, the Content-Encoding header
        // of the response doesn't represent the piped content, so don't pass it.
        if (!self.gzip || i !== 'content-encoding') {
            dest.setHeader(i, response.headers[i])
        }
        }
        dest.statusCode = response.statusCode
    }
    ...
    }

axios 处理gzip

axios在判断返回头 content-encoding 值为 gzip、compress、deflate，会直接进行解压处理。由于axios的过度封装导致只能通过更改transport的方式进行拦截处理，但不知道是否会有其他影响。

• axios 处理gzip请求

    var req = transport.request(options, function handleResponse(res) {
      if (req.aborted) return;

      // uncompress the response body transparently if required
      var stream = res;
      switch (res.headers['content-encoding']) {
      /*eslint default-case:0*/
      case 'gzip':
      case 'compress':
      case 'deflate':
        // add the unzipper to the body stream processing pipeline
        stream = (res.statusCode === 204) ? stream : stream.pipe(zlib.createUnzip());

        // remove the content-encoding in order to not confuse downstream operations
        delete res.headers['content-encoding'];
        break;
      }
      ....

• transport 拦截处理

var http = require("http");
var https = require("https");
var httpFollow = require("follow-redirects").http;
var httpsFollow = require("follow-redirects").https;
const getTransport = function (isHttpsProxy, config = {}) {
    var transport;
    if (config.maxRedirects === 0) {
        transport = isHttpsProxy ? https : http;
    } else {
        if (config.maxRedirects) {
            options.maxRedirects = config.maxRedirects;
        }
        transport = isHttpsProxy ? httpsFollow : httpFollow;
    }
    var _transportRequest = transport.request.bind(transport);
    transport.request = function (options, handleResponse) {
        return _transportRequest(options, function (res) {
            var encoding = res.headers["content-encoding"];
            var has = ["gzip", "compress", "deflate"].includes(encoding);
            has && delete res.headers["content-encoding"];
            handleResponse(res);
            has && (res.headers["content-encoding"] = encoding);
        });
    };
    return transport;
};

...
    axios({
        url: "http://local.com/nginx/2.json",
        method: "get",
        responseType: "stream",
        headers: req.headers,
        transport: getTransport(false)
    })
...

关于网关开启gzip

• gzip本身是一种cpu操作，不建议在node网关中使用
• gzip最好是有缓存的
• gzip一般用于静态文件，静态文件会提前生成gzip压缩文件，根据请求头判断是否访问添加gzip后缀进行查找文件
• 接口请求不建议用gzip进行压缩，尤其是nodejs网关
• 建议nodejs网关碰到gzip请求，直接做流代理，不解压也不压缩，返回原生数据，gzip功能由上游实现