1小时后，istio-proxy关闭长时间运行的TCP连接 [英] istio-proxy closing long running TCP connection after 1 hour

问题描述

TL; DR:我们如何配置istio sidecar注入/istio-proxy/envoy-proxy/istio出口网关，以允许较长的寿命(> 3小时)，可能是空闲的TCP连接?

TL;DR: How can we configure istio sidecar injection/istio-proxy/envoy-proxy/istio egressgateway to allow long living (>3 hours), possibly idle, TCP connections?

一些细节:

我们正在尝试将数据库迁移到PostgreSQL，这是由配置了Spring Boot + Flyway的一个应用程序触发的，该迁移预计将持续约3个小时.

We're trying to perform a database migration to PostgreSQL which is being triggered by one application which has Spring Boot + Flyway configured, this migration is expected to last ~3 hours.

我们的应用程序部署在我们的kubernetes集群中，该集群配置了istio sidecar注入.经过一小时的迁移，连接始终处于关闭状态.

Our application is deployed inside our kubernetes cluster, which has configured istio sidecar injection. After exactly one hour of running the migration, the connection is always getting closed.

当我们尝试从没有istio sidecar注入的Pod迁移时，我们确定它是istio-proxy关闭连接，并且运行时间超过一小时，但是这不是一个可行的选择，因为这可能会导致某些停机时间在生产中我们无法考虑.

We're sure it's istio-proxy closing the connection as we attempted the migration from a pod without istio sidecar injection and it was running for longer than one hour, however this is not an option going forward as this may imply some downtime in production which we can't consider.

我们怀疑这应该可以在istio代理中配置为设置参数idle_timeout-已在此处.但是，这不起作用，或者我们没有正确配置它，我们正在尝试在istio安装期间通过在头盔模板中添加--set gateways.istio-ingressgateway.env.ISTIO_META_IDLE_TIMEOUT=5s来配置它.

We suspect this should be configurable in istio proxy setting the parameter idle_timeout - which was implemented here. However this isn't working, or we are not configuring it properly, we're trying to configure this during istio installation by adding --set gateways.istio-ingressgateway.env.ISTIO_META_IDLE_TIMEOUT=5s to our helm template.

推荐答案

如果您使用的istio版本高于1.7，则可以尝试使用 github 上有答案和示例.

If you use istio version higher than 1.7 you might try use envoy filter to make it work. There is answer and example on github provided by @ryant1986.

我们在1.7上遇到了相同的问题，但是我们注意到ISTIO_META_IDLE_TIMEOUT设置仅在输出端而不是输入端被拾取.通过添加一个附加的过滤器，该过滤器应用于请求的INBOUND端，我们能够成功地增加超时时间(我们使用了24小时)

We ran into the same problem on 1.7, but we noticed that the ISTIO_META_IDLE_TIMEOUT setting was only getting picked up on the OUTBOUND side of things, not the INBOUND. By adding an additional filter that applied to the INBOUND side of the request, we were able to successfully increase the timeout (we used 24 hours)

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: listener-timeout-tcp
  namespace: istio-system
spec:
  configPatches:
  - applyTo: NETWORK_FILTER
    match:
      context: SIDECAR_INBOUND
      listener:
        filterChain:
          filter:
            name: envoy.filters.network.tcp_proxy
    patch:
      operation: MERGE
      value:
        name: envoy.filters.network.tcp_proxy
        typed_config:
          '@type': type.googleapis.com/envoy.config.filter.network.tcp_proxy.v2.TcpProxy
          idle_timeout: 24h

我们还创建了一个类似的过滤器以应用于直通群集(因此超时仍适用于我们没有服务条目的外部流量)，因为配置也未在其中被拾取.

We also created a similar filter to apply to the passthrough cluster (so that timeouts still apply to external traffic that we don't have service entries for), since the config wasn't being picked up there either.

这篇关于1小时后，istio-proxy关闭长时间运行的TCP连接的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！