几天后播放框架资源匮乏的情况 [英] Play framework resource starvation after a few days

问题描述

我在Play 2.5.8(Java)中遇到了一个问题，即使服务器CPU&内存使用情况似乎很好.无法访问数据库的端点仍然可以正常工作.

I am experiencing an issue in Play 2.5.8 (Java) where database related service endpoints starts timing out after a few days even though the server CPU & memory usage seems fine. Endpoints that does not access the DB continue to work perfectly.

该应用程序在具有t2.medium MySQL RDS的t2.medium EC2实例上运行，两者均位于同一可用区中.大多数HTTP调用以每秒约8-12个请求的速度对数据库进行查找/更新，并且还有±800个WebSocket连接/角色，每秒有±8个请求(90％的WebSocket消息无法访问数据库).数据库操作大多是简单的查找& ;;更新大约需要100毫秒.

The application runs on a t2.medium EC2 instance with a t2.medium MySQL RDS, both in the same availability zone. Most HTTP calls do lookups/updates to the database with around 8-12 requests per second, and there are also ±800 WebSocket connections/actors with ±8 requests/second (90% of the WebSocket messages does not access the database). DB operations are mostly simple lookups & updates taking around 100ms.

仅使用默认线程池时，大约需要2天才能达到死锁，并且按照

When using only the default thread pool it took about 2 days to reach the deadlock, and after moving the database requests to a separate thread pool as per https://www.playframework.com/documentation/2.5.x/ThreadPools#highly-synchronous, it improved but only to about 4 days.

这是我当前在application.conf中的线程配置:

This is my current thread config in application.conf:

akka {
  actor {
    guardian-supervisor-strategy = "actors.RootSupervisionStrategy"
  }
  loggers = ["akka.event.Logging$DefaultLogger",
    "akka.event.slf4j.Slf4jLogger"]
  loglevel = WARNING

  ## This pool handles all HTTP & WebSocket requests
  default-dispatcher {
      executor = "thread-pool-executor"
      throughput = 1
      thread-pool-executor {
        fixed-pool-size = 64 
      }
  }

  db-dispatcher {
    type = Dispatcher
    executor = "thread-pool-executor"
    throughput = 1
    thread-pool-executor {
      fixed-pool-size = 210 
    }
  }
}

数据库配置:

play.db.pool="default"
play.db.prototype.hikaricp.maximumPoolSize=200
db.default.driver=com.mysql.jdbc.Driver

我一直在研究数据库池中的连接数量.调整默认值& db-dispatcher池大小，但似乎没有任何区别.感觉上我缺少有关Play线程池&的基本知识.配置，因为我认为服务器上的负载不应成为Play处理的问题.

I have played around with the amount of connections in the DB pool & adjusting the size of the default & db-dispatcher pool size but it doesn't seem to make any difference. It feels I'm missing something fundamental about Play's thread pools & configuration as I don't think the load on the server should not be an issue for Play to handle.

推荐答案

经过更多调查，我发现此问题根本与线程池配置无关，而与由于WS重新连接直到服务器(或Play框架)不能再接受任何连接.发生这种情况时，仅服务于已建立的TCP连接，其中大多数都包括已建立的WebSocket连接.

After more investigation I found that the issue is not related to thread pool configuration at all, but rather TCP connections that build up due to WS reconnections until the server (or Play framework) cannot accept any more connections. When this happens, only established TCP connections are serviced which mostly includes the established WebSocket connections.

我还不能确定为什么不正确管理/关闭连接.

I could not yet determine why the connections are not managed/closed properly.

我的问题与此问题有关:

My issue relates to this question:

播放2.5 WebSocket连接版本

这篇关于几天后播放框架资源匮乏的情况的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！