Google Cloud 数据流:如何为每个工作人员(单例)仅初始化一次 Hikari 连接池? [英] Google Cloud dataflow : How to initialize Hikari connection pool only once per worker (singleton)?

问题描述

Hibernate Utils 正在创建会话工厂和 Hikari 配置.目前我们正在 ParDo 的 @Setup 方法内部进行，但它打开了太多连接.那么有没有什么好的例子来初始化每个 worker 的连接池?

解决方案

如果您在 DoFn 中使用 @Setup 方法创建数据库连接，请记住 ApacheBeam 将为每个工作实例线程创建连接池.这可能会导致大量数据库连接，具体取决于启动的 worker 数量.

您可以在 ParDo 中使用 Singleton 类来创建连接池并控制每个工作器的连接数.您可以在此处查看示例实现

norelrnref="3Eollow"http://mail-archives.apache.org/mod_mbox/beam-issues/201905.mbox/[e-mail]%3E

Hibernate Utils is creating the session factory along with Hikari configuration. Currently we are doing inside @Setup method of ParDo, but it opens way too many connections. So is there any good example to initialize connection pool per worker ?

解决方案

If you are using @Setup method inside DoFn to create a database connection keep in mind that Apache Beam would create connection pool per worker instance thread. This might result in a lot of database connections depending on the number of workers that spin up.

You can use a Singleton class inside a ParDo to create your connection pool and control the number of connections per worker. You can take a look here for a sample implementation

http://mail-archives.apache.org/mod_mbox/beam-issues/201905.mbox/[e-mail]%3E

这篇关于Google Cloud 数据流:如何为每个工作人员(单例)仅初始化一次 Hikari 连接池?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！