如何设计带有MySQL DB连接的守护进程 [英] How to design a daemon with a MySQL DB connection

问题描述

假设您正在写一个服务作业队列的守护进程。各种其他软件将守护进程的作业写入队列。守护程序每隔几秒轮询队列以查看挂起作业。假设队列被实现为MySQL数据库中的表，并且守护程序是一个简单的循环：

1. 从队列中获取所有到期作业


2. 执行作业


3. 睡眠N秒


4. goto 1

守护程序必须在MySQL DB服务器中断服务并且数据库连接中断后才能生效。

守护进程每个周期连接到DB服务器一次？即在1.之前连接，并在2之间断开连接3？

或者你有守护进程保持连接打开吗？在这种情况下，它还需要a）检测服务器或连接何时不工作，b）断开并重新连接，以及c）这样做而不累积DB连接，dud连接描述符或其他死资源。

如果您有偏好，为什么？

利弊？

进入设计的因素？

任何其他方法？

这里的答案：来自php中的守护进程的mysql连接没有说明为什么最好保持连接打开。我在别的地方看到MySQL的每个连接开销非常轻。所以不明显为什么永久消费一个服务器连接比每几秒钟连接/断开更好。

在我的例子中，守护进程是用PHP编写的。

解决方案

我实际上在某事非常接近你所描述的，但在我的情况下，守护进程不轮询事件，它通过XMPP异步地（但除了这一点）。

剪下中间人

我认为不是将事件存储在数据库中，您可以使用 Gearman 异步从客户端发送（ example ）。

垃圾收集 b
$ b

PHP并非真正设计为作为守护程序运行，而是直到PHP 5.3，它才有循环引用垃圾回收，它成为一个可行的选择。

另外要注意的一点是，如果你想要长期运行时没有内存泄漏，那么非常 GC是，内存只有免费，如果它不再被引用（任何地方）。因此，如果你为全局作用域分配一个变量，它将一直存在，直到守护进程退出。重要的是，您创建或使用的任何代码不会在地方构建变量（即静态日志，不删除旧数据等）。

$ b

Stat Cache

另一件事是， $ c> clearstatcache 。由于您的PHP进程未重新启动，因此手动执行此调用以防止获取旧的统计数据（这可能会影响或可能不会影响）非常重要。根据文档这些功能被缓存。

受影响的函数包括stat（），lstat（），file_exists（），is_writable（），is_readable（），is_executable（），is_file（），is_dir （），filectime（），fileatime（），filemtime（），fileinode（），filegroup（），fileowner（），filesize（），filetype（）和fileperms blockquote>

资源管理

如果你在使用MySQL的过程中你的过程，我建议在启动时建立一个连接，并保持活着。即使它可能在MySQL端使用更多的RAM，您也可以通过不用每1秒连接一次来减少一些延迟和CPU开销。

否URL请求

这可能很明显，但是使用CLI PHP没有URL请求信息。

LooPHP

我将在这里弹出一个无耻的插件，我写了一个框架来帮助管理PHP守护进程。 LooPHP 是一个运行循环框架，可让您安排事件发生或创建对抽象源的侦听（套接字，流，等）。在我的例子中，我有守护进程做了超过1的事情，所以它是非常有帮助的系统跟踪所有的计时器为我，使我可以有效地轮询 stream_select XMPP连接。

Say you were writing a daemon that services a job queue. Various other software writes jobs for the daemon into the queue. The daemon polls the queue every few seconds for pending jobs. Assume the queue is implemented as a table in a MySQL database and that the daemon is a simple loop:

1. get all due jobs from the queue
2. do the jobs
3. sleep for N seconds
4. goto 1

The daemon must survive interrupted service from the MySQL DB server and disruption to DB connections.

Would you design the daemon to connect to the DB server once per cycle? i.e. connect before 1. and disconnect between 2 an 3?

Or would you have daemon keep a connection open? In which case it needs also to a) detect when the server or connection is not working, b) disconnect and reconnect, and c) do so without accumulating DB connections, dud connection descriptors or other dead resources.

If you have a preference, why?

Pros and cons?

Factors that enter into the design?

Any other approaches?

The answer here: mysql connection from daemon written in php doesn't say why it's better to keep the connection open. I read elsewhere that the per-connection overhead in MySQL is very light. So it isn't obvious why permanently consuming one server connection is better than connecting/disconnecting every few seconds.

In my case the daemon is written in PHP.

解决方案

I'm actually work on something very close to what you described, but in my case the daemon doesn't poll for event it get's them asynchronously via XMPP (but that's besides the point).

Cut out the Middle Man

I think instead of storing the events in the database and polling them w/ MySQL you could probably use Gearman to send them from the client asynchronously (example).

Garbage Collection

PHP isn't really designed to run as a daemon, and it wasn't until PHP 5.3 when it got circular reference garbage collection that it became a viable option. It's very important that you use PHP 5.3 if you want any chance at long term running without memory leaks.

Another thing to keep in mind about GC is that memory is only free if it's not longer referenced (anywhere). So if you assign a variable to the global scope, it'll be there until the daemon exits. It's important that any code you create or use doesn't built up variables in places (ie, static log, not removing old data, etc).

Stat Cache

Another thing is that it's important to run clearstatcache every so often. Since your PHP process isn't restarted it's important to do this call manually to prevent getting old stat data (which may or may not effect you). According to the documentation these functions are cached.

Affected functions include stat(), lstat(), file_exists(), is_writable(), is_readable(), is_executable(), is_file(), is_dir(), is_link(), filectime(), fileatime(), filemtime(), fileinode(), filegroup(), fileowner(), filesize(), filetype(), and fileperms().

Resource management

If your going to be using thing like MySQL during the lifetime of your process, I'd suggest making one connection at startup and keeping it alive. Even though it might use more ram on the MySQL side, you'll cut out some latency and CPU overhead by not having to connect every 1 second.

No URL request

This may seem obvious, but with CLI PHP there is no URL request info. Some libraries aren't written with this in mind, and this can cause some problems.

LooPHP

I'm going to pop a shameless plug in here for a framework I wrote to help with the management of PHP daemons. LooPHP is a run loop framework that lets you schedule event to happen or create listens for abstract sources (socket, stream, etc). In my case, I have the daemon doing more than 1 thing, so it's very helpful to have system keep track of all the timers for me so that I can effectively poll stream_select for the XMPP connection.

这篇关于如何设计带有MySQL DB连接的守护进程的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！