多个线程在C中写入单个文件 [英] Multiple threads writing to a single file in C

问题描述

我在C应用程序中有几个线程，我希望它们写入一个日志文件而不会阻止

它们的运行。

我可能需要做一个工作线程，当它想要写入文件然后写入从信令线程获得的数据时，每个线程都会发出信号



写作顺序很重要



我怎样才能实现这个目标？



谢谢！



dj



我的尝试：



使用事件但我无法理解如何发送写入数据

Hi,
I have several threads in a C application and i want them to write to a single log file without blocking
their run.
I probably need to do a worker thread which gets signaled by each thread when it wants to write to the file and then write the data which it got from the signaling thread

The order of the writing is important

How can i achieve this?

thanks!

dj

What I have tried:

Using Events but i cant understand how to send the writing data

推荐答案

我写了一个库和实用程序来完成这个多年前。它用于事件消息记录。我最后在共享内存中使用一个大缓冲区来保存记录的消息数据，然后我有一个单独的进程监视共享内存并定期将其写入文件。我们使用互斥锁来控制对缓冲区的访问。



这意味着记录事件的正常事件序列是获取互斥锁，将消息文本复制到第一个可用缓冲区槽，并释放互斥锁。它也很快。我们有几十个进程在运行，所以它的工作非常繁重。文件写入过程一直等到缓冲区几乎已满，然后将缓冲区写入文件并清除它。当这一天翻身时，它还在午夜写了缓冲区。这是因为我们为一天的所有事件保留了一个文件。



消息的缓冲区大约是1MB，并且将它写入文件的速度很慢。编写器必须打开日志文件，寻找它的结尾，写入所有数据，然后关闭文件，同时按住互斥锁。这花费了相当长的时间来阻碍工作进程和线程。为了加快速度，我让它将整个缓冲区复制到本地分配的缓冲区，然后从那里写入数据。这需要更长的时间，但这意味着互斥锁被保留（并且工作人员被阻止）不到一毫秒，仅在内存复制的持续时间内。然后，用于写入消息缓冲区的事件序列为副本分配内存，获取互斥锁，复制缓冲区，重置缓冲区的计数器，释放互斥锁，打开文件并搜索结束，写入数据，关闭文件，释放分配的内存。



重要的是要注意我的系统使用多个进程，所以我使用互斥锁来控制对缓冲区的访问。我还为多线程环境编写了这个，为此我使用了一个关键部分来控制访问，因为它比互斥锁更有效。这是因为互斥锁是一个可以在多个进程之间工作的内核级对象。关键部分不能并且只能在单个进程的多个线程之间工作。由于您的系统是一个单一过程，因此关键部分最好。两种设计的逻辑和实现基本相同，除了文件编写器位于单独的线程而不是单独的进程中，并且使用临界区而不是互斥锁进行访问控制。此外，缓冲区可以在本地堆内存而不是共享内存中，但这是可选的。

I wrote a library and utility to do exactly this many years ago. It was for event message logging. I ended up using a big buffer in shared memory to hold the logged message data and then I had a separate process that monitored the shared memory and wrote it to a file periodically. We used a mutex to control access to the buffer.

This meant the normal sequence of events to log an event was acquire the mutex, copy the message text to the first available buffer slot, and release the mutex. It was pretty fast too. We had dozens of processes running so it was worked pretty heavily. The file writing process waited until the buffer was almost full and then wrote the buffer to a file and cleared it. It also wrote the buffer at midnight when the day rolled over. This is because we kept one file for all the events of one day.

The buffer of messages was around 1MB and writing it to the file was slow. The writer had to open the log file, seek to the end of it, write all the data, and then close the file, all while holding the mutex. That took a surprisingly long amount of time which held up the worker processes and threads. To speed things up, I had it copy the entire buffer to a locally allocated buffer and then write the data from there. This took a little longer BUT it meant the mutex was held (and workers blocked) for less than a millisecond, just for the duration of the memory copy. The sequence of events for writing the message buffer was then allocate memory for the copy, acquire the mutex, copy the buffer, reset the buffer's counters, release the mutex, open the file and seek to its end, write the data, close the file, release the allocated memory.

It is important to note my system used multiple processes so I used a mutex to control access to the buffer. I also wrote this for a multiple threaded environment and for it I used a critical section to control access because it is much more efficient than a mutex. This is because a mutex is a kernel-level object that can work between multiple processes. A critical section is not and can work only between multiple threads of a single process. Since your system is a single process, a critical section would be best. The logic and implementation is essentially the same for both designs except the file writer is in a separate thread instead of a separate process and a critical section is used instead of a mutex for access control. Also, the buffer can be in local heap memory instead of shared memory but that is optional.

这篇关于多个线程在C中写入单个文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！