同时采用Boost库我的程序不支持线程安全日志记录 [英] My program doesn't support thread safe logging while using boost library
问题描述
我目前创建应该用于记录目的的一类。
I am currently creating a class that should be used for logging purpose.
但这里有一个问题，它不支持线程同步。当我由一个线程运行一个是没有问题的。但有问题的，而我同时运行两个或多个线程。
But here one problem , it doesn't support thread synchronization. There is no problem when I run one by one thread. But there is problem while I run two or more threads at a time.
我现在面临的问题是：如果我创建了两个线程创建两个单独的日志，但两个线程碰撞，写入其日志消息两个文件
Problem I am facing is: "If I create two threads to create two separate logs , but Two threads are colliding and write its log messages to both the files".
如果任何人发现问题，请帮我解决这个问题。
If anyone find the problem , please help me to resolve it.
BoostLogger.h：
BoostLogger.h :
#pragma once
......
///////////////////////////////////////
//Defining Macros
///////////////////////////////////////
#define AddCommonAttr() logging::add_common_attributes()
#define GetLoggingCore() logging::core::get()
#define LoggingSeverity logging::trivial::severity
#define AddFileLog logging::add_file_log
#define ThreadValueType logging::attributes::current_thread_id::value_type
#define Record logging::record
#define Extract logging::extract
#define ExprStream expr::stream
#define ExprAttr expr::attr
#define ExprFormatDateTime expr::format_date_time
#define PosixTimeType boost::posix_time::ptime
#define ExprMessage expr::smessage
#define FileName keywords::file_name
#define RotationSize keywords::rotation_size
#define TimeBasedRotation keywords::time_based_rotation
#define Format keywords::format
#define Target keywords::target
#define MaxSize keywords::max_size
#define MinFreeSpace keywords::min_free_space
#define RotationAtTimeInterval sinks::file::rotation_at_time_interval
#define Reset_Filter reset_filter /*The reset_filter method removes the global logging filter.*/
#define Set_Filter set_filter /*The set_filter method sets the global logging filter to every log record that is processed.*/
#define SetFormatter set_formatter
#define RecordView logging::record_view
#define FormattingOstream logging::formatting_ostream
#define SharedPtr boost::shared_ptr
#define MakeShared boost::make_shared
#define SinkFileBackend sinks::text_file_backend
#define LockedBackend locked_backend
#define SetFileCollector set_file_collector
#define MakeCollector sinks::file::make_collector
#define AddSink add_sink /*The add_sink method adds a new sink. The sink is included into logging process immediately after being added and until being removed. No sink can be added more than once at the same time. If the sink is already registered, the call is ignored.*/
#define RemoveSink remove_sink /*The remove_sink method removes the sink from the output. The sink will not receive any log records after removal. The call has no effect if the sink is not registered.*/
#define RemoveAllSinks remove_all_sinks /*The remove_all_sinks method removes all registered sinks from the output. The sinks will not receive any log records after removal.*/
#define Flush flush
#define ScanForFiles scan_for_files
#define ScanAll sinks::file::scan_all
#define ScanMatching sinks::file::scan_matching
#define SetExceptionHandler set_exception_handler
#define ExceptionSuppressor logging::make_exception_suppressor
#define MakeExceptionHandler logging::make_exception_handler
typedef sinks::synchronous_sink < SinkFileBackend > sink_type;
static src::logger lg;
#define WriteToLog BOOST_LOG(lg)
/*Defining Macros for Writing log with Severity*/
//BOOST_LOG_INLINE_GLOBAL_LOGGER_DEFAULT(my_logger, src::logger_mt)
//static src::severity_logger< logging::trivial::severity_level > slg;
#define LogTrace BOOST_LOG_SEV(obj->slg, logging::trivial::trace)
#define LogDebug BOOST_LOG_SEV(obj->slg, logging::trivial::debug)
#define LogInfo BOOST_LOG_SEV(obj->slg, logging::trivial::info)
#define LogWarning BOOST_LOG_SEV(obj->slg, logging::trivial::warning)
#define LogError BOOST_LOG_SEV(obj->slg, logging::trivial::error)
#define LogFatal BOOST_LOG_SEV(obj->slg, logging::trivial::fatal)
#define _1MB (1 * 1024 * 1024)
#define _10MB (10 * 1024 * 1024)
#define datefmt ("_%Y-%b-%d")
#define timefmt ("_%H-%M-%S")
using namespace std;
class CBoostLogger
{
private:
SharedPtr< SinkFileBackend > backend;
SharedPtr< sink_type > sink;
public:
src::severity_logger< logging::trivial::severity_level > slg;
CBoostLogger(void);
~CBoostLogger(void);
bool StartLogger(struct FileFormat *sff);
bool StopLogger();
bool SetFilter(short severitylevel);
bool SetFormat(struct LogFormat *sle);
private:
friend void Formatter(logging::record_view const& rec, logging::formatting_ostream& strm);
};
/*This Structure is used to set the formats for file*/
struct FileFormat
{
bool includedatetofile;
bool includetimetofile;
string filename;
string filelocation;
unsigned long rotationsize;
unsigned long maxsize;
FileFormat() : includedatetofile(false),
includetimetofile(false),
filename("log") ,
filelocation("C:/Log") ,
rotationsize(_1MB) ,
maxsize(_10MB) {};
};
struct LogFormat
{
bool Set_LineID;
bool Set_Time;
bool Set_Severity;
bool Set_ThreadID;
bool Set_Message;
LogFormat() : Set_LineID(true) ,
Set_Time(true) ,
Set_Severity(true) ,
Set_ThreadID(true) ,
Set_Message(true) {};
LogFormat(bool lineid, bool time, bool severity, bool threadid, bool message)
: Set_LineID(lineid) ,
Set_Time(time) ,
Set_Severity(severity) ,
Set_ThreadID(threadid) ,
Set_Message(message) {};
};
BoostLogger.cpp：
BoostLogger.cpp:
#pragma once
#include "BoostLogger.h"
////////////////////////////////////
//Global Declarations
////////////////////////////////////
bool SetLineID, SetTime, SetSeverity, SetThreadID, SetMessage ;
CBoostLogger::CBoostLogger(void)
{
cout << "Calling CBoostLogger Constructor..." << endl;
SetFilter(2);
//GetLoggingCore()->SetExceptionHandler(MakeExceptionHandler<std::runtime_error,std::exception>(handler()));
GetLoggingCore()->SetExceptionHandler(ExceptionSuppressor());
}
CBoostLogger::~CBoostLogger(void)
{
GetLoggingCore() -> Reset_Filter();
GetLoggingCore() -> RemoveAllSinks();
}
bool CBoostLogger::StartLogger(struct FileFormat *sff )
{
if(sff->includedatetofile)
sff->filename += datefmt;
if(sff->includetimetofile)
sff->filename += timefmt;
sff->filename += ".log";
backend = MakeShared < SinkFileBackend >(
FileName = sff->filename, /*< file name pattern >*/
RotationSize = sff->rotationsize /*< rotate files for every 1M >*/
);
sink = MakeShared < sink_type > (backend);
LogFormat sle;
SetFormat(&sle);
sink->LockedBackend()->SetFileCollector
(
MakeCollector
(
Target = sff->filelocation , /*File Storage Location*/
MaxSize = sff->maxsize /*Limit for folder : maxsize, where initially maxsize = 10M*/
)
);
sink->LockedBackend()->ScanForFiles(ScanAll);
GetLoggingCore()->AddSink(sink);
AddCommonAttr();
BOOST_LOG_SEV(this->slg, logging::trivial::info) << "Logger Starts";
return true;
}
/*This function used to remove the registered sink from core.*/
bool CBoostLogger::StopLogger()
{
BOOST_LOG_SEV(this->slg, logging::trivial::info) << "Logger Stops";
GetLoggingCore()->RemoveSink(sink);
GetLoggingCore()->Flush();
return true;
}
/*This function is used to set filter level. */
bool CBoostLogger::SetFilter(short severitylevel)
{
GetLoggingCore()->Set_Filter
(
LoggingSeverity >= severitylevel
);
return true;
}
/*This function is used to set format for log. */
bool CBoostLogger::SetFormat(struct LogFormat *sle)
{
SetLineID = sle->Set_LineID;
SetTime = sle->Set_Time;
SetSeverity = sle->Set_Severity;
SetThreadID = sle->Set_ThreadID;
SetMessage = sle->Set_Message;
sink->SetFormatter(&Formatter);
return true;
}
/*This function is used to set format for the log file.*/
void Formatter(RecordView const& rec, FormattingOstream& strm)
{
if(SetLineID)
{
strm << Extract < unsigned int > ("LineID", rec) << "\t"; // Get the LineID attribute value and put it into the stream
}
if(SetTime)
{
strm << Extract < PosixTimeType > ("TimeStamp", rec) << "\t"; // Get the TimeStamp attribute value and put it into the stream
}
if(SetSeverity)
{
strm << "[ " << rec[LoggingSeverity] << " ]\t"; // Get the Severity attribute value and put it into the stream
}
if(SetThreadID)
{
strm << Extract < ThreadValueType > ("ThreadID", rec )<<"\t"; // Get the ThreadID attribute value and put into the stream
}
if(SetMessage)
{
strm << rec[ExprMessage]; // Finally, put the record message to the stream
}
}
struct handler
{
void operator()(const runtime_error &ex) const
{
std::cerr << "\nRuntime_error: " << ex.what() << '\n';
}
void operator()(const exception &ex) const
{
std::cerr << "Exception: " << ex.what() << '\n';
}
};
Source.cpp：
Source.cpp :
#include "BoostLogger.h"
void func_thread(std::string fn,string fl,int num)
{
std::string buf = "";
char str[20];
buf += itoa(num, str, 10);
fn += buf;
CBoostLogger *obj = new CBoostLogger();
FileFormat formatobj;
formatobj.filename = fn;
formatobj.filelocation = fl;
formatobj.includedatetofile = true;
formatobj.includetimetofile = true;
obj->StartLogger(&formatobj);
for(int i=0;i<10000;i++)
{
LogTrace << "Trace message new " << fn;
BOOST_LOG_SEV(obj->slg,logging::trivial::trace) << "Test";
LogDebug << "Debug Message new" << fn;
LogInfo << "Info message" << fn;
LogWarning << "Warning message new" << fn;
LogError << "An error message new" << fn;
LogFatal << "A fatal message new" << fn;
}
LogFormat sle(true,false,false,false,true);
obj->SetFormat(&sle);
for(int i=0;i<10000;i++)
{
LogTrace << "Trace message new " << fn;
LogDebug << "Debug Message new" << fn;
LogInfo << "Info message" << fn;
LogWarning << "Warning message new" << fn;
LogError << "An error message new" << fn;
LogFatal << "A fatal message new" << fn;
}
obj->StopLogger();
delete obj;
}
int main()
{
//This following code makes problem.
boost::thread *thread1 = new boost::thread(&func_thread,"Thread_","C:/BoostLog",1);
boost::thread *thread2 = new boost::thread(&func_thread,"Thread_","C:/BoostLog",2);
thread1->join();
thread2->join();
/*
//This following is not making that problem.
boost::thread_group t_groups;
for(int i=1;i<=5;i++)
{
t_groups.create_thread(boost::bind(&func_thread,"Thread","C:/BoostLog",i));
t_groups.join_all();
}
boost::thread_group tgroup;
boost::thread *threads;
for(int i=0;i<20;i++)
{
threads=new boost::thread(&func_thread,"Thread","C:/BoostLog",i);
tgroup.add_thread(threads);
std::cout << "\nThread "<<i<<" is created whose id is : "<<threads->get_id();
threads->join();
}
*/
return 0;
}
如果您需要这方面的信息了，请叫我。
If you need anymore information regarding this, please ask me.
我想使用boost库中创建线程安全记录。如果你能帮助我。
I want to create thread safe logger using boost library. Help me if you can.
和另外一件事，如果可能的话，必须线程并行运行。
And another one thing , if possible threads must run concurrently.
感谢。
推荐答案
作为一个在需要做低延迟记录从多个线程我最近做了一些狩猎周围的位置。
Being in a position of needing to do low latency logging from multiple threads I've recently done a bit of hunting around.
基本要求
就我而言一个基本要求是，记录事件的时候应该是固定的，而且越短越好。很多我所见过的多线程日志记录选项倒在这（log4cpp，提振吨伐木者只要我可以告诉）。
As far as I'm concerned a basic requirement is that the time to log an event should be fixed, and the shorter the better. A lot of the "multi-threaded" logging options I've seen fall down on this (log4cpp, boost mt loggers so far as I can tell).
这具有某种嵌入，以确保使用它与保证最高的日志记录时间线程安全不提供每个线程的互斥体的单个记录。当一个线程试图登录它可能是有其他十几个线程都在竞争同一个互斥。所以线程安全不扩大，也保证低延迟。
A single logger that has a mutex of some sort embedded in it to ensure thread safety is not providing each thread using it with a guaranteed maximum logging time. When one thread tries to log it may be that there's a dozen other threads all contending on the same mutex. So thread safe doesn't scale up and also guarantee low latency.
理想
现在需要的是一些地方的记录器的总是可用的线程，事件被塞进某种队列，并有在另一端一个单独的线程拉事件从队列和写他们到文件或什么的。
What's needed is something where the logger is always available to the thread, events are stuffed into some sort of queue, and there's a separate thread at the other end pulling events off the queue and writing them to file or whatever.
阿帕奇log4cpp
阿帕奇log4cpp得到接近这个 - 它有一个AsyncAppender。我说接近;虽然它有什么其他的appender一个人连接后端线程调用，前端不能被一堆没有他们，所有的线程争夺同一个互斥之间共享。但至少写入文件的行为被伐木脱钩。这又大大有助于确保一个线程不阻止日志。
Apache log4cpp gets close to this - it has an AsyncAppender. I say close; whilst it has the back end thread calling whatever other appenders one has attached, the front end cannot be shared between a bunch of threads without them all fighting over the same mutex. But at least the act of writing to file is decoupled from logging. This goes a long way towards ensuring that a thread isn't blocked by a log.
Log4cpp
Log4cpp有BufferingAppender（在该文档中没有提及，但它的存在在code）。这有一个队列，但没有调度线程像log4cxx的AsyncAppender。这的可能的作为模板来打造更好的东西。
Log4cpp has a BufferingAppender (not mentioned in the docs, but it's there in the code). This has a queue, but no dispatcher thread like log4cxx's AsyncAppender. This could be used as a template to build something better.
升压登录
这是一个讨厌的恐怖大丑一堆难以揣测和不完全记载code的。我还没能参透出到底是什么实际上可以做相一相线程和我的理想，但我认为它asynchronous_sink做这项工作。如Apache log4cxx它看起来像记录前端还采用了互斥体，使其线程安全的。
This is a nasty horrible big ugly pile of hard to fathom and incompletely documented code. I've not yet been able to fathom out exactly what it can actually do vis a vis threads and my ideal, but I think that its asynchronous_sink does the job. Like Apache log4cxx it looks like the logging front end also uses a mutex to make it thread safe.
ZeroMQ
我Log4cxx和升压日志中看到的问题是，没有太多的空间日志架构的灵活性。我真正想要的是你可以做ZeroMQ模式的类型。
The problem I see with Log4cxx and Boost log is that there's not much room for logging architectural flexibility. What I really want is the type of patterns you can do with ZeroMQ.
我已经变成确信，我需要写我自己的日志库。 ZeroMQ有一些非常好的模式（特别是PUB / SUB），并以此来推动从线程到中央记录线程日志消息似乎是一个不错的主意，尤其是考虑ZeroMQ的架构灵活性。
I'm becoming convinced that I need to write my own logging library. ZeroMQ has some very nice patterns (particularly PUB/SUB), and using this to push log messages from threads to a central logging thread would seem to be a neat idea, especially given ZeroMQ's architectural flexibility.
该ZeroMQ插座将作为缓冲来吸收突发记录的需求，而中央线程沿线所有写入磁盘plods。提供记录的持续率不超过硬盘的带宽，它会被罚款。
The ZeroMQ sockets would act as buffers to soak up sudden logging demand, whilst the central thread plods along writing it all to disk. Provided the sustained rate of logging doesn't exceed the harddrive bandwidth it'll be fine.
我可能会尝试把它挤入log4cpp字面上使用它作为一个中央追加程序，做实际输出每个线程类/附加目的地之间的交通和。这将是有点棘手，但。 ZeroMQ传输的字节，而不是对象。所以，我可能要连载log4cpp事件......这则提出了用什么系列化，等等，等等的问题之前，我知道它，它会成为一个讨厌的大怪物低效
I may try to squeeze it into log4cpp and literally use it as a transport between per thread category/appenders, and a central appender that does the actual output. This is going to be slightly tricky though. ZeroMQ transports bytes, not objects. So I may have to serialise log4cpp events... This then raises the question of what serialisation to use, etc. etc. and before I know it it'll become a nasty big inefficient monster.
这篇关于同时采用Boost库我的程序不支持线程安全日志记录的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
