NLog 的线程安全性如何? [英] How Thread-Safe is NLog?
问题描述
嗯，
我已经等了好几天才决定发布这个问题，因为我不知道如何说明这一点，所以我写了一篇很长的详细帖子.但是，我认为此时寻求社区的帮助是有意义的.
I have waited for days before deciding to post this issue, as I was not sure how to state this, resutling into a long detailed post. However, I think it is relevant to ask for the community's help at this point.
基本上，我尝试使用 NLog 为数百个线程配置记录器.我认为这会非常简单，但在几十秒后我得到了这个异常:"InvalidOperationException : 集合被修改；枚举操作可能无法执行"
Basically, I tried to use NLog to configure loggers for hundreds of threads. I thought this would be very straightforward, But I got this exception after few tens of seconds : "InvalidOperationException : Collection was modified; enumeration operation may not execute"
这是代码.
//Launches threads that initiate loggers
class ThreadManager
{
//(...)
for (int i = 0; i<500; i++)
{
myWorker wk = new myWorker();
wk.RunWorkerAsync();
}
internal class myWorker : : BackgroundWorker
{
protected override void OnDoWork(DoWorkEventArgs e)
{
// "Logging" is Not static - Just to eliminate this possibility
// as an error culprit
Logging L = new Logging();
//myRandomID is a random 12 characters sequence
//iLog Method is detailed below
Logger log = L.iLog(myRandomID);
base.OnDoWork(e);
}
}
}
public class Logging
{
//ALL THis METHOD IS VERY BASIC NLOG SETTING - JUST FOR THE RECORD
public Logger iLog(string loggerID)
{
LoggingConfiguration config;
Logger logger;
FileTarget FileTarget;
LoggingRule Rule;
FileTarget = new FileTarget();
FileTarget.DeleteOldFileOnStartup = false;
FileTarget.FileName = "X:\" + loggerID + ".log";
AsyncTargetWrapper asyncWrapper = new AsyncTargetWrapper();
asyncWrapper.QueueLimit = 5000;
asyncWrapper.OverflowAction = AsyncTargetWrapperOverflowAction.Discard;
asyncWrapper.WrappedTarget = FileTarget;
//config = new LoggingConfiguration(); //Tried to Fool NLog by this trick - bad idea as the LogManager need to keep track of all config content (which seems to cause my problem;
config = LogManager.Configuration;
config.AddTarget("File", asyncWrapper);
Rule = new LoggingRule(loggerID, LogLevel.Info, FileTarget);
lock (LogManager.Configuration.LoggingRules)
config.LoggingRules.Add(Rule);
LogManager.Configuration = config;
logger = LogManager.GetLogger(loggerID);
return logger;
}
}
所以我完成了我的工作，而不是仅仅在这里发布我的问题并享受家庭时光，我整个周末都在研究这个(幸运男孩！)我下载了 NLOG 2.0 的最新稳定版本并将其包含在我的项目中.我能够追踪它爆炸的确切位置:
So I did my job an rather than just posting my question here and having a family-quality-time, I spent the week-end digging on that (Lucky Boy ! ) I downloaded the latest stable release of NLOG 2.0 and included it in my project. I was able to trace the exact place where it blowed :
在 LogFactory.cs 中:
in LogFactory.cs :
internal void GetTargetsByLevelForLogger(string name, IList<LoggingRule> rules, TargetWithFilterChain[] targetsByLevel, TargetWithFilterChain[] lastTargetsByLevel)
{
//lock (rules)//<--Adding this does not fix it
foreach (LoggingRule rule in rules)//<-- BLOWS HERE
{
}
}
在 LoggingConfiguration.cs 中:
in LoggingConfiguration.cs :
internal void FlushAllTargets(AsyncContinuation asyncContinuation)
{
var uniqueTargets = new List<Target>();
//lock (LoggingRules)//<--Adding this does not fix it
foreach (var rule in this.LoggingRules)//<-- BLOWS HERE
{
}
}
我认为的问题
因此，根据我的理解，会发生什么情况是 LogManager 混淆了，因为 从不同线程调用 config.LoggingRules.Add(Rule) 而 GetTargetsByLevelForLogger 和 FlushAllTargets 正在被调用.我试图搞砸 foreach 并用 for 循环替换它，但记录器变成了流氓(跳过了许多日志文件的创建)
The problem according to me
So, based on my understanding, what happens is that the LogManager gets mixed up as there are calls to config.LoggingRules.Add(Rule) from different threads while GetTargetsByLevelForLogger and FlushAllTargets are being called.
I tried to screw the foreach and replace it by a for loop but the logger turned rogue (skipping many log files creation)
SOoooo 终于
到处都写着 NLOG 是线程安全的，但我已经阅读了一些帖子，这些帖子进一步挖掘并声称这取决于使用场景.我的情况呢?我必须创建数千个记录器(不是同时创建所有记录器，但速度仍然非常快).
SOoooo FINALLY
It is written everywhere that NLOG is thread-safe, but I've come through a few posts that dig things further and claim that this depends on the utilization scenario. What about my case ?
I have to create thousands of loggers (not all at the same time, but still at a very high pace).
我发现的解决方法是在同一主线程中创建所有记录器；这真的不方便，因为我在应用程序开始时创建了我所有的应用程序记录器(有点像记录器池).虽然它工作得很好，但它只是不可接受的设计.
The workaround I have found is to create all loggers in the SAME MASTER THREAD; this is REALLY unconvenient, as I create all my app loggers at the start of the app (sort of a Logger Pool). And though it works sweet, it is just NOT an acceptable design.
所以你们都知道，伙计们.请帮助编码员再次见到他的家人.
So you know it all Folks. Please help a coder see his family again.
推荐答案
对于你的问题，我真的没有答案，但我确实有一些观察和一些问题:
I don't really have an answer to your problem, but I do have some observations and some questions:
根据您的代码，您似乎想为每个线程创建一个记录器，并且您希望该记录器记录到以某个传入 id 值命名的文件中.因此，id 为abc"的记录器将记录到x:abc.log"，def"将记录到x:def.log"，依此类推.我怀疑您可以通过 NLog 配置而不是通过编程来做到这一点.我不知道它是否会更好用，或者 NLog 是否会遇到与您相同的问题.
According to your code, it looks like you want to create a logger per thread and you want to have that logger log to a file named for some passed-in id value. So, the logger whose id is "abc" would log to "x:abc.log", "def" would log to "x:def.log", and so on. I suspect that you can do this via NLog configuration rather than programmatically. I don't know if it would work any better, or if NLog would have the same issue as you are having.
我的第一印象是你做了很多工作:为每个线程创建一个文件目标，为每个线程创建一个新规则，获取一个新的记录器实例等，你可能不需要做这些工作来完成它显示的内容你想要完成的.
My first impression is that you are doing a lot of work: creating a file target per thread, creating a new rule per thread, getting a new logger instance, etc, that you might not need to do to accomplish what it appears that you want to accomplish.
我知道 NLog 允许动态命名输出文件，至少基于一些 NLog LayoutRenderer.例如，我知道这行得通:
I know that NLog allows the output file to be named dynamically, based on at least some of the NLog LayoutRenderers. For example, I know that this works:
fileName="${level}.log"
并且会给你这样的文件名:
and will give you filenames like this:
Trace.log
Debug.log
Info.log
Warn.log
Error.log
Fatal.log
因此，例如，您似乎可以使用这样的模式根据线程 ID 创建输出文件:
So, for example, it seems that you could use a pattern like this to create output file(s) based on thread id:
fileName="${threadid}.log"
如果您最终拥有线程 101 和 102，那么您将拥有两个日志文件:101.log 和 102.log.
And if you ended up having threads 101 and 102, then you would have two log files: 101.log and 102.log.
在您的情况下，您想根据自己的 ID 命名文件.您可以将 id 存储在 MappedDiagnosticContext(这是一个允许您存储线程本地名称-值对的字典)中，然后在您的模式中引用它.
In your case, you want to name the file based on your own id. You could store the id in the MappedDiagnosticContext (which is a dictionary that allows you to store thread-local name-value pairs) and then reference that in your pattern.
您的文件名模式如下所示:
Your pattern for your filename would look something like this:
fileName="${mdc:myid}.log"
因此，在您的代码中，您可以这样做:
So, in your code you might do this:
public class ThreadManager
{
//Get one logger per type.
private static readonly Logger logger = LogManager.GetCurrentClassLogger();
protected override void OnDoWork(DoWorkEventArgs e)
{
// Set the desired id into the thread context
NLog.MappedDiagnosticsContext.Set("myid", myRandomID);
logger.Info("Hello from thread {0}, myid {1}", Thread.CurrentThread.ManagedThreadId, myRandomID);
base.OnDoWork(e);
//Clear out the random id when the thread work is finished.
NLog.MappedDiagnosticsContext.Remove("myid");
}
}
这样的事情应该允许你的 ThreadManager 类有一个名为ThreadManager"的记录器.每次它记录一条消息时，它都会在 Info 调用中记录格式化的字符串.如果记录器被配置为记录到文件目标(在配置文件中制定一个规则，将*.ThreadManager"发送到文件目标，其文件名布局是这样的:
Something like this should allow your ThreadManager class to have a single logger named "ThreadManager". Each time it logs a message, it will log the formatted string in the Info call. If the logger is configured to log to the File target (in the config file make a rule that sends "*.ThreadManager" to a File target whose filename layout is something like this:
fileName="${basedir}/${mdc:myid}.log"
在记录消息时，NLog 将根据 fileName 布局的值确定文件名应该是什么(即它在记录时应用格式标记).如果文件存在，则将消息写入其中.如果该文件尚不存在，则会创建该文件并将消息记录到该文件中.
At the time a message is logged, NLog will determine what the filename should be, based on the value of the fileName layout (i.e. it applies the formatting tokens at log time). If the file exists, then the message is written to it. If the file does not exist yet, the file is created and the message is logged to it.
如果每个线程都有一个随机的 id，比如aaaaaaaaaaaa"、aaaaaaaaaaab"、aaaaaaaaaaac"，那么你应该得到这样的日志文件:
If each thread has a random id like "aaaaaaaaaaaa", "aaaaaaaaaaab", "aaaaaaaaaaac", then you should get log files like this:
aaaaaaaaaaaa.log
aaaaaaaaaaab.log
aaaaaaaaaaac.log
等等.
如果您可以这样做，那么您的生活应该会更简单，因为您不必进行 NLog 的所有编程配置(创建规则和文件目标).您可以让 NLog 担心创建输出文件名.
If you can do it this way, then your life should be simpler as you do not have to all of that programmatic configuration of NLog (creating rules and file targets). And you can let NLog worry about creating the output file names.
我不确定这会比你之前做的更好.或者，即使是这样，您可能真的需要在更大的范围内了解您正在做的事情.测试它是否有效应该很容易(即，您可以根据 MappedDiagnosticContext 中的值命名输出文件).如果它适用于此，那么您可以在创建数千个线程的情况下尝试使用它.
I don't know for sure that this will work any better than what you were doing. Or, even if it does, you might really need to what you are doing in your bigger picture. It should be easy enough to test to see that it even works (i.e. that you can name your output file based a value in the MappedDiagnosticContext). If it works for that, then you can try it for your case where you are creating thousands of threads.
更新:
这是一些示例代码:
使用这个程序:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using NLog;
using System.Threading;
using System.Threading.Tasks;
namespace NLogMultiFileTest
{
class Program
{
public static Logger logger = LogManager.GetCurrentClassLogger();
static void Main(string[] args)
{
int totalThreads = 50;
TaskCreationOptions tco = TaskCreationOptions.None;
Task task = null;
logger.Info("Enter Main");
Task[] allTasks = new Task[totalThreads];
for (int i = 0; i < totalThreads; i++)
{
int ii = i;
task = Task.Factory.StartNew(() =>
{
MDC.Set("id", "_" + ii.ToString() + "_");
logger.Info("Enter delegate. i = {0}", ii);
logger.Info("Hello! from delegate. i = {0}", ii);
logger.Info("Exit delegate. i = {0}", ii);
MDC.Remove("id");
});
allTasks[i] = task;
}
logger.Info("Wait on tasks");
Task.WaitAll(allTasks);
logger.Info("Tasks finished");
logger.Info("Exit Main");
}
}
}
还有这个 NLog.config 文件:
And this NLog.config file:
<?xml version="1.0" encoding="utf-8" ?>
<!--
This file needs to be put in the application directory. Make sure to set
'Copy to Output Directory' option in Visual Studio.
-->
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<targets>
<target name="file" xsi:type="File" layout="${longdate} | ${processid} | ${threadid} | ${logger} | ${level} | id=${mdc:id} | ${message}" fileName="${basedir}/log_${mdc:item=id}.txt" />
</targets>
<rules>
<logger name="*" minlevel="Debug" writeTo="file" />
</rules>
</nlog>
我能够为委托的每次执行获取一个日志文件.日志文件以存储在 MDC 中的id"命名(MappedDiagnosticContext).
I am able to get one log file for each execution of the delegate. The log file is named for the "id" stored in the MDC (MappedDiagnosticContext).
因此，当我运行示例程序时，我得到了 50 个日志文件，每个文件中有三行Enter..."、Hello..."、Exit...".每个文件都被命名为 log__X_.txt，其中 X 是捕获的计数器 (ii) 的值，所以我有 log_0.txt, log_1.txt、log_1.txt 等，log_49.txt.每个日志文件仅包含与一次委托执行相关的日志消息.
So, when I run the sample program, I get 50 log files, each of which has three lines in it "Enter...", "Hello...", "Exit...". Each file is named log__X_.txt where X is the value of the captured counter (ii), so I have log_0.txt, log_1.txt, log_1.txt, etc, log_49.txt. Each log file contains only those log messages pertaining to one execution of the delegate.
这与您想做的相似吗?我的示例程序使用任务而不是线程，因为我前段时间已经编写了它.我认为该技术应该很容易适应你正在做的事情.
Is this similar to what you want to do? My sample program uses Tasks rather than threads because I had already written it some time ago. I think that the technique should adapt to what you are doing easily enough.
您也可以使用相同的 NLog.config 文件这样做(为委托的每次执行获取一个新的记录器):
You could also do it this way (getting a new logger for each excecution of the delegate), using the same NLog.config file:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using NLog;
using System.Threading;
using System.Threading.Tasks;
namespace NLogMultiFileTest
{
class Program
{
public static Logger logger = LogManager.GetCurrentClassLogger();
static void Main(string[] args)
{
int totalThreads = 50;
TaskCreationOptions tco = TaskCreationOptions.None;
Task task = null;
logger.Info("Enter Main");
Task[] allTasks = new Task[totalThreads];
for (int i = 0; i < totalThreads; i++)
{
int ii = i;
task = Task.Factory.StartNew(() =>
{
Logger innerLogger = LogManager.GetLogger(ii.ToString());
MDC.Set("id", "_" + ii.ToString() + "_");
innerLogger.Info("Enter delegate. i = {0}", ii);
innerLogger.Info("Hello! from delegate. i = {0}", ii);
innerLogger.Info("Exit delegate. i = {0}", ii);
MDC.Remove("id");
});
allTasks[i] = task;
}
logger.Info("Wait on tasks");
Task.WaitAll(allTasks);
logger.Info("Tasks finished");
logger.Info("Exit Main");
}
}
}
这篇关于NLog 的线程安全性如何?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
