为什么LongRunning任务（TPL）与JpegBitmapDe codeR耗尽资源？ [英] Why does LongRunning task (TPL) with JpegBitmapDecoder run out of resources?

问题描述

我们有一个托管的.NET / C＃应用程序创建TPL任务对JPEG图像进行JPEG编码的元数据。每个任务都与TaskCreationOptions.LongRunning选项构造，如

 工作任务=新任务（（）=＆GT; TaskProc（），cancelToken，TaskCreationOptions.LongRunning）;
 

TaskProc（）利用JpegBitmapDe codeR和JpegBitmapEn codeR类来添加JPEG元数据和新的图像保存到磁盘。 我们允许多达2这样的任务是活动在任何一个时间，这个过程应该继续下去。

在执行我们得到了上述的一段时间的没有足够的存储可用 处理此命令的试图创造JpegBitmapDe codeR类的一个实例时异常：

  

System.ComponentModel.Win32Exception（0X80004005）：没有足够的存储   可用于在处理此命令   MS.Win32.UnsafeNativeMethods.RegisterClassEx（WNDCLASSEX_D wc_d）
  在MS.Win32.HwndWrapper..ctor（的Int32 classStyle，的Int32风格，的Int32   扩展风格，INT3 2个，的Int32 Y，的Int32宽度，高度的Int32，字符串名称，   IntPtr的父母，HwndWrapperHoo K表[]钩子）的   System.Windows.Threading.Dispatcher..ctor（）在   System.Windows.Threading.Dispatcher.get_CurrentDispatcher（）在   System.Windows.Media.Imaging.BitmapDe coder..ctor（流bitmapStream，   BitmapC reateOptions createOptions，BitmapCacheOption cacheOption，   GUID expectedClsId）在   System.Windows.Media.Imaging.JpegBitmapDe coder..ctor（流   bitmapStream，位mapCreateOptions createOptions，BitmapCacheOption   cacheOption）

发生错误的仅当我们使用JpegBitmapDe codeR添加元数据。换句话说，如果任务只想EN code和;保存一个位图图像到文件，没有任何问题出现。使用进程管理器，进程监视器，或其他诊断工具时没有什么明显的显露。没有线程，内存或句柄泄漏，观察在所有。当这种错误发生时，没有新的应用可以被启动，例如，记事本，字等。 一旦我们的应用程序被终止，一切都恢复了正常。

LongRunning的任务创建选项MSDN中定义的指定一个任务将是一个长期运行的，粗粒度的操作。它提供一个提示到的TaskScheduler该超额预订可能有必要。的这意味着该线程选择要运行的任务可能无法从线程池，即，它会为任务的目的而创建的。其他任务创建选项将导致一个线程池线程被选中的任务。

经过一段时间的分析和测试，我们改变了任务创建选项以外的任何其他的 LongRunning 的，例如， preferFairness 的。没有其他改变code都发了言。这种解决的问题，即没有更多的用光存储的错误。

我们困惑的是实际的原因LongRunning线程是罪魁祸首。 下面是我们的一些问题，对这样的：

1. 为什么要该线程选择执行来自线程池或不工作的事实？如果线程终止，应该不是它的资源回收随着时间的推移GC和其原产地返回到OS，不管？

2. 有什么特别之处一个LongRunning任务，JpegBitmapDe codeR的功能，导致错误的组合？

解决方案

课堂时间是在 System.Windows.Media.Imaging 命名空间的基础上的的调度线程架构。对于默认行为的更好或更坏的部分是要启动一个新的调度在任何线程执行每当某个组件通过静态请求当前调度 Dispatcher.Current 属性。这意味着整个调度程序运行正在启动的线程和各种资源得到分配，如果不正确清理，会造成管理漏洞。该调度运行还预计，其执行的线程是STA线程标准的消息抽水事情和工作运行，在默认情况下，没有启动STA线程。

所以，所有的说，为什么它发生在LongRunning，而不是一个常规线程池为主线？原因LongRunning意味着你旋转起来每次每一次，这意味着新的调度资源，每一次一个新的线程和。最后，如​​果你让默认的任务调度（线程池基于一种）运行足够长的时间它也将用尽空间，因为没有被抽为调度运行时的消息，以便能够收拾东西，它需要为好。

因此​​，如果你想使用调度 -thread基础班这样，你真的需要有一个自定义的的TaskScheduler 被设计为在线程中管理的调度运行正常池运行方面的工作。好消息是你很幸运，因为我已经写了一个可以抢这里 。 FWIW，我用在生产code表示处理每天数以十万计的图像三个非常大体积部这个实现。

实施更新

我已经更新了再次实施日前所以这是它兼容了.NET 4.5中新的异步功能。原来的实施是不是合作与的SynchronizationContext 的概念，因为它没有要。现在，你可能会使用的是调度线程上执行的方法在计谋的关键字在C＃中，我需要能够与合作。在previous实施将死锁在这种情况下，这一最新的实施没有。

We have a managed .Net / C# application that creates TPL tasks to perform JPEG metadata encoding on JPEG images. Each task is constructed with TaskCreationOptions.LongRunning option, e.g.,

Task task = new Task( () => TaskProc(), cancelToken, TaskCreationOptions.LongRunning );

TaskProc() utilizes JpegBitmapDecoder and JpegBitmapEncoder classes to add JPEG metadata and save new images to disk. We allow up to 2 such tasks to be active at any one time, and this process should continue indefinitely.

After some time of performing the aforementioned we get Not enough storage is available to process this command exception when trying to create an instance of JpegBitmapDecoder class:

System.ComponentModel.Win32Exception (0x80004005): Not enough storage is available to process this command at MS.Win32.UnsafeNativeMethods.RegisterClassEx(WNDCLASSEX_D wc_d)
at MS.Win32.HwndWrapper..ctor(Int32 classStyle, Int32 style, Int32 exStyle, Int3 2 x, Int32 y, Int32 width, Int32 height, String name, IntPtr parent, HwndWrapperHoo k[] hooks) at System.Windows.Threading.Dispatcher..ctor() at System.Windows.Threading.Dispatcher.get_CurrentDispatcher() at System.Windows.Media.Imaging.BitmapDecoder..ctor(Stream bitmapStream, BitmapC reateOptions createOptions, BitmapCacheOption cacheOption, Guid expectedClsId) at System.Windows.Media.Imaging.JpegBitmapDecoder..ctor(Stream bitmapStream, Bit mapCreateOptions createOptions, BitmapCacheOption cacheOption)

The error occurred only when we utilized JpegBitmapDecoder to add metadata. In other words, if the task would just encode & save a Bitmap image to file, no problems arose. Nothing obvious was revealed when using Process Explorer, Process Monitor, or other diagnostics tools. No thread, memory, or handle leaks were observed at all. When such error occurs, no new applications can be launched, e.g., notepad, word, etc. Once our application is terminated, everything goes back to normal.

The task creation option of LongRunning is defined in MSDN as Specifies that a task will be a long-running, coarse-grained operation. It provides a hint to the TaskScheduler that oversubscription may be warranted. This implies that the thread chosen to run the task may not be from the ThreadPool, i.e., it will be created for the purpose of the task. The other task creation options will result in a ThreadPool thread being selected for the task.

After some time analyzing and testing, we changed the task creation option to anything other than LongRunning, e.g., PreferFairness. No other changes to the code were made at all. This "resolved" the problem, i.e., no more running out of storage errors.

We are puzzled as to the actual reason for LongRunning threads being the culprit. Here are some of our questions on this:

1. Why should the fact that the threads chosen to execute the task come from the ThreadPool or not? If the thread terminates, shouldn't its resources be reclaimed over time by GC and returned back to the OS, regardless of its origin?

2. What is so special about the combination of a LongRunning task and JpegBitmapDecoder's functionality that causes the error?

解决方案

Classes in the System.Windows.Media.Imaging namespace are based on the Dispatcher threading architecture. For better or worse part of the default behavior is to start up a new Dispatcher on whatever thread is executing whenever some component requests the current dispatcher via the static Dispatcher.Current property. This means that the entire Dispatcher "runtime" is started up for the thread and all sorts of resources get allocated and, if not properly cleaned up, will result in managed leaks. The Dispatcher "runtime" also expects the thread its executing on to be an STA thread with standard message pumping going on and the Task runtime, by default, is not starting STA threads.

So, all that said, why does it happen with LongRunning and not a "regular" ThreadPool based thread? Cause LongRunning means you're spinning up a new thread each and every time which means new Dispatcher resources each and every time. Eventually if you let the default task scheduler (the ThreadPool based one) run long enough it too would run out of space because nothing is pumping messages for the Dispatcher runtime to be able to clean up things it needs to as well.

Therefore, if you want to use Dispatcher-thread based classes like this, you really need to do so with a custom TaskScheduler that is designed to run that kind of work on a pool of threads that are managing the Dispatcher "runtime" properly. The good news is you're in luck cause I've already written one that you can grab here. FWIW, I use this implementation in three very high volume portions of production code that process hundreds of thousands of images a day.

Implementation Update

I've updated the implementation again recently so that is it compatible with the new async features of .NET 4.5. The original implementation was not cooperative with the SynchronizationContext concept because it did not have to be. Now that you might be using the await keyword in C# within a method that is executing on the on the Dispatcher thread, I need to be able to cooperate with that. The previous implementation would deadlock in this situation, this latest implementation does not.

这篇关于为什么LongRunning任务（TPL）与JpegBitmapDe codeR耗尽资源？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！