在非托管资源上执行P/Invoke时需要GC.KeepAlive(this)吗? [英] When GC.KeepAlive(this) is needed when doing P/Invoke on unmanaged resources?
问题描述
我有一个本机组件的TestNet
包装器.本机组件公开了一个 blocking TestNative::Foo()
和一个较弱的GCHandle
,后者通过调用托管回调与托管部件进行通信,而GCHandle
用于检索对.NET包装器的引用并提供上下文. GCHandle
是弱的,因为.NET包装器旨在隐藏向用户处理非托管资源并且故意不实现IDisposable
接口的事实:如果为非弱,则它将阻止在以下位置收集TestNet
实例:全部,造成内存泄漏.发生的事情是,在 Release 版本中,甚至在TestNative::Foo()
和令人惊讶的TestNet::Foo()
解除阻塞之前,仅垃圾收集器将在执行托管回调时收集对.NET包装的引用.我了解了我自己的问题,可以通过在P/Invoke调用后发出GC.KeepAlive(this)
来解决此问题,但是由于对此问题的了解还不是很广泛,所以似乎很多人都在做错了.我有几个问题:
- 如果最后一条指令是对非托管资源的P/Invoke调用,还是在托管方法中始终需要
GC.KeepAlive(this)
,或者在这种特殊情况下(即切换到托管执行上下文,同时将托管回调从本机代码整理起来),是否总是需要在托管方法中使用GC.KeepAlive(this)
?问题可能是:我应该把GC.KeepAlive(this)
放在各处吗?这个旧的Microsoft 博客(原始链接是404,这是 CA2115 指向在使用P/Invoke访问非托管资源时通常使用GC.KeepAlive(this)
any .通常,在处理Dll1.cpp :
#include <iostream> extern "C" typedef void (*Callback)(void *data); class TestNative { public: void SetCallback(Callback callback1, void *data); void Foo(); private: Callback m_callback; void *m_data; }; void TestNative::SetCallback(Callback callback, void * data) { m_callback = callback; m_data = data; } void TestNative::Foo() { // Foo() will never end while (true) { m_callback(m_data); } } extern "C" { __declspec(dllexport) TestNative * CreateTestNative() { return new TestNative(); } __declspec(dllexport) void FreeTestNative(TestNative *obj) { delete obj; } __declspec(dllexport) void TestNativeSetCallback(TestNative *obj, Callback callback1, void * data) { obj->SetCallback(callback1, data); } __declspec(dllexport) void TestNativeFoo(TestNative *obj) { obj->Foo(); } }
输出一致:
this.Foo() begins TestNet.callback() begins this.callback() this.~TestNet() TestNet.callback() ends TestNet.callback() begins System.NullReferenceException: Object reference not set to an instance of an object.
如果取消注释
TestNet.Foo()
中的GC.KeepAlive(this)
调用,程序将永远不会结束.解决方案总结非常有用的评论并完成研究:
1)如果最后一条指令是使用实例拥有的非托管资源的P/Invoke调用,那么在托管实例方法中是否总是需要
GC.KeepAlive(this)
?是的,如果您不希望API的用户在病理情况下对托管对象的实例承担不可收集的引用的最后责任,请查看以下示例.但这不是唯一的方法:
HandleRef
或SafeHandle
技术可以在执行P/Invoke Interop时,还可以用来延长被管理对象的寿命.该示例随后将通过拥有本地资源的托管实例调用本地方法:
using System; using System.Diagnostics; using System.Runtime.InteropServices; using System.Threading; namespace ConsoleApp1 { class Program { static void Main(string[] args) { new Thread(delegate() { // Run a separate thread enforcing GC collections every second while(true) { GC.Collect(); Thread.Sleep(1000); } }).Start(); while (true) { var test = new TestNet(); test.Foo(); TestNet.Dump(); } } } class TestNet { static ManualResetEvent _closed; static long _closeTime; static long _fooEndTime; IntPtr _nativeHandle; public TestNet() { _closed = new ManualResetEvent(false); _closeTime = -1; _fooEndTime = -1; _nativeHandle = CreateTestNative(); } public static void Dump() { // Ensure the now the object will now be garbage collected GC.Collect(); GC.WaitForPendingFinalizers(); // Wait for current object to be garbage collected _closed.WaitOne(); Trace.Assert(_closeTime != -1); Trace.Assert(_fooEndTime != -1); if (_closeTime <= _fooEndTime) Console.WriteLine("WARN: Finalize() commenced before Foo() return"); else Console.WriteLine("Finalize() commenced after Foo() return"); } ~TestNet() { _closeTime = Stopwatch.GetTimestamp(); FreeTestNative(_nativeHandle); _closed.Set(); } public void Foo() { // The native implementation just sleeps for 250ms TestNativeFoo(_nativeHandle); // Uncomment to have all Finalize() to commence after Foo() //GC.KeepAlive(this); _fooEndTime = Stopwatch.GetTimestamp(); } [DllImport("Dll1", CallingConvention = CallingConvention.Cdecl)] static extern IntPtr CreateTestNative(); [DllImport("Dll1", CallingConvention = CallingConvention.Cdecl)] static extern void FreeTestNative(IntPtr obj); [DllImport("Dll1", CallingConvention = CallingConvention.Cdecl)] static extern void TestNativeFoo(IntPtr obj); } }
为确保本机调用始终安全,我们希望仅在
Foo()
返回之后才调用终结器.相反,我们可以通过在后台线程中手动调用垃圾回收来轻松实施违规.输出如下:Finalize() commenced after Foo() return WARN: Finalize() commenced before Foo() return Finalize() commenced after Foo() return Finalize() commenced after Foo() return Finalize() commenced after Foo() return WARN: Finalize() commenced before Foo() return Finalize() commenced after Foo() return
2)在哪里可以找到文档?
GC.KeepAlive()
的文档提供了这个例子与原始问题中的托管回调非常相似.HandleRef
也有非常有趣的考虑因素关于被管理对象和Interop的生命周期:如果使用平台调用来调用托管对象,并且该对象是 在平台调用之后没有在其他地方引用,它是 垃圾收集器最终确定托管对象的可能性. 此操作释放资源并使句柄无效,从而导致 平台调用失败.用HandleRef包装手柄 保证直到被管理对象不会被垃圾收集 平台调用调用完成.
通过
@GSerg找到的链接[1]解释了何时有资格进行对象收集,并指出 this
引用不在根集中,从而允许在没有返回实例方法的情况下也进行收集.3)为什么仅在发布版本中会发生这种情况?
这是一种优化,也可以在调试版本中启用@SimonMourier指出的启用优化功能.默认情况下,它也未在Debug中启用,因为它可能阻止调试当前方法范围内的变量,如以下答案.
... >[1] https://devblogs.microsoft.com/oldnewthing/20100810-00/? p = 13193 ?
I have a
TestNet
wrapper for a native component. The native component exposes a blockingTestNative::Foo()
that communicates with managed part through calling managed callbacks and a weakGCHandle
that is used to retrieve the reference to the .NET wrapper and provides a context. TheGCHandle
is weak since the .NET wrapper is meant to hide the fact that is handling unmanaged resources to user and deliberately doesn't implement theIDisposable
interface: being non weak it would preventTestNet
instances from being collected at all, creating a memory leak. What's happening is that in Release build only the garbage collector will collect reference to .NET wrapper while executing the managed callback, even before bothTestNative::Foo()
and surprisinglyTestNet::Foo()
unblocks. I understood the problem my self and I can fix it by issuing aGC.KeepAlive(this)
after the P/Invoke call but since the knowledge of this is not very widespread, it seems a lot of people are doing it wrong. I have few questions:- Is
GC.KeepAlive(this)
always needed in a managed method if last instruction is a P/Invoke call on unmanaged resources or it's just needed in this special case, namely the switch to managed execution context while marshaling the managed callback from native code? The question could be: should I putGC.KeepAlive(this)
everywhere? This old microsoft blog (original link is 404, here is cached) seems to suggest so! But this would be game changer and basically it would mean that most people never did P/Invoke correctly, because this would require reviewing most P/Invoke calls in wrappers. Is there for example a rule that say that garbage collector (EDIT: or better the finalizer) can't run for objects that belong to the current thread while execution context is unamanaged (native)? - Where I can find proper documentation? I could find CodeAnalysis policy CA2115 pointing to generically use
GC.KeepAlive(this)
any time a unmanaged resource is accessed with P/Invoke. In generalGC.KeepAlive(this)
seems to be very rarely needed when dealing with finalizers. - Why is this happening only in Release build? It looks like an optimization but not being needed at all in Debug build hides an important behavior of the garbage collector.
NOTE: I have no problem with delegates being collected, that is a different issue which I know how to handle properly. The issue here is with objects holding unmanaged resources being collected when P/Invoke calls are not finished yet.
It follows code that clearly manifest the problem. Creates a C# console application and a C++ Dll1 project and build them in Release mode:
Program.cs:
using System; using System.Runtime.InteropServices; namespace ConsoleApp1 { class Program { static void Main(string[] args) { var test = new TestNet(); try { test.Foo(); } catch (Exception ex) { Console.WriteLine(ex); } } } class TestNet { [UnmanagedFunctionPointer(CallingConvention.Cdecl)] delegate void Callback(IntPtr data); static Callback _callback; IntPtr _nativeHandle; GCHandle _thisHandle; static TestNet() { // NOTE: Keep delegates references so they can be // stored persistently in unmanaged resources _callback = callback; } public TestNet() { _nativeHandle = CreateTestNative(); // Keep a weak handle to self. Weak is necessary // to not prevent garbage collection of TestNet instances _thisHandle = GCHandle.Alloc(this, GCHandleType.Weak); TestNativeSetCallback(_nativeHandle, _callback, GCHandle.ToIntPtr(_thisHandle)); } ~TestNet() { Console.WriteLine("this.~TestNet()"); FreeTestNative(_nativeHandle); _thisHandle.Free(); } public void Foo() { Console.WriteLine("this.Foo() begins"); TestNativeFoo(_nativeHandle); // This is never printed when the object is collected! Console.WriteLine("this.Foo() ends"); // Without the following GC.KeepAlive(this) call // in Release build the program will consistently collect // the object in callback() and crash on next iteration //GC.KeepAlive(this); } static void callback(IntPtr data) { Console.WriteLine("TestNet.callback() begins"); // Retrieve the weak reference to self. As soon as the istance // of TestNet exists. var self = (TestNet)GCHandle.FromIntPtr(data).Target; self.callback(); // Enforce garbage collection. On release build self = null; GC.Collect(); GC.WaitForPendingFinalizers(); Console.WriteLine("TestNet.callback() ends"); } void callback() { Console.WriteLine("this.callback()"); } [DllImport("Dll1", CallingConvention = CallingConvention.Cdecl)] static extern IntPtr CreateTestNative(); [DllImport("Dll1", CallingConvention = CallingConvention.Cdecl)] static extern void FreeTestNative(IntPtr obj); [DllImport("Dll1", CallingConvention = CallingConvention.Cdecl)] static extern void TestNativeSetCallback(IntPtr obj, Callback callback, IntPtr data); [DllImport("Dll1", CallingConvention = CallingConvention.Cdecl)] static extern void TestNativeFoo(IntPtr obj); } }
Dll1.cpp:
#include <iostream> extern "C" typedef void (*Callback)(void *data); class TestNative { public: void SetCallback(Callback callback1, void *data); void Foo(); private: Callback m_callback; void *m_data; }; void TestNative::SetCallback(Callback callback, void * data) { m_callback = callback; m_data = data; } void TestNative::Foo() { // Foo() will never end while (true) { m_callback(m_data); } } extern "C" { __declspec(dllexport) TestNative * CreateTestNative() { return new TestNative(); } __declspec(dllexport) void FreeTestNative(TestNative *obj) { delete obj; } __declspec(dllexport) void TestNativeSetCallback(TestNative *obj, Callback callback1, void * data) { obj->SetCallback(callback1, data); } __declspec(dllexport) void TestNativeFoo(TestNative *obj) { obj->Foo(); } }
The output is consistently:
this.Foo() begins TestNet.callback() begins this.callback() this.~TestNet() TestNet.callback() ends TestNet.callback() begins System.NullReferenceException: Object reference not set to an instance of an object.
If one uncomment the
GC.KeepAlive(this)
call inTestNet.Foo()
the program correctly never ends.解决方案Summarizing very useful comments and research done:
1) Is
GC.KeepAlive(this)
always needed in a managed instance method if last instruction is a P/Invoke call using unmanaged resources hold by the instance?Yes, if you don't want the user of the API to have last responsibility of holding a non-collectible reference for the instance of the managed object in pathological cases, look the example below. But it's not the only way:
HandleRef
orSafeHandle
techiniques can also be used to prolong the lifetime of a managed object when doing P/Invoke Interop.The example will subsequently call native methods through managed instances holding native resources:
using System; using System.Diagnostics; using System.Runtime.InteropServices; using System.Threading; namespace ConsoleApp1 { class Program { static void Main(string[] args) { new Thread(delegate() { // Run a separate thread enforcing GC collections every second while(true) { GC.Collect(); Thread.Sleep(1000); } }).Start(); while (true) { var test = new TestNet(); test.Foo(); TestNet.Dump(); } } } class TestNet { static ManualResetEvent _closed; static long _closeTime; static long _fooEndTime; IntPtr _nativeHandle; public TestNet() { _closed = new ManualResetEvent(false); _closeTime = -1; _fooEndTime = -1; _nativeHandle = CreateTestNative(); } public static void Dump() { // Ensure the now the object will now be garbage collected GC.Collect(); GC.WaitForPendingFinalizers(); // Wait for current object to be garbage collected _closed.WaitOne(); Trace.Assert(_closeTime != -1); Trace.Assert(_fooEndTime != -1); if (_closeTime <= _fooEndTime) Console.WriteLine("WARN: Finalize() commenced before Foo() return"); else Console.WriteLine("Finalize() commenced after Foo() return"); } ~TestNet() { _closeTime = Stopwatch.GetTimestamp(); FreeTestNative(_nativeHandle); _closed.Set(); } public void Foo() { // The native implementation just sleeps for 250ms TestNativeFoo(_nativeHandle); // Uncomment to have all Finalize() to commence after Foo() //GC.KeepAlive(this); _fooEndTime = Stopwatch.GetTimestamp(); } [DllImport("Dll1", CallingConvention = CallingConvention.Cdecl)] static extern IntPtr CreateTestNative(); [DllImport("Dll1", CallingConvention = CallingConvention.Cdecl)] static extern void FreeTestNative(IntPtr obj); [DllImport("Dll1", CallingConvention = CallingConvention.Cdecl)] static extern void TestNativeFoo(IntPtr obj); } }
For the native call to be always safe we expect finalizer to be called only after
Foo()
return. Instead we can easily enforce violations by manually invoking garbage collection in a background thread. Output follows:Finalize() commenced after Foo() return WARN: Finalize() commenced before Foo() return Finalize() commenced after Foo() return Finalize() commenced after Foo() return Finalize() commenced after Foo() return WARN: Finalize() commenced before Foo() return Finalize() commenced after Foo() return
2) Where I can find documentation?
Documentation of
GC.KeepAlive()
provides an example very similar to the managed callback in the original question.HandleRef
has also very interesting considerations about lifecycle of managed objects and Interop:If you use platform invoke to call a managed object, and the object is not referenced elsewhere after the platform invoke call, it is possible for the garbage collector to finalize the managed object. This action releases the resource and invalidates the handle, causing the platform invoke call to fail. Wrapping a handle with HandleRef guarantees that the managed object is not garbage collected until the platform invoke call completes.
Also link[1] found by @GSerg explains when an object is eligible for collection, pointing that
this
reference is not in the root set, allowing it to be collected also when instance method has not returned.3) Why is this happening only in Release build?
It's an optimization and can happen also in Debug build, with optimization enabled, as pointed by @SimonMourier. It's not enabled by default also in Debug because it could prevent debugging of variables in the current method scope, as explained in these other answers.
[1] https://devblogs.microsoft.com/oldnewthing/20100810-00/?p=13193?
这篇关于在非托管资源上执行P/Invoke时需要GC.KeepAlive(this)吗?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
- Is