串联的Delphi字符串是否保存在保留对字符串的引用的隐藏临时变量中? [英] Are concatenated Delphi strings held in a hidden temporary variable that retains a reference to the string?

问题描述

我试图了解Delphi服务器应用程序中的内存问题:最初我怀疑是内存泄漏，但现在相信由于动态编译器在动态连接时使用了编译器，因此内存滞留的时间比应该的更长带有+的字符串，导致痛苦的自由空间内存碎片.

背景:

这是Windows上的一整套32位服务器应用程序，Delphi版本相当旧，我认为它是7，但可以肯定是Unicode之前的版本，并使用Nexus 3内存管理器(我在其中编写了DLL)进行挂钩所有分配/免费调用(以及GB的内存跟踪).

我有应用程序源代码，但没有编译器；我不是该应用程序的开发人员(甚至不是Delphi开发人员)，但创建了广泛的自定义工具来监视，跟踪和分析内存.我一直在IDA Pro反汇编程序中选择.EXE.

一些示例代码:

我已经尝试将这种情况减少到最低限度.该代码无意于编译:

 过程TaskThread.RunWorkLoop开始虽然不被终止开始tsk:= WaitForWorkToDo();//一次可以坐几分钟SetThreadName('正在使用'+ tsk.Name);tsk.Run();//这可能需要很长时间SetThreadName('Idle');结尾结尾; 

SetThreadName()使用一个const字符串参数并将其挂起，以便系统的其他部分知道此线程在做什么.

我对代码的反汇编表明，编译器已分配了一个隐藏的本地临时变量，以接收工作中"部分和任务名称部分的串联，这是传递给 SetThreadName 的地方，它还保留了字符串的句柄.

任务正在运行时-可能需要20分钟-我相信该字符串有两个句柄.一个保存在 SetThreadName 中，另一个保存在隐藏的临时文件中.

这一切都很好，很好.

然后，当任务结束且线程名称设置为'Idle'时， SetThreadName()释放原始字符串并分配文字 Idle.

但是:我相信隐藏的本地临时文件仍然保留了该字符串的句柄，且引用计数为refcount = 1，因此它将占用空间，直到过程返回或 next 循环到来可以覆盖该隐藏的本地临时文件，从而释放旧的值.

在此期间，程序无法访问它，无法将其显式释放，并且没有用处，但仍在消耗内存.

对于大多数过程而言，这并不重要，因为它们的开始和结束彼此之间是相对接近的，因此所有内容都可以一次发布，但是在循环服务器应用程序中，这些过程可能会停留更长的时间.这导致我们内存碎片.

情况变得更糟

在实际的应用程序中，它更像是:

  SetThreadName(tsk.Name +'-'+ FormatDateTime('mm/dd/yy hh:nn:ss'，Now)); 

在这种情况下，有两个隐藏的临时变量:一个是 FormatDateTime 的结果，另一个是整体串联结果的，实际上是这样运行的:

  tmp1:字符串；tmp2:字符串；...tmp1:= FormatDateTime('...');tmp2:= tsk.Name +'-'+ tmp1;SetThreadName(tmp2); 

我敢肯定，在任务完成很长时间之后，我就已经看到 FormatDateTime 的字符串结果在内存中徘徊了，而且我已经从字面上看位于1 MB内存部分中间的30字节分配空间，周围是可用空间；Nexus3MM使用 VirtualAlloc 分配更大的OS级块.

单个30字节的字符串 最终会在下一个循环或过程退出时被释放，所以我确定它不是泄漏，但是我宁愿在完成一个兆字节分区的中间时，就不再分配一个30字节的分配，这样整个分区就可以发布到操作系统上.

但是，如果它停留的时间足够长，则内存管理器将从中分配其他内容，这在内存中的漏洞将变得更加永久.

我们有非常详细的忙/闲内存映射，并确保这种碎片正在杀死我们(这当然不是唯一的原因).

我的问题:

1)我理解正确吗?

2)如果是这样，则是通过使用显式临时表消除隐藏临时表的唯一解决方法，

  tmp1:字符串；tmp2:字符串；...tmp1:= FormatDateTime('...');tmp2:= tsk.Name +'-'+ tmp1;SetThreadName(tmp2);tmp1:='';//释放日期/时间字符串tmp2:='';//释放整体线程名称字符串 

我非常有信心我必须对 FormatDateTime 中间结果(我已经看过它)进行此操作，但是不确定整体串联.

这感觉很不对.

几周后才更新.我们已经重写了中央循环以使用显式的临时性，实际上它使某些关键服务器进程的内存碎片产生了明显的(尽管不是主要的)差异.我们还有其他事情需要研究，但是对我来说很明显，这是一条值得走的路.

解决方案

根据我的经验，它确实可以像这样工作.我不确定这是通过合同还是通过实施.我猜想随着最近增加的内联变量声明，现在可能会略有不同.但是我相信在Unicode之前的版本中，它完全可以按照您的描述进行工作.

所有使用托管类型的变量(隐式或显式)或包含一个变量的记录的例程，都会在例程中生成一个隐式的 try/finally 块，并带有 finally 部分清除参考.您的代码真正的作用是:

 过程TaskThread.RunWorkLoop变种sImplicit:字符串；开始简化:='';尝试虽然不被终止开始tsk:= WaitForWorkToDo();//一次可以坐几分钟sImplicit:='正在使用'+ tsk.Name;SetThreadName(sImplicit);tsk.Run();//这可能需要很长时间SetThreadName('Idle');结尾;最后简化:='';结尾;结尾; 

在您的情况下，由于您永远不会退出使用隐式变量的例程，因此它确实会保留在内存中.

至于解决方案，我相信您的建议是可行的.但是您也可以简单地将代码移至另一个方法(或本地过程).

 过程TaskThread.RunWorkLoop过程JustKeepWorking；开始tsk:= WaitForWorkToDo();//一次可以坐几分钟SetThreadName('正在使用'+ tsk.Name);tsk.Run();//这可能需要很长时间SetThreadName('Idle');结尾;开始虽然不被终止开始JustKeepWorking；结尾结尾; 

此外，您可能想查看此问题以获取更多的见识.>

I'm trying to understand memory issues in a Delphi server application: originally I suspected an outright leak, but now believe we're seeing memory hanging around longer than it should due to the compiler's use of a hidden temporary when dynamically concatenating strings with +, causing painful free-space memory fragmentation.

Background:

This is a suite of 32-bit server applications on Windows, Delphi version is quite old, I think it's 7 but is for sure pre-Unicode, and uses the Nexus 3 memory manager where I've written a DLL to hook all the allocate/free calls (and gigabytes of memory traces).

I have application source code but not the compiler; I am not the developer of this app (or even a Delphi dev) but have created extensive custom tools to monitor, trace, and analyze memory. I've been picking the .EXE apart in the IDA Pro disassembler.

Some sample code:

I've tried to whittle this down to the bare minimum case; this code is not intended to compile:

procedure TaskThread.RunWorkLoop
begin
    while not Terminated do
    begin

      tsk := WaitForWorkToDo();  // this could sit for minutes at a time

      SetThreadName('Working on ' + tsk.Name);

      tsk.Run(); // THIS COULD TAKE A LONG TIME

      SetThreadName('Idle');
   end
end;

SetThreadName() takes a const string parameter and hangs onto it so that other parts of the system know what this thread is doing.

My disassembly of the code shows that the compiler has allocated a hidden local temporary variable to receive the concatenation of the "Working on" and task name parts, and this is what's passed to SetThreadName, where it also retains a handle to the string.

While the task is running - and this could be 20 minutes - I believe there are two handles to the string. One is held within SetThreadName, the other is in the hidden temporary.

This is all fine and good.

Then, when the task is over and the thread name is set to 'Idle', SetThreadName() releases the original string and assigns the literal Idle.

BUT: I believe the hidden local temporary still retains a handle to that string, with a refcount=1, so it's going to take up space until either the procedure returns, or the next loop comes around to overwrite that hidden local temporary, releasing the old value.

And during this time, it's not accessible to the program, can't be explicitly released, and is serving no useful purpose but is still consuming memory.

For most procedures this doesn't matter because they start and finish relatively close to each other, so everything is released all at once, but in a looping server app, these can hang around much longer. This is causing us memory fragmentation.

It gets worse

In the actual application, it's more along the lines of:

SetThreadName(tsk.Name + '-' + FormatDateTime('mm/dd/yy hh:nn:ss', Now));

In this case, there are two hidden temporaries: one for the result of FormatDateTime, and the other for the overall concatenation result, in effect running as:

tmp1: String;
tmp2: String;
...
  tmp1 := FormatDateTime('...');
  tmp2 := tsk.Name + '-' + tmp1;
  SetThreadName(tmp2);

I am certain I'm seeing the string result of FormatDateTime hanging around in memory long after the task has completed, and I've seen it literally be a single ~30-byte allocation sitting in the middle of a 1 megabyte memory section, surrounded by free space; Nexus3MM uses VirtualAlloc to allocate larger OS-level chunks.

That single 30-byte string will be released eventually, either on the next loop or when the procedure exits, so I'm certain it's not a leak, but I would rather that single 30-byte allocation sitting in the middle of a lonely one megabyte section actually go away when we're done with it so the whole section could be released to the OS.

But if it sticks around long enough, the memory manager is going to allocate something else from it, and this hole in memory gets more permanent.

We have very detailed busy/free memory maps and are sure that this fragmentation is killing us (this is certainly not the only cause).

My Questions:

1) Am I understanding this correctly?

2) If so, is the only workaround to elide the hidden temporaries by using explicit ones, where we do things like:

tmp1: String;
tmp2: String;
...
  tmp1 := FormatDateTime('...');
  tmp2 := tsk.Name + '-' + tmp1;
  SetThreadName(tmp2);
  tmp1 := '';  // release the date/time string
  tmp2 := '';  // release the overall thread name string

I'm pretty confident I have to do this with the FormatDateTime intermediate result (I've seen it specifically), but am not sure about the overall concatenation.

This just feels wrong.

EDIT: Just an update a few weeks later. We've rewritten the central loop to use explicit temporaries, and it's actually made a noticeable (though not major) difference in memory fragmentation of some key server processes. We still have other things to look into, but it's clear to me that this was a road worth going down.

解决方案

From my experience, it does work exactly like that. I'm not sure if this is by contract or by implementation. I guess with the recent addition of inline variable declaration, this might be slightly different now. But in pre-unicode Delphi, I believe it works exactly as you described.

All routines using variables (implicit or explicit) of a managed type, or a record containing one, will generate an implicit try/finally block in the routine, with the finally part clearing the reference. What your code really does is :

procedure TaskThread.RunWorkLoop
var
  sImplicit : string;
begin
  sImplicit := '';
  try
    while not Terminated do
    begin
      tsk := WaitForWorkToDo();  // this could sit for minutes at a time

      sImplicit := 'Working on ' + tsk.Name;

      SetThreadName(sImplicit);

      tsk.Run(); // THIS COULD TAKE A LONG TIME

      SetThreadName('Idle');
    end;
  finally
    sImplicit := '';
  end;
end;

In your situation, since you never exit the routine where the implicit variable is used, it does remain in memory.

As for a solution, I believe what you propose would work. But you could also simply move the code to another method (or a local procedure).

procedure TaskThread.RunWorkLoop
  procedure JustKeepWorking;
  begin
    tsk := WaitForWorkToDo();  // this could sit for minutes at a time
    SetThreadName('Working on ' + tsk.Name);
    tsk.Run(); // THIS COULD TAKE A LONG TIME
    SetThreadName('Idle');
  end;
begin
  while not Terminated do
  begin
    JustKeepWorking;
  end
end;

Also, you might want to check this question for additional insight.

这篇关于串联的Delphi字符串是否保存在保留对字符串的引用的隐藏临时变量中?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！