扫描文件夹/子文件夹/文件时如何减少CPU使用情况？ [英] How to reduce CPU usage when scanning for folders/sub-folders/files?

问题描述

我开发了一个应用程序，可以扫描文件或文件列表的所有位置。
当我扫描小文件夹，如10 000个文件和子文件没有问题。但是当我扫描例如我的整个用户文件夹超过10万项，我的处理器非常沉重。这需要大约40％的处理器功率。

I have developed an application that scans basically everywhere for a file or list of files. When I scan small folders like 10 000 files and sub files there is no problem. But when I scan for instance my entire users folder with more than 100 000 items, it is very heavy on my processor. It takes about 40% of my processor's power.

有没有办法优化这个代码，以减少CPU？

Is there a way to optimize this code so that it uses less CPU?

procedure GetAllSubFolders(sPath: String);
var
  Path: String;
  Rec: TSearchRec;
begin
  try
    Path := IncludeTrailingBackslash(sPath);
    if FindFirst(Path + '*.*', faAnyFile, Rec) = 0 then
      try
        repeat
          Application.ProcessMessages;
          if (Rec.Name <> '.') and (Rec.Name <> '..') then
          begin
            if (ExtractFileExt(Path + Rec.Name) <> '') And
              (ExtractFileExt(Path + Rec.Name).ToLower <> '.lnk') And
              (Directoryexists(Path + Rec.Name + '\') = False) then
            begin
              if (Pos(Path + Rec.Name, main.Memo1.Lines.Text) = 0) then
              begin
                main.ListBox1.Items.Add(Path + Rec.Name);
                main.Memo1.Lines.Add(Path + Rec.Name)
              end;
            end;

            GetAllSubFolders(Path + Rec.Name);
          end;
        until FindNext(Rec) <> 0;
      finally
        FindClose(Rec);
      end;
  except
    on e: Exception do
      ShowMessage(e.Message);
  end;
end;

我的应用程序搜索所选文件夹和子文件夹中的所有文件，并将其复制到您指定的另一个位置。

My app searches for all the files in a selected folder and sub-folder, zip's them and copies them to another location you specify.

Application.ProcessMessages 命令用于确保应用程序不会喜欢挂起，用户关闭它。因为找到100 000个文件可能需要一个小时左右的时间...

The Application.ProcessMessages command is there to make sure the application doesn't look like it is hanging and the user closes it. Because finding 100 000 files for instance can take an hour or so...

我关心处理器的使用情况，内存不是真的受到影响。

I am concerned about the processor usage, The memory is not really affected.

注意：备忘录是为了确保相同的文件没有被选中两次。

Note: The memo is to make sure the same files are not selected twice.

推荐答案

我看到以下性能问题：

1. 调用 Application.ProcessMessages 有点贵。您正在轮询邮件，而不是使用阻止等待，即 GetMessage 。除了性能问题之外，使用 Application.ProcessMessages 通常是由于各种原因而导致设计不良，一般应避免使用它。


2. 非虚拟列表框对很多文件执行得很糟。


3. 使用备忘录控件（GUI控件）存储列表字符串非常昂贵。


4. 每次添加到GUI控件时，它们都会更新和刷新，这是非常昂贵的。


5. 评价 Memo1.Lines.Text 非常昂贵。


6. Pos 同样非常昂贵。


7. 使用 DirectoryExists 是昂贵的虚假的搜索记录中返回的属性包含该信息。

1. The call to Application.ProcessMessages is somewhat expensive. You are polling for messages rather than using a blocking wait, i.e. GetMessage. As well as the performance issue, the use of Application.ProcessMessages is generally an indication of poor design for various reasons and one should, in general, avoid the need to call it.
2. A non-virtual list box performs badly with a lot of files.
3. Using a memo control (a GUI control) to store a list of strings is exceptionally expensive.
4. Every time you add to the GUI controls they update and refresh which is very expensive.
5. The evaluation of Memo1.Lines.Text is extraordinarily expensive.
6. The use of Pos is likewise massively expensive.
7. The use of DirectoryExists is expensive and spurious. The attributes returned in the search record contain that information.

我将进行以下更改：

• 将搜索代码移动到一个线程中，以避免需要 ProcessMessages 。您将需要设计一些方法将信息传输回主线程以在GUI中显示。


• 使用虚拟列表视图显示文件。


• 将要搜索重复项的文件列表存储在一个字典中，该字典将为您提供 O（1）查找。注意文件名的不区分大小写，这个问题到目前为止你可能被忽略了。这将取代备忘录。


• 使用 Rec.Attr 检查项目是否是目录。这是检查 Rec.Attr和faDirectory<> 0 。

• Move the search code into a thread to avoid the need for ProcessMessages. You'll need to devise some way to transport the information back to the main thread for display in the GUI.
• Use a virtual list view to display the files.
• Store the list of files that you wish to search for duplicates in a dictionary which gives you O(1) lookup. Take care with case-insensitivity of file names, an issue that you have perhaps neglected so far. This replaces the memo.
• Check whether an item is a directory by using Rec.Attr. That is check that Rec.Attr and faDirectory <> 0.

这篇关于扫描文件夹/子文件夹/文件时如何减少CPU使用情况？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！