如何在Delphi中使用Pipeline模式 [英] How to use Pipeline pattern in Delphi

问题描述

我正在尝试在我的测试项目中实现Pipeline模式（如何进行Mutlithreded idhttp调用以在StringList上工作），但我正在努力将 TThread 代码改编为Pipeline模式代码。关于如何使用它的资源并不多。

I am trying to implement a Pipeline pattern in my test project (How to make a Mutlithreded idhttp calls to do work on a StringList), but am having a struggle adapting TThread code to Pipeline pattern code. There are not many resources about how to use it.

我在下面尽力了，请不要downvote，我知道我的代码很乱但我会编辑我的问题如果需要。

I tried my best below, please DO NOT downvote, I know my code is messy but I'll edit my question if needed.

type
  TForm2 = class(TForm)
    ...
  private
    procedure Retriever(const input: TOmniValue; var output: TOmniValue);
    procedure Inserter(const input, output: IOmniBlockingCollection);
    function HttpGet(url: string; var page: string): boolean;
  end;

procedure TForm2.startButton1Click(Sender: TObject);
var
  pipeline: IOmniPipeline;
  i       : Integer;
  v       : TOmniValue;
  s       : string;
  urlList : TStringList;
begin
  pipeline := Parallel.Pipeline;
  pipeline.Stage(Retriever);
  pipeline.Stage(Inserter).NumTasks(10);
  pipeline.Run;
  for s in urlList do
    pipeline.Input.Add(s);
  pipeline.Input.CompleteAdding;
  // wait for pipeline to complete
  pipeline.WaitFor(INFINITE);
end;

function TForm2.HttpGet(url: string; var page: string): boolean;
var
  lHTTP: TIdHTTP;
  i : integer;
  X : Tstrings;
  S,M,fPath : String;
begin
  lHTTP := TIdHTTP.Create(nil);
  X := TStringList.Create;
  try
    X.Text := lHTTP.Get('https://instagram.com/'+fPath);
    S:= ExtractDelimitedString(X.Text);
    X.Clear;
    Memo2.Lines.Add(fPath+ ' :     '+ M ); //how to pass the result to Inserter
  finally
    lHttp.Free;
  end;
end;

procedure TForm2.Inserter(const input, output: IOmniBlockingCollection);
var
  result   : TOmniValue;
  lpage     : string;
begin
  for result in input do begin
    Memo2.Lines.Add(lpage);
    FreeAndNil(lpage);
  end;
  // correect?
end;

procedure TForm2.Retriever(const input: TOmniValue; var output: TOmniValue);
var
  pageContents: string;
begin
  if HttpGet(input.AsString, pageContents) then
    output := //???
end;

推荐答案

首先 - 描述您的具体问题。没有人可以站在你的背后，看着你的电脑，看看你在做什么。
http://www.catb.org/esr/ faqs / smart-questions.html #beprecise

First of all - describe what is your specific problem. No one can stand behind your back and look at your computer and see what you are doing. http://www.catb.org/esr/faqs/smart-questions.html#beprecise

你的意思是你的程序行为不端。但是你没有描述如何以及为什么。我们不知道。

You do imply your program misbehaves. But you do not describe how and why. And we do not know it.

作为一般性评论，你过度使用了管道。

As general remarks, you overuse the pipeline a bit.

1. 您传递给OTL的所有工作程序 - 在您的情况下，插入器和检索器在随机线程。这意味着如果没有同步，他们都不应该触摸GUI - VCL不是多线程的。
   另外使用 TThread.Synchronize 是一个糟糕的选择，正如我在链接问题中向您解释的那样。它使程序变慢，并使表单不可读。要更新表单，请使用固定帧率进行轮询。不要从OTL工作者内部更新您的表单。

1. all the worker procedures you pass to OTL - in your case those are Inserter and Retriever work in random threads. That means none of them should touch GUI without synchronizing - VCL is not multithreaded. Also using TThread.Synchronize is a poor choice as I explained to you in the linked question. It makes program slow and it makes forms unreadable. To update your form use polling with fixed framerate. Do not update your form from inside OTL workers.

换句话说，插入器不是您所需要的。这里你需要的只是它的输入集合，下载程序和输出集合。是的，管道复杂的事情是非常简单的任务，这就是为什么我之前提到了两个更简单的模式。

In other words, Inserter is not what you need. All you need from the pipeline here is its Input collection, a downloader procedure and the Output collection. Yes it is very simple task for the complex things pipelines are, that is why I mentioned two other simpler patterns before it.

你需要 TTimer 在你的表单上以固定帧速率每秒2-3次轮询输出集合，并检查集合是否尚未最终确定（如果是 - 管道已停止）并且应该更新GUI主线程。

You need TTimer on your form that would poll the Output collection at fixed framerate 2-3 times per second, and check that the collection is not finalized yet ( if it is - the pipeline got stopped ) and that should update GUI from a main thread.

1. 您不应该等待管道在主VCL线程内完成。相反，你应该拆卸pipeleine并让它完全在后台运行。将对创建的管道的引用保存到Form的成员变量中，以便您可以从 TTimer 事件访问其Output集合，并且还可以在其进程运行后释放管道。

1. You should not wait for a pipeline to finish inside your main VCL thread. Instead You should detach the pipeleine and let it run totally in background. Save the reference to the created pipeline into the Form's member variable so you could access its Output collection from the TTimer event and also can free the pipeline after its process run over.

您应该保持该变量链接到管道对象，直到下载结束并设置为 nil （释放对象）之后，但不是之前。 你知道Delphi中的接口和引用计数，对吗？

You should keep that variable linked to the pipeline object until the downloading is over and set to nil (Free the objects) after that, but not before. You know about interfaces and reference-counting in Delp right?

对于其他OTL模式，例如parallel-FOR读取OTL文档他们的 .NoWait（）来电。

1. 您应该将此表单设为双模式，以便在下载运行时和不运行时具有不同的启用控件集。我通常使用特殊的布尔属性来做，就像我在你链接的主题中向你展示的那样。
   当管道正在进行时，您的用户不应该更改列表和设置（除非您实现实时任务更改，但您还没有）。当切换从工作模式进入空闲模式时，此模式切换器也是释放完成的管道对象的好地方。

1. You should make this Your form bi-modal, to have different set of enabled controls when downloading is running and when it is not. I usually do it with special Boolean property like I shown to you in the topic you linked. Your user is not supposed to change the lists and settings while the pipeline is in progress (unless you would implement that realtime task changing, but you did not yet). This mode switcher would also be a good place to free the finished pipeline object when the switching is going from working mode to idle mode.

如果你想要玩管道工作者链接，然后你可以输入输入集合而不是URL字符串本身，但那些数组 - Memo1.Lines.ToArray（），然后您可以从 Unpacker 阶段开始，该阶段从输入集合中获取字符串数组（实际上只有一个）并枚举它并将字符串放入阶段输出集合中。
然而这没什么实际价值，它甚至会使你的程序减慢一点点，因为 Memo1.Lines.ToArray（）函数仍然可以在主VCL线程。但只是为了试验这些管道，这可能很有趣。

If you would want to play with the pipeline workers chaining, then you can put into the Input Collection not the URL strings themselves, but the array of those - the Memo1.Lines.ToArray(), then you can start with Unpacker stage that gets string arrays from the input collection (there would be only one, actually) and enumerate it and put the strings into stage-output collection. This however has little practical value, it would even slow your program down a tiny bit, as the Memo1.Lines.ToArray() function would still work in the main VCL thread. But just to experiment with the pipelines this might be funny.

因此草案就像那样，

 TfrmMain = class(TForm)
  private
    var pipeline: IOmniPipeline;

    property inProcess: Boolean read ... write SetInProcess;
...
  end.

procedure Retriever(const input: TOmniValue; var output: TOmniValue);
var
  pageContents, URL: string;
  lHTTP: TIdHTTP;
begin
  URL := input.AsString;

  lHTTP := TIdHTTP.Create(nil);
  try
    lHTTP.ReadTimeout := 30000;
    lHTTP.HandleRedirects := True;

    pageContents := ExtractDelimitedString( lHTTP.Get('https://instagram.com/' + URL) );

    if pageContents > '' then
       Output := pageContents;
  finally
    lHTTP.Destroy;
  end;
end;

procedure TfrmMain.FormCloseQuery(Sender: TObject; var CanClose: Boolean);
begin
  if InProgress then begin
     CanClose := False;
     ShowMessage( 'You cannot close this window now.'^M^J+
                  'Wait for downloads to complete first.' ); 
  end;
end;

procedure TfrmMain.SetInProcess(const Value: Boolean);
begin
  if Value = InProcess then exit; // form already is in this mode

  FInProcess := Value;

  memo1.ReadOnly := Value;
  StartButton.Enabled := not Value;
  if Value then 
     Memo2.Lines.Clear;

  Timer1.Delay := 500; // twice per second
  Timer1.Enabled := Value;

  If not Value then  // for future optimisation - make immediate mode change 
     FlushData;      // when last worker thread quits, no waiting for timer event

  If not Value then
     pipeline := nil; // free the pipeline object

  If not Value then
     ShowMessage('Work complete');
end;

procedure TfrmMain.Timer1Timer(const Sender: TObject);
begin
  If not InProcess then exit;

  FlushData;

  if Pipeline.Output.IsFinalized then
     InProcess := False;
end;

procedure TForm2.startButton1Click(Sender: TObject);
var
  s       : string;
  urlList : TStringList;
begin
  urlList := Memo1.Lines;

  pipeline := Parallel.Pipeline;

  pipeline.Stage(Retriever).NumTasks(10).Run;

  InProcess := True; // Lock the input data GUI - user no more can edit it
  for s in urlList do
    pipeline.Input.Add(s);
  pipeline.Input.CompleteAdding;
end;

procedure TfrmMain.FlushData;
var v: TOmniValue;
begin
  if pipeline = nil then exit;
  if pipeline.Output = nil then exit;
  if pipeline.Output.IsFinalized then
  begin
    InProcess := False;  
    exit;
  end;

  Memo2.Lines.BeginUpdate;
  try
    while pipeline.Output.TryTake(v) do
      Memo2.Lines.Add( v.AsString );
  finally
    Memo2.Lines.EndUpdate;
  end;

  // optionally - scroll output memo2 to the last line 
end;

注意一些细节，考虑一下并了解它们的本质：

Note few details, think about them and understand the essence of those:

1. 只有 FlushData 正在更新输出备忘录。从 TTimer 事件或表单模式属性设置器调用FlushData。它们都只是从主VCL线程中调用过来的。因此， FlushData 永远不会被称为表格后台线程。

1. Only FlushData is updating the output memo. FlushData is called from the TTimer event or from the form mode property setter. Both of them only are ever called from the main VCL thread. Thus FlushData is NEVER called form background threads.

Retriever 是一个免费的独立函数，它不是表单的成员，它对表单一无所知，也没有对表单实例的引用。这样你就可以实现这两个目标：避免紧耦合并避免错误地从后台线程访问表单控件，这在VCL中是不允许的。
检索器函数在后台线程中工作，它们确实加载数据，它们存储数据，但它们从不接触GUI。这就是这个想法。

Retriever is a free standalone function, it is not a member of the form and it knows nothing about the form and has no reference to your form instance(s). That way you achieve both goals: you avoid "tight coupling" and you avoid a chance of mistakingly access the form's controls from a background thread, which is not allowed in VCL. Retriever functions work in background threads, they do load the data, they do store the data, but they never touch the GUI. That is the idea.

经验法则 - 表单的所有方法都只是从主VCL线程调用。所有管道阶段子例程 - 后台线程的主体 - 都被声明并在任何VCL表单之外工作，并且不能访问任何这些子程序。这些领域之间应该没有混合。

Rule of thumb - all methods of the form are only called from the main VCL thread. All pipeline stage subroutines - bodies of the background threads - are declared and work outside of any VCL forms and have no access to none of those. There should be no mix between those realms.

1. 您将GUI更新限制为固定刷新率。而且这个比率应该不会太频繁。 Windows GUI和用户眼睛应该有时间赶上。

1. you throttle GUI update to a fixed refresh rate. And that rate should be not too frequent. Windows GUI and user eyes should have time to catch up.

您的表单以两种明确描述的模式运行 - InProcess 和不是InProcess 。在这些模式中，用户可以使用不同的功能和控件集。它还管理模式到模式的转换，例如清除输出备忘录文本，提醒用户状态更改，释放使用线程的内存 - 管理对象（此处：管道）等。因此，此属性仅更改（调用setter）从主要的VCL线程，从不从后台工作者。而＃2也有助于此。

Your form operates in two clearly delineated modes - InProcess and not InProcess. In those modes different sets of functions and controls are available to the user. It also manages mode-to-mode transitions like clearing output-memo text, alerting user of status changes, freeing memory of used threads-managing objects (here: pipelines), etc. Consequently, this property only is changed (setter is called) from main VCL thread, never from background workers. And #2 helps with that too.

未来可能的增强功能是使用 pipeline.OnStop 事件发出 PostMessage 使用自定义Windows消息到表单，因此它将在工作完成后立即切换模式，而不是等待下一个计时器olling事件。这可能是管道知道表单任何内容并且有任何引用的唯一地方。但这打开了Windows消息传递，HWND娱乐和其他微妙的事情，我不想放在这里。

The possible future enhancement would be to use pipeline.OnStop event to issue a PostMessage with a custom Windows Message to your form, so it would switch the mode immediately as the work is done, not waiting for the next timer olling event. This might be the ONLY place where pipeline knows anything about the form and has any references to it. But this open the can of Windows messaging, HWND recreation and other subtle things that I do not want to put here.

这篇关于如何在Delphi中使用Pipeline模式的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！