无法从 Datasnap 服务器检索大于 260.000 字节的 TStreams [英] Can't retrieve TStreams bigger than around 260.000 bytes from a Datasnap Server

问题描述

我有一个 Delphi 10.1 Berlin Datasnap Server，它无法返回大于 260.000 字节的数据包(通过 TStream).

I have a Delphi 10.1 Berlin Datasnap Server, that can't return Data packets (through a TStream) bigger than around 260.000 bytes.

我已经按照 Delphi 的 Object PascalDataSnapFireDAC 示例对其进行了编程，该示例也显示了此问题.

I have programmed it following the Object PascalDataSnapFireDAC sample from Delp which also shows this problem.

只需打开该示例，将 ServerMethodsUnit.pas 上 qOrders 组件的 IndexFieldName 设置为空白，并将其 SQL 属性更改为:

The problem can be seen just opening that sample, setting blank the IndexFieldName of the qOrders component on ServerMethodsUnit.pas, and changing its SQL property to :

select * from Orders
union 
select * from Orders

现在要发送的数据量超过 260.000 字节，这似乎是您无法从客户端检索它的地方.获得 EFDException [FireDAC][Stan]-710.二进制存储格式无效.

Now the amount of data to be send is beyond 260.000 bytes, which seems to be the point where you can't retrieve it from the client. Getting a EFDException [FireDAC][Stan]-710. Invalid binary storage format.

数据作为流发送，您从服务器上的 FDSchemaAdapter 获取，然后加载到客户端上的另一个 FDSchemaAdpater.Client和Server之间的连接也是FireDAC.

The data is sent as a Stream that you get from a FDSchemaAdapter on the server, and you load on another FDSchemaAdpater on the client. The connection between Client and Server is also FireDAC.

这是服务器返回该流的方式:

This is how the Server returns that Stream :

function TServerMethods.StreamGet: TStream;
begin
  Result := TMemoryStream.Create;
  try
    qCustomers.Close;
    qCustomers.Open;
    qOrders.Close;
    qOrders.Open;
    FDSchemaAdapter.SaveToStream(Result, TFDStorageFormat.sfBinary);
    Result.Position := 0;
  except
    raise;
  end;
end;

这就是客户端检索它的方式:

And this is how the Client retrieves it :

procedure TClientForm.GetTables;
var
  LStringStream: TStringStream;
begin
  FDStoredProcGet.ExecProc;
  LStringStream := TStringStream.Create(FDStoredProcGet.Params[0].asBlob);
  try
    if LStringStream <> nil then
    begin
      LStringStream.Position := 0;
      DataModuleFDClient.FDSchemaAdapter.LoadFromStream(LStringStream, TFDStorageFormat.sfBinary);
    end;
  finally
    LStringStream.Free;
  end;
end;

客户端不会获取 Blob 参数的所有数据.我把Stream的内容保存在Server上，和Client上到达Blob参数的内容，大小一样，但是Blob参数的内容被截断了，最后几Kbytes为零.

The Client doesn't get all the data on the Blob parameter. I save the content of the Stream on the Server, and the content that arrives at the Blob parameter on the Client, and they have the same size, but the content of the Blob parameter has its content truncated, and the last few Kbytes are zeroes.

这是我如何在服务器上保存将进入流的内容:

This is how I save on the Server the content that will go to the Stream:

FDSchemaAdapter.SaveToFile('C:TempJSON_Server.json', TFDStorageFormat.sfJSON);

这是我检查客户端 blob 参数的方式:

This is how I check what I get on the Client blob parameter:

TFile.WriteAllText('C:TempJSON_Client.json', FDStoredProcGet.Params[0].asBlob);

我可以看到客户端截断了数据.

I can see that the Client gets the data truncated.

您是否知道如何修复它，或将所有流内容从 Datasnap 服务器检索到我的客户端的解决方法?.

Do you know how to fix it, or a workaround to retrieve all the Stream content from the Datasnap Server to my Client ?.

更新:我已更新到 Delphi 10.1 Berlin Update 2，但问题仍然存在.

Update: I have updated to Delphi 10.1 Berlin Update 2, but the problem remains.

谢谢.

推荐答案

我编写了一个解决方法.看到我无法传递大于 255Kb 的数据，然后我将其拆分为不同的 255Kb 数据包并分别发送(我还添加了压缩以最小化带宽和往返次数).

I have coded a workaround. Seeing that I can't pass data bigger than 255Kb then I split it in different 255Kb packets and send them separately (I have also added compression to minimize the bandwidth and roundtrips).

在服务器上，我将 StremGet 更改为两个不同的调用:StreamGet 和 StreamGetNextPacket.

On the server I have changed StremGet to two different calls : StreamGet and StreamGetNextPacket.

function TServerMethods.StreamGet(var Complete: boolean): TStream;
var Data: TMemoryStream;
    Compression: TZCompressionStream;
begin
  try
    // Opening Data
    qCustomers.Close;
    qCustomers.Open;
    qOrders.Close;
    qOrders.Open;

    // Compressing Data
    try
      if Assigned(CommStream) then FreeAndNil(CommStream);
      CommStream := TMemoryStream.Create;
      Data := TMemoryStream.Create;
      Compression := TZCompressionStream.Create(CommStream);
      FDSchemaAdapter.SaveToStream(Data, TFDStorageFormat.sfBinary);
      Data.Position := 0;
      Compression.CopyFrom(Data, Data.Size);
    finally
      Data.Free;
      Compression.Free;
    end;

    // Returning First 260000 bytes Packet
    CommStream.Position := 0;
    Result := TMemoryStream.Create;
    Result.CopyFrom(CommStream, Min(CommStream.Size, 260000));
    Result.Position := 0;

    // Freeing Memory if all sent
    Complete := (CommStream.Position = CommStream.Size);
    if Complete then FreeAndNil(CommStream);
  except
    raise;
  end;
end;

function TServerMethods.StreamGetNextPacket(var Complete: boolean): TStream;
begin
  // Returning the rest of 260000 bytes Packets
  Result := TMemoryStream.Create;
  Result.CopyFrom(CommStream, Min(CommStream.Size - CommStream.Position, 260000));
  Result.Position := 0;

  // Freeing Memory if all sent
  Complete := (CommStream.Position = CommStream.Size);
  if Complete then FreeAndNil(CommStream);
end;

CommStream:TStream 在 TServerMethods 上被声明为私有.

CommStream: TStream is declared as private on TServerMethods.

客户端以这种方式检索它:

And the Client retrieves it this way :

procedure TClientForm.GetTables;
var Complete: boolean;
    Input: TStringStream;
    Data: TMemoryStream;
    Decompression:  TZDecompressionStream;
begin
  Input := nil;
  Data := nil;
  Decompression := nil;

  try
    // Get the First 260000 bytes Packet
    spStreamGet.ExecProc;
    Input := TStringStream.Create(spStreamGet.ParamByName('ReturnValue').AsBlob);
    Complete := spStreamGet.ParamByName('Complete').AsBoolean;

    // Get the rest of 260000 bytes Packets
    while not Complete do begin
      spStreamGetNextPacket.ExecProc;
      Input.Position := Input.Size;
      Input.WriteBuffer(TBytes(spStreamGetNextPacket.ParamByName('ReturnValue').AsBlob), Length(spStreamGetNextPacket.ParamByName('ReturnValue').AsBlob));
      Complete := spStreamGetNextPacket.ParamByName('Complete').AsBoolean;
    end;

    // Decompress Data
    Input.Position := 0;
    Data := TMemoryStream.Create;
    Decompression := TZDecompressionStream.Create(Input);
    Data.CopyFrom(Decompression, 0);
    Data.Position := 0;

    // Load Datasets
    DataModuleFDClient.FDSchemaAdapter.LoadFromStream(Data, TFDStorageFormat.sfBinary);
  finally
    if Assigned(Input) then FreeAndNil(Input);
    if Assigned(Data) then FreeAndNil(Data);
    if Assigned(Decompression) then FreeAndNil(Decompression);
  end;
end;

现在一切正常.

这篇关于无法从 Datasnap 服务器检索大于 260.000 字节的 TStreams的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！