德尔福 - 快速（ER）的XML解析器 [英] Delphi - Fast(er) XML parser
问题描述
我在寻找德尔福非常快的XML解析器，非常简单的数据。
I'm looking for a very fast XML parser for Delp for very simple data.
考虑以下类型的数据：
<node>
<datatype1>randomdata</datatype1>
<datatype2>randomdata</datatype2>
<datatype3>randomdata</datatype3>
<datatype4>randomdata</datatype4>
<datatype5>randomdata</datatype5>
<datatype6>randomdata</datatype6>
<datatype7>randomdata</datatype7>
<datatype8>randomdata</datatype8>
<datatype9>randomdata</datatype9>
<datatype10>randomdata</datatype10>
<datatype11>randomdata</datatype11>
<datatype12>randomdata</datatype12>
<datatype13>randomdata</datatype13>
<datatype14>randomdata</datatype14>
<datatype15>randomdata</datatype15>
<datatype16>randomdata</datatype16>
<datatype17>randomdata</datatype17>
<datatype18>randomdata</datatype18>
<datatype19>randomdata</datatype19>
<datatype20>randomdata</datatype20>
</node>
复制这10000次（数据类型和数据是在真实的场景中明显不同）。想想也该数据包含统一code。
Copy this 10000 times (datatypes and the data being obviously different in a real scenario). Consider also the data contains Unicode.
这会像
Type MyData = record
d1,d2,d3,d4,d5,
d6,d7,d8,d9,d10,
d11,d12,d13,d14,d15,
d16,d17,d18,d19,d20: string;
end;
我写了这个定制的分析器，它在我的电脑需要约。 115ms整个过程中，从文件加载到具有10000条记录填写。
I wrote a custom parser for this, which in my computer takes approx. 115ms for the entire process, from loading the file to having the 10,000 records filled.
所以我在寻找的东西，可以做到这一点快。
So I am looking for something that can accomplish this faster.
相关问题：
<一个href=\"http://stackoverflow.com/questions/14168974/delphi-perform-pos-within-utf8-string-boundaries\">Delphi - UTF8字符串范围内进行波什（）
推荐答案
首先，让我告诉你，你在这里优化错误的东西：除非你这样做是为了娱乐的目的，那么你的做法是错误的。 XML是不是一个困难的格式，但它确实有它的怪癖，它需要它的自由。这是专为外国应用之间的数据交换格式，所以需要将重点放在兼容性，而不是速度！有什么好处是非标准超快分析器时略有改变XML文件都面临着给出错误的结果？
First let me tell you that you're optimizing the wrong thing here: unless you're doing this for recreational purposes, then your approach is wrong. XML is not a difficult format but it does have it quirks and it takes it's liberties. It's a format designed for data exchange between foreign applications, so the emphasis needs to be put on COMPATIBILITY, not on SPEED! What good is a non-standard ultra-fast parser that gives the wrong result when confronted with a slightly altered XML file?
如果你能找到的保证与任何东西在那里，可以在你一半的硬盘驱动器可以读取它的速度解析您的数据兼容的XML解析库，然后简单地实现生产者 - 消费者多线程应用程序，其中一个线程不断从磁盘读取数据，而其他两个简单地做了解析。在最后，你会只受硬盘，同时保持兼容性速度的限制。如果你只想找一个速度你是容易犯错，跳过XML功能，取决于你对付示例XML文件的某些特殊性。您的应用程序可能突破的原因很多。
If you can find a XML parsing LIBRARY that's guaranteed to be compatible with anything out there that can parse your data at HALF the speed your HDD can read it, then simply implement a producer-consumer multi-threaded application where one thread constantly reads the data from disk while the other two simply do the parsing. In the end you'll only be limited by the speed of the HDD while maintaining compatibility. If you're only looking for speed you're liable to make mistakes, skip XML features, depend on certain particularities of the sample XML file you're dealing with. Your application is likely to break for numerous reasons.
请记住，应用程序中最昂贵的周期维护，而不是生产。你可以通过做一个50％的速度获得了今日的啄那是200％％以上难以维持会丢失在一年左右的时间，当电脑获得更快的50％（归零你的优势在竞争中）。此外，还有超过在这样一些过程的自然限制，比如硬盘的速度是没有意义的。这是不相关的，你从一个RAM的驱动器中的文件进行测试 - 当应用程序投入生产将与从硬盘上的文件使用，并且应用程序的性能会被你的硬盘的速度被限制
Remember that the most costly cycle for an application is MAINTENANCE, not production. What you might gain today by making a 50% faster thingy that's 200% percent more difficult to maintain will be lost in a year or so, when computers get 50% faster (nulling your edge over the competition). Besides, there's no point in exceeding natural limits for such processes, like the speed of the HDD. It's irrelevant that you're testing with a file from a RAM-drive - when the application goes into production it will be used with files from a HDD, and your application's performance will be limited by the speed of your HDD.
不管怎样，我在一段时间做喜欢挑战一次，我真的很喜欢的解析器。下面是一个非常简单的解析器的实现，只有着眼于每个字符输入字符串的一次的只有复印件的东西在需要的地方：复制节点，以决定下一步做什么，并拷贝做名节点的有效载荷在适当的时候，以将其推入的阵列。在我的谦虚的i7 @ 3.4GHz的解析复制示例数据10000次内置了字符串采用63毫秒。它清楚地击败你的时间，但提醒一句，这code是脆弱的：它的依赖的上有一个XML文件，这是一定的形式。周围没办法。
Anyhow, I do like a challenge once in a while and I really like parsers. What follows is a very simple parser implementation that only looks at each character in the input string once and only copies stuff where needed: copies the name of the nodes in order to decide what to do next and copies the node's "Payload" when appropriate, in order to push it into the array. On my "modest" i7 @ 3.4 Ghz parsing a string built by copying your sample data 10,000 times takes 63 ms. It clearly beats your time, BUT a word of warning, this code is fragile: it depends on having a XML file that's a certain form. No way around that.
program Project28;
{$APPTYPE CONSOLE}
uses SysUtils, DateUtils, Windows;
const SampleData =
'<node>'#13#10+
' <datatype1>randomdata</datatype1>'#13#10+
' <datatype2>randomdata</datatype2>'#13#10+
' <datatype3>randomdata</datatype3>'#13#10+
' <datatype4>randomdata</datatype4>'#13#10+
' <datatype5>randomdata</datatype5>'#13#10+
' <datatype6>randomdata</datatype6>'#13#10+
' <datatype7>randomdata</datatype7>'#13#10+
' <datatype8>randomdata</datatype8>'#13#10+
' <datatype9>randomdata</datatype9>'#13#10+
' <datatype10>randomdata</datatype10>'#13#10+
' <datatype11>randomdata</datatype11>'#13#10+
' <datatype12>randomdata</datatype12>'#13#10+
' <datatype13>randomdata</datatype13>'#13#10+
' <datatype14>randomdata</datatype14>'#13#10+
' <datatype15>randomdata</datatype15>'#13#10+
' <datatype16>randomdata</datatype16>'#13#10+
' <datatype17>randomdata</datatype17>'#13#10+
' <datatype18>randomdata</datatype18>'#13#10+
' <datatype19>randomdata</datatype19>'#13#10+
' <datatype20>randomdata</datatype20>'#13#10+
'</node>'#13#10;
const NodeIterations = 10000;
type
TDummyRecord = record
D1, D2, D3, D4, D5, D6, D7, D8, D9, D10, D11, D12, D13,
D14, D15, D16, D17, D18, D19, D20: string;
end;
TDummyRecordArray = array[1..NodeIterations] of TDummyRecord;
procedure ParseDummyXMLToRecordArray(const InputText:string; var A: TDummyRecordArray);
var PInputText: PChar;
cPos, TextLen: Integer;
C: Char;
State: Integer;
tag_starts_at: Integer;
last_payload_starts_at: Integer;
FlagEndTag: Boolean;
NodeName, Payload: string;
cNode: Integer;
const st_not_in_node = 1;
st_in_node = 2;
begin
cPos := 1;
TextLen := Length(InputText);
PInputText := @InputText[1];
State := st_not_in_node;
last_payload_starts_at := 1;
cNode := 0;
// This is the lexer/parser loop. It's a finite-state machine with only
// two states: st_not_in_node and st_in_node
while cPos < TextLen do
begin
C := PInputText[cPos-1];
case State of
// What happens when we're NOT currently inside a node?
// Not much. We only jump to st_in_node if we see a "<"
st_not_in_node:
case C of
'<':
begin
// A node starts here. Switch state and set up some simple
// flags.
state := st_in_node;
tag_starts_at := cPos + 1;
FlagEndTag := False;
end;
end;
// What happens while inside a node? Again, not much. We only care about
// the "/" - as it signals an closing tag, and we only care about the
// ">" because that means the end of the ndoe.
st_in_node:
case C of
'/': FlagEndTag := True;
'>':
begin
// This is where the magic haepens. We're in one of possibly two states:
// We're ither seeing the first <name> of a pair, or the second </name>
//
if FlagEndTag then
begin
// This is the closing pair of a tag pair, ie, it's the </NodeName> What we'll do
// depends on what node is closing, so we retreive the NodeName:
NodeName := System.Copy(InputText, tag_starts_at+1, cPos - tag_starts_at-1);
if NodeName <> 'node' then // SAMPLE-DATA-SPECIFIC: I know I don't care about "node" tags.
begin
// SAMPLE-DATA-SPECIFIC: I know there are only two kinds of nodes:
// "node" and "datatypeN". I retreive the PAYLOAD for the node because
// I know it's not "ndoe" and I know I'll need it.
Payload := System.Copy(InputText,last_payload_starts_at, tag_starts_at - last_payload_starts_at -1);
// Make sure we're dealing with a valid node
if (cNode > 0) and (cNode <= High(A)) then
begin
// Based on NodeName, copy the Payload into the appropriate field.
if NodeName = 'datatype1' then A[cNode].D1 := Payload
else if NodeName = 'datatype2' then A[cNode].D2 := Payload
else if NodeName = 'datatype3' then A[cNode].D3 := Payload
else if NodeName = 'datatype4' then A[cNode].D4 := Payload
else if NodeName = 'datatype5' then A[cNode].D5 := Payload
else if NodeName = 'datatype6' then A[cNode].D6 := Payload
else if NodeName = 'datatype7' then A[cNode].D7 := Payload
else if NodeName = 'datatype8' then A[cNode].D8 := Payload
else if NodeName = 'datatype9' then A[cNode].D9 := Payload
else if NodeName = 'datatype10' then A[cNode].D10 := Payload
else if NodeName = 'datatype11' then A[cNode].D11 := Payload
else if NodeName = 'datatype12' then A[cNode].D12 := Payload
else if NodeName = 'datatype13' then A[cNode].D13 := Payload
else if NodeName = 'datatype14' then A[cNode].D14 := Payload
else if NodeName = 'datatype15' then A[cNode].D15 := Payload
else if NodeName = 'datatype16' then A[cNode].D16 := Payload
else if NodeName = 'datatype17' then A[cNode].D17 := Payload
else if NodeName = 'datatype18' then A[cNode].D18 := Payload
else if NodeName = 'datatype19' then A[cNode].D19 := Payload
else if NodeName = 'datatype20' then A[cNode].D20 := Payload
else
raise Exception.Create('Unknown node: ' + NodeName);
end
else
raise Exception.Create('cNode out of bounds.');
end;
// Repeat :-)
state := st_not_in_node;
end
else
begin
// Node start. Retreive node name. I only care about the start of the "NODE" - if I see that
// I'll increment the current node counter so I'll go on filling the next position in the array
// with whatever I need.
NodeName := System.Copy(InputText, tag_starts_at, cPos - tag_starts_at);
last_payload_starts_at := cPos+1;
if NodeName = 'node' then Inc(cNode);
state := st_not_in_node;
end;
end;
end;
end;
Inc(cPos);
end;
end;
var DataString: string;
SB: TStringBuilder;
i: Integer;
DummyArray: TDummyRecordArray;
T1, T2, F: Int64;
begin
try
try
// Prepare the sample string; 10.000 iterations of the sample data.
SB := TStringBuilder.Create;
try
for i:=1 to NodeIterations do
SB.Append(SampleData);
DataString := SB.ToString;
finally SB.Free;
end;
// Invoke the simple parser using the string constant.
QueryPerformanceCounter(T1);
ParseDummyXMLToRecordArray(DataString, DummyArray);
QueryPerformanceCounter(T2);
QueryPerformanceFrequency(F);
WriteLn(((T2-T1) * 1000) div F);
// Test parse validity.
for i:=1 to NodeIterations do
begin
if DummyArray[i].D1 <> 'randomdata' then raise Exception.Create('Bug. D1 doesn''t have the proper value, i=' + IntToStr(i));
if DummyArray[i].D2 <> 'randomdata' then raise Exception.Create('Bug. D2 doesn''t have the proper value, i=' + IntToStr(i));
if DummyArray[i].D3 <> 'randomdata' then raise Exception.Create('Bug. D3 doesn''t have the proper value, i=' + IntToStr(i));
if DummyArray[i].D4 <> 'randomdata' then raise Exception.Create('Bug. D4 doesn''t have the proper value, i=' + IntToStr(i));
if DummyArray[i].D5 <> 'randomdata' then raise Exception.Create('Bug. D5 doesn''t have the proper value, i=' + IntToStr(i));
if DummyArray[i].D6 <> 'randomdata' then raise Exception.Create('Bug. D6 doesn''t have the proper value, i=' + IntToStr(i));
if DummyArray[i].D7 <> 'randomdata' then raise Exception.Create('Bug. D7 doesn''t have the proper value, i=' + IntToStr(i));
if DummyArray[i].D8 <> 'randomdata' then raise Exception.Create('Bug. D8 doesn''t have the proper value, i=' + IntToStr(i));
if DummyArray[i].D9 <> 'randomdata' then raise Exception.Create('Bug. D9 doesn''t have the proper value, i=' + IntToStr(i));
if DummyArray[i].D10 <> 'randomdata' then raise Exception.Create('Bug. D10 doesn''t have the proper value, i=' + IntToStr(i));
if DummyArray[i].D11 <> 'randomdata' then raise Exception.Create('Bug. D11 doesn''t have the proper value, i=' + IntToStr(i));
if DummyArray[i].D12 <> 'randomdata' then raise Exception.Create('Bug. D12 doesn''t have the proper value, i=' + IntToStr(i));
if DummyArray[i].D13 <> 'randomdata' then raise Exception.Create('Bug. D13 doesn''t have the proper value, i=' + IntToStr(i));
if DummyArray[i].D14 <> 'randomdata' then raise Exception.Create('Bug. D14 doesn''t have the proper value, i=' + IntToStr(i));
if DummyArray[i].D15 <> 'randomdata' then raise Exception.Create('Bug. D15 doesn''t have the proper value, i=' + IntToStr(i));
if DummyArray[i].D16 <> 'randomdata' then raise Exception.Create('Bug. D16 doesn''t have the proper value, i=' + IntToStr(i));
if DummyArray[i].D17 <> 'randomdata' then raise Exception.Create('Bug. D17 doesn''t have the proper value, i=' + IntToStr(i));
if DummyArray[i].D18 <> 'randomdata' then raise Exception.Create('Bug. D18 doesn''t have the proper value, i=' + IntToStr(i));
if DummyArray[i].D19 <> 'randomdata' then raise Exception.Create('Bug. D19 doesn''t have the proper value, i=' + IntToStr(i));
if DummyArray[i].D20 <> 'randomdata' then raise Exception.Create('Bug. D20 doesn''t have the proper value, i=' + IntToStr(i));
end;
except on E: Exception do Writeln(E.ClassName, ': ', E.Message);
end;
finally
WriteLn('ENTER to Exit');
ReadLn;
end;
end.
这篇关于德尔福 - 快速（ER）的XML解析器的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
