需要一种快速的方法来将大量的double转换为string [英] Need a fast method to convert large amount of double to string
问题描述
我正在编写一个用于高速计算程序的结果输出模块.
I am writing a results output module for a high speed computing program.
我的计划是:
- 我的任务是以相对较快的速度将结果插入数据库(PostgreSQL).
- 我使用libpq的[从STDIN复制]，这被告知是最快的方法.
- 该方法需要将结果转换为char *格式.
结果看起来像这样:
- 接下来的106年的每月现金流(总计为1272倍).
- 每个条目大约有14个现金流.
- 大约2800个实体(用于测试数据的实体为2790个).
数据库中的表如下所示:
And the table in database looks like this:
- 表的每一行包含一个实体.
- 有一些前缀可以标识不同的实体.
- 现金流是在前缀(PGSQL中为float8 []类型)之后的双精度数组.
以下显示了用于在数据库中创建表的代码:
The following present the code for creating the table in database:
create table AgentCF(
PlanID int4,
Agent int4,
Senario int4,
RM_Prev float8[], DrvFac_Cur float8[], Prem float8[],
Comm float8[], CommOR float8[], FixExp float8[],
VarExp float8[], CIRCFee float8[], SaftyFund float8[],
Surr float8[], Benefit_1 float8[], Benefit_2 float8[],
Benefit_3 float8[], Benefit_4 float8[], Benefit_5 float8[],
Benefit_6 float8[], Benefit_7 float8[], Benefit_8 float8[],
Benefit_9 float8[], Benefit_10 float8[]
);
用于准备插入的CashFlow的函数的呈现代码:
Presenting code for function that prepare the inserted CashFlow:
void AsmbCF(char *buffer, int size, int ProdNo, int i, int Pos, int LineEnd)
{
int j, Step = sizeof(nodecf) / sizeof(double), PosST, Temp;
double *LoopRate = &AllHeap[ProdNo].Heap.AgentRes[i].CF.NodeCF[0].Prem;
strcpy_s(buffer, size, "{");
for (j = 0; j < TOTLEN / 10; j++) {
PosST = j * 10 * Step + Pos;
sprintf_s(&buffer[strlen(buffer)], size - strlen(buffer), "%f,%f,%f,%f,%f,%f,%f,%f,%f,%f,",
LoopRate[PosST],
LoopRate[PosST + 1 * Step],
LoopRate[PosST + 2 * Step],
LoopRate[PosST + 3 * Step],
LoopRate[PosST + 4 * Step],
LoopRate[PosST + 5 * Step],
LoopRate[PosST + 6 * Step],
LoopRate[PosST + 7 * Step],
LoopRate[PosST + 8 * Step],
LoopRate[PosST + 9 * Step]
);
}
Temp = j * 10;
PosST = Temp * Step + Pos;
sprintf_s(&buffer[strlen(buffer)], size - strlen(buffer), "%f", LoopRate[PosST]);
Temp = Temp + 1;
for (j = Temp; j < TOTLEN; j++) {
PosST = j * Step + Pos;
sprintf_s(&buffer[strlen(buffer)], size - strlen(buffer), ",%f", LoopRate[PosST]);
}
if (LineEnd) {
strcat_s(buffer, size, "}\n");
}
else {
strcat_s(buffer, size, "}\t");
}
}
以下是用于速度测试的代码:
The following are the code for speed testing:
void ThreadOutP(LPVOID pM)
{
char *buffer = malloc(BUFFLEN), sql[SQLLEN];
int Status, ProdNo = (int)pM, i, j, ben;
PGconn *conn = NULL;
PGresult *res;
clock_t begin, end;
fprintf_s(fpOutP, "PlanID %d Start inseting...\n", AllHeap[ProdNo].PlanID);
begin = clock();
DBConn(&conn, CONNSTR, fpOutP);
#pragma region General cashflow
//============================== Data Query ==============================
//strcpy_s(&sql[0], SQLLEN, "COPY AgentCF(PlanID,Agent,Senario,Prem,Comm,CommOR,CIRCFee,SaftyFund,FixExp,VarExp,Surr");
//for (ben = 1; ben <= AllHeap[ProdNo].Heap.TotNo.NoBenft; ben++) {
// strcat_s(&sql[0], SQLLEN, ",Benefit_");
// _itoa_s(ben, &sql[strlen(sql)], sizeof(sql) - strlen(sql), 10);
//}
//strcat_s(&sql[0], SQLLEN, ") FROM STDIN;");
//res = PQexec(conn, &sql[0]);
//if (PQresultStatus(res) != PGRES_COPY_IN) {
// fprintf_s(fpOutP, "Not in COPY_IN mode\n");
//}
//PQclear(res);
//============================== Data Apply ==============================
for (i = 0; i < AllHeap[ProdNo].MaxAgntPos + AllHeap[ProdNo].Heap.TotNo.NoSensi; i++) {
sprintf_s(buffer, BUFFLEN, "%d\t%d\t%d\t", AllHeap[ProdNo].PlanID, AllHeap[ProdNo].Heap.AgentRes[i].Agent, AllHeap[ProdNo].Heap.AgentRes[i].Sensi);
//Status = PQputCopyData(conn, buffer, (int)strlen(buffer));
//if (1 != Status) {
// fprintf_s(fpOutP, "PlanID %d inserting error for agent %d\n", AllHeap[ProdNo].PlanID, AllHeap[ProdNo].Heap.AgentRes[i].Agent);
//}
for (j = 0; j < 8 + AllHeap[ProdNo].Heap.TotNo.NoBenft; j++) {
if (j == 7 + AllHeap[ProdNo].Heap.TotNo.NoBenft) {
AsmbCF(buffer, BUFFLEN, ProdNo, i, j, 1);
}
else {
AsmbCF(buffer, BUFFLEN, ProdNo, i, j, 0);
}
//Status = PQputCopyData(conn, buffer, (int)strlen(buffer));
//if (1 != Status) {
// fprintf_s(fpOutP, "PlanID %d inserting error for agent %d\n", AllHeap[ProdNo].PlanID, AllHeap[ProdNo].Heap.AgentRes[i].Agent);
//}
}
}
//Status = PQputCopyEnd(conn, NULL);
#pragma endregion
#pragma region K cashflow
#pragma endregion
PQfinish(conn);
FreeProd(ProdNo);
free(buffer);
end = clock();
fprintf_s(fpOutP, "PlanID %d inserted, total %d rows inserted, %d millisecond cost\n", AllHeap[ProdNo].PlanID, i, end - begin);
AllHeap[ProdNo].Printed = 1;
}
请注意，我禁用了涉及插入的代码.
Please note that I disable the code that involving inserting.
测试结果为:
- 仅组装字符串的成本为45930毫秒.
- 组装字符串和插入件的成本为54829毫秒.
因此，大部分成本都在于将double转换为char.
So the most of the cost lie on converting double to char.
因此，我想问一下是否有一种更快的将double系列转换为字符串的方法，因为与计算成本相比，瓶颈实际上是结果的输出.
Therefore I would like to ask if there is a faster way of converting series of double into string, because compare to the calculation cost, the bottleneck actually is the outputting of results.
顺便说一下，我的平台是Windows 10，PostgreSQL 11，Visual Studio 2017.
By the way, my platform is Windows 10, PostgreSQL 11, Visual Studio 2017.
非常感谢！
推荐答案
将大量double转换为字符串的快速方法
fast method to convert large amount of double to string
对于完整的 double 应用程序，请使用 sprintf(buf，％a"，some_double).如果需要十进制输出，请使用％e" .
For full double range application, use sprintf(buf, "%a", some_double). If decimal output required, use "%e".
任何其他代码只有包含准确性或允许的输入范围 ，它才会更快.
Any other code will only be faster if it is comprises accuracy or allowable input range somehow.
常规方法是将 double x 转换为某个宽比例的整数并将其转换为字符串.这意味着OP尚未明确表达对 x 的限制.
The usual approach is to convert double x to some wide scaled integer and convert that to a string. This implies limitations on x not clearly expressed yet by OP.
即使某些其他方法出现得更快，也可能不会随着代码的发展或移植而变得更快.
Even if some other approaches appears faster, it may not be faster as code evolves or is ported.
OP需要发布的是用于客观性能评估的速度测试代码.
What OP needs to post is speed test code for objective performance assessment.
这篇关于需要一种快速的方法来将大量的double转换为string的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
