散列破坏者... [英] Hashing mindbreaker ...

问题描述

你好

我想分享一些想法，并征询一些人的意见.

I would like to share some thoughts and get maybe some people's opinion.

我们正在使用Function Apps将XML文件直接处理到我们的SQL数据仓库中.基本上，该文件具有一个File_UID字段(每个到达的文件都获得一个BIGINT)，一个xPath(XML字段路径)和一个Value字段(还有其他字段可以找到父目录..).全部 这些数据存储在我们构建EDW模式的ods模式中.

We are processing XML files directly into our SQL Datawarehouse with Function Apps. Basically the file has a File_UID field (each arriving file get a BIGINT), a xPath (The XML Fields Path) and a Value Field (there are other fields to find parents ..). All this data get stored in a ods schema from where we build our EDW schema.

在构建EDW(报告)表时，我们将记录与父项进行匹配，并产生一个更关键的结果来说明维度和事实(就表格设计而言，仍然存在).因此，所有联接始终都包含file_UID.

When building the EDW(reporting) tables we match records with parents and produce lets say a more pivoted result into dimensions and facts (as far as in tabular design this still exists). So all joining always include the file_UID.

因此，对我而言，最合乎逻辑的哈希将是file_uid，否则我们将拥有大量数据移动量来处理每个文件.相反，在将文件上传到暂存并将其从暂存传输到ods时，我们只会从60台服务器中收费"一个....

So to me the most logical hash would be file_uid, if not we have plenty of data-movement to process each file. At the opposite side when uploading a file into staging and transferring it from staging to ods we would only 'charge' one from the 60 servers....

round_robin似乎有点杂乱无章，但所有服务器都在逐个服务器处理...所以可以说，这时可以清楚选择什么，但我对此表示怀疑...

Doing everything round_robin seems a bit messy, but all processing server by server ... So lets say that at this moment it clear what to choose but I have doubts ...

推荐答案

哈利，

您是否正在寻找一种更有效的方式来完成此数据加载?

Are you seeking a more efficient way to accomplish this data load?

这篇关于散列破坏者...的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！