Django可信服务器进行文件处理 - 分发检索 [英] Django trusted server for file processing-distributing-retrieving

问题描述

我是Django中的一名新手，有一个项目，以完成并想知道如何在以下情况下采取正确的架构和路径：

我有一个值得信赖的具有以下职责的服务器：
1.验证用户
2.从用户接收文件并通过执行我的算法将其转换为n个段（假定像Rabin的IDA或All-或 - 无变换（AONT）），然后将每个生成的段存储在其他预定义的不同服务器上。
3.在下载阶段，服务器对用户进行身份验证，从存储服务器检索相应的请求文件的段，组合段以构建文件并将其传递给用户。

对于我来说，不透明的是存储方面需要的元素：
可信服务器如何与存储服务器通信并检索分段？我如何提供信任的服务器来请求存储服务器的属于用户Y的文件X？

我所想到的是为每个段提供一个特定的可以使用具有文件ID和用户ID作为输入的哈希函数生成的元数据。

如果您提供了路线图和元素I，我将不胜感激需要考虑这个项目的每个方面。一视图就足够了。非常感谢你。

解决方案

有很多方法可以解决，每个都有自己的优点和缺点。

对于我来说，解决方案的共同点是这样的设置：

• 可信服务器）首先接收文件（上传到此服务器）


• TS应用您的算法并获取段（也是文件）列表


• 分段可以单独加密（？）



• TS将元数据存储在数据库中，如下所示：

  
  
  

  用户| - < |文件| - < |段|> - < |服务器|




• 文件获取ID，例如：338977FE-36F3-4397-BD60-A9CE94146AF0（UUID），需要任何元数据

  / li>
  
• 一个段也获得一个UUID，它属于它的文件的ForeignKey，以及它所存储的服务器


• 数据库按顺序存储段所以他们可以再次组装成一个文件

文件ID可能会返回给用户以备将来检索，或者他可以得到一个我的基本思路是TS可以从存储服务器发送和检索分段的两种方式（让我们称之为SA，SB，SC ...），它们的TS文件列表来自TS。

< ：

在存储服务器上使用简单的Web服务来读取和写入细分，例如写你的POST POST：

  sa.storagesyste.com/segments/7FD960E2-7C77-4A17-BB67-813377A83443/ 
  

要阅读细分，只需HTTPS即可获得相同的URL。这需要在每个存储服务器上运行简单的Web服务。无论您想要存储服务器上的身份验证取决于您，您都需要知道获取文件的确切密钥，这可能就足够了。使用这种方法，您还可以让桌面客户端检索分段本身 - 下载后，只需获取分段ids列表以及下载它们的位置。或者，如果您使用的是Unix或Linux，则可以使用 scp 在TS和存储服务器。这样，您可以使用Unix / Linux的 SSH PKI基础架构以可管理的方式添加额外的安全性。使用这种方法，TS需要收集细分并重新生成文件，然后用户可以下载。

I am a newbie in Django having a project in mind to accomplish and wondering about the proper architecture and path to take towards the following scenario:

I have a trusted server that has the following duties: 1. Authenticates a user 2. Receives a file from the user and transforms it into n segments by performing an algorithm of mine (presume one like Rabin's IDA or All-or-nothing transform (AONT)), and then stores each produced segment on other pre-defined distinct servers. 3. IN downloading phase, the server authenticates the user, retrieves the segments of the corresponding requested file from storage servers, combines the segments to construct the file and delivers it to the user.

What is opaque for me is the needed elements on the storage sides: How does the trusted server communicate with the storage severs and retrieves the segments? How can I provide the trusted server to request the storage servers for the file X that belongs to User Y?

What I had in mind was to provide each segment with a specific metadata that could be generated using a hash function that has the file ID and Users ID as inputs.

I would be grateful if you provided me with a road map and elements I would need to consider on each side for this project. An over view suffices. Thank you so much.

解决方案

There are many ways to go about this, each with their strengths and weaknesses.

Common for the solutions that comes to me is this setup:

• The trusted server (TS) receives the File in the first place (upload to this server)
• TS applies your algorithm to and get a list of segments (also files)
• Segments may be individually encrypted(?)

• TS stores the metadata in a database like so:

  | User |--<| File |--<| Segment |>-<| Server |

• A File gets an id, say: 338977FE-36F3-4397-BD60-A9CE94146AF0 (UUID) and whatever metadata is needed

• A Segment gets a UUID too, a ForeignKey to the File it belongs to, and the servers it's stored on
• The database stores the Segments in order so they can be assembled into a File again

The File id may be returned to the user for future retrieval, or he can get a list of his Files from TS.

I basically think of two ways TS can send and retrieve Segments from storage servers (let's call them SA, SB, SC ...):

Use a simple web service on storage servers to read and write Segments, e.g. to write you HTTPS POST:

sa.storagesyste.com/segments/7FD960E2-7C77-4A17-BB67-813377A83443/

To read the Segment, simply HTTPS GET the same URL. This requires a simple web service running on each storage server. Whether you want authentication on the storage servers is up to you, you need to know the exact key to get the file, that may be enough. With this approach, you could also have a desktop client retrieve the Segments itself - on download it would simply get the list of Segment ids and where to download them.

Alternatively, if you're on Unix or Linux, you could use scp to transfer the files between TS and storage servers. This way you can use the SSH PKI infrastructure of Unix/Linux to add extra security in a manageable way. With this approach the TS need to collect the Segments and regenerate the File, then the user can download.

这篇关于Django可信服务器进行文件处理 - 分发检索的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！