多节下载TEMP文件 [英] Multi - Section Downloads TEMP files

问题描述

大家好！

我正在创建一个用于多段下载的应用程序(每次下载有多个线程)...但是我有一个大问题:

我的程序分析了远程文件，并创建了具有相等范围的段/节
所以每个线程都下载自己的段...说，我想下载一个大小为70MB的文件，并且
我的程序创建了两个线程来下载范围(0-35 MB； 35-70 MB). 每个线程将下载的内容保存在其自己的临时文件中...

下载完成后，我的程序仅合并这些文件，创建它们的最终结果文件，完成后将这些临时文件删除...

每次下载有几个文件有效吗?拥有一个更有效吗
临时文件并使用LOCKS仅允许一个线程写入下载的内容
及时，而不是尝试以某种方式对字节排序并创建结果文件???

您如何看待更好的选择???

我个人认为，每次下载使用多个文件时，我可以轻松合并它们
导致我的程序跟踪每个段并为其分配索引...因此在下载时
完成后，它会按索引对这些段进行排序并轻松合并文件，但在其他
上 侧面，许多文件和许多线程使硬盘驱动器在硬盘驱动器群集和磁道之间切换...

请帮助...

Hi guys!

I''m creating an application for multi-segment downloading (many threads per single download) ...but I have a BIG QUESTION:

My program analyzes the remote file and creates segments/sections with equal ranges
so each thread downloads its own segment...Say, I want to download a file 70MB in size and
my program creates two threads to download ranges ( 0 - 35 MB; 35 - 70 MB)..
Each thread saves downloaded content in its OWN TEMPORARY FILE...

When download is complete my program simply merges those files, creates the final resulting FILE of them and when done DELETES those temporary files...

Is it effective to have several files per download?? IS it more effective to have one
temporary file and to use LOCKS to allow only one thread write downloaded content
in time and than try to sort bytes somehow and create the resulting file???

How do you think what choice is better???

I personally think that when I use many files per download I can easily merge them
cause my program tracks each segment and assigns index to it...so when download
is complete it sorts those segments by index and merges files easily BUT on other
side many files and many threads make hard drive to switch between hard drive clusters and tracks...

PLEASE HELP...

推荐答案

Nick，

我个人的选择是运行每个文件的每个线程.单个文件不需要多个线程.因为应用要求.

您的应用程序需要同时下载多个文件.对?在这种情况下，每个文件线程是更好的选择.关键原因是性能.由于文件分割算法+对单个文件的多线程管理+以后的合并，因此增加了CPU时钟周期.我认为由于每个文件都具有多线程功能，因此不会大大提高性能值.

Nick,

My personal choice would be running each thread per file. No need to have multiple threads for single file. Becaz of the application requirement.

Your application needs to download multiple files at the same time. Right? In that scenario, thread per file is the better option. Key reason is performance. CPU clock cycle is added up due to the file split algorithm + multi thread management for the single file + later merging. I don''t think it will greatly add the performance value due to multithread per file.

Hello Nick，

一段时间前，我学到了很多有关创建自己的下载器的知识，我想提供一些建议.

首先，每次下载1个线程.
在同一线程中为段创建任意数量的连接.
我实际上每段尝试了1个线程，完全没有必要.

其次，如果每次下载也使用1个线程，则每次下载1个文件会更容易，并且不需要锁定.
每次下载要管理许多文件可能是一个挑战，如果下载量很大，合并将需要一些时间.

祝您代码顺利.

Hello Nick,

I learned a lot creating my own downloader some time ago and I''d like to offer some suggestions.

First, 1 thread per download.
Create any number of connections you need for the segments in the same thread.
I actually tried 1 thread per segment, totally unnecessary.

Second, 1 file per download would be easier and requires no locking if you also use 1 thread per download.
Managing many files per download can be challenging, and if the download is big, merging would take some time.

Good luck with your code.

这篇关于多节下载TEMP文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！