线程问题 [英] Thread Question
问题描述
我对线程有一些基本的怀疑。
我有一个列表，里面有需要的项目从
下载互联网。
让我们说列表是：
list_items []其中有100个项目。
我有一个函数download_from_web（）可以完成
的工作，从网上下载这些项目。它会在错误处理的情况下进行，因为它会导致错误，例如404等等。
目前，这就是我使用它的方式。
for list_items中的项目：
download_from_web（item）
这样，一次下载一个项目。 />
我打算在我的应用程序中实现线程，以便可以同时下载多个
项目。我希望线程选项是
用户定义。
查看线程文档（Core Python Programming），我已经
注意到所有线程都执行了一次。取决于他们b
的作用，有些早完成，有些则稍后完成。
但是我想实现类似的东西：
list_items中$ item
：
for thread_args中的num：
thread [num] .start（）
thread [num] .start（）
这是线程应用程序的正确方法吗？
这是我第一次进行线程化。所以当时正在寻找
的意见和建议。
谢谢，
Ritesh
Hi,
I have some basic doubts about thread.
I have a list which has items in it which need to be downloaded from
the internet.
Let''s say list is:
list_items[] which has 100 items in it.
I have a function download_from_web() which does the work of
downloading the items from the web. It does error handling in case it
gets errors like 404 et cetera.
Currently, this is how I''m using it.
for item in list_items:
download_from_web(item)
This way, one items is downloaded at a time.
I''m planning to implement threads in my application so that multiple
items can be downloaded concurrently. I want the thread option to be
user-defined.
Looking at the documentation of threads (Core Python Programming), I''ve
noticed that all threads are executed a once. Depending upon what they
are doing, some finish early and some later.
But I want to implement something like:
for item in list_items:
for num in thread_args:
thread[num].start()
thread[num].start()
Is this the correct way of threading applications ?
This is the first time I''d be doing threading. So was looking for
comments and suggestions.
Thanks,
Ritesh
推荐答案
Ritesh Raj Sarraf写道：
Ritesh Raj Sarraf wrote:
我打算在我的应用程序中实现线程以便多个
项目可以同时下载。我希望线程选项是
用户定义。
查看线程文档（Core Python Programming），我已经
注意到所有线程都执行了一次。取决于他们b
的作用，有些早完成，有些则稍后完成。
但是我想实现类似的东西：
list_items中$ item
：
for thread_args中的num：
thread [num] .start（）
thread [num] .start（）
这是线程应用程序的正确方法吗？
这是我第一次进行线程化。所以一直在寻找
的意见和建议。
I''m planning to implement threads in my application so that multiple
items can be downloaded concurrently. I want the thread option to be
user-defined.
Looking at the documentation of threads (Core Python Programming), I''ve
noticed that all threads are executed a once. Depending upon what they
are doing, some finish early and some later.
But I want to implement something like:
for item in list_items:
for num in thread_args:
thread[num].start()
thread[num].start()
Is this the correct way of threading applications ?
This is the first time I''d be doing threading. So was looking for
comments and suggestions.
你想要的是使用一个线程池，以便你可以配置如何
一次发出许多请求（你不想尝试并行发出100个
请求）。您可以通过
队列与线程进行通信。
因此，如果线程的代码如下：
def run（请求，响应）：
而1：
item = request.get（）
如果item为None：< br $>
休息
response.put（download_from_web（item））
＃你的主循环可以是这样的：
requestQueue = Queue（）
responseQueue = Queue（）
thread_pool = [
线程（目标=运行） ，args =（requestQueue，responseQueue）
for i in range（numthreads）]
for t in thread_pool：t.start（）
list_items项目
：
requestQueue.put（item）
for i in range（len（list_items））：
response = responseQueue.get（）
handle_response（响应）
＃然后在你完成后关闭线程：
for t in thread_pool：
requestQueue.put（None）
for t in thr ead_pool：
t.join（）
What you want is to use a pool of threads so that you can configure how
many requests are issued at a time (you don''t want to try to issue 100
requests all in parallel). You can communicate with the threads through a
Queue.
So if the code for a thread looks like:
def run(request, response):
while 1:
item = request.get()
if item is None:
break
response.put(download_from_web(item))
# your main loop can be something like:
requestQueue = Queue()
responseQueue = Queue()
thread_pool = [
Thread(target=run, args=(requestQueue, responseQueue)
for i in range(numthreads)]
for t in thread_pool: t.start()
for item in list_items:
requestQueue.put(item)
for i in range(len(list_items)):
response = responseQueue.get()
handle_response(response)
# and then to shut down the threads when you''ve finished:
for t in thread_pool:
requestQueue.put(None)
for t in thread_pool:
t.join()
Duncan，
我不能从代码中得到很多。
相反，这就是我所做的。
threads = []
nloops = range（len（lRawData））
我在nloops中的
：
（sUrl，sFile，download_size，checksum）=
stripper（lRawData [i]）
t = threading.Thread（target = download_from_web，args =（sUrl，
sFile，sSourceDir，None））
＃ = pypt_thread（download_from_web，i，
stripper（lRawData [i]））
threads.append（t）
i = 0
join_i = 0
而i< nloops：
counter = 0
while counter< 3：
线程[i] .start（）
计数器+ = 1
i + = 1
counter = 0
join_i = i - 3
而计数器< 3：
个主题[join_i] .join（）
计数器+ = 1
join_i + = 1
这是对的吗？评论!!
Ritesh
Duncan Booth写道：
Duncan,
I couldn''t make out much from the code.
Instead this is what I did.
threads = []
nloops = range(len(lRawData))
for i in nloops:
(sUrl, sFile, download_size, checksum) =
stripper(lRawData[i])
t = threading.Thread(target=download_from_web, args=(sUrl,
sFile, sSourceDir, None))
# = pypt_thread(download_from_web, i,
stripper(lRawData[i]))
threads.append(t)
i = 0
join_i = 0
while i < nloops:
counter = 0
while counter < 3:
threads[i].start()
counter += 1
i += 1
counter = 0
join_i = i - 3
while counter < 3:
threads[join_i].join()
counter += 1
join_i += 1
Is this correct ? Comments!!
Ritesh
Duncan Booth wrote:
Ritesh Raj Sarraf写道：
Ritesh Raj Sarraf wrote:
我打算在我的应用程序中实现线程，以便可以同时下载多个
项目。我希望线程选项是
用户定义。
查看线程文档（Core Python Programming），我已经
注意到所有线程都执行了一次。取决于他们b
的作用，有些早完成，有些则稍后完成。
但是我想实现类似的东西：
list_items中$ item
：
for thread_args中的num：
thread [num] .start（）
thread [num] .start（）
这是线程应用程序的正确方法吗？
这是我第一次进行线程化。所以一直在寻找
的意见和建议。
I''m planning to implement threads in my application so that multiple
items can be downloaded concurrently. I want the thread option to be
user-defined.
Looking at the documentation of threads (Core Python Programming), I''ve
noticed that all threads are executed a once. Depending upon what they
are doing, some finish early and some later.
But I want to implement something like:
for item in list_items:
for num in thread_args:
thread[num].start()
thread[num].start()
Is this the correct way of threading applications ?
This is the first time I''d be doing threading. So was looking for
comments and suggestions.
您想要的是使用一个线程池，以便您可以配置
一次发出多少请求（你不要我不想尝试并行发出100
请求。您可以通过
队列与线程进行通信。
因此，如果线程的代码如下：
def run（请求，响应）：
而1：
item = request.get（）
如果item为None：< br $>
休息
response.put（download_from_web（item））
＃你的主循环可以是这样的：
requestQueue = Queue（）
responseQueue = Queue（）
thread_pool = [
线程（目标=运行） ，args =（requestQueue，responseQueue）
for i in range（numthreads）]
for t in thread_pool：t.start（）
list_items项目
：
requestQueue.put（item）
for i in range（len（list_items））：
response = responseQueue.get（）
handle_response（响应）
＃然后在你完成后关闭线程：
for thread_pool中的t：
requestQueue.put（无）
对于thread_pool中的t，
：
t.join（）
What you want is to use a pool of threads so that you can configure how
many requests are issued at a time (you don''t want to try to issue 100
requests all in parallel). You can communicate with the threads through a
Queue.
So if the code for a thread looks like:
def run(request, response):
while 1:
item = request.get()
if item is None:
break
response.put(download_from_web(item))
# your main loop can be something like:
requestQueue = Queue()
responseQueue = Queue()
thread_pool = [
Thread(target=run, args=(requestQueue, responseQueue)
for i in range(numthreads)]
for t in thread_pool: t.start()
for item in list_items:
requestQueue.put(item)
for i in range(len(list_items)):
response = responseQueue.get()
handle_response(response)
# and then to shut down the threads when you''ve finished:
for t in thread_pool:
requestQueue.put(None)
for t in thread_pool:
t.join()
Ritesh Raj Sarraf写道：
Ritesh Raj Sarraf wrote:
Duncan，
我无法从代码中得到很多。
Duncan,
I couldn''t make out much from the code.
请再试一次，了解Duncan的代码。你做了什么比
要好得多。
Please, try again to understand Duncan''s code. It''s much better than
what you did.
相反，这就是我所做的。
threads = []
nloops = range（len（lRawData））
我在nloops中的
：
（sUrl，sFile ，download_size，checksum）=
stripper（lRawData [i]）
t = threading.Thread（target = download_from_web，args =（sUrl，
sFile，sSourceDir，无））
＃= pypt_thread（download_from_web，i，
stripper（lRawData [i]））
个线程。追加（t）
i = 0
join_i = 0
而i< nloops：
counter = 0
while counter< 3：
线程[i] .start（）
计数器+ = 1
i + = 1
counter = 0
join_i = i - 3
而计数器< 3：
个主题[join_i] .join（）
计数器+ = 1
join_i + = 1
这是对的吗？评论！！
Instead this is what I did.
threads = []
nloops = range(len(lRawData))
for i in nloops:
(sUrl, sFile, download_size, checksum) =
stripper(lRawData[i])
t = threading.Thread(target=download_from_web, args=(sUrl,
sFile, sSourceDir, None))
# = pypt_thread(download_from_web, i,
stripper(lRawData[i]))
threads.append(t)
i = 0
join_i = 0
while i < nloops:
counter = 0
while counter < 3:
threads[i].start()
counter += 1
i += 1
counter = 0
join_i = i - 3
while counter < 3:
threads[join_i].join()
counter += 1
join_i += 1
Is this correct ? Comments!!
这真是太痛苦了。它并不完全是不正确的，但我认为（我是* b $ b *想*，即使在阅读了3次之后也很难说）它是
一次只下载三个请求，如果一个（或多个）
请求需要很长时间，它将会阻止所有其他请求。另外，
你在lRawData中为每个项目创建一个线程，即使你只需要
一次需要/使用三个。
如果你在Duncan的代码中设置numthreads = 3，那么你一次只能下载3件东西，但你只会使用三个线程
和长时间运行的请求不会影响其他请求。
如果您需要帮助，请提出问题。我，其中一个，
很乐意评论它，以解释它是如何工作的。它是如此美好和优雅，我已经把它粘贴到我自己的
"笔记本电脑中了。很酷的有用python模式将来使用。
慢慢重读，想想它正在做什么，如果出现问题，请写下来然后问他们。邓肯的代码很漂亮。非常值得
你的理解时间。
和平，
~Simon
This is just painful. It''s not exactly "incorrect", but I think (I
*think*, it''s hard to tell even after reading it 3 times) that it''s
going to only download three requests at a time and if one (or more) of
the requests takes a long time it will hold up all the others. Also,
you''re creating one thread per item in lRawData even though you only
need/use three at a time.
if you set numthreads = 3 in Duncan''s code, you would only be
downloading 3 things at a time, but you''d only be using three threads
and long running requests would not affect the others.
If you need help understanding it please ask questions. I, for one,
would be happy to comment it for you to explain how it works. It''s so
nice and elegant that I''ve already cut-and-pasted it into my own
"notebook" of cool useful python "patterns" to use in the future.
Reread it slowly, think about what it''s doing, if questions arise write
them down and ask them. Duncan''s code is beautiful. It''s well worth
your time to understand it.
Peace,
~Simon
>
Ritesh
Duncan Booth写道：
>
Ritesh
Duncan Booth wrote:
Ritesh Raj Sarraf写道：
Ritesh Raj Sarraf wrote:
我打算在我的应用程序中实现线程，以便可以下载多个
项目同时。我希望线程选项是
用户定义。
>
查看线程文档（Core Python Programming），我'$
注意到所有线程都执行了一次。取决于他们在b
的作用，有些早完成，有些则稍后完成。
>
但是我想实现类似的东西：
>
for list_items中的项目：
for thread_args中的num：
thread [num] .start（）
thread [num] .start（）
>
这是线程化应用程序的正确方法吗？
这是我第一次做线程。所以一直在寻找
的意见和建议。
>
I''m planning to implement threads in my application so that multiple
items can be downloaded concurrently. I want the thread option to be
user-defined.
>
Looking at the documentation of threads (Core Python Programming), I''ve
noticed that all threads are executed a once. Depending upon what they
are doing, some finish early and some later.
>
But I want to implement something like:
>
for item in list_items:
for num in thread_args:
thread[num].start()
thread[num].start()
>
Is this the correct way of threading applications ?
This is the first time I''d be doing threading. So was looking for
comments and suggestions.
>
您想要的是使用一个线程池，以便您可以配置
一次发出多少请求（你不要我不想尝试并行发出100
请求。您可以通过
队列与线程进行通信。
因此，如果线程的代码如下：
def run（请求，响应）：
而1：
item = request.get（）
如果item为None：< br $>
休息
response.put（download_from_web（item））
＃你的主循环可以是这样的：
requestQueue = Queue（）
responseQueue = Queue（）
thread_pool = [
线程（目标=运行） ，args =（requestQueue，responseQueue）
for i in range（numthreads）]
for t in thread_pool：t.start（）
list_items项目
：
requestQueue.put（item）
for i in range（len（list_items））：
response = responseQueue.get（）
handle_response（响应）
＃然后当你关闭线程'完成了：
for thread_pool：
requestQueue.put（无）
for thread inpool：
t.join（）
What you want is to use a pool of threads so that you can configure how
many requests are issued at a time (you don''t want to try to issue 100
requests all in parallel). You can communicate with the threads through a
Queue.
So if the code for a thread looks like:
def run(request, response):
while 1:
item = request.get()
if item is None:
break
response.put(download_from_web(item))
# your main loop can be something like:
requestQueue = Queue()
responseQueue = Queue()
thread_pool = [
Thread(target=run, args=(requestQueue, responseQueue)
for i in range(numthreads)]
for t in thread_pool: t.start()
for item in list_items:
requestQueue.put(item)
for i in range(len(list_items)):
response = responseQueue.get()
handle_response(response)
# and then to shut down the threads when you''ve finished:
for t in thread_pool:
requestQueue.put(None)
for t in thread_pool:
t.join()
这篇关于线程问题的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
