foreach，IEnumerable和修改内容 [英] foreach, IEnumerable and modifying contents
问题描述
我有一个相当复杂的需求。
我有一个类解析网页并提取所有相关文件
地址。它允许我在网页上下载每个pdf，用于
实例。我想合并线程，以便我可以单独下载N
文件。
显而易见的解决方案是线程池。但是，我需要确保
我下载文件Async - 所以我可以获得百分比和状态
信息到我的界面。
我已经决定最好的方法是让我的下载（一个
类表示要下载的文件）在
时举起活动完了。我希望在下载完成后让我的线程重新加入
线程池。
但是，我的下载实例来自
IEnumerable<从WebExtractor类接收下载
（使用yield return即动解析HTML）。
我认为我缺乏一些关于线程池的基础知识。如何使用
线程池并使下载的事件仍然到达
界面？是否有办法在foreach或IEnumerator代码块中为实例添加事件处理程序
？
任何帮助都会让我更接近完成我的第二个
版本的软件。提前致谢！
~Travis
2007-11-27 20:24:28 -0800，je ********** @ gmail.com
< je ********** @ gmail.comsaid：
我有一个相当复杂的需求。
也许。虽然，我怀疑你创造了一个复杂的b
需求，但实际上并没有必要这样做。
我有一个类解析网页并提取所有相关文件
地址。它允许我在网页上下载每个pdf，用于
实例。我想合并线程，以便我可以单独下载N
文件。
一个相当常见的操作。
显而易见的解决方案是线程池。但是，我需要确保
我下载文件Async - 所以我可以获得百分比和状态
信息到我的界面。
在我看来，一个不同的显而易见的解决方案就是在HttpWebRequest类上使用
async方法，或者甚至只使用一个简单的
TcpClient或Socket实例，以及一个队列。
队列的生产者将添加要下载的URL，而消费者将保持跟踪有多少活动下载（通过HttpWebRequest，
）
TcpClient或Socket）。
每次生产者向队列添加东西时，它都会向消费者发出信号
。作为响应的消费者将从
队列中删除项目，当队列为空或已达到最大数量
并发操作时停止，以先到者为准。 br />
项目完成后，消费者也会收到信号，
允许它从队列中提取新项目。
在上面，我想的是消费者和制作人作为个人
线程。但是你可以很容易地实现它，而不需要一个专用的线程来支持
，而消费者和生产者类只是被称为
，无论当时管理它们的是什么线程。在那个
的情况下，信令消费者更多的只是执行尝试出列更多下载操作的方法。
我已经确定了最好的这样做的方法是让我的下载（一个
类表示要下载的文件）在
完成时引发事件。我希望在下载完成后让我的线程重新加入
线程池。
如果在上述类中使用async方法，则免费获得
线程池行为。
但是，我的下载实例来自
IEnumerable<从WebExtractor类收到的下载
（其中使用yield return即时解析HTML。
这是我认为队列会更好的另一个原因。没有
技术原因你不能实现异步枚举器，但是在这种情况下这样做的
似乎使问题过于复杂。一个
队列对我来说似乎更自然，并且没有你似乎遇到的相同复杂因素的
。
< blockquote class =post_quotes>
我认为我缺乏一些关于线程池的基础知识。如何使用
线程池并使下载的事件仍然到达
界面？
我认为你可以完全避免这个问题，但基本答案是
那个线程池的概念并且有事件......到达
界面是正交的想法。由于线程池，您可能需要处理线程同步问题。但基本的问题是以一些某些
接口的实现者接收它们的方式提出事件并不受是否存在多个
$的影响b $ b涉及的线程。
是否有办法在foreach或IEnumerator代码块中为实例添加事件处理程序
？
您可以随时订阅活动。
任何帮助都会让我感到惊讶我的第二个
版软件更接近完成。提前致谢！
见上文。我建议放弃这个异步枚举器的想法
并使用一个漂亮，简单的队列。
Pete
11月27日晚上10点04分，Peter Duniho< NpOeStPe ... @ NnOwSlPiAnMk.com>
写道：
2007-11-27 20:24:28 -0800，" jehugalea ... @ gmail.com"
< jehugalea ... @ gmail.comsaid：
我有一个相当复杂的需求。
也许吧。虽然，我怀疑你创造了一个复杂的b
需求，但实际上并没有必要这样做。
我有一个类解析网页并提取所有相关文件
地址。它允许我在网页上下载每个pdf，用于
实例。我想合并线程，以便我可以单独下载N
文件。
一个相当常见的操作。
显而易见的解决方案是线程池。但是，我需要确保
我下载文件Async - 所以我可以获得百分比和状态
信息到我的界面。
在我看来，一个不同的显而易见的解决方案就是在HttpWebRequest类上使用
async方法，或者甚至只使用一个简单的
TcpClient或Socket实例，以及一个队列。
队列的生产者将添加要下载的URL，而消费者将保持跟踪有多少活动下载（通过HttpWebRequest，
）
TcpClient或Socket）。
每次生产者向队列添加东西时，它都会向消费者发出信号
。作为响应的消费者将从
队列中删除项目，当队列为空或已达到最大数量
并发操作时停止，以先到者为准。 br />
项目完成后，消费者也会收到信号，
允许它从队列中提取新项目。
在上面，我想的是消费者和制作人作为个人
线程。但是你可以很容易地实现它，而不需要一个专用的线程来支持
，而消费者和生产者类只是被称为
，无论当时管理它们的是什么线程。在那个
的情况下，信令消费者更多的只是执行尝试出列更多下载操作的方法。
我已经确定了最好的这样做的方法是让我的下载（一个
类表示要下载的文件）在
完成时引发事件。我希望在下载完成后让我的线程重新加入
线程池。
如果在上述类中使用异步方法，则免费获得
线程池行为。
但是，我的下载实例来自
IEnumerable<从WebExtractor类收到的下载
（其中使用yield return即时解析HTML。
这是我认为队列会更好的另一个原因。没有
技术原因你不能实现异步枚举器，但是在这种情况下这样做的
似乎使问题过于复杂。一个
队列对我来说似乎更自然，并且没有你似乎遇到的相同复杂因素的
。
< blockquote class =post_quotes>
我认为我缺乏一些关于线程池的基础知识。如何使用
线程池并使下载的事件仍然到达
界面？
我认为你可以完全避免这个问题，但基本的答案是
那个线程池的概念并且有事件......到达
界面是正交的想法。由于线程池，您可能需要处理线程同步问题。但基本的问题是以一些某些
接口的实现者接收它们的方式提出事件并不受是否存在多个
$的影响b $ b涉及的线程。
是否有办法在foreach或IEnumerator代码块中为实例添加事件处理程序
？
您可以随时订阅活动。
任何帮助都会让我感到惊讶我的第二个
版软件更接近完成。提前致谢！
见上文。我建议放弃这个异步枚举器的想法
并使用一个漂亮，简单的队列。
Pete
当我收到下载完成后，我的第一个实现实际上有一个队列<下载了
消耗了。但是，我的代码很难说，嘿，不要再尝试消费了！我结束了有一个非常严格的代码集，我希望摆脱它。
我从一个下载完成事件中遇到了大问题/>
在另一个线程处于锁定区块的中间时中断。
我偶尔会遇到死锁。
我的希望我的新设计是为了摆脱对这么多的并发管理的需求。在某种意义上，我通过使用yield return语句
并将其作为我的队列来实现。它还使得终止
点更容易看到。然而，没有一种说法，嘿，
我们还没有准备好开始下载你 - 等一下，我
正在下载我的电脑可以处理多个文件。所以
我希望找到一种说法，嘿等待而不需要
必须手动管理线程/并发下载数量
。
我可以尝试管理下载再次手动。我确实移动了很多
的代码来将界面与下载区分开来，所以它现在比以前更容易
。第二次，ThreadPools似乎更直观了。也许我的第一种方法是更好的
一个。
感谢您的想法，
Travis
在2007-11-27 21:23:40 -0800，je ********** @ gmail.com
< je**********@gmail.com说：
我的第一个实现实际上有一个队列<下载那是
当我收到下载完成时消耗了。但是，我的代码很难说，嘿，不要再尝试消费了！
通常使用队列时，该点是队列为空时。它的价格通常不会很难。
我结束了
有一个非常严格的代码集和我希望摆脱它。
我从一次下载完成事件中遇到大问题
中断而另一个线程处于锁定块的中间。
我不停地偶尔死锁。
嗯，不管怎样，你似乎还在处理线程问题
。死锁是实施错误的结果。如果你在前一个设计中遇到麻烦处理线程同步问题，那么你可能会遇到任何其他涉及
$ b的设计问题。 $ b threads。
我对新设计的希望是远离需要这么多的b $ b并发管理。
你打算如何通过引入自己的线程池来实现这一点，我是不是很清楚。 :)
我是通过使用yield return语句
并在某种意义上将其作为我的队列来实现的。它还使得终止
点更容易看到。然而，没有一种说法，嘿，
我们还没有准备好开始下载你 - 等一下，我
正在下载我的电脑可以处理多个文件。所以
我希望找到一种说法，嘿等待而不需要
必须手动管理线程/并发下载的数量
。
使用我提到的队列/异步范例管理它将是简单的
。特别是考虑到在网络类上使用异步
i / o方法的效率优势，在我看来，通过创建自己的线程池来管理
并发消费者数量是更多
复杂且容易出错。
我会说你似乎把自己放在这里的困惑是
很好地表明了这一点。 :)
我可以尝试再次手动管理下载。我确实移动了很多
的代码来将界面与下载区分开来，所以它现在比以前更容易
。第二次，ThreadPools似乎更直观了。也许我的第一种方法是更好的
一。
如果它像我的建议，显然我同意。 :)
Pete
I have a rather complex need.
I have a class that parses web pages and extracts all relevant file
addresses. It allows me to download every pdf on a web page, for
instance. I would like to incorporate threads so that I can download N
files separately.
The obvious solution is a thread pool. However, I need to make sure
that I download the files Async - so I can get percentage and status
information to my interface.
I have decided that the best way to do this is to have my Download (a
class representing the file to download) to have events raised when
they are finished. I was hoping to have my threads rejoin with the
thread pool when the downloads are finished.
However, I have my Download instances coming out of an
IEnumerable<Downloadthat is recieved from the WebExtractor class
(which parses the HTML) on-the-fly using "yield return".
I think I am lacking some basics about Thread Pools. How can I use a
thread pool and have the events fired by the Downloads still reach the
interface? Is there are way to add an event handler to an instance
while in a foreach or IEnumerator code block?
Any help would put me one step closer to being done with my second
release of the software. Thanks in advance!
~Travis
On 2007-11-27 20:24:28 -0800, "je**********@gmail.com"
<je**********@gmail.comsaid:
I have a rather complex need.Perhaps. Though, I suspect it''s more that you''ve created a complex
need, where it wasn''t really necessary to do so.
I have a class that parses web pages and extracts all relevant file
addresses. It allows me to download every pdf on a web page, for
instance. I would like to incorporate threads so that I can download N
files separately.A reasonably common operation.
The obvious solution is a thread pool. However, I need to make sure
that I download the files Async - so I can get percentage and status
information to my interface.It seems to me that a different "obvious" solution would be to just use
the async methods on the HttpWebRequest class, or even just a plain
TcpClient or Socket instance, along with a queue. The producer of the
queue would add URLs to be downloaded, while the consumer would keep
track of how many active downloads are going on (via HttpWebRequest,
TcpClient, or Socket).
Every time the producer adds something to the queue, it would signal
the consumer. The consumer in response would remove items from the
queue, stopping when either the queue is empty or your maximum number
of concurrent operations has been reached, whichever comes first.
Upon completion of an item, the consumer would also be signaled,
allowing it to pull a new item from the queue.
In the above, I''m thinking of the consumer and producer as individual
threads. But you could easily implement it without a thread dedicated
to either, with the consumer and producer classes simply being called
by whatever thread happens to be managing them at the time. In that
case, "signaling" the consumer would be more a matter of just executing
the method that attempts to dequeue more download operations.
I have decided that the best way to do this is to have my Download (a
class representing the file to download) to have events raised when
they are finished. I was hoping to have my threads rejoin with the
thread pool when the downloads are finished.If you use the async methods on the above-mentioned classes, you get
the thread pooling behavior for free.
However, I have my Download instances coming out of an
IEnumerable<Downloadthat is recieved from the WebExtractor class
(which parses the HTML) on-the-fly using "yield return".This is another reason I think a queue would be better. There''s no
technical reason you can''t implement an asynchronous enumerator, but
having done so in this case seems to have overcomplicated the issue. A
queue seems like a much more natural fit to me, and wouldn''t have the
same complicating factors you seem to be running into.
I think I am lacking some basics about Thread Pools. How can I use a
thread pool and have the events fired by the Downloads still reach the
interface?I think you can avoid the question altogether, but the basic answer is
that the idea of a thread pool and having "the events...reach the
interface" are orthogonal ideas. Because of the thread pool, you may
have thread synchronization issues to deal with. But the basic
question of raising an event in a way that some implementer of some
interface receives them isn''t affected by whether there are multiple
threads involved.
Is there are way to add an event handler to an instance
while in a foreach or IEnumerator code block?You can subscribe to an event at any time you find convenient.
Any help would put me one step closer to being done with my second
release of the software. Thanks in advance!See above. I recommend abandoning this asynchronous enumerator idea
and going with a nice, simple queue.
Pete
On Nov 27, 10:04 pm, Peter Duniho <NpOeStPe...@NnOwSlPiAnMk.com>
wrote:On 2007-11-27 20:24:28 -0800, "jehugalea...@gmail.com"
<jehugalea...@gmail.comsaid:
I have a rather complex need.
Perhaps. Though, I suspect it''s more that you''ve created a complex
need, where it wasn''t really necessary to do so.
I have a class that parses web pages and extracts all relevant file
addresses. It allows me to download every pdf on a web page, for
instance. I would like to incorporate threads so that I can download N
files separately.
A reasonably common operation.
The obvious solution is a thread pool. However, I need to make sure
that I download the files Async - so I can get percentage and status
information to my interface.
It seems to me that a different "obvious" solution would be to just use
the async methods on the HttpWebRequest class, or even just a plain
TcpClient or Socket instance, along with a queue. The producer of the
queue would add URLs to be downloaded, while the consumer would keep
track of how many active downloads are going on (via HttpWebRequest,
TcpClient, or Socket).
Every time the producer adds something to the queue, it would signal
the consumer. The consumer in response would remove items from the
queue, stopping when either the queue is empty or your maximum number
of concurrent operations has been reached, whichever comes first.
Upon completion of an item, the consumer would also be signaled,
allowing it to pull a new item from the queue.
In the above, I''m thinking of the consumer and producer as individual
threads. But you could easily implement it without a thread dedicated
to either, with the consumer and producer classes simply being called
by whatever thread happens to be managing them at the time. In that
case, "signaling" the consumer would be more a matter of just executing
the method that attempts to dequeue more download operations.
I have decided that the best way to do this is to have my Download (a
class representing the file to download) to have events raised when
they are finished. I was hoping to have my threads rejoin with the
thread pool when the downloads are finished.
If you use the async methods on the above-mentioned classes, you get
the thread pooling behavior for free.
However, I have my Download instances coming out of an
IEnumerable<Downloadthat is recieved from the WebExtractor class
(which parses the HTML) on-the-fly using "yield return".
This is another reason I think a queue would be better. There''s no
technical reason you can''t implement an asynchronous enumerator, but
having done so in this case seems to have overcomplicated the issue. A
queue seems like a much more natural fit to me, and wouldn''t have the
same complicating factors you seem to be running into.
I think I am lacking some basics about Thread Pools. How can I use a
thread pool and have the events fired by the Downloads still reach the
interface?
I think you can avoid the question altogether, but the basic answer is
that the idea of a thread pool and having "the events...reach the
interface" are orthogonal ideas. Because of the thread pool, you may
have thread synchronization issues to deal with. But the basic
question of raising an event in a way that some implementer of some
interface receives them isn''t affected by whether there are multiple
threads involved.
Is there are way to add an event handler to an instance
while in a foreach or IEnumerator code block?
You can subscribe to an event at any time you find convenient.
Any help would put me one step closer to being done with my second
release of the software. Thanks in advance!
See above. I recommend abandoning this asynchronous enumerator idea
and going with a nice, simple queue.
PeteMy first implementation actually had a Queue<Downloadthat was
consumed when I recieved that a download had finished. However, it was
difficult for my code to say, "Hey, stop trying to consume!" I ended
up having a very rigid code set and I was hoping to get away from it.
I was having BIG issues with the events of from one download finishing
interrupting while another thread was in the middle of a locked block.
I kept getting the occasional dead lock.
My hope in my new design was to get away from the need for so much
concurrency management. I did that by using the yield return statement
and making that my Queue, in a sense. It also makes the termination
point a lot easier to see. However, without a way of saying, "Hey,
we''re not ready to start downloading you yet - wait for a moment", I
was downloading as many files at once as my computer could handle. So
my hope was to find a way to say, "Hey wait" while not needing to
necessarily manage the number of threads/concurrent downloads
manually.
I could try to manage the downloads manually again. I did move a lot
of code around to separate the interface from the downloading, so it
might be easier now than before. ThreadPools seemed more intuitive to
me the second time around. Perhaps my first approach is the better
one.
Thanks for your thoughts,
Travis
On 2007-11-27 21:23:40 -0800, "je**********@gmail.com"
<je**********@gmail.comsaid:
My first implementation actually had a Queue<Downloadthat was
consumed when I recieved that a download had finished. However, it was
difficult for my code to say, "Hey, stop trying to consume!"Typically with a queue, that point is when the queue is empty. It''s
not usually difficult.
I ended
up having a very rigid code set and I was hoping to get away from it.
I was having BIG issues with the events of from one download finishing
interrupting while another thread was in the middle of a locked block.
I kept getting the occasional dead lock.Well, for what it''s worth you seem to be dealing with threading issues
anyway. Dead lock is a consequence of a buggy implementation. If you
had trouble dealing with thread synchronization in the previous design,
you''re likely to have trouble with any other design that also involves
threads.
My hope in my new design was to get away from the need for so much
concurrency management.How you intended to do that by introducing your own thread pool, I''m
not really clear on. :)
I did that by using the yield return statement
and making that my Queue, in a sense. It also makes the termination
point a lot easier to see. However, without a way of saying, "Hey,
we''re not ready to start downloading you yet - wait for a moment", I
was downloading as many files at once as my computer could handle. So
my hope was to find a way to say, "Hey wait" while not needing to
necessarily manage the number of threads/concurrent downloads
manually.Managing that with a the queue/async paradigm I mentioned would be
simple. Especially given the efficiency advantages of using the async
i/o methods on the network classes, it seems to me that managing the
concurrent consumer count by creating your own thread pool is much more
complicated and error-prone.
I''d say the puzzlement you appear to have put yourself into here is a
good indication of that. :)
I could try to manage the downloads manually again. I did move a lot
of code around to separate the interface from the downloading, so it
might be easier now than before. ThreadPools seemed more intuitive to
me the second time around. Perhaps my first approach is the better
one.If it''s like what I suggested, obviously I''d agree. :)
Pete
这篇关于foreach，IEnumerable和修改内容的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
