Haskell的Control.Concurrent.Async.mapConcurrent可以有限制吗? [英] Can Haskell's Control.Concurrent.Async.mapConcurrently have a limit?

问题描述

我试图在Haskell中并行运行多个下载，通常只使用Control.Concurrent.Async.mapConcurrently函数.但是，这样做会打开约3000个连接，这将导致Web服务器拒绝所有连接.是否可以同时完成与mapConcurrent相同的任务，但是一次只能打开有限数量的连接(即一次只能打开2或4个连接)?

I'm attempting to run multiple downloads in parallel in Haskell, which I would normally just use the Control.Concurrent.Async.mapConcurrently function for. However, doing so opens ~3000 connections, which causes the web server to reject them all. Is it possible to accomplish the same task as mapConcurrently, but only have a limited number of connections open at a time (i.e. only 2 or 4 at a time)?

推荐答案

一种快速的解决方案是使用信号量以限制并发操作的数量.这不是最佳选择(所有线程都立即创建然后等待)，但是可以正常工作:

A quick solution would be to use a semaphore to restrict the number of concurrent actions. It's not optimal (all threads are created at once and then wait), but works:

import Control.Concurrent.MSem
import Control.Concurrent.Async
import Control.Concurrent (threadDelay)
import qualified Data.Traversable as T

mapPool :: T.Traversable t => Int -> (a -> IO b) -> t a -> IO (t b)
mapPool max f xs = do
    sem <- new max
    mapConcurrently (with sem . f) xs

-- A little test:
main = mapPool 10 (\x -> threadDelay 1000000 >> print x) [1..100]

这篇关于Haskell的Control.Concurrent.Async.mapConcurrent可以有限制吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！