为什么 Clojure 在执行我的计算后挂起? [英] Why does Clojure hang after having performed my calculations?

问题描述

我正在尝试并行过滤元素.对于每个元素，我需要执行距离计算，以查看它是否足够接近目标点.不要介意已经存在用于执行此操作的数据结构，我现在只是在做初步实验.

无论如何，我想运行一些非常基本的实验，在其中生成随机向量并对其进行过滤.这是我的实现，可以完成所有这些

(defn pfilter [pred coll](地图第二(先过滤(pmap (fn [item] [(pred item) item]) coll))))(定义随机n向量[n](取n(反复兰特)))(定义距离 [u v](Math/sqrt (reduce + (map #(Math/pow (- %1 %2) 2) u v))))(defn -main [& args](让 [[n-str 向量-str 阈值-str] argsn (Integer/parseInt n-str)向量(Integer/parseInt 向量-str)阈值(双/解析双阈值-str)随机向量(部分随机 n 向量 n)u(随机向量)](时间(打印 n 个向量(数数(过滤器(fn [v] (< (距离 u v) 阈值))(取向量(重复随机向量))))))))

代码执行并返回我期望的值，即参数 n(向量的长度)、向量(向量的数量)以及比阈值更接近目标向量的向量的数量.我不明白的是为什么程序在终止前会再挂一分钟.

这是一个运行的输出，演示了错误

<前>$ time lein run 10 100000 1.0[空] 10 100000 12283[null]经过的时间:3300.856 毫秒"真正的 1m6.336s用户 0m7.204s系统 0m1.495s

任何关于如何并行过滤的评论也非常受欢迎，因为我还没有确认 pfilter 是否真的有效.

解决方案

您需要调用 shutdown-agents 来终止支持 pmap 使用的线程池的线程.

关于 pfilter，它应该可以工作，但运行速度比 filter 慢，因为你的谓词很简单.并行化不是免费的，因此您必须为每个线程分配中等强度的任务以抵消多线程开销.过滤之前先对项目进行批处理.

I'm experimenting with filtering through elements in parallel. For each element, I need to perform a distance calculation to see if it is close enough to a target point. Never mind that data structures already exist for doing this, I'm just doing initial experiments for now.

Anyway, I wanted to run some very basic experiments where I generate random vectors and filter them. Here's my implementation that does all of this

(defn pfilter [pred coll]
  (map second
    (filter first
      (pmap (fn [item] [(pred item) item]) coll))))

(defn random-n-vector [n]
  (take n (repeatedly rand)))

(defn distance [u v]
  (Math/sqrt (reduce + (map #(Math/pow (- %1 %2) 2) u v))))

(defn -main [& args]
  (let [[n-str vectors-str threshold-str] args
        n (Integer/parseInt n-str)
        vectors (Integer/parseInt vectors-str)
        threshold (Double/parseDouble threshold-str)
        random-vector (partial random-n-vector n)
        u (random-vector)]
    (time (println n vectors 
      (count 
        (pfilter 
          (fn [v] (< (distance u v) threshold))
          (take vectors (repeatedly random-vector))))))))

The code executes and returns what I expect, that is the parameter n (length of vectors), vectors (the number of vectors) and the number of vectors that are closer than a threshold to the target vector. What I don't understand is why the programs hangs for an additional minute before terminating.

Here is the output of a run which demonstrates the error

$ time lein run 10 100000 1.0
     [null] 10 100000 12283
     [null] "Elapsed time: 3300.856 msecs"

real    1m6.336s
user    0m7.204s
sys 0m1.495s

Any comments on how to filter in parallel in general are also more than welcome, as I haven't yet confirmed that pfilter actually works.

解决方案

You need to call shutdown-agents to kill the threads backing the threadpool used by pmap.

About pfilter, it should work but run slower than filter, since your predicate is simple. Parallelization isn't free so you have to give each thread moderately intensive tasks to offset the multithreading overhead. Batch your items before filtering them.

这篇关于为什么 Clojure 在执行我的计算后挂起?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！