是clojure的读取文件结构，即with-open和clojure.java.io/reader，足够有效的频繁访问？ [英] Is clojure's read file structure, i.e. with-open and clojure.java.io/reader, efficient enough for frequent access?

问题描述

假设我写了一个函数来解析来自txt文件的数据，该文件包含 with-open和clojure .java.io / reader，，然后我写了另一个函数来调用reader函数多次，以便处理数据，例如

 （defn grabDataFromFile [file patternString] 
（let [data（atom []）] 
（with-open [rdr（clojure.java.io/reader file）] 
（doseq [line（line-seq rdr）] 
（if（重新匹配（重新模式patternString）行）（swap！数据共轭）））
 @data））
 
 
（defn myCalculation [file] 
（let [data1（grabDataFromFile filepattern1）
 data2（grabDataFromFile filepattern2）
 data3 grabDataFromFile filepattern3）] 
; data1，data2，data3 ....的计算或过程））
  

b $ b

我的问题是，在这个myCalculation函数中，是底层代码聪明地足以使用clojure读取器打开文件一次，并获得一次性所需的所有数据？或者它打开和关闭文件的次数与函数grabDataFromFile的调用次数？ （在这个例子中，3）

后续问题是，如果读者不够聪明， c> grabDataFromFile 解析方法 将在每次被调用时打开和关闭读取器（在退出时）。

让 grabDataFromFile c $ c>接受另一个函数，它是每个行上操作的解析器逻辑（或者它可以是任何你想在每行上执行的函数）

 （defn grabDataFromFile [file patternString process-fn] 
（with-open [rdr（clojure.java.io/reader file）] 
（doseq [line（line- seq rdr）] 
（process-fn line））））
 
 
 
 
（defn myCalculation [file] 
（let [模式[pattern] [b] [b] [b] [b] [b] [b] [b] （fn [[p data]] 
（fn [line] 
（if（重新匹配（重新模式p）行）
 ）
 pattern-fn（apply juxt pattern-fns）] 
（grabDataFromFile文件模式-fn）
;对模式原子执行计算
））
  




Suppose I write a function to parse data from a txt file with with-open and clojure.java.io/reader, and then I wrote another function to call the reader function multiple of times in order to process data, e.g.

(defn grabDataFromFile [file patternString]   
     (let [data (atom [])]
        (with-open [rdr (clojure.java.io/reader file)] 
         (doseq [line (line-seq rdr)] 
           (if  (re-matches  (re-pattern  patternString) line) (swap! data conj line))))
        @data))


(defn myCalculation [file ]
  (let [data1 (grabDataFromFile file "pattern1")
        data2 (grabDataFromFile file "pattern2")
        data3 (grabDataFromFile file "pattern3")]
    ;calculations or processes of data1, data2, data3....))

My question is, inside this myCalculation function, is the underlying code smart enough to open the file just once with clojure reader, and get all data needed in one shot? Or does it open and close the file as many times as number of calls for function grabDataFromFile ? ( In this example, 3)

A follow up question would be, what can I do to speed up if the reader is not smart enough, and if I have to intentionally separate "parser" code with "processing" code?

解决方案

grabDataFromFile will open and close reader (on exit) every time it is called. The underlying code cannot be that smart such that a function can detect the context of its caller without some explicitly provided information.

Make grabDataFromFile to accept another function which is your parser logic that operates on each line (or it could be any function that you want to perform on each line)

(defn grabDataFromFile [file patternString process-fn]   
  (with-open [rdr (clojure.java.io/reader file)] 
    (doseq [line (line-seq rdr)] 
      (process-fn line))))




(defn myCalculation [file]
  (let [patterns [["pattern1" (atom [])]
                  ["pattern2" (atom [])]
                  ["pattern3" (atom [])]]
        pattern-fns (map (fn [[p data]]
                           (fn [line]
                             (if  (re-matches (re-pattern  p) line)                              
                               (swap! data conj line)))) patterns)
        pattern-fn (apply juxt pattern-fns)]
    (grabDataFromFile file pattern-fn)
    ;perform calc on patterns atoms
    ))

这篇关于是clojure的读取文件结构，即with-open和clojure.java.io/reader，足够有效的频繁访问？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！