使用Golang读取csv，对列进行重新排序，然后使用并发将结果写入新的csv [英] Using Golang to read csv, reorder columns then write result to a new csv with Concurrency

问题描述

这是我的出发点.

这是一个Golang脚本，用于读取具有3列的csv，重新排序各列并将结果写入新的csv文件中.

It is a Golang script to read in a csv with 3 columns, re-order the columns and write the result to a new csv file.

package main

import (
   "fmt"
   "encoding/csv"
   "io"
   "os"
   "math/rand"
   "time"
)

func main(){
  start_time := time.Now()

  // Loading csv file
  rFile, err := os.Open("data/small.csv") //3 columns
  if err != nil {
    fmt.Println("Error:", err)
    return
   }
  defer rFile.Close()

  // Creating csv reader
  reader := csv.NewReader(rFile)

  lines, err := reader.ReadAll()
  if err == io.EOF {
      fmt.Println("Error:", err)
      return
  }

  // Creating csv writer
  wFile, err := os.Create("data/result.csv")
  if err != nil {
      fmt.Println("Error:",err)
      return
  }
  defer wFile.Close()
  writer := csv.NewWriter(wFile)

  // Read data, randomize columns and write new lines to results.csv
  rand.Seed(int64(time.Now().Nanosecond()))
  var col_index []int
  for i,line :=range lines{
      if i == 0 {
        //randomize column index based on the number of columns recorded in the 1st line
        col_index = rand.Perm(len(line))
    }
    writer.Write([]string{line[col_index[0]], line[col_index[1]], line[col_index[2]]}) //3 columns
    writer.Flush()
}

//print report
fmt.Println("No. of lines: ",len(lines))
fmt.Println("Time taken: ", time.Since(start_time))

}

问题:

1. 我的代码是Golang惯用的吗?

1. Is my code idiomatic for Golang?

如何在此代码中添加并发?

How can I add concurrency to this code?

推荐答案

您的代码正常.并发的情况不多.但是您至少可以减少运行时的内存消耗重新排序.只需使用 Read()而不是 ReadAll()来避免为​​孔输入文件分配切片.

Your code is OK. There are no much case for concurrency. But you can at least reduce memory consumption reordering on the fly. Just use Read() instead of ReadAll() to avoid allocating slice for hole input file.

for line, err := reader.Read(); err == nil; line, err = reader.Read(){
    if err = writer.Write([]string{line[col_index[0]], line[col_index[1]], line[col_index[2]]}); err != nil {
            fmt.Println("Error:", err)
            break
    }
    writer.Flush()
}

这篇关于使用Golang读取csv，对列进行重新排序，然后使用并发将结果写入新的csv的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！