panic：运行时错误：当并行作为goroutine运行时，slice限制超出范围 [英] panic: runtime error: slice bounds out of range when concurrently running as goroutine

问题描述

我将一个函数作为一个goroutine调用，并使用一个WaitGroup来阻止共享扫描器在完成之前关闭。 myfunc（）函数迭代文件。我想记忆映射这个文件并在所有goroutine之间共享它，而不是每次都有从磁盘读取的I / O chokepoint。有人告诉我，这种方法可以解决另一个问题的答案。然而，虽然这个功能独立运行，但它不是兼职工作。我收到错误：

  panic：运行时错误：切片范围超出范围
  

但错误是当我调用 Scan（）方法时（不在片上）这是令人困惑的。

这是一个MWE：

  //包装声明;进口; yada yada 
 
 //实际尺寸图更有意义，这仅仅是MWE 
 var Sizes = map [int] string {
 10：Ten， 
 20：Twenty，
 30：三十，
 40：四十，
} 
 
类型FileScanner结构{
 io.Closer 
 * bufio.Scanner 
} 
 
 func main（）{
 // ...验证存储在filePath变量中的文件的路径
 filePath：=/path/to/file.txt
 
 //获取单词列表扫描器以便在goroutines之间共享
 scanner：= getScannerPtr（&filePath）
 
 //为每个参数调用myfunc（）传递
 var wg sync.WaitGroup $ b $ ch：= make（chan string）
 for _，param：= range os.Args [ 1：] {
 wg.Add（1）
 go myfunc（& param，scanner，ch）
 wg.Done（）
} 
 
 //打印从频道
收到的范围为os.Args的结果[1：] {
 fmt.Println（< -ch）//打印从频道收到的数据ch 
 } 
 
 //在所有goroutines完成之前，不要关闭扫描器
 wg.Wait（）
 defer scanner.Close（）
} 
 
 func getScannerPtr（filePath * string）* FileScanner {
f，err：= os.Open（* filePath）
 if err！= nil {
 fmt.Fprint（os.Stderr， 错误打开文件\）
恐慌（错误）
} 
扫描器：= bufio.NewScanner（f）
返回& FileScanner {f，扫描器} 
 
 $ b func myfunc（param * string，scanner * FileScanner，ch chan< -string）{
 for scanner.Scan（）{
 line：= strings.TrimSpace （scanner.Text（））
 // ...使用行进行某些操作（只读）
 // ...访问共享执行此操作时的大小映射（只读）
 ch < ;  - 一些字符串结果在这里
} 
} 
  

认为这个问题是对共享大小映射的并发访问，但是它在 myfunc（）中移动（并且低效地重新声明/每次重新定义它）仍导致相同的错误，这与调用 Scan（）有关。我试图按照我收到的指导在这个答案中。

  panic：运行时错误：切片边界超出范围
 
 goroutine 6 [running]：
 bufio。（* Scanner）.Scan（0xc42008a000，0x80）
 /usr/local/go/src/bufio/scan.go:139+ 0xb3e 
 main.crack（0xc42004c280,0xc42000a080,0xc42001c0c0）
 /Users/dan/go/src/crypto_ctf_challenge/main.go:113+ 0x288 
由main.main创建
 / Users /dan/go/src/crypto_ctf_challenge/main.go:81 + 0x1d8 
退出状态2 
  

<第81行是：

  go myfunc（& param，scanner，ch）
  
 $ b> 
 
 
第113行是： 
 
 
  .Scan（）{
  
 
 
解决方案
 code>扫描来源，它似乎不是threa d安全。您可以通过从扫描器中读取一个例程来解决这个问题，而其他任何例程都会消耗行并处理它们： 
 
 
  func main（）{
 // ...验证存放在filePath变量中的文件的路径
 filePath：=/path/to/file.txt
 
 //获取单词列表扫描器以便在goroutines之间共享
 scanner：= getScannerPtr（&filePath）
 defer scanner.Close（）
 
 //为每个传递的参数调用myfunc（） 
 var wg sync.WaitGroup $ b $ ch：make（chan string）
 lines：= make（chan string）
去func（）{
 for scanner.Scan （）{
 lines<  -  scanner.Text（）
} 
 close（lines）
（）
 for _，param：= range os.Args [1：] {
 wg.Add（1）
 go myfunc（param，lines，ch）
 wg.Done（）
} 
 
 //打印从频道
收到的范围为os.Args的结果[1：] {
 fmt.Println（&l t; -ch）//打印从频道接收到的数据ch 
} 
 
 //在所有goroutines完成之前不关闭扫描器
 wg.Wait（）
} 
 
 func myfunc（param string，chan [] byte，ch chan< -string）{
 for line：= range lines {
 line = strings.TrimSpace（行）
 // ...用行（只读）执行某些操作
 // ...访问共享执行操作时的大小映射（只读）
 ch < - 一些字符串结果在这里

} 
  
另外请注意， 延迟函数中的最后一行;整个 defer 点就是在函数体内的某处调用它，并知道函数返回后会调用它。由于您使用 WaitGroup 来防止函数返回，直到完成扫描程序，所以您可以安全地推迟关闭。
 
I'm calling a function as a goroutine and using a WaitGroup to prevent closing a shared Scanner before they all finish. The myfunc() function iterates over a file. I wanted to memory-map this file and share it between all of the goroutines rather than have the I/O chokepoint of reading from disk each time. I was told that this approach would work in an answer to another question. However, while this function worked fine standalone, it's not working concurrently. I am receiving the error:
panic: runtime error: slice bounds out of range
but the error is when I call the Scan() method (not on a slice) which is confusing.

Here is a MWE:
// ... package declaration; imports; yada yada

// the actual Sizes map is much more meaningful, this is just for the MWE
var Sizes = map[int]string {
    10: "Ten",
    20: "Twenty",
    30: "Thirty",
    40: "Forty",
}

type FileScanner struct {
    io.Closer
    *bufio.Scanner
}

func main() {
    // ... validate path to file stored in filePath variable
    filePath := "/path/to/file.txt"

    // get word list scanner to be shared between goroutines
    scanner := getScannerPtr(&filePath)

    // call myfunc() for each param passed
    var wg sync.WaitGroup
    ch := make(chan string)
    for _, param := range os.Args[1:] {
        wg.Add(1)
        go myfunc(&param, scanner, ch)
        wg.Done()
    }

    // print results received from channel
    for range os.Args[1:] {
        fmt.Println(<-ch)  // print data received from channel ch
    }

    // don't close scanner until all goroutines are finished
    wg.Wait()
    defer scanner.Close()
}

func getScannerPtr(filePath *string) *FileScanner {
    f, err := os.Open(*filePath)
    if err != nil {
        fmt.Fprint(os.Stderr, "Error opening file\n")
        panic(err)
    }
    scanner := bufio.NewScanner(f)
    return &FileScanner{f, scanner}
}

func myfunc(param *string, scanner *FileScanner, ch chan<-string) {
    for scanner.Scan() {
        line := strings.TrimSpace(scanner.Text())
        // ... do something with line (read only)
        // ... access shared Sizes map when doing it (read only)
        ch <- "some string result goes here"
    }
}
I originally thought the issue was concurrent access to the shared Sizes map, but moving it inside myfunc() (and inefficiently redeclaring/redefining it every time) still resulted in the same error, which has to do with calling Scan(). I'm attempting to follow the guidance I received in this answer.

Here's the full stack trace of the panic:
panic: runtime error: slice bounds out of range

goroutine 6 [running]:
bufio.(*Scanner).Scan(0xc42008a000, 0x80)
        /usr/local/go/src/bufio/scan.go:139 +0xb3e
main.crack(0xc42004c280, 0xc42000a080, 0xc42001c0c0)
        /Users/dan/go/src/crypto_ctf_challenge/main.go:113 +0x288
created by main.main
        /Users/dan/go/src/crypto_ctf_challenge/main.go:81 +0x1d8
exit status 2
Line 81 is:
go myfunc(&param, scanner, ch)
Line 113 is:
for scanner.Scan() {

解决方案
Actually after review of the Scan source, it doesn't appear to be thread-safe. You can get around this by having one routine read off of the scanner, and any number of other routines consume lines and process them:
func main() {
    // ... validate path to file stored in filePath variable
    filePath := "/path/to/file.txt"

    // get word list scanner to be shared between goroutines
    scanner := getScannerPtr(&filePath)
    defer scanner.Close()

    // call myfunc() for each param passed
    var wg sync.WaitGroup
    ch := make(chan string)
    lines := make(chan string)
    go func() {
        for scanner.Scan() {
            lines <- scanner.Text()
        }
        close(lines)
    }()
    for _, param := range os.Args[1:] {
        wg.Add(1)
        go myfunc(param, lines, ch)
        wg.Done()
    }

    // print results received from channel
    for range os.Args[1:] {
        fmt.Println(<-ch)  // print data received from channel ch
    }

    // don't close scanner until all goroutines are finished
    wg.Wait()
}

func myfunc(param string, lines chan []byte, ch chan<-string) {
    for line := range lines {
        line = strings.TrimSpace(line)
        // ... do something with line (read only)
        // ... access shared Sizes map when doing it (read only)
        ch <- "some string result goes here"
    }
}
Also note that there's no point in defering the last line in a function; the whole point of defer is to call it somewhere in the body of the function and know it will be called after the function returns. Since you're using a WaitGroup to prevent the function returning until you're done with your scanner, you can safely defer the close immediately.

                        
这篇关于panic：运行时错误：当并行作为goroutine运行时，slice限制超出范围的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！