并发写入文件 [英] Concurrent writing to a file

问题描述

在执行过程中，如何控制并发写入文本文件?

In go, how can I control the concurrent writing to a text file?

我之所以这样问，是因为我将使用同一个文件处理程序将多个goroutine写入文本文件.

I ask this because I will have multiple goroutines writing to a text file using the same file handler.

我写了这段代码来尝试看看会发生什么，但是我不确定是否做到了正确":

I wrote this bit of code to try and see what happens but I'm not sure if I did it "right":

package main

import (
    "os"
    "sync"
    "fmt"
    "time"
    "math/rand"
    "math"
)


func WriteToFile( i int, f *os.File, w *sync.WaitGroup ){
    //sleep for either 200 or 201 milliseconds
    randSleep := int( math.Floor( 200 + ( 2 * rand.Float64() ) ) )
    fmt.Printf( "Thread %d waiting %d\n", i, randSleep )
    time.Sleep( time.Duration(randSleep) * time.Millisecond )

    //write to the file
    fmt.Fprintf( f, "Printing out: %d\n", i )
    //write to stdout
    fmt.Printf( "Printing out: %d\n", i )
    w.Done()
}

func main() {
    rand.Seed( time.Now().UnixNano() )

    d, err := os.Getwd()
    if err != nil {
        fmt.Println( err )
    }
    filename := d + "/log.txt"

    f, err := os.OpenFile( filename, os.O_CREATE | os.O_WRONLY | os.O_TRUNC, 0666 )

    if err != nil {
        fmt.Println( err )
    }
    var w *sync.WaitGroup = new(sync.WaitGroup)
    w.Add( 10 )

    //start 10 writers to the file
    for i:=1; i <= 10; i++ {
        go WriteToFile( i, f, w )
    }

    //wait for writers to finish
    w.Wait()

}

我一半希望输出在文件中显示类似这样的内容，而不是我得到的连贯输出:

I half expected that the output would show something like this in the file instead of the coherent output I got:

Printing Printing out: 2
out: 5
Poriuntitng: 6

从本质上讲，我希望角色能够不连贯地出现，并且由于缺乏同步而交织在一起.我是否没有编写可以哄骗这种行为的代码?还是在调用 fmt.Fprintf 同步编写过程中有某种机制?

Essentially, I expected the characters to come out incoherently and interweaved due to a lack of synchronization. Did I not write code that would coax this behavior out? Or is some mechanism during calls to fmt.Fprintf synchronizing the writing?

推荐答案

有很多方法可以控制并发访问.最简单的方法是使用 Mutex :

There are many ways to control concurrent access. The easiest is to use a Mutex:

var mu sync.Mutex

func WriteToFile( i int, f *os.File, w *sync.WaitGroup ){
    mu.Lock()
    defer mu.Unlock()
    // etc...
}

关于为什么看不到问题，Go使用操作系统调用来实现文件访问，而这些系统调用是

As to why you're not seeing problems, Go uses operating system calls to implement file access, and those system calls are thread safe (emphasis added):

根据POSIX.1-2008/SUSv4第XSI 2.9.7节(与常规文件操作的线程交互"):

According to POSIX.1-2008/SUSv4 Section XSI 2.9.7 ("Thread Interactions with Regular File Operations"):

以下所有功能相对于彼此在POSIX.1-2008中指定的效果时对常规文件或符号链接进行操作:...

All of the following functions shall be atomic with respect to each other in the effects specified in POSIX.1-2008 when they operate on regular files or symbolic links: ...

随后列出的API中有write()和writev(2).和在线程间应该是原子性的影响之中(以及进程)是文件偏移量的更新.但是，在Linux之前版本3.14则不是这样:如果两个进程共享一个打开文件描述(请参见open(2))执行write()(或writev(2))同时，那么I/O操作不是原子的尊重更新文件偏移量，结果是两个进程输出的数据可能(不正确)重叠.此问题已在Linux 3.14中修复.

Among the APIs subsequently listed are write() and writev(2). And among the effects that should be atomic across threads (and processes) are updates of the file offset. However, on Linux before version 3.14, this was not the case: if two processes that share an open file description (see open(2)) perform a write() (or writev(2)) at the same time, then the I/O operations were not atomic with respect updating the file offset, with the result that the blocks of data output by the two processes might (incorrectly) overlap. This problem was fixed in Linux 3.14.

尽管如此，我仍然会使用锁，因为Go代码不会自动进行线程安全.(两个goroutine修改同一个变量会导致奇怪的行为)

I would still use a lock though, since Go code is not automatically thread safe. (two goroutines modifying the same variable will result in strange behavior)

这篇关于并发写入文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！