Java：如何通过线程同步文件修改 [英] Java: how to synchronize file modification by threads
问题描述
我不知道要使用哪种文件锁定或同步方法。我从来没有在Java中做过文件锁定，我没有太多的Java或编程经验。
我看了一下java的NIO，我读到：文件锁是代表整个Java虚拟机进行保存的，它们不适合用来控制对多个文件的访问同一个虚拟机内的线程。马上我知道我需要专家的帮助，因为这是生产代码，我几乎不知道我在做什么（而且我必须在今天完成）
这是我的代码上传一些东西（档案文件）到服务器的简要概述。它从文件中获取要从文件上传的文件列表（称为listFile） - 当这个方法正在读取的时候listFile可以被修改。我通过将listFile复制到临时文件并在其后使用该临时文件来最大限度地减少这种情况的机会。但我想我需要在这个复制过程中锁定文件（或类似的东西）。
package myPackage;
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;
import com.example.my.FileHelper;
import com.example.my.Logger;
$ b $ public class BatchUploader implements Runnable {
$ b $ private int processUploads（）{
File myFileToUpload;
文件copyOfListFile = null;
try {
copyOfListFile = new File（/ path / to / temp / workfile）;
文件origFile =新文件（/ path / to / listFile）; //listFile - 包含要上传文件列表的文件
DataWriter.copyFile（origFile，copyOfListFile）; //查看下面的代码
} catch（IOException ex）{
Logger .LOG（前）;
尝试{
BufferedReader input = new BufferedReader（new FileReader（copyOfListFile））;
while（！stopRunning&& amp;（fileToUploadName = input.readLine（））！= null）{
upload（new File（fileToUploadName））;
}
} finally {
input.close（）;
isUploading = false;
}
}
return filesUploadedCount;
$ / code $ / pre
这是将文件列表修改为在上面的代码中使用：
public class DataWriter {
public void modifyListOfFilesToUpload（String uploadedFilename ）{
StringBuilder content = new StringBuilder（）;
尝试{
文件listOfFiles =新文件（/ path / to / listFile）;如果（！listOfFiles.exists（））{
//一些代码
}
BufferedReader输入=新的文件BufferedReader（new FileReader（listOfFiles））;
尝试{
String line =; （（line = input.readLine（））！= null）{
if（！line.isEmpty（）& line.endsWith（FILE_EXTENSION））{
if（$ ！line.contains（uploadedFilename））{
content.append（String.format（％1 $ s％n，line））;
} else {
// some code
}
} else {
// some code
}
}
}最后{
input.close（）;
}
this.write（/ path / to /，listFile，content.toString（），false，false，false）;
} catch（IOException ex）{
Logger.debug（Error reading / writing uploads logfile：+ ex.getMessage（））;
$ b $ public static void copyFile（File in，File out）throws IOException {
FileChannel inChannel = new FileInputStream（in）.getChannel（）;
FileChannel outChannel = new FileOutputStream（out）.getChannel（）;
尝试{
inChannel.transferTo（0，inChannel.size（），outChannel）;
} catch（IOException e）{
throw e;
} finally {
if（inChannel！= null）{
inChannel.close（）; $（$ out）
（outChannel！= null）{
outChannel.close（）;
$ b private void write（String path，String fileName，String data，boolean append，boolean addNewLine，boolean doLog）{
尝试{
File file = FileHelper.getFile（fileName，path）;
BufferedWriter bw = new BufferedWriter（new FileWriter（file，append））;
bw.write（data）;
if（addNewLine）{
bw.newLine（）;
}
bw.flush（）;
bw.close（）;
if（doLog）{
Logger.debug（String.format（Wired％1 $ s％2 $ s，path，fileName））;
}
} catch（java.lang.Exception ex）{
Logger.log（ex）;
$ b 解决方案我建议采用稍微不同的方法。 Linux上的Afair文件重命名（mv）操作在本地磁盘上是原子的。一个进程没有机会看到半写文件。
让XXX是一个包含三个（或更多）数字的序列号。你可以让你的DataWriter追加到一个名为listFile-XXX.prepare的文件中，并在其中写入一个固定的N个文件名。当写入N个名字时，关闭该文件并将其重命名为atomicFile-XXX。使用下一个文件名，开始写入listFile-YYY，其中YYY = XXX + 1。
您的BatchUploader可以在任何时候检查是否找到匹配模式的文件listFile-XXX ，打开它们，阅读它们上传指定的文件，关闭并删除它们。
实现提示：
$ b $这些线程没有机会混淆彼此的文件。 b - 确保在BatchUploader中使用轮询机制，如果没有找到可供上传的文件（等待空闲等待），则等待1秒或更长时间。
- 您可能想要请务必按照XXX对listFile-XXX进行排序，以确保上传顺序保持不变。
- 当然，您可以在listFile-XXX.prepare关闭时改变协议。如果DataWriter长时间没有任何功能，则不想让文件准备好上传，因为还没有准备好。
好处：不锁定（这将是一个很难得到的权利），不需要复制，简单地浏览整个工作队列，并且在文件系统中处理状态。
Only one instance of my Java application can run at a time. It runs on Linux. I need to ensure that one thread doesn't modify the file while the other thread is using it.
I don't know which file locking or synchronization method to use. I have never done file locking in Java and I don't have much Java or programming experience.
I looked into java NIO and I read that "File locks are held on behalf of the entire Java virtual machine. They are not suitable for controlling access to a file by multiple threads within the same virtual machine." Right away I knew that I needed expert help because this is production code and I have almost no idea what I'm doing (and I have to get it done today).
Here's a brief outline of my code to upload some stuff (archive files) to a server. It gets the list of files to upload from a file (call it "listFile") -- and listFile can be modified while this method is reading from it. I minimize the chances of that by copying listFile to a temp file and using that temp file thereafter. But I think I need to lock the file during this copy process (or something like that).
package myPackage;
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;
import com.example.my.FileHelper;
import com.example.my.Logger;
public class BatchUploader implements Runnable {
private int processUploads() {
File myFileToUpload;
File copyOfListFile = null;
try {
copyOfListFile = new File("/path/to/temp/workfile");
File origFile = new File("/path/to/listFile"); //"listFile" - the file that contains a list of files to upload
DataWriter.copyFile(origFile, copyOfListFile);//see code below
} catch (IOException ex) {
Logger.log(ex);
}
try {
BufferedReader input = new BufferedReader(new FileReader(copyOfListFile));
try {
while (!stopRunning && (fileToUploadName = input.readLine()) != null) {
upload(new File(fileToUploadName));
}
} finally {
input.close();
isUploading = false;
}
}
return filesUploadedCount;
}
}
Here is the code that modifies the list of files to be uploaded used in the above code:
public class DataWriter {
public void modifyListOfFilesToUpload(String uploadedFilename) {
StringBuilder content = new StringBuilder();
try {
File listOfFiles = new File("/path/to/listFile"); //file that contains a list of files to upload
if (!listOfFiles.exists()) {
//some code
}
BufferedReader input = new BufferedReader(new FileReader(listOfFiles));
try {
String line = "";
while ((line = input.readLine()) != null) {
if (!line.isEmpty() && line.endsWith(FILE_EXTENSION)) {
if (!line.contains(uploadedFilename)) {
content.append(String.format("%1$s%n", line));
} else {
//some code
}
} else {
//some code
}
}
} finally {
input.close();
}
this.write("/path/to/", "listFile", content.toString(), false, false, false);
} catch (IOException ex) {
Logger.debug("Error reading/writing uploads logfile: " + ex.getMessage());
}
}
public static void copyFile(File in, File out) throws IOException {
FileChannel inChannel = new FileInputStream(in).getChannel();
FileChannel outChannel = new FileOutputStream(out).getChannel();
try {
inChannel.transferTo(0, inChannel.size(), outChannel);
} catch (IOException e) {
throw e;
} finally {
if (inChannel != null) {
inChannel.close();
}
if (outChannel != null) {
outChannel.close();
}
}
}
private void write(String path, String fileName, String data, boolean append, boolean addNewLine, boolean doLog) {
try {
File file = FileHelper.getFile(fileName, path);
BufferedWriter bw = new BufferedWriter(new FileWriter(file, append));
bw.write(data);
if (addNewLine) {
bw.newLine();
}
bw.flush();
bw.close();
if (doLog) {
Logger.debug(String.format("Wrote %1$s%2$s", path, fileName));
}
} catch (java.lang.Exception ex) {
Logger.log(ex);
}
}
}
解决方案 My I suggest a slightly different approach. Afair on Linux the file rename (mv) operation is atomic on local disks. No chance for one process to see a 'half written' file.
Let XXX be a sequence number with three (or more) digits. You could let your DataWriter append to a file called listFile-XXX.prepare and write a fixed number N of filenames into it. When N names are written, close the file and rename it (atomic, see above) to listFile-XXX. With the next filename, start writing to listFile-YYY where YYY=XXX+1.
Your BatchUploader may at any time check whether it finds files matching the pattern listFile-XXX, open them, read them upload the named files, close and delete them. There is no chance for the threads to mess up each other's file.
Implementation hints:
- Make sure to use a polling mechanism in BatchUploader that waits 1 or more seconds if it does not find a file ready for upload (prevent idle wait).
- You may want to make sure to sort the listFile-XXX according to XXX to make sure the uploading is kept in sequence.
- Of course you could variate the protocol of when listFile-XXX.prepare is closed. If DataWriter has nothing to do for a longer time, you don't want to have files ready for upload hang around just because there are not yet N ready.
Benefits: no locking (which will be a pain to get right), no copying, easy overview over the work queue and it state in the file system.
这篇关于Java：如何通过线程同步文件修改的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
