在Linux bash脚本中使用tee随机更新文件失败 [英] updating a file using tee randomly fails in linux bash script

问题描述

使用 sed -e 更新配置文件的某些参数并将其通过管道传输到 |时，tee (将更新的内容写入文件)，这会随机中断并导致文件无效(大小为0).

when using sed -e to update some parameters of a config file and pipe it to | tee (to write the updated content into the file), this randomly breaks and causes the file to be invalid (size 0).

在摘要中，此代码用于更新参数:

In Summary, this code is used for updating parameters:

# based on the provided linenumber, add some comments, add the new value, delete old line

sed -e "$lineNr a # comments" -e "$lineNr a $newValue" -e "$lineNr d" $myFile | sudo tee $myFile

我设置了一个脚本，该脚本会调用此更新命令100次.

I set up an script which calls this update command 100 times.

• 在与OSX共享的目录中的Ubuntu VM(Parallels Desktop)中，此行为最多发生50次
• 在Ubuntu VM(Parallels Desktop)上，在Ubuntu分区上，此行为最多发生40次
• 在本机系统(带有Ubuntu的IntelNUC)上，此行为最多发生15次

有人可以解释为什么会这样吗?

Can someone explain why this is happening?

这是一个功能齐全的脚本，您也可以在其中运行实验.(所有必需的文件都是由脚本生成的，因此您只需将其复制/粘贴到bashscript文件中并运行它即可)

Here is a fully functional script where you can run the experiment as well. (All necessary files are generated by the script, so you can simply copy/paste it into a bashscriptfile and run it)

#!/bin/bash
# main function at bottom

#====================
#===HELPER METHOD====
#====================

# This method updates parameters with a new value. The replacement is performed linewise.
doUpdateParameterInFile()
{
  local valueOfInterest="$1"
  local newValue="$2"
  local filePath="$3"

  # stores all matching linenumbers
  local listOfLines=""
  # stores the linenumber which is going to be replaced
  local lineToReplace=""

  # find value of interest in all non-commented lines and store related lineNumber
  lineToReplace=$( grep -nr "^[^#]*$valueOfInterest" $filePath | sed -n 's/^\([0-9]*\)[:].*/\1/p' )

  # Update parameters
  # replace the matching line with the desired value
  oldValue=$( sed -n "$lineToReplace p" $filePath )
  sed -e "$lineToReplace a # $(date '+%Y-%m-%d %H:%M:%S'): replaced: $oldValue with: $newValue" -e "$lineToReplace a $newValue" -e "$lineToReplace d" $filePath | sudo tee $filePath >/dev/null

  # Sanity check to make sure file did not get corrupted by updating parameters
  if [[ ! -s $filePath ]] ; then
    echo "[ERROR]: While updating file it turned invalid."
    return 31
  fi

}

#===============================
#=== Actual Update Function ====
#===============================

main_script()
{
  echo -n "Update Parameter1 ..."
  doUpdateParameterInFile "Parameter1" "Parameter1 YES" "config.txt"
  if [[ "$?" == "0" ]] ; then echo "[ OK ]" ; else echo "[FAIL]"; return 33 ; fi

  echo -n "Update Parameter2 ..."
  doUpdateParameterInFile "Parameter2" "Parameter2=90" "config.txt"
  if [[ "$?" == "0" ]] ; then echo "[ OK ]" ; else echo "[FAIL]"; return 34 ; fi

  echo -n "Update Parameter3 ..."
  doUpdateParameterInFile "Parameter3" "Parameter3 YES" "config.txt"
  if [[ "$?" == "0" ]] ; then echo "[ OK ]" ; else echo "[FAIL]"; return 35 ; fi
}

#=================
#=== Main Loop ===
#=================

#generate file config.txt
printf "# Configfile with 3 Parameters\n#[Parameter1]\n#only takes YES or NO\nParameter1 NO \n\n#[Parameter2]\n#Parameter2 takes numbers\nParameter2 = 100 \n\n#[Parameter3]\n#Parameter3 takes YES or NO \nParameter3 YES\n" > config.txt
cp config.txt config.txt.bkup

# Start the experiment and let it run 100 times
cnt=0
failSum=0
while [[ $cnt != "100" ]] ; do
  echo "==========run: $cnt; fails: $failSum======="
  main_script 
  if [[ $? != "0" ]] ; then cp config.txt.bkup config.txt ; failSum=$(($failSum+1)) ; fi
  cnt=$((cnt+1))
  sleep 0.5
done

致谢DonPromillo

regards DonPromillo

推荐答案

问题是您正在使用 tee 覆盖 $ filepath 与 sed 试图从中读取.如果 tee 首先将其截断，则 sed 将得到一个空文件，而另一端的文件长度为 0 .

The problem is that you're using tee to overwrite $filepath at the same time as sed is trying to read from it. If tee truncates it first then sed gets an empty file and you end up with a 0 length file at the other end.

如果您具有GNU sed ，则可以使用 -i 标志让 sed 修改文件(其他版本支持 -i ，但需要一个参数).如果您的 sed 不支持，则可以将其写入临时文件，然后将其移回原始名称，例如

If you have GNU sed you can use the -i flag to have sed modify the file in place (other versions support -i but require an argument to it). If your sed doesn't support it you can have it write to a temp file and move it back to the original name like

tmpname=$(mktemp)
sed -e "$lineToReplace a # $(date '+%Y-%m-%d %H:%M:%S'): replaced: $oldValue with: $newValue" -e "$lineToReplace a $newValue" -e "$lineToReplace d" "$filePath" > "$tmpname"
sudo mv "$tmpname" "$filePath"

或者如果您想保留原始权限，则可以

or if you want to preserve the original permissions you could do

sudo sh -c "cat '$tmpname' > '$filePath'"
rm "$tmpname"

或使用您的 tee 方法

sudo tee "$filePath" >/dev/null <"$tmpname"
rm "$tmpname"

这篇关于在Linux bash脚本中使用tee随机更新文件失败的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！