批处理文件多任务处理问题与父级共享变量 [英] Batch file multitasking issues sharing variables with parent

问题描述

我正在尝试在x265中对视频文件进行大规模编码，并想提出一个可以自动执行该过程的批处理文件.为了加快操作速度，我发现用3个ffmpeg实例调用2个线程会产生理想的编码时间，但是昨天我整天试图找到一种方法来获取批处理文件，该批处理文件将调用3个实例，然后完成后再打电话给新朋友.到目前为止，这是我的位置:

I'm trying to mass encode video files in x265 and wanted to come up with a batch file that could automate the process. In order to expedite things, I found that having 3 instances of ffmpeg called with 2 threads resulted in ideal encoding times, however I've tried all day yesterday to come up with a way to get a batch file that will call 3 instances and then call new ones when they complete. So far, this is where I am:

父批

@echo off
Setlocal EnableDelayedExpansion
SET /A COUNT=0
for %%a in (*.mkv) do (
    CALL :CHECK
    SET /A COUNT+=1
    START CALL "child.bat" "%%a"
    )
EXIT /B 0

:CHECK
IF !COUNT! EQU 3 (
    TIMEOUT /T 5
    GOTO :CHECK
)
EXIT /B 0

儿童批次

ffmpeg <COMMAND LINE ARGS>
SET /A COUNT-=1
EXIT /B 0

我有两个问题. 1)COUNT变量不会在父进程中进行更新，并且在子进程完成时它不会产生新的实例. 2)子进程不会干净地退出.它将留下一个单独的cmd.exe窗口，并带有DOS提示符.

I have two problems. 1) The COUNT variable isn't being updated in the parent process and it never spawns new instances when the child processes finish. 2) The child process doesn't cleanly exit. It leaves a separate cmd.exe window open with a DOS prompt.

有什么想法吗?

替换嵌套的GOTO以防止FOR循环中断

Replaced nested GOTO to prevent FOR loop breakage

以下解决方法

Setlocal EnableDelayedExpansion
SET /A COUNT=0
for %%a in (*.mkv) do (
    IF !COUNT! EQU 3 CALL :CHECK
    SET /A COUNT+=1
    START CALL "child.bat" "%%a"
    )
EXIT /B 0

:CHECK
IF EXIST DONE.TXT (
    SET /A COUNT-=1
    DEL DONE.TXT
    EXIT /B 0
    ) ELSE (
    TIMEOUT /T 5
    GOTO :CHECK
    )
EXIT /B 0

儿童批次

ffmpeg <COMMAND LINE ARGS>

:DONE
IF EXIST DONE.TXT (
    TIMEOUT /T 1
    GOTO :DONE
)
echo 1 >> DONE.TXT
EXIT 0

推荐答案

关于您陈述的问题:

1)子进程无法在父进程中修改环境变量.您将需要一种不同的机制来检测孩子何时终止.另外，正如Squashman在评论中指出的那样，循环中的GOTO会中断(终止)循环，这就是为什么在第一个3之后没有新的子进程启动的原因.

1) A child process cannot modify environment variables in the parent process. You will need a different mechanism to detect when the child has terminated. Also, as Squashman states in his comment, a GOTO within a loop will break (terminate) the loop, which is why no new child processes are launched after the first 3.

2)您的子窗口不会终止，因为您使用的是EXIT/B.改用EXIT，窗口将关闭.

2) Your child window does not terminate because you use EXIT /B. Use EXIT instead and the window will close.

要找到可行的解决方案，您还有很长的路要走；-)

You have a long way to go before you have a working solution ;-)

也许最大的障碍是检测子进程何时终止.

Perhaps the biggest hurdle is detecting when a child process terminates.

我知道3种策略:

1)使用TASKLIST和FIND/C一起计算当前正在运行的ffmpeg进程的数量.这也许是最简单的解决方案，但是它无法区分脚本启动的进程和其他机制可能启动的进程.

1) Use TASKLIST coupled with FIND /C to count the number of ffmpeg processes that are currently running. This is perhaps the simplest solution, but it cannot differentiate between processes that your script launches vs processes that may have been launched by some other mechanism.

2)使用文件作为信号.为每个进程创建一个空文件，然后在进程完成时删除它.您的主脚本可以通过查找文件来监视哪些进程处于活动状态.这也很简单，但是如果您的进程之一在可以删除文件之前就崩溃了，它的行为就不会很好.这会使您的系统处于不正常的状态.

2) Use a file as a signal. Create an empty file for each process, and then when the process finishes, have it delete the file. Your main script can monitor which processes are active by looking for the files. This is also simple, but it does not behave well if one of your processes crashes before it can delete the file. That leaves your system in an unhealthy state.

3)我最喜欢的是使用锁定文件.每个子进程都通过重定向锁定文件，并且当该进程终止时(崩溃，正常退出，无论如何)，都将释放该锁定.主进程可以尝试锁定相同的文件.如果锁定成功，它将知道子项已终止，否则子项仍在运行.此策略最复杂，使用了奥术语法，但我发现它非常有效.

3) My favorite is to use lock files. Each child process locks a file via redirection, and when the process terminates (crash, normal exit, it doesn't matter how), then the lock is released. The main process can attempt to lock the same files. It knows the child has terminated if the lock is successful, else the child is still running. This strategy is the most complicated, and it uses arcane syntax, but I find it highly effective.

我已经在并行执行Shell进程中使用了选项3)实现了一个解决方案.下面是针对您情况的代码的改编/简化.

I have already implemented a solution at Parallel execution of shell processes that uses option 3). Below is an adaptation/simplification of that code for your situation.

我使用START/B在父窗口中启动每个子进程，并将所有输出重定向到锁定文件.完成后，我输入输出文件，以便您可以看到发生了什么.我还列出了每个子进程的开始和停止时间.

I launch each child process in the parent window using START /B, and I redirect all output to the lock file. When finished, I type the output file so you can see what happened. I also list the start and stop times for each child process.

您只需要调整3个顶级环境变量即可满足您的需求.其余的应该很好.但是，如果任何文件名包含!字符，则编写的代码将失败.可以通过更多的工作来消除此限制.

You just need to adjust the 3 top environment variables to suit your needs. The remainder should be good to go. However, the code as written will fail if any file names contain the ! character. This limitation can be removed with a bit more work.

脚本中包含大量文档. %= COMMENT =%语法是一种在不使用REM的情况下将注释安全地嵌入循环中的方法.

There is extensive documentation within the script. The %= COMMENT =%syntax is one way of safely embedding comments within a loop without using REM.

@echo off
setlocal enableDelayedExpansion

:: Define the command that will be run to obtain the list of files to process
set listCmd=dir /b /a-d *.mkv

:: Define the command to run for each file, where "%%F" is an iterated file name from the list
::   something like YOUR_COMMAND -i "%%F"
set runCmd=ffmpeg  [** command arguments go here **]

:: Define the maximum number of parallel processes to run.
set "maxProc=3"

::---------------------------------------------------------------------------------
:: The remainder of the code should remain constant
::

:: Get a unique base lock name for this particular instantiation.
:: Incorporate a timestamp from WMIC if possible, but don't fail if
:: WMIC not available. Also incorporate a random number.
  set "lock="
  for /f "skip=1 delims=-+ " %%T in ('2^>nul wmic os get localdatetime') do (
    set "lock=%%T"
    goto :break
  )
  :break
  set "lock=%temp%\lock%lock%_%random%_"

:: Initialize the counters
  set /a "startCount=0, endCount=0"

:: Clear any existing end flags
  for /l %%N in (1 1 %maxProc%) do set "endProc%%N="

:: Launch the commands in a loop
  set launch=1
  for /f "tokens=* delims=:" %%F in ('%listCmd%') do (
    if !startCount! lss %maxProc% (
      set /a "startCount+=1, nextProc=startCount"
    ) else (
      call :wait
    )
    set cmd!nextProc!=%runCmd%
    echo -------------------------------------------------------------------------------
    echo !time! - proc!nextProc!: starting %runCmd%
    2>nul del %lock%!nextProc!
    %= Redirect the lock handle to the lock file. The CMD process will     =%
    %= maintain an exclusive lock on the lock file until the process ends. =%
    start /b "" cmd /c >"%lock%!nextProc!" 2^>^&1 %runCmd%
  )
  set "launch="

:wait
:: Wait for procs to finish in a loop
:: If still launching then return as soon as a proc ends
:: else wait for all procs to finish
  :: redirect stderr to null to suppress any error message if redirection
  :: within the loop fails.
  for /l %%N in (1 1 %startCount%) do 2>nul (
    %= Redirect an unused file handle to the lock file. If the process is    =%
    %= still running then redirection will fail and the IF body will not run =%
    if not defined endProc%%N if exist "%lock%%%N" 9>>"%lock%%%N" (
      %= Made it inside the IF body so the process must have finished =%
      echo ===============================================================================
      echo !time! - proc%%N: finished !cmd%%N!
      type "%lock%%%N"
      if defined launch (
        set nextProc=%%N
        exit /b
      )
      set /a "endCount+=1, endProc%%N=1"
    )
  )
  if %endCount% lss %startCount% (
    timeout /t 1 /nobreak >nul
    goto :wait
  )

2>nul del %lock%*
echo ===============================================================================
echo Thats all folks

这篇关于批处理文件多任务处理问题与父级共享变量的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！