bash:如何在脚本的多个实例之间保持一些延迟 [英] bash: how to keep some delay between multiple instances of a script

问题描述

我正在尝试使用脚本下载 100 个文件

I am trying to download 100 files using a script

我不希望在任何时候下载不超过 4 次.

I dont want at any point of time not more than 4 downloads are happening.

所以我创建了一个文件夹/home/user/file_limit.在脚本中，它在下载之前在此处创建一个文件，下载完成后它将删除它.

So i have create a folder /home/user/file_limit. In the script it creates a file here before the download and after the download is complete it will delete it.

脚本将检查文件夹中的文件数小于 4，然后才允许在文件夹中创建文件 /home/user/file_limit

The script will check the number of files in the folder is less than 4 then only it will allow to create a file in the folder /home/user/file_limit

我正在运行这样的脚本

    today=`date +%Y-%m-%d-%H_%M_%S_%N`;
    while true
    do
        sleep 1
        # The below command will find number of files in the folder /home/user/file_limit
        lines=$(find /home/user/file_limit -iname 'download_*' -type f| wc -l)
        if [ $lines -lt 5 ]; then
            echo "Create file"
            touch "/home/user/file_limit/download_${today}"
            break;
        else
            echo "Number of files equals 4"
        fi
    done

    #After this some downloading happens and once the downloading is complete

    rm "/home/user/file_limit/download_${today}"

我面临的问题是当 100 个这样的脚本正在运行时.例如，当文件夹中的文件数小于 4 时，许多 touch "/home/user/file_limit/download_${today}" 会同时执行，所有这些都会创建文件.所以文件总数变成了 4 个以上我不想要的，因为更多的下载导致我的系统变慢.

The problem i am facing is when 100 such scripts are running. Eg when the number of files in the folder are less than 4, then many touch "/home/user/file_limit/download_${today}" gets executed simultaneously and all of them creates files. So the total number of files become more than 4 which i dont want because more downloads cause my system get slower.

如何确保每个脚本之间存在延迟以检查lines=$(find/home/user/file_limit -iname 'download_*' -type f| wc -l) 以便只执行一个触摸命令.

How to ensure there is a delay between each script for checking the lines=$(find /home/user/file_limit -iname 'download_*' -type f| wc -l) so that only one touch command get executed.

或者如何确保 lines=$(find/home/user/file_limit -iname 'download_*' -type f| wc -l) 命令被队列中的每个脚本检查.没有两个脚本可以同时检查它.

Or HOw to ensure the lines=$(find /home/user/file_limit -iname 'download_*' -type f| wc -l) command is checked by each script in a queue. No two scripts can check it at the same time.

推荐答案

如何确保检查lines=$(find ... | wc -l)的每个脚本之间存在延迟，以便只执行一个触摸命令

How to ensure there is a delay between each script for checking the lines=$(find ... | wc -l) so that only one touch command get executed

添加延迟并不能解决问题.您需要一个锁、互斥体或信号量来确保以原子方式执行文件的检查和创建.

Adding a delay won't solve the problem. You need a lock, mutex, or semaphore to ensure that the check and creation of files is executed atomically.

锁将并行进程的数量限制为 1.可以使用 flock(通常预先安装)创建锁.信号量是通用锁，将并发进程的数量限制为任意 N 个.可以使用 sem 创建信号量(GNU 并行的一部分，必须安装).

Locks limit the number of parallel processes to 1. Locks can be created with flock (usually pre-installed). Semaphores are generalized locks limiting the number concurrent processes to any number N. Semaphores can be created with sem (part of GNU parallel, has to be installed).

以下脚本允许并行下载 4 次.如果正在运行 4 次下载并且您第 5 次启动脚本，则第 5 次下载将暂停，直到 4 次正在运行的下载之一完成.

The following script allows 4 downloads in parallel. If 4 downloads are running and you start the script a 5th time then that 5th download will pause until one of the 4 running downloads finish.

#! /usr/bin/env bash

main() {
  # put your code for downloading here
}
export -f main
sem --id downloadlimit -j4 main

这篇关于bash:如何在脚本的多个实例之间保持一些延迟的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！