批量For循环不刷新它是从拉动文件 [英] Batch For loop doesn't refresh the file it's pulling from
问题描述
所以我有一个for循环，确实为每行一个SQL存储过程的迭代在文件 queue.txt ，现在不过所有的伟大工程，是什么犯规是，如果它是迭代和另一行被添加到它使用，因为它是迭代的标准那么它只是忽略它。该文件的底部
So I have a for loop that does an iteration of a SQL stored procedure for every line in a file queue.txt, now that all works great, what DOESNT however is that if it is iterating and another line is added to the bottom of the file it uses as it's iteration criteria then it just ignores it.
我已经是这样的：
@echo off
cd "%UserProfile%\Desktop\Scripting\"
echo words > busy.txt
FOR /f "delims=" %%a in ('type queue.txt') DO (
IF NOT EXIST reset.sql (
::Create SQL command
echo USE dbname> reset.sql
echo EXEC dbo.sp_ResetSubscription @ClientName = '%%a'>> reset.sql
echo EXEC dbo.sp_RunClientSnapshot @ClientName = '%%a'>> reset.sql
echo #################### %date% - %time% ####################################################>> log.txt
echo Reinitialising '%%a'>> log.txt
sqlcmd -i "reset.sql">> log.txt
echo. >> log.txt
echo ####################################################################################################>> log.txt
echo. >> log.txt
type queue.txt | findstr /v %%a> new.txt
type new.txt> queue.txt
echo New list of laptops waiting:>> log.txt
type queue.txt>> log.txt
echo. >> log.txt
echo ####################################################################################################>> log.txt
echo. >> log.txt
if exist reset.sql del /f /q reset.sql
)
)
if exist busy.txt del /f /q busy.txt
if exist queue.txt del /f /q queue.txt
if exist new.txt del /f /q new.txt那么，这样做是拉文件 queue.txt ，使迭代每个那些，现在说，这与2号线在该文件中启动，这是伟大的，它开始运行的程序他们。
So what this does is pulls the file queue.txt and makes an iteration for each of those, now say that it starts with 2 lines in the file, this is great, it starts running the procedures for them.
现在，说我添加另一条线路 queue.txt 循环播放过程中运行，它只是忽略该行因此它看起来不至于从文件更新在每次迭代它只是进口一次。
Now, say I add another line to queue.txt WHILE the loop is running it just ignores that line so it looks like the for doesn't update from the file on each iteration it just imports once.
一个办法，我想解决这个是计算行数在循环的第一次迭代，然后在反对它认为的价值应该是每个迭代结束检查，如果超过预计再回到上面的for循环（使用goto语句或类似），但是goto方法不逻辑的前pressions工作。
One way i was thinking to solve this was to count the number of lines at the first iteration of the loop and then check it at the end of each iteration against what it thinks the value should be, and if it is more than it expects then go back to above the for loop (using a goto or some such) but gotos don't work in logic expressions.
咨询任何人都好吗？
推荐答案
@Myles格雷 - 你的解决方案有一些问题
@Myles Gray - Your solution has some problems.
首先是小问题：
1）队列循环的每次迭代之后，重新创建队列作为原始队列减去您目前正在使用就行了（你希望！后来更多）。当您创建队列你把它附加到您的日志。这将工作，但它似乎很innefficient并具有使日志大规模和unweildy的潜力。假设你有1万行队列。到时候你处理你的队列，你会写99989998队列线，包括49994999队列行到你的日志！这将需要很长的时间来处理，即使没有实际做的工作。
1) After each iteration of the queue loop, you recreate the queue as the original queue minus the line you are currently working on (you hope! more on that later). After you recreate the queue you append it to your log. That will work, but it seems very innefficient and has the potential of making the log massive and unweildy. Suppose you have a queue with 10,000 lines. By the time you have processed your queue you will have written 99,989,998 queue lines, including 49,994,999 queue lines to your log! That will take a long time to process, even without actually doing your work.
2）通过使用FINDSTR，preserving不符合当前的ID的所有行创建队列。但是，这也将去掉以后的行，如果他们正好符合当前ID。这可能不是一个问题。但是，你正在做一个字符串匹配。您也FINDSTR将消除包含在其内的任何地方你当前的ID后续行。我不知道你的ID的样子。但是，如果你的电流ID是123，则以下所有的ID的将被错误地剥离 - 31236，12365等。即一个潜在devestating问题。我说，这是潜力，因为FOR循环已经缓冲队列，所以它并不关心 - 除非你中止循环，因为新的工作已追加到late.txt文件 - 那么你实际上将跳过这些失踪的ID！这可以通过添加/ X选项FINDSTR固定。至少这样你才会真正跳过重复。
2) You recreate the queue by using FINDSTR, preserving all lines that don't match your current ID. But this will also strip out subsequent lines if they happen to match your current ID. That might not be a problem. But you are doing a substring match. Your FINDSTR will also eliminate subsequent lines that contain your current ID anywhere within it. I have no idea what your IDs look like. But if your current ID is 123, then all of the following IDs will be stripped erroneously - 31236, 12365 etc. That is a potentially devestating problem. I say it is potential because the FOR loop has already buffered the queue, so it doesn't care - unless you abort the loop because new work has been appended to the late.txt file - then you actually will skip those missing IDs! This could be fixed by adding the /X option to FINDSTR. At least then you will only be skipping true duplicates.
现在的主要问题 - 所有的事实所产生的只有一个进程可以打开任何一种写一个文件（或删除）操作
Now the major problems - all stemming from the fact only one process can have a file open for any kind of write (or delete) operation.
3）即使一个FOR / F循环不会写入文件，它的设计，如果该文件正在由另一个进程写入失败。所以，如果你的循环试图读取队列，而另一个过程是追加到它，你的队列处理脚本将失败。你有busy.txt文件检查，但您的队列作家可能已经开始已创建了busy.txt文件之前写作。写操作可能需要一段时间，特别是如果被追加多条线路。虽然被写入行队列的处理器可能会启动，然后你有你的碰撞和失败。
3) Even though a FOR /F loop does not write to the file, it is designed to fail if the file is actively being written to by another process. So if your FOR loop attempts to read the queue while another process is appending to it, your queue processing script will fail. You have the busy.txt file check, but your queue writer might have already started writing before the busy.txt file has been created. The write operation might take a while, especially if many lines are being appended. While the lines are being written your queue processor could start and then you have your collision and failure.
4）您的队列处理器追加late.txt你的队列，然后删除late.txt。但的追加和删除一个队列作者可以添加额外的行late.txt之间的时间点。这迟来的行会，而不必经过处理被删除！
4) Your queue processor appends the late.txt to your queue and then deletes late.txt. But there is point of time between the append and the delete where a queue writer could append an additional line to late.txt. This late arriving line will be deleted without having been processed!
5）另一种可能性是一个作家可以试图写入late.txt而它是在由队列处理器被删除的处理。写会失败，并重新您的队列会丢失工作。
5) Another possibility is a writer may attempt to write to the late.txt while it is in the process of being deleted by the queue processor. The write will fail, and again your queue will be missing work.
6）另一种可能是您的队列可能试图删除late.txt而队列作家被追加到它。删除会失败，你最终会在接下来的时间排队处理器追加到late.txt您queue.txt重复的队列
6) Yet another possibility is your queue may attempt to delete late.txt while a queue writer is appending to it. The delete will fail, and you will end up with duplicates in your queue the next time the queue processor appends late.txt to queue.txt.
综上所述，并发问题可能会导致既缺少队列的工作，以及队列中的重复劳动。只要你有多个进程同时进行更改到一个文件，你必须建立某种锁定机制序列化的事件。
In summary, concurrency issues can lead both to missing work in your queue, as well as duplicate work in your queue. Whenever you have multiple processes making changes to a file simultaneously, you MUST establish some kind of locking mechanism to serialize the events.
您已经在使用SQLServer数据库。做最顺理成章的事情是将您的队列出来的文件系统和数据库中。关系数据库从地上爬起来，处理并发建造的。
You are already using a SqlServer database. The most logical thing to do is to move your queue out of the file system and into the database. Relational databases are built from the ground up to deal with concurrency.
话虽这么说，这是不是太困难，只要你使用一个锁定策略使用文件作为Windows批处理中的一个队列。你必须确保两个队列处理器和队列作家遵循相同的锁定策略。
That being said, it is not too difficult to use a file as a queue within Windows batch as long as you employ a locking strategy. You must make sure both your queue processor and your queue writers follow the same locking strategy.
下面是基于文件的解决方案。我会假设你只有一个队列处理器，以及可能有多个队列作家。随着更多的工作，你可以修改文件队列解决方案来支持多个队列处理器。但多个队列处理器是可能更容易使用，我在
Below is a file based solution. I'm going to assume you only have one queue processor, and possibly multiple queue writers. With additional work you can adapt the file queue solution to support multiple queue processors. But multiple queue processors is probably easier to implement using the folder based queue that I described at the end of my first answer.
代替具有队列作家写要么queue.txt或late.txt，更容易具有队列处理器重命名现有队列并对其进行处理，以完成，而队列作家总是写入queue.txt
Instead of having the queue writers write to either queue.txt or late.txt, it is easier to have the queue processor rename the existing queue and process it to completion, while the queue writers always write to queue.txt.
该解决方案将当前状态到status.txt文件。您可以通过发出在命令窗口中键入status.txt中监控你的队列处理器状态。
This solution writes the current status to a status.txt file. You can monitor your queue processor status by issuing TYPE STATUS.TXT from a command window.
我做了一些延迟扩展来回切换，以防止腐败，由于！您的数据。如果你知道！将永远不会出现，那么你可以简单的SETLOCAL EnableDelayedExpansion移动到顶部，并放弃了切换。
I do some delayed expansion toggling to protect against corruption due to ! in your data. If you know that ! will never appear, then you can simply move the SETLOCAL EnableDelayedExpansion to the top and forgo the toggling.
另外一个优化 - 这是更快地只是一次为一组语句打开和关闭该文件为每个语句而不是重定向输出
One other optimisation - it is faster to redirect output just once for a group of statements instead of opening and closing the file for each statement.
这code是没有经过测试的，所以很容易被一些愚蠢的错误。但其概念是健全的。希望你的想法。
This code is totally untested, so there could easily be some silly bugs. But the concepts are sound. Hopefully you get the idea.
queueProcessor.bat
queueProcessor.bat
@echo off
setlocal disableDelayedExpansion
cd "%UserProfile%\Desktop\Scripting\"
:rerun
::Safely get a copy of the current queue, exit if none or error
call :getQueue || exit /b
::Get the number of lines in the queue to be used in status updates
for /f %%n in ('find /v "" ^<inProcess.txt') do set /a "record=0, recordCount=%%n"
::Main processing loop
for /f "delims=" %%a in (inProcess.txt) do (
rem :: Update the status. Need delayed expansion to access the current record number.
rem :: Need to toggle delayed expansion in case your data contains !
setlocal enableDelayedExpansion
set /a "record+=1"
> status.txt echo processing !record! out of %recordCount%
endlocal
rem :: Create SQL command
> reset.sql (
echo USE dbname
echo EXEC dbo.sp_ResetSubscription @ClientName = '%%a'
echo EXEC dbo.sp_RunClientSnapshot @ClientName = '%%a'
)
rem :: Log this action and execute the SQL command
>> log.txt (
echo #################### %date% - %time% ####################################################
echo Reinitialising '%%a'
sqlcmd -i "reset.sql"
echo.
echo ####################################################################################################
echo.
)
)
::Clean up
delete inProcess.txt
delete status.txt
::Look for more work
goto :rerun
:getQueue
2>nul (
>queue.lock (
if not exist queue.txt exit /b 1
if exist inProcess.txt (
echo ERROR: Only one queue processor allowed at a time
exit /b 2
)
rename queue.txt inProcess.txt
)
)||goto :getQueue
exit /b 0
queueWriter.bat
queueWriter.bat
::Whatever your code is
::At some point you want to append a VALUE to the queue in a safe way
call :appendQueue VALUE
::continue on until done
exit /b
:appendQueue
2>nul (
>queue.lock (
>>queue.txt echo %*
)
)||goto :appendQueue
锁定code的说明：
Explanation of the lock code:
:retry
::First redirect any error messages that occur within the outer block to nul
2>nul (
rem ::Next redirect all stdout within the inner block to queue.lock
rem ::No output will actually go there. But the file will be created
rem ::and this process will have a lock on the file until the inner
rem ::block completes. Any other process that tries to write to this
rem ::file will fail. If a different process already has queue.lock
rem ::locked, then this process will fail to get the lock and the inner
rem ::block will not execute. Any error message will go to nul.
>queue.lock (
rem ::you can now safely manipulate your queue because you have an
rem ::exclusive lock.
>>queue.txt echo data
rem ::If some command within the inner block can fail, then you must
rem ::clear the error at the end of the inner block. Otherwise this
rem ::routine can get stuck in an endless loop. You might want to
rem ::add this to my code - it clears any error.
verify >nul
) && (
rem ::I've never done this before, but if the inner block succeeded,
rem ::then I think you can attempt to delete queue.lock at this point.
rem ::If the del succeeds then you know that no process has a lock
rem ::at this point. This could be useful if you are trying to monitor
rem ::the processes. If the del fails then that means some other process
rem ::has already grabbed the lock. You need to clear the error at
rem ::this point to prevent the endless loop
del queue.lock || verify >nul
)
) || goto :retry
:: If the inner block failed to get the lock, then the conditional GOTO
:: activates and it loops back to try again. It continues to loop until
:: the lock succeeds. Note - the :retry label must be above the outer-
:: most block.
如果你有一个唯一的进程ID，你可以把它写在内部块中queue.lock。然后，你可以从另一个窗口中键入queue.lock要找出哪个进程目前拥有（或者最近有）锁。这应该只是一个问题，如果某个进程挂起。
If you have a unique process ID, you can write it to queue.lock within the inner block. Then you can type queue.lock from another window to find out which process currently has (or most recently had) the lock. That should only be an issue if some process hangs.
这篇关于批量For循环不刷新它是从拉动文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
