在Windows上使用通配符处理带有jq的文件 [英] Using wildcard for files with jq on Windows

问题描述

我正在Windows 8.1上使用jq 1.6，并且遇到与此处报告的问题相同的问题 https://github.com/stedolan/jq/issues/1644

I am using jq 1.6 on Windows 8.1 and facing same issue as reported here https://github.com/stedolan/jq/issues/1644

简单至jq . *.json的命令失败，并显示以下错误:

Command as simple as jq . *.json is failing with following error:

断言失败！

Assertion failed!

程序:jq.exe 文件:src/main.c，第256行 表达式:wargc == argc

Program: jq.exe File: src/main.c, Line 256 Expression: wargc == argc

此应用程序已请求运行时 以一种不寻常的方式终止它.请联系应用程序的 支持团队以获取更多信息.

This application has requested the Runtime to terminate it in an unusual way. Please contact the application's support team for more information.

有人对此有解决方案吗?在Windows文件夹中的所有文件中使用jq的正确方法是什么?

Does anyone have solution to this? What is the correct way to use jq with all the files in the folder on windows?

推荐答案

这是一个奇怪的问题.

由高峰回答 ，cmd.exe不会扩展通配符，而将这项工作留给了程序.并且jq不处理通配符(来自问题列表，更多信息请参见后面的内容).

As alredy answered by peak, cmd.exe does not expand wildcards, leaving this work to the programs. And jq does not handle wildcards (from the issues list, more later).

但这不是失败的全部原因.

But this is not the full reason for this failure.

问题在于，源代码在断言中失败:wargc == argc.读取源代码时，在Windows jq中尝试使用

As the question points, source code fails in an assert: wargc == argc. When reading the source code, in windows jq tries to process the original command line with

 wchar_t **wargv = CommandLineToArgvW(GetCommandLineW(), &wargc);

尝试检索与argv[]和argc相同的内容，但处理多字节参数.

trying to retrieve the equivalent to argv[] and argc but handling multibyte arguments.

由于cmd没有扩展通配符，因此将存在三个参数(有问题的命令行)

As cmd is not expanding the wildcard there will be three arguments (command line in question)

jq  .  *.json
^^  ^  ^....^  
0   1  2      

都在argv[]和wargv[]中，因此argc和wargc应该匹配.

both in argv[] and wargv[], so argc and wargc should match.

然后，为什么它失败了?为什么argc与wargc不同?

Then, why does it fail? Why argc is different to wargc?

因为使用了GCC来编译程序.

Because GCC was used to compile the program.

不，问题不在于GCC本身. 问题"是，GCC运行时中的参数处理确实扩展了通配符(Microsoft编译器运行时没有作用，但是没有关系，因为它也无法解决问题).

And no, the problem is not GCC itself. The "problem" is that the argument handling in GCC runtime DOES expand the wildcard (Microsoft compiler runtime does not, but it doesn't matter as it will also not solve the problem).

这意味着argc和argv(由具有通配符扩展名的GCC代码确定)将包含根据与通配符匹配的文件数的信息，而wargc和wargv(由没有通配符扩展名的MS代码确定) )不会.

That means that argc and argv (determined by GCC code with wildcard expansion) will contain information according to the number of files matching the wildcard while wargc and wargv (determined by MS code without wildcard expansion) will not.

一种简单的探测方法是尝试上一个命令时只有一个.json文件.断言不会失败，但jq会因jq: error: Could not open file *.json: Invalid argument而失败，因为它不处理通配符.

A simple way to probe it is to have only one .json file when trying the previous command. The assert will not fail but jq will fail with jq: error: Could not open file *.json: Invalid argument as it does not handle wildcards.

jq . test.json       As seen in argv   argc  = 3
jq . *.json          As seen in wargv  wargc = 3

那么，如何处理呢?不修改jq的源代码，最好的选择是连接文件列表并将其传递给jq.峰的答案和您的评论中的参考应该可以解决该问题.

So, how to deal with it? Without modifying the jq's source code your best option is to concatenate the list of files and pass it to jq. References in peak's answer and in your comment should deal with the problem.

但是请记住，在cmd和批处理文件中，命令行限制为8191个字符.如果不足以解决您的问题，您可以尝试使用类似的方法(是的，很多行，其中大多数是注释和命令用法)

But remember that in cmd and batch files your command lines are limited to 8191 characters. If it is not enough to deal with your problem you can try with something like (yes, a lot of lines, most of them comments and command usage)

@if (@this==@isBatch) @then /* ------------------------------------- batch code
@echo off
    setlocal enableextensions disabledelayedexpansion

    rem This is an hybrid batch/javascript file. The batch part will retrieve
    rem the required information and start the cscript engine to execute the 
    rem javascript part of this file.

    rem Retrieve a safe reference to current batch file 
    call :getCurrentFile thisFile fileName

    rem Arguments to current batch file are the command line to execute later
    rem Using an environment variable to avoid losing quotes when using the
    rem internal Wscript argumeng handling routines
    set [commandLine]=%*

    if not defined [commandLine] (
        echo(
        echo usage: command1 ^| "%fileName%" command2
        echo(
        echo where:
        echo     command1   is a command generating a set of lines that will be
        echo                concatenated to pass as arguments to command2
        echo(
        echo     command2   is the command to execute with all the lines from 
        echo                command1 as command line arguments
        echo(
        echo examples:
        echo(
        echo    dir /b ^| "%fileName%" cmd /c echo
        echo    dir /b *.json ^| "%fileName%" jq . 
        echo(
        goto :eof
    )

    rem Execute the javascript part of this script
    "%windir%\system32\cscript.exe" //nologo //e:JScript "%thisFile%" 
    goto :eof


:getCurrentFile fullPath fileName
    set "%~1=%~f0"
    set "%~2=%~nx0"
    goto :eof

------------------------------------------------------------- end of batch code
*/@end //------------------------------------------------------ javascript code

/*
    This script will read all lines from standard input and execute the 
    command stored by the batch code above into the [commandLine] environment 
    variable, passing as command lien arguments the concatenation of all lines 
    present in the standard input.
*/

var shell = WScript.CreateObject('WScript.Shell')
  , commandLine = shell.Environment("PROCESS").Item('[commandLine]')
  , stdIn = WScript.StdIn
  , stdOut = WScript.StdOut
  , stdErr = WScript.StdErr
  , line = ''
  , buffer = []
  ;
    // read the list of arguments from standard input
    while ( !stdIn.AtEndOfStream ){ 
        if ( 
            line = stdIn.ReadLine().replace(/"/g, '') 
        ) buffer.push( ' "' + line + '"' );
    };

    // concatenate all arguments 
    buffer = buffer.join('');

    // if we don't have a command line, output what we have contatenated 
    // but if we have a command line, execute it, get its output and show it
    // as it is possible that we are piping it to another process.

    if ( commandLine ){
        try {
            stdOut.WriteLine(
                shell.Exec( commandLine + buffer ).StdOut.ReadAll()
            );
        } catch ( e ){
            stdErr.WriteLine( 'ERROR: Command line exec failed when running:' );
            stdErr.WriteLine( '---------------------------------------------' );
            stdErr.WriteLine( commandLine + buffer );
            stdErr.WriteLine( '---------------------------------------------' );
        };
    } else {
        stdOut.WriteLine( buffer );
    };

将其另存为cmd文件(例如list2args.cmd)并按建议使用

Save it as a cmd file (ex. list2args.cmd) and use it as suggested

dir /b *.json | list2args.cmd jq .

区别在于，在脚本部分内部进行串联并使用WScript.Shell.Exec方法启动进程，我们可以使用最大32KB(命令行的Windows限制)的命令行.

The difference is that doing the concatenation inside the script part and starting the process using the WScript.Shell.Exec method we can use command lines up to 32KB (the windows limit for command lines).

这篇关于在Windows上使用通配符处理带有jq的文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！