同时解析python中的多个子命令或以其他方式对解析的参数进行分组 [英] Parse multiple subcommands in python simultaneously or other way to group parsed arguments

问题描述

我正在将 Bash shell 安装程序实用程序转换为 Python 2.7，并且需要实现复杂的 CLI，因此我能够解析数十个参数(可能高达 ~150).除了在 shell 版本中可用的十几个通用部署选项之外，这些是 Puppet 类变量的名称.

I am converting Bash shell installer utility to Python 2.7 and need to implement complex CLI so I am able to parse tens of parameters (potentially up to ~150). These are names of Puppet class variables in addition to a dozen of generic deployment options, which where available in shell version.

然而，在我开始添加更多变量后，我面临着几个挑战:1. 我需要将参数分组到单独的字典中，以便将部署选项与 Puppet 变量分开.如果它们被扔到同一个桶中，那么我将不得不编写一些逻辑来对它们进行排序，可能会重命名参数，然后字典合并将不是微不足道的.2. 可能存在名称相同但属于不同 Puppet 类的变量，所以我认为子命令可以让我过滤哪些内容并避免名称冲突.

However after I have started to add more variables I faced are several challenges: 1. I need to group parameters into separate dictionaries so deployment options are separated from Puppet variables. If they are thrown into the same bucket, then I will have to write some logic to sort them, potentially renaming parameters and then dictionary merges will not be trivial. 2. There might be variables with the same name but belonging to different Puppet class, so I thought subcommands would allow me to filter what goes where and avoiding name collisions.

目前我已经通过简单地添加多个解析器来实现参数解析:

At the momment I have implemented parameter parsing via simply adding multiple parsers:

parser = argparse.ArgumentParser(description='deployment parameters.')
env_select = parser.add_argument_group(None, 'Environment selection')
env_select.add_argument('-c', '--client_id',  help='Client name to use.')
env_select.add_argument('-e', '--environment', help='Environment name to use.')
setup_type = parser.add_argument_group(None, 'What kind of setup should be done:')
setup_type.add_argument('-i', '--install', choices=ANSWERS, metavar='', action=StoreBool, help='Yy/Nn Do normal install and configuration')
# MORE setup options
...
args, unk = parser.parse_known_args()
config['deploy_cfg'].update(args.__dict__)

pup_class1_parser = argparse.ArgumentParser(description=None)
pup_class1 = pup_class1_parser.add_argument_group(None, 'Puppet variables')
pup_class1.add_argument('--ad_domain', help='AD/LDAP domain name.')
pup_class1.add_argument('--ad_host', help='AD/LDAP server name.')
# Rest of the parameters

args, unk = pup_class1_parser.parse_known_args()
config['pup_class1'] = dict({})
config['pup_class1'].update(args.__dict__)
# Same for class2, class3 and so on.

这种方法的问题是它不能解决问题 2.此外，第一个解析器使用-h"选项，其余参数未显示在帮助中.

The problem with this approach that it does not solve issue 2. Also first parser consumes "-h" option and rest of parameters are not shown in help.

我尝试使用 选择作为答案的示例 但我无法同时使用这两个命令.

I have tried to use example selected as an answer but I was not able to use both commands at once.

## This function takes the 'extra' attribute from global namespace and re-parses it to create separate namespaces for all other chained commands.
def parse_extra (parser, namespace):
  namespaces = []
  extra = namespace.extra
  while extra:
    n = parser.parse_args(extra)
    extra = n.extra
    namespaces.append(n)

  return namespaces

pp = pprint.PrettyPrinter(indent=4)

argparser=argparse.ArgumentParser()
subparsers = argparser.add_subparsers(help='sub-command help', dest='subparser_name')

parser_a = subparsers.add_parser('command_a', help = "command_a help")
## Setup options for parser_a
parser_a.add_argument('--opt_a1', help='option a1')
parser_a.add_argument('--opt_a2', help='option a2')

parser_b = subparsers.add_parser('command_b', help = "command_b help")
## Setup options for parser_a
parser_b.add_argument('--opt_b1', help='option b1')
parser_b.add_argument('--opt_b2', help='option b2')


## Add nargs="*" for zero or more other commands
argparser.add_argument('extra', nargs = "*", help = 'Other commands')

namespace = argparser.parse_args()
pp.pprint(namespace)
extra_namespaces = parse_extra( argparser, namespace )
pp.pprint(extra_namespaces)

结果我:

$ python argtest.py command_b --opt_b1 b1 --opt_b2 b2 command_a --opt_a1 a1
usage: argtest.py [-h] {command_a,command_b} ... [extra [extra ...]]
argtest.py: error: unrecognized arguments: command_a --opt_a1 a1

同样的结果是当我尝试用两个子解析器定义父级时.

The same result was when I tried to define parent with two child parsers.

问题

1. 我可以以某种方式使用 parser.add_argument_group 进行参数解析还是仅用于帮助打印中的分组?它将解决问题 1，而不会错过帮助副作用.将它作为 parse_known_args(namespace=argument_group) 传递(如果我正确地回忆了我的实验)获取所有变量(没问题)但也会获取结果字典中的所有 Python 对象内容(这对 hieradata YAML 不利)
2. 我在第二个示例中缺少什么以允许使用多个子命令?或者 argparse 不可能做到这一点?
3. 对命令行变量进行分组还有其他建议吗?我查看了 Click，但没有发现我的任务比标准 argparse 有任何优势.

1. Can I somehow use parser.add_argument_group for argument parsing or is it just for the grouping in help print out? It would solve issue 1 without missing help side effect. Passing it as parse_known_args(namespace=argument_group) (if I correctly recall my experiments) gets all the variables (thats ok) but also gets all Python object stuff in resulting dict (that's bad for hieradata YAML)
2. What I am missing in the second example to allow to use multiple subcommands? Or is that impossible with argparse?
3. Any other suggestion to group command line variables? I have looked at Click, but did not find any advantages over standard argparse for my task.

注意:我是系统管理员，不是程序员，所以请对我进行非对象风格的编码.:)

Note: I am sysadmin, not a programmer so be gently on me for the non object style coding. :)

谢谢

已解决通过 hpaulj 建议的答案解决了参数分组.

RESOLVED Argument grouping solved via the answer suggested by hpaulj.

import argparse
import pprint
parser = argparse.ArgumentParser()

group_list = ['group1', 'group2']

group1 = parser.add_argument_group('group1')
group1.add_argument('--test11', help="test11")
group1.add_argument('--test12', help="test12")

group2 = parser.add_argument_group('group2')
group2.add_argument('--test21', help="test21")
group2.add_argument('--test22', help="test22")

args = parser.parse_args()
pp = pprint.PrettyPrinter(indent=4)

d = dict({})

for group in parser._action_groups:
    if group.title in group_list:
        d[group.title]={a.dest:getattr(args,a.dest,None) for a in group._group_actions}

print "Parsed arguments"
pp.pprint(d)

这让我在问题 1 中得到了想要的结果.直到我将有多个同名参数.解决方案可能看起来很丑陋，但至少它按预期工作.

This gets me desired result for the issue No.1. until I will have multiple parameters with the same name. Solution may look ugly, but at least it works as expected.

python argtest4.py --test22 aa  --test11 yy11 --test21 aaa21
Parsed arguments
{   'group1': {   'test11': 'yy11', 'test12': None},
    'group2': {   'test21': 'aaa21', 'test22': 'aa'}}

推荐答案

您的问题太复杂了，无法一一解答.但我会抛出一些初步的想法.

Your question is too complicated to understand and respond to in one try. But I'll throw out some preliminary ideas.

是的，argument_groups 只是帮助中对参数进行分组的一种方式.它们对解析没有影响.

Yes, argument_groups are just a way of grouping arguments in the help. They have no effect on parsing.

另一个最近的 SO 询问解析参数组:

Another recent SO asked about parsing groups of arguments:

是否可以只用argparse解析一个参数组的参数?

那个发帖人最初想使用一个组作为解析器，但是 argparse 类结构不允许这样做.argparse 以对象样式编写.parser=ArguementParser... 创建一类对象，parser.add_arguement... 创建另一类，add_argument_group... 再创建一个.您可以通过子类化 ArgumentParser 或 HelpFormatter 或 Action 类等来自定义它.

That poster initially wanted to use a group as a parser, but the argparse class structure does not allow that. argparse is written in object style. parser=ArguementParser... creates one class of object, parser.add_arguement... creates another, add_argument_group... yet another. You customize it by subclassing ArgumentParser or HelpFormatter or Action classes, etc.

我提到了一个 parents 机制.您定义一个或多个父解析器，并使用它们来填充您的主"解析器.它们可以独立运行(使用 parse_known_args)，而 'main' 用于处理帮助.

I mentioned a parents mechanism. You define one or more parent parsers, and use those to populate your 'main' parser. They could be run indepdently (with parse_known_args), while the 'main' is used to handle help.

我们还讨论了在解析后对参数进行分组.namespace 是一个简单的对象，其中每个参数都是一个属性.它也可以转换为字典.从字典中提取项目组很容易.

We also discussed grouping the arguments after parsing. A namespace is a simple object, in which each argument is an attribute. It can also be converted to a dictionary. It is easy to pull groups of items from a dictionary.

关于使用多个子解析器有很多问题.这是一个尴尬的提议.有可能，但并不容易.子解析器就像向系统程序发出命令.您通常每次调用发出一个命令.您不会嵌套它们或发出序列.您让 shell 管道和脚本处理多个操作.

There have SO questions about using multiple subparsers. That's an awkward proposition. Possible, but not easy. Subparsers are like issueing a command to a system program. You generally issue one command per call. You don't nest them or issue sequences. You let shell piping and scripts handle multiple actions.

IPython 使用 argparse 来解析它的输入.它首先捕获帮助，然后发布自己的信息.大多数参数来自配置文件，因此可以使用默认配置、自定义配置和命令行设置值.这是命名大量参数的示例.

IPython uses argparse to parse its inputs. It traps help first, and issues its own message. Most arguments come from config files, so it is possible to set values with default configs, custom configs and in the commandline. It's an example of naming a very large set of arguments.

子解析器允许您使用相同的参数名称，但不能在一次无用的调用中调用多个子解析器.即使您可以调用多个子解析器，它们仍然会将参数放在同一个命名空间中.此外 argparse 尝试以独立于顺序的方式处理标记的参数.因此，命令行末尾的 --foo 被解析为与开头相同.

Subparsers let you use the same argument name, but without being able to invoke multiple subparsers in one call that doesn't help much. And even if you could invoke several subparsers, they would still put the arguments in the same namespace. Also argparse tries to handle flaged arguments in an order independent manner. So a --foo at the end of the command line gets parsed the same as though it were at the start.

我们讨论过使用参数名称 ('dest') 的问题，例如 'group1.argument1'，我什至讨论过使用嵌套命名空间.如果有帮助，我可以查一下.

There was SO question where we discussed using argument names ('dest') like 'group1.argument1', and I've even discussed using nested namespaces. I could look those up if it would help.

另一个想法 - 在将它传递给一个或多个解析器之前加载 sys.argv 并对其进行分区.您可以将其拆分为某个关键字或前缀等.

Another thought - load sys.argv and partition it before passing it to one or more parsers. You could split it on some key word, or on prefixes etc.

这篇关于同时解析python中的多个子命令或以其他方式对解析的参数进行分组的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！