同时解析python中的多个子命令或以其他方式对已解析的参数进行分组 [英] Parse multiple subcommands in python simultaneously or other way to group parsed arguments

问题描述

我正在将Bash shell安装程序实用程序转换为Python 2.7，并且需要实现复杂的CLI，因此我能够解析数十个参数(可能多达150个).这些是Puppet类变量的名称，此外还有十几个常规部署选项(在Shell版本中可用).

I am converting Bash shell installer utility to Python 2.7 and need to implement complex CLI so I am able to parse tens of parameters (potentially up to ~150). These are names of Puppet class variables in addition to a dozen of generic deployment options, which where available in shell version.

但是，在我开始添加更多变量之后，我面临着一些挑战: 1.我需要将参数分组到单独的词典中，以便将部署选项与Puppet变量分开.如果将它们扔到同一个存储桶中，那么我将不得不编写一些逻辑来对它们进行排序，可能会重命名参数，然后字典合并将变得不容易. 2.可能存在名称相同但属于不同Puppet类的变量，因此我认为子命令将允许我过滤行进内容并避免名称冲突.

However after I have started to add more variables I faced are several challenges: 1. I need to group parameters into separate dictionaries so deployment options are separated from Puppet variables. If they are thrown into the same bucket, then I will have to write some logic to sort them, potentially renaming parameters and then dictionary merges will not be trivial. 2. There might be variables with the same name but belonging to different Puppet class, so I thought subcommands would allow me to filter what goes where and avoiding name collisions.

现在，我已经通过简单地添加多个解析器来实现参数解析:

At the momment I have implemented parameter parsing via simply adding multiple parsers:

parser = argparse.ArgumentParser(description='deployment parameters.')
env_select = parser.add_argument_group(None, 'Environment selection')
env_select.add_argument('-c', '--client_id',  help='Client name to use.')
env_select.add_argument('-e', '--environment', help='Environment name to use.')
setup_type = parser.add_argument_group(None, 'What kind of setup should be done:')
setup_type.add_argument('-i', '--install', choices=ANSWERS, metavar='', action=StoreBool, help='Yy/Nn Do normal install and configuration')
# MORE setup options
...
args, unk = parser.parse_known_args()
config['deploy_cfg'].update(args.__dict__)

pup_class1_parser = argparse.ArgumentParser(description=None)
pup_class1 = pup_class1_parser.add_argument_group(None, 'Puppet variables')
pup_class1.add_argument('--ad_domain', help='AD/LDAP domain name.')
pup_class1.add_argument('--ad_host', help='AD/LDAP server name.')
# Rest of the parameters

args, unk = pup_class1_parser.parse_known_args()
config['pup_class1'] = dict({})
config['pup_class1'].update(args.__dict__)
# Same for class2, class3 and so on.

这种方法不能解决问题2的问题.此外，第一个解析器使用"-h"选项，其余参数未显示在帮助中.

The problem with this approach that it does not solve issue 2. Also first parser consumes "-h" option and rest of parameters are not shown in help.

我尝试使用示例作为答案，但我无法一次使用这两个命令.

I have tried to use example selected as an answer but I was not able to use both commands at once.

## This function takes the 'extra' attribute from global namespace and re-parses it to create separate namespaces for all other chained commands.
def parse_extra (parser, namespace):
  namespaces = []
  extra = namespace.extra
  while extra:
    n = parser.parse_args(extra)
    extra = n.extra
    namespaces.append(n)

  return namespaces

pp = pprint.PrettyPrinter(indent=4)

argparser=argparse.ArgumentParser()
subparsers = argparser.add_subparsers(help='sub-command help', dest='subparser_name')

parser_a = subparsers.add_parser('command_a', help = "command_a help")
## Setup options for parser_a
parser_a.add_argument('--opt_a1', help='option a1')
parser_a.add_argument('--opt_a2', help='option a2')

parser_b = subparsers.add_parser('command_b', help = "command_b help")
## Setup options for parser_a
parser_b.add_argument('--opt_b1', help='option b1')
parser_b.add_argument('--opt_b2', help='option b2')


## Add nargs="*" for zero or more other commands
argparser.add_argument('extra', nargs = "*", help = 'Other commands')

namespace = argparser.parse_args()
pp.pprint(namespace)
extra_namespaces = parse_extra( argparser, namespace )
pp.pprint(extra_namespaces)

导致我进入

$ python argtest.py command_b --opt_b1 b1 --opt_b2 b2 command_a --opt_a1 a1
usage: argtest.py [-h] {command_a,command_b} ... [extra [extra ...]]
argtest.py: error: unrecognized arguments: command_a --opt_a1 a1

相同的结果是当我尝试使用两个子解析器定义父对象时.

The same result was when I tried to define parent with two child parsers.

问题

1. 我可以以某种方式使用parser.add_argument_group进行参数解析还是仅用于帮助打印输出中的分组?它将解决问题1而不会丢失帮助副作用.将其作为parse_known_args(namespace=argument_group)传递(如果我正确地记得我的实验)可以获取所有变量(没关系)，还可以获取结果字典中的所有Python对象内容(这对hieradata YAML不利)
2. 第二个示例中我缺少允许使用多个子命令的内容吗?还是用argparse不可能?
3. 还有其他建议将命令行变量分组吗?我已经看过Click，但是没有发现我的任务比标准argparse有任何优势.

1. Can I somehow use parser.add_argument_group for argument parsing or is it just for the grouping in help print out? It would solve issue 1 without missing help side effect. Passing it as parse_known_args(namespace=argument_group) (if I correctly recall my experiments) gets all the variables (thats ok) but also gets all Python object stuff in resulting dict (that's bad for hieradata YAML)
2. What I am missing in the second example to allow to use multiple subcommands? Or is that impossible with argparse?
3. Any other suggestion to group command line variables? I have looked at Click, but did not find any advantages over standard argparse for my task.

注意:我是sysadmin，不是程序员，所以请对我进行非对象样式编码. :)

Note: I am sysadmin, not a programmer so be gently on me for the non object style coding. :)

谢谢

已解决 参数分组通过 hpaulj 建议的答案解决.

RESOLVED Argument grouping solved via the answer suggested by hpaulj.

import argparse
import pprint
parser = argparse.ArgumentParser()

group_list = ['group1', 'group2']

group1 = parser.add_argument_group('group1')
group1.add_argument('--test11', help="test11")
group1.add_argument('--test12', help="test12")

group2 = parser.add_argument_group('group2')
group2.add_argument('--test21', help="test21")
group2.add_argument('--test22', help="test22")

args = parser.parse_args()
pp = pprint.PrettyPrinter(indent=4)

d = dict({})

for group in parser._action_groups:
    if group.title in group_list:
        d[group.title]={a.dest:getattr(args,a.dest,None) for a in group._group_actions}

print "Parsed arguments"
pp.pprint(d)

这使我获得了期望的结果No.1.直到我将有多个具有相同名称的参数.解决方案可能看起来很丑陋，但至少可以按预期工作.

This gets me desired result for the issue No.1. until I will have multiple parameters with the same name. Solution may look ugly, but at least it works as expected.

python argtest4.py --test22 aa  --test11 yy11 --test21 aaa21
Parsed arguments
{   'group1': {   'test11': 'yy11', 'test12': None},
    'group2': {   'test21': 'aaa21', 'test22': 'aa'}}

推荐答案

您的问题太复杂了，无法一口气理解和回答.但是，我会提出一些初步的想法.

Your question is too complicated to understand and respond to in one try. But I'll throw out some preliminary ideas.

是的，argument_groups只是对帮助中的参数进行分组的一种方式.它们对解析没有影响.

Yes, argument_groups are just a way of grouping arguments in the help. They have no effect on parsing.

最近又有一个SO询问有关解析参数组的信息:

Another recent SO asked about parsing groups of arguments:

是否可以仅用argparse解析一个参数组的参数?

该发帖人最初希望使用组作为解析器，但是argparse类结构不允许这样做. argparse以对象样式编写. parser=ArguementParser...创建一个类的对象，parser.add_arguement...创建另一个类，add_argument_group...另一个.您可以通过将ArgumentParser或HelpFormatter或Action类等子类化来对其进行自定义.

That poster initially wanted to use a group as a parser, but the argparse class structure does not allow that. argparse is written in object style. parser=ArguementParser... creates one class of object, parser.add_arguement... creates another, add_argument_group... yet another. You customize it by subclassing ArgumentParser or HelpFormatter or Action classes, etc.

我提到了parents机制.您定义一个或多个父解析器，并使用这些解析器填充主"解析器.它们可以独立运行(使用parse_known_args)，而"main"用于处理帮助.

I mentioned a parents mechanism. You define one or more parent parsers, and use those to populate your 'main' parser. They could be run indepdently (with parse_known_args), while the 'main' is used to handle help.

我们还讨论了解析后对参数进行分组的问题. namespace是一个简单的对象，其中的每个参数都是一个属性.也可以将其转换为字典.从字典中提取项目组很容易.

We also discussed grouping the arguments after parsing. A namespace is a simple object, in which each argument is an attribute. It can also be converted to a dictionary. It is easy to pull groups of items from a dictionary.

对于使用多个子解析器有一些疑问.这是一个尴尬的主张.可能，但不容易.子解析器就像向系统程序发出命令.通常，每个调用发出一个命令.您不嵌套它们或发出序列.您可以让外壳管道和脚本处理多个操作.

There have SO questions about using multiple subparsers. That's an awkward proposition. Possible, but not easy. Subparsers are like issueing a command to a system program. You generally issue one command per call. You don't nest them or issue sequences. You let shell piping and scripts handle multiple actions.

IPython使用argparse解析其输入.它首先捕获帮助，然后发出自己的消息.大多数参数来自配置文件，因此可以使用默认配置，自定义配置以及在命令行中设置值.这是命名大量参数的示例.

IPython uses argparse to parse its inputs. It traps help first, and issues its own message. Most arguments come from config files, so it is possible to set values with default configs, custom configs and in the commandline. It's an example of naming a very large set of arguments.

子解析器使您可以使用相同的参数名称，但无法在一次调用中调用多个子解析器，这并没有多大帮助.即使您可以调用多个子解析器，它们仍会将参数放在相同的名称空间中.同样，argparse尝试以顺序无关的方式处理标记的参数.因此，在命令行末尾的--foo的解析与开始时的解析相同.

Subparsers let you use the same argument name, but without being able to invoke multiple subparsers in one call that doesn't help much. And even if you could invoke several subparsers, they would still put the arguments in the same namespace. Also argparse tries to handle flaged arguments in an order independent manner. So a --foo at the end of the command line gets parsed the same as though it were at the start.

有一个问题，我们在其中讨论了使用像'group1.argument1'这样的参数名称(目标")，甚至讨论了使用嵌套名称空间的问题.如果可以的话，我可以查一下.

There was SO question where we discussed using argument names ('dest') like 'group1.argument1', and I've even discussed using nested namespaces. I could look those up if it would help.

另一种想法-加载sys.argv并对其进行分区，然后再将其传递给一个或多个解析器.您可以将其拆分为某些关键字或前缀等.

Another thought - load sys.argv and partition it before passing it to one or more parsers. You could split it on some key word, or on prefixes etc.

这篇关于同时解析python中的多个子命令或以其他方式对已解析的参数进行分组的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！