如何在Python Luigi中使用参数 [英] How to use Parameters in Python Luigi

问题描述

如何将参数传递给Luigi?如果我有一个名为FileFinder.py的python文件，其类名为getFIles:

How do I pass in parameters to Luigi? if I have a python file called FileFinder.py with a class named getFIles:

class getFiles(luigi.Task):

，我想将目录传递给此类，例如:

and I want to pass in a directory to this class such as:

C://Documents//fileName

然后在我的运行方法中使用此参数

and then use this parameter in my run method

def run(self):

如何在命令行中运行此命令并添加要在我的代码中使用的参数?我习惯于在命令行中运行该文件，如下所示:

how do I run this in command line and add the parameter for use in my code? I am accustomed to running this file in command line like this:

python FileFinder.py getFiles --local-scheduler

我要在代码中添加哪些内容以使用参数，以及如何将该参数添加至命令行参数?

What do I add to my code to use a parameter, and how do I add that parameter to the command line argument?

此外，作为此问题的扩展，我将如何使用多个参数?或不同数据类型的参数(例如字符串或列表)?

Also, as an extension of this question, how would I use multiple arguments? or arguments of different data types such as strings or lists?

推荐答案

您已经知道，可以通过以下方式将参数传递给luigi

As you have already figured out, you can pass arguments to luigi via

--param-name param-value

在命令行中输入

.在代码内部，您必须通过实例化Parameter类或其子类之一来声明这些变量.子类用于告诉luigi变量是否具有非字符串的数据类型.这是一个使用两个命令行参数的示例，一个为Int和一个为List:

in the command line. Inside your code, you have to declare these variables by instantiating the Parameter class or one of it's subclasses. The subclasses are used to tell luigi if the variable has a data-type that is not string. Here is an example which uses two command line arguments, one Int and one List:

import luigi

class testClass(luigi.Task):
  int_var = luigi.IntParameter()
  list_var = luigi.ListParameter()

  def run(self):
      print('Integer Param + 1 = %i' % (self.int_var + 1))

      list_var = list(self.list_var)
      list_var.append('new_elem')
      print('List Param with added element: ' + str(list_var))

请注意，Luigi实际上将ListParams转换为元组，因此，如果要对其进行列表操作，则必须先将其转换回原先(这是

Note that ListParams actually get converted to tuples by luigi, so if you want to do list operations on them, you have to convert them back first (This is a known issue, but doesn't look like it will be fixed soon).

您可以像这样从命令行调用上述模块(我已将代码保存为名为"testmodule.py"的文件，并从同一目录内进行了调用):

You can invoke the above module from the command line like this (i have saved the code as a file called "testmodule.py" and made the call from inside the same directory):

luigi --module testmodule testClass --int-var 3 --list-var '[1,2,3]'  --local-scheduler

请注意，对于包含_的变量，必须将其替换为-. 呼叫产生(伴随许多状态消息):

Note here that for variables containing a _, this has to be replaced by -. The call yields (along with many status messages):

Integer Param + 1 = 4
List Param with added element: [1, 2, 3, 'new_elem']

这篇关于如何在Python Luigi中使用参数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！