如何解决Azure数据工厂ARM模板参数限制? [英] How to get around Azure Data Factory ARM Template parameter limits?

问题描述

我已经在Azure中创建了一个数据工厂，其中包含大约10-15个管道，并且它们都可以正常工作，并且我可以调试并运行它们，而不会在已设置的测试数据工厂上出现问题.我还使用CI/CD管道设置了与数据工厂的Azure DevOps集成.

I have created an Data Factory in Azure with a about 10-15 pipelines in it and they all work fine and I can debug and run them without issue on the Test Data Factory that I have setup. I have also setup Azure DevOps integration with the Data Factory with a CI/CD pipeline.

最初，该管道也运行良好，我能够使用生成的ARM模板在生产数据工厂实例上重新创建Pipelines/Triggers/etc.

Initially this pipeline worked fine as well and I was able to use the generated ARM template to recreate the Pipelines/Triggers/etc on a production Data Factory instance.

现在，我向数据工厂添加了一些内容，当它生成ARM模板时，它现在创建一个带有293参数的模板，该模板在通过CI/CD管道运行时会失败，因为仅允许256个参数带有ARM模板.

Now I have added a few more things to the Data Factory and when it generates the ARM template now it creates a template that with 293 parameters, which when it is run through the CI/CD pipeline fails because only 256 parameters are allowed with an ARM template.

我放入参数文件中的许多项目都不需要参数化，例如文件名和文件路径.我真正想要参数化的是各种连接信息.

A lot of the items that it put in the parameter file I do not need to be parameterized, like filename and file paths. All I really want to have parameterized is the various connection information.

我尝试创建第二个ARM模板参数文件并删除不需要的参数，因为我阅读了一些文档，该文档能够添加一些附加参数，但这是行不通的，因为它没有这样做. t从ARM模板本身中删除参数.

I tried to create a second ARM Template parameter file and remove the parameters that I didn't want because I read some documentation about being able to do this to add some additional parameters, but this doesn't work because it doesn't remove the parameters out of the ARM Template itself.

所以我的问题是，有没有办法解决这个问题?我能想到的事情是创建多个数据工厂，并且每个工厂只有几个管道.我真的不喜欢这个主意，因为随着时间的流逝，它会变得非常庞大和繁琐.我可以从参数文件中手动删除模板，也可以从模板本身中删除模板，但是我真的不喜欢这种解决方案，因为它是手动的，而且很容易出错.

So my question is, is there a way to handle this? The things that I can think of doing are creating multiple Data Factories and only having a couple of pipelines in each one. I don't really like this idea because it would become very large and cumbersome over time. I could manually remove the templates out of the parameter file and also out of the template itself, but I really don't like this solution as it is manual and really error prone.

我理想地希望做的是定义我想在管道中参数化的项目，就像我提到的90％的参数化参数我不需要参数化，但是我可以找不到任何方法(缺少手动操作).

What I would ideally like to be able to do is to define which items I want to be parameterized in the Pipeline, like I mentioned 90% of the things that it parameterized I don't need to be parameterized, but I can't find any way to do this (short of manually doing it).

推荐答案

2种方式:

1. 显而易见-删除不需要的参数
2. 不那么明显-使用对象而不是字符串.

让我解释一下，假设您有5个输入参数，分别称为param1,2,3,4,5.您可以将它们压缩"为单个参数，如下所示:

let me explain, imagine you have 5 input parameters called param1,2,3,4,5. you can "compress" them into a single parameter like this:

"param": {
    "type": object,
    "defaultValue": {
        "param1": "something",
        xxx
        "param5": "otherthing"
    }
}

看到您有290个参数，我可能会走对象路线，因为那样您就可以对它们进行逻辑分组.

seeing you have 290 parameters, I'd probably go the object route, because that way you can logically group them.

关于不对事物进行参数化-您可以为它们定义默认值的变量(但是，如果您定义默认值，它们仍会计入参数)，因此变量允许您执行此操作.

as for not parameterizing thing - you can define variable of default values for them (but if you define default values they still count for parameters), so variables allow you to do that.

这篇关于如何解决Azure数据工厂ARM模板参数限制?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！