如何绕过 Azure 数据工厂 ARM 模板参数限制? [英] How to get around Azure Data Factory ARM Template parameter limits?

问题描述

我在 Azure 中创建了一个数据工厂，其中有大约 10-15 个管道，它们都运行良好，我可以在我设置的测试数据工厂上调试和运行它们而不会出现问题.我还使用 CI/CD 管道设置了 Azure DevOps 与数据工厂的集成.

I have created an Data Factory in Azure with a about 10-15 pipelines in it and they all work fine and I can debug and run them without issue on the Test Data Factory that I have setup. I have also setup Azure DevOps integration with the Data Factory with a CI/CD pipeline.

最初这个管道也运行良好，我能够使用生成的 ARM 模板在生产数据工厂实例上重新创建管道/触发器/等.

Initially this pipeline worked fine as well and I was able to use the generated ARM template to recreate the Pipelines/Triggers/etc on a production Data Factory instance.

现在我向数据工厂添加了更多内容，现在当它生成 ARM 模板时，它会创建一个包含 293 个参数的模板，当它通过 CI/CD 管道运行时会失败，因为只允许使用 256 个参数使用 ARM 模板.

Now I have added a few more things to the Data Factory and when it generates the ARM template now it creates a template that with 293 parameters, which when it is run through the CI/CD pipeline fails because only 256 parameters are allowed with an ARM template.

它放在参数文件中的很多项目我都不需要参数化，比如文件名和文件路径.我真正想要参数化的是各种连接信息.

A lot of the items that it put in the parameter file I do not need to be parameterized, like filename and file paths. All I really want to have parameterized is the various connection information.

我尝试创建第二个 ARM 模板参数文件并删除我不想要的参数，因为我阅读了一些关于能够执行此操作以添加一些附加参数的文档，但这不起作用，因为它不起作用't 从 ARM 模板本身中删除参数.

I tried to create a second ARM Template parameter file and remove the parameters that I didn't want because I read some documentation about being able to do this to add some additional parameters, but this doesn't work because it doesn't remove the parameters out of the ARM Template itself.

所以我的问题是，有没有办法处理这个问题?我能想到的事情是创建多个数据工厂，并且每个工厂只有几个管道.我不太喜欢这个想法，因为随着时间的推移它会变得非常庞大和笨重.我可以手动从参数文件和模板本身中删除模板，但我真的不喜欢这个解决方案，因为它是手动的，而且非常容易出错.

So my question is, is there a way to handle this? The things that I can think of doing are creating multiple Data Factories and only having a couple of pipelines in each one. I don't really like this idea because it would become very large and cumbersome over time. I could manually remove the templates out of the parameter file and also out of the template itself, but I really don't like this solution as it is manual and really error prone.

我最希望能够做的是定义我想在流水线中参数化的项目，就像我提到的 90% 它参数化的东西我不需要参数化，但我可以找不到任何方法来执行此操作(手动操作的不足).

What I would ideally like to be able to do is to define which items I want to be parameterized in the Pipeline, like I mentioned 90% of the things that it parameterized I don't need to be parameterized, but I can't find any way to do this (short of manually doing it).

推荐答案

2 种方式:

1. 明显 - 删除不需要的参数
2. 不太明显 - 使用对象而不是字符串.

让我解释一下，假设您有 5 个名为 param1,2,3,4,5 的输入参数.您可以将它们压缩"为一个参数，如下所示:

let me explain, imagine you have 5 input parameters called param1,2,3,4,5. you can "compress" them into a single parameter like this:

"param": {
    "type": object,
    "defaultValue": {
        "param1": "something",
        xxx
        "param5": "otherthing"
    }
}

看到你有 290 个参数，我可能会选择对象路由，因为这样你就可以对它们进行逻辑分组.

seeing you have 290 parameters, I'd probably go the object route, because that way you can logically group them.

至于不参数化的东西 - 你可以为它们定义默认值的变量(但如果你定义默认值，它们仍然算作参数)，所以变量允许你这样做.

as for not parameterizing thing - you can define variable of default values for them (but if you define default values they still count for parameters), so variables allow you to do that.

这篇关于如何绕过 Azure 数据工厂 ARM 模板参数限制?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！