是否可以在AWS Cloudformation中指定数据格式转换？ [英] Is it possible to specify data format conversion in AWS Cloudformation?

问题描述

AWS文档使您看起来好像可以为 AWS :: KinesisFirehose :: DeliveryStream 指定 DataFormatConversionConfiguration 以cloudformation的形式提供，但没有关于该属性应该去哪里的文档。尝试将其添加到属性， ExtendedS3DestinationConfiguration ， ProcessingConfiguration 和处理器之一。每次CF都会抱怨-

The AWS docs makes it look as if you can specify DataFormatConversionConfiguration for a AWS::KinesisFirehose::DeliveryStream in cloudformation, but is not documentation on where the property is supposed to go. Tried adding it under Properties, ExtendedS3DestinationConfiguration, ProcessingConfiguration, and one of the Processors. Each time, CF complains with-

以下资源未能更新：[EventFirehose]。
12:24:32 UTC-0500

The following resource(s) failed to update: [EventFirehose]. 12:24:32 UTC-0500

UPDATE_FAILED AWS :: KinesisFirehose :: DeliveryStream EventFirehose遇到不受支持的属性DataFormatConversionConfiguration

UPDATE_FAILED AWS::KinesisFirehose::DeliveryStream EventFirehose Encountered unsupported property DataFormatConversionConfiguration

有自己的文档说-

如果您想让Kinesis Data Firehose转换格式从JSON到Parquet或ORC的输入数据中，请在ExtendedS3DestinationConfiguration或ExtendedS3DestinationUpdate中指定可选的DataFormatConversionConfiguration元素。

If you want Kinesis Data Firehose to convert the format of your input data from JSON to Parquet or ORC, specify the optional DataFormatConversionConfiguration element in ExtendedS3DestinationConfiguration or in ExtendedS3DestinationUpdate.

我在做什么错

推荐答案

根据SDK 文档，它应位于 ExtendedS3DestinationConfiguration 或 ExtendedS3DestinationUpdate 。但是，根据不支持此属性-extendeds3destinationconfiguration.html rel = nofollow noreferrer>文档。这是cloudformation与其他AWS服务之间非常常见的差异。这里提到了一个类似的问题 AWS ECS：严重ECS Service Cloudformation模板中的错误（最近已解决）。

As per SDK documentation, it should be inside ExtendedS3DestinationConfiguration or ExtendedS3DestinationUpdate. However, the cloudformation is currently NOT supporting this property as per this docs. This is a very common discrepancy between cloudformation and other AWS services. One similar issue is mentioned here AWS ECS: Severe bug in ECS Service Cloudformation template (which got resolved recently).

目前，您可以通过SDK更新或等待一段时间以等待cloudformation赶上。

For the time being, you can update via SDK or wait for some time for cloudformation to catch up.

如果要让Kinesis Data Firehose将输入数据的格式从JSON转换为Parquet或ORC，请在ExtendedS3DestinationConfiguration或ExtendedS3DestinationUpdate中指定可选的DataFormatConversionConfiguration元素

If you want Kinesis Data Firehose to convert the format of your input data from JSON to Parquet or ORC, specify the optional DataFormatConversionConfiguration element in ExtendedS3DestinationConfiguration or in ExtendedS3DestinationUpdate

编辑

截至2019年6月， DataFormatConversionConfiguration 属性在CloudFormation中可用。请参阅更改日志： https://docs.aws.amazon.com/ AWSCloudFormation / latest / UserGuide / ReleaseHistory.html

As of June 2019, the DataFormatConversionConfiguration property is in available in CloudFormation. See change log: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/ReleaseHistory.html

这篇关于是否可以在AWS Cloudformation中指定数据格式转换？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！