AWS Redshift-无法将外部表合并到本地目录中 [英] AWS Redshift - Failed to incorporate external table into local catalog

问题描述

我们的一个外部表在redshift中有问题.

Having a problem with one of our external tables in redshift.

我们在AWS Glue中有300多个表，这些表已作为名为 events 的外部架构添加到我们的redshift集群中. events 中的大多数表都可以进行查询.但是，当查询名为 item_loaded 的表之一时，会出现以下错误；

We have over 300 tables in AWS Glue which have been added to our redshift cluster as an external schema called events. Most of the tables in events can be queries fine. But when querying one of the tables called item_loaded we get the following error;

select * from events.item_loaded limit 1;
ERROR:  XX000: Failed to incorporate external table "events"."item_loaded" into local catalog.
LOCATION:  localize_external_table, /home/ec2-user/padb/src/external_catalog/external_catalog_api.cpp:358

奇怪的是它们在目录中；

What's weird is that they are in the catalog;

select *
from SVV_EXTERNAL_TABLES
where tablename = 'item_loaded';

-[ RECORD 1 ]-----+------------------------------------------
schemaname        | events
tablename         | item_loaded
location          | s3://my_bucket/item_loaded
input_format      | org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat
output_format     | org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat
serialization_lib | org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe 
serde_parameters  | {"serialization.format":"1"}
compressed        | 0
parameters        | {"EXTERNAL":"TRUE","parquet.compress":"SNAPPY","transient_lastDdlTime":"1504792238"}

AFAICT，此表的配置方式与相同模式中工作正常的其他表完全相同.我尝试重新创建一个指向相同AWS Glue数据库的新外部架构，但是会出现相同的问题.

AFAICT, this table is configured the exact same way as the other tables in the same schema which are working fine. I've tried recreating a new external schema pointing to the same AWS Glue database but the same issue occurs.

我还可能检查什么?是否有可能导致表从目录中删除的任何事情?

What else could I potentially check? Is there anything that could occur which would cause a table to removed from the catalog?

推荐答案

外部表中的列数超过了Redshift限制:

The external table has a number of columns which exceed the Redshift limits:

每个表
• 1,600列用于本地Redshift表
• 用于Redshift Spectrum外部表的1,598列

您可以通过查询 svv_external_columns

这篇关于AWS Redshift-无法将外部表合并到本地目录中的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！