为什么 Flink SQL 对所有表都使用 100 行的基数估计? [英] Why does Flink SQL use a cardinality estimate of 100 rows for all tables?

问题描述

我不确定为什么在中没有正确评估逻辑计划这个例子.

我更深入地查看了 Flink 基本代码，并在 calcite 评估/估计对象中查询的行数时检查了这一点.出于某种原因，对于任何表源，它总是返回100.

I looked more deeply in the Flink base code and I checked that when calcite evaluate/estimate the number of rows for the query in object. For some reason it returns always 100 for any table source.

实际上在 Flink 中，在程序计划创建的过程中，对于每个转换后的规则，它被称为 VolcanoPlanner 类由 TableEnvironment.runVolcanoPlanner.规划器尝试通过调用 RelMetadataQuery.getRowCount

In Flink in fact, during the process of the program plan creation, for each transformed rule it is called the VolcanoPlanner class by the TableEnvironment.runVolcanoPlanner. The planner try to optimise and calculate some estimation by calling RelMetadataQuery.getRowCount

我通过创建一个失败的test 应该断言 0 作为关系表 'S' 的行数，但它总是返回 100.

I reproduced the error by creating a failing test which should assert 0 as row count for relation table 'S' but it returns always 100.

为什么会这样?有人对这个问题有答案吗?

Why this is happening? Does anyone has an answer to this issue?

推荐答案

在当前版本(1.7.1，2019 年 1 月)中，Flink 的关系 API(表 API 和 SQL)不会尝试估计基表的基数.因此，Calcite 使用其默认值 100.

In the current version (1.7.1, Jan 2019), Flink's relational APIs (Table API and SQL) do not attempt to estimate the cardinality of base tables. Hence, Calcite uses its default value which is 100.

这适用于过滤器和投影下推等基本优化，目前已经足够了，因为 Flink(尚未)重新排序连接.

This works fine for basic optimizations like filter and projection push-down and is currently sufficient because Flink does not (yet) reorder joins.

为表注入基数估计的唯一方法是通过 ExternalCatalog.

The only way to inject cardinality estimates for tables is via an ExternalCatalog.

这篇关于为什么 Flink SQL 对所有表都使用 100 行的基数估计?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！