星火是否支持subqqueries？ [英] Does Spark support subqqueries?

问题描述

当我运行此查询我得到这个类型的错误

  SELECT * FROM raw_2其中ip NOT IN（选择raw_1 *）;
 

  

org.apache.spark.sql.AnalysisException：

在查询不支持的语言功能：

  SELECT * FROM raw_2其中ip NOT IN（选择raw_1 *）
  TOK_QUERY 1，0,24，14
    TOK_FROM 1，4,6，14
      TOK_TABREF 1，6,6，14
        TOK_TABNAME 1，6,6，14
          raw_2 1，6,6，14
    TOK_INSERT 0，-1,24，0
      TOK_DESTINATION 0，-1，-1,0
       TOK_DIR 0，-1，-1,0
          TOK_TMP_FILE 0，-1，-1,0
      TOK_SELECT 0，0,2，0
        TOK_SELEXPR 0，2,2，0
          TOK_ALLCOLREF 0，2,2，0
      TOK_WHERE 1，8,24，29
       不是1，10,24，29
          TOK_SUBQUERY_EXPR 1，14,10，33
            TOK_SUBQUERY_OP 1，14,14，33
              1，14,14，33
            TOK_QUERY 1，16,24，51
              TOK_FROM 1，21,23，51
                TOK_TABREF 1，23,23，51
                  TOK_TABNAME 1，23,23，51
                    raw_1 1，23,23，51
              TOK_INSERT 0，-1,19，0
          TOK_DESTINATION 0，-1，-1,0
            TOK_DIR 0，-1，-1,0
              TOK_TMP_FILE 0，-1，-1,0
          TOK_SELECT 0，17,19，0
            TOK_SELEXPR 0，19,19，0
              TOK_ALLCOLREF 0，19,19，0
      TOK_TABLE_OR_COL 1，10,10，26
        IP 1，10,10，26
 

  

scala.NotImplementedError：为ASTNode类型没有解析规则：817，文本：

  TOK_SUBQUERY_EXPR：
  TOK_SUBQUERY_EXPR 1，14,10，33
    TOK_SUBQUERY_OP 1，14,14，33
      1，14,14，33
    TOK_QUERY 1，16,24，51
      TOK_FROM 1，21,23，51
        TOK_
 

解决方案

  

星火是否支持subqqueries？

一般来说它。像构造SELECT * FROM（SELECT * FROM foo其中巴= 1）星火SQL作为TMP 完全有效的查询。

据我可以从<一讲href=\"https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/SqlParser.scala#L241\"相对=nofollow>催化剂解析器源它不支持在 NOT IN 子句内的查询：

  | termEx pression〜（不〜在〜（〜＆GT; rep1sep（termEx pression，））＆LT;〜）^^ {
    案例E1〜E2 =＆GT;不（在（E1，E2））
  }
 

它仍然有可能使用外部联接随后过滤，得到同样的效果。

When I am running this query i got this type of error

  select * from raw_2 where ip NOT IN (select * from raw_1);

org.apache.spark.sql.AnalysisException:

Unsupported language features in query:

 select * from raw_2 where ip NOT IN (select * from raw_1)
  TOK_QUERY 1, 0,24, 14
    TOK_FROM 1, 4,6, 14
      TOK_TABREF 1, 6,6, 14
        TOK_TABNAME 1, 6,6, 14
          raw_2 1, 6,6, 14
    TOK_INSERT 0, -1,24, 0
      TOK_DESTINATION 0, -1,-1, 0
       TOK_DIR 0, -1,-1, 0
          TOK_TMP_FILE 0, -1,-1, 0
      TOK_SELECT 0, 0,2, 0
        TOK_SELEXPR 0, 2,2, 0
          TOK_ALLCOLREF 0, 2,2, 0
      TOK_WHERE 1, 8,24, 29
       NOT 1, 10,24, 29
          TOK_SUBQUERY_EXPR 1, 14,10, 33
            TOK_SUBQUERY_OP 1, 14,14, 33
              IN 1, 14,14, 33
            TOK_QUERY 1, 16,24, 51
              TOK_FROM 1, 21,23, 51
                TOK_TABREF 1, 23,23, 51
                  TOK_TABNAME 1, 23,23, 51
                    raw_1 1, 23,23, 51
              TOK_INSERT 0, -1,19, 0
          TOK_DESTINATION 0, -1,-1, 0
            TOK_DIR 0, -1,-1, 0
              TOK_TMP_FILE 0, -1,-1, 0
          TOK_SELECT 0, 17,19, 0
            TOK_SELEXPR 0, 19,19, 0
              TOK_ALLCOLREF 0, 19,19, 0
      TOK_TABLE_OR_COL 1, 10,10, 26
        ip 1, 10,10, 26

scala.NotImplementedError: No parse rules for ASTNode type: 817, text:

  TOK_SUBQUERY_EXPR :
  TOK_SUBQUERY_EXPR 1, 14,10, 33
    TOK_SUBQUERY_OP 1, 14,14, 33
      IN 1, 14,14, 33
    TOK_QUERY 1, 16,24, 51
      TOK_FROM 1, 21,23, 51
        TOK_

解决方案

Does Spark support subqqueries?

Generally speaking it does. Constructs like SELECT * FROM (SELECT * FROM foo WHERE bar = 1) as tmp perfectly valid queries in the Spark SQL.

As far as I can tell from the Catalyst parser source it doesn't support inner queries in a NOT IN clause:

| termExpression ~ (NOT ~ IN ~ "(" ~> rep1sep(termExpression, ",")) <~ ")" ^^ {
    case e1 ~ e2 => Not(In(e1, e2))
  }

It is still possible to use outer join followed by filter to obtain the same effect.

这篇关于星火是否支持subqqueries？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！