如何使用 BigQuery 找到最常见的二元组? [英] How to find the most common bi-grams with BigQuery?

问题描述

我想在我的表中找到最常见的二元组(词对).如何使用 BigQuery 执行此操作?

I want to find the most common bi-grams (pair of words) in my table. How can I do this with BigQuery?

推荐答案

现在有了一个新函数:ML.NGRAMS():

Now with a new function: ML.NGRAMS():

WITH data AS  (
  SELECT REGEXP_EXTRACT_ALL(LOWER(title), '[a-z]+') title_arr
  FROM `fh-bigquery.reddit_posts.2019_08`  
  WHERE title LIKE '% %'
  AND score>1
)


SELECT APPROX_TOP_COUNT(bigram, 10) top
FROM (
  SELECT ML.NGRAMS(title_arr, [2,2]) x
  FROM data
), UNNEST(x) bigram
WHERE LENGTH(bigram) > 10

文档:

• https://cloud.google.com/bigquery-ml/docs/reference/standard-sql/bigqueryml-preprocessing-functions#mlngrams

这篇关于如何使用 BigQuery 找到最常见的二元组?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！