如何在 BigQuery 中获取给定存储库的 GitHub 星总数? [英] How to get total number of GitHub stars for a given repo in BigQuery?

问题描述

我的目标是随着时间的推移跟踪我的 BigQuery 存储库的受欢迎程度.

我想使用公开可用的 BigQuery 数据集，例如

天真计数:有些人明星和取消明星，然后再次明星.这会创建重复的 WatchEvent.

按演员 id 计数唯一:每个人只能出演一次.我们可以计算这些(但我们不知道它们是否未加星标，所以总数会低于这个).

由演员登录唯一:某些历史月份缺少actor.id"字段.我们可以改为查看actor.login"字段(但有些人更改了他们的登录名).

或者，感谢 GHTorrent 项目:

#standardSQL选择 COUNT(*) 颗星来自`ghtorrent-bq.ght_2017_01_19.watchers` a加入`ghtorrent-bq.ght_2017_01_19.projects` bON a.repo_id=b.idWHERE url = 'https://api.github.com/repos/angular/angular'限制 10

20567，截至 2017 年 1 月 19 日.

<小时>

相关:

• 当项目更改名称时会发生什么?

https://stackoverflow.com/a/42935592/132438

• 如何在更新之前获取更新的 GHtorrent 数据?

https://stackoverflow.com/a/42935662/132438

My goal is to track over time the popularity of my BigQuery repo.

I want to use publicly available BigQuery datasets, like GitHub Archive or the GitHub dataset

The GitHub dataset sample_repos does not contain a snapshot of the star counts:

SELECT
  watch_count
FROM
  [bigquery-public-data:github_repos.sample_repos]
WHERE
  repo_name == 'angular/angular'

returns 5318.

GitHub Archive is a timeline of event. I can try to sum them all, but the numbers do not match with the numbers in the GitHub UI. I guess because it does not count unstar actions. Here is the query I used:

SELECT
  COUNT(*)
FROM
  [githubarchive:year.2011],
  [githubarchive:year.2012],
  [githubarchive:year.2013],
  [githubarchive:year.2014],
  [githubarchive:year.2015],
  [githubarchive:year.2016],
  TABLE_DATE_RANGE([githubarchive:day.], TIMESTAMP('2017-01-01'), TIMESTAMP('2017-03-30') )
WHERE
  repo.name == 'angular/angular'
  AND type = "WatchEvent"

returns 24144

The real value is 21,921

解决方案

#standardSQL
SELECT 
  COUNT(*) naive_count,
  COUNT(DISTINCT actor.id) unique_by_actor_id, 
  COUNT(DISTINCT actor.login) unique_by_actor_login 
FROM `githubarchive.month.*` 
WHERE repo.name = 'angular/angular'
AND type = "WatchEvent"

Naive count: Some people star and un-star, and star again. This creates duplicate WatchEvents.

Unique by actor id count: Each person can only star once. We can count those (but we don't know if they un-starred, so the total count will be lower than this).

Unique by actor login: Some historical months are missing the 'actor.id' field. We can look at the 'actor.login' field instead (but some people change their logins).

Alternatively, thanks to GHTorrent project:

#standardSQL
SELECT COUNT(*) stars
FROM `ghtorrent-bq.ght_2017_01_19.watchers` a
JOIN `ghtorrent-bq.ght_2017_01_19.projects` b
ON a.repo_id=b.id
WHERE url = 'https://api.github.com/repos/angular/angular'
LIMIT 10

20567, as of 2017/01/19.

---

Related:

• What happens when a project changes it's name?

https://stackoverflow.com/a/42935592/132438

• How to get updated GHtorrent data, before they update it?

https://stackoverflow.com/a/42935662/132438

这篇关于如何在 BigQuery 中获取给定存储库的 GitHub 星总数?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！