使用rtweet get_timeline()避免速率限制 [英] Avoid Rate limit with rtweet get_timeline()

问题描述

有没有阻止我的循环被速率限制打断的方法?我希望我的代码等待执行，直到可能的时限过去为止.

Is there anyway to stop my loop from being interrupted by the rate limit? I would like my code to wait to execute until the time limit has passed if possible.

另一个问题:我考虑过并行化for循环.您认为这是个好主意吗?我不确定是否有机会将数据写入错误的文件.

A side question: I thought about parallelizing the for loop. Do you think this would be a good idea? I was not sure if there would be a chance for data to be written to the wrong file.

library(rtweet)
create_token(app="Arconic Influential Followers",consumer_key,consumer_secret) 

flw <- get_followers("arconic")
fds <- get_friends("arconic")
usrs <- lookup_users(c(flw$user_id, fds$user_id))

for(i in 1:length(usrs$user_id)){

    a<-tryCatch({get_timeline(usrs$user_id[i])},
                error=function(e){message(e)}
       )
    tryCatch({save_as_csv(a,usrs$user_id[i])},
                error=function(e){message(e)}
       )

}

推荐答案

我最终要做的是创建一个while循环，检查我留在Users向量中的记录数，运行for循环，然后将系统睡眠15分钟.这种方法很好，但是需要考虑一些事项.我的while循环在200处中断，以防万一有用户没有任何数据要保存到csv中.事实证明这是一个好举动，因为如果您注意到for循环在80开始迭代.当您开始遍历用户矢量时，将反复删除好用户.这仅导致引起错误的用户.对于要完成任务的人来说，一个改进就是以编程的方式处理此问题.

What I ended up doing was create a while loop that checked the number of records I had left in my Users vector, ran my for loop, and then put the system to sleep for 15 mins. This approach is good, but there are some things to account for. I have the while loop breaking at 200 just in case there were users that didn't have any data to save into a csv. This turned out to be a good move because if you notice the for loop starts iterating at 80. As you start moving across your vector of users the good users are removed iteratively. This leaves only the users that cause errors. An improvement for someone up to the task would be to handle this programatically.

Users <- usrs$user_id
goodUsers <- substring(list.files(),1,nchar(list.files())-11)
Users <- setdiff(Users,goodUsers)

while(length(Users)>200){
    for(i in 80:length(Users)){

        a<-tryCatch({get_timeline(Users[i],usr=FALSE)},
                error=function(e){message(e)}
           )
        tryCatch({save_as_csv(a,Users[i])
                goodUsers <- append(goodUsers,Users[i])},
                    error=function(e){message(e)}
        )


    }
Users <- setdiff(Users,goodUsers)
Sys.sleep(900)
}

length(Users)
length(goodUsers)

这篇关于使用rtweet get_timeline()避免速率限制的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！