node.js Get.Request &分页&异步 [英] node.js Get.Request & Pagination & Async

问题描述

我在这里组织流程非常困难，因为我是自学的，所以想知道是否有人可以提供帮助.

I'm having a tremendously tough time organizing the flow here as I'm self-taught so wondering if someone might be able to assist.

var channelIds = ['XYZ','ABC','QRS']
var playlistIds = [];
var videoIds = [];

流程顺序

1.获取所有播放列表 ID: 如果返回的获取请求 JSON 包含 nextPageToken，则在转到 (2) 之前再次使用该页面运行获取请求

2.获取所有视频 ID: 如果返回的获取请求 JSON 包含 nextPageToken，则在转到 (3) 之前再次使用该页面运行获取请求

3.聚合为最终数组: 我需要将所有内容放入一个数组中，例如:var ArrFinal = [{channelId,playlistID,videoId},{channelId,playlistID,videoId},{channelId,playlistID,videoId}];

我不一定需要有人来写整件事.我试图更好地了解了解上一步何时完成的最有效方法，同时还要处理 nextPageToken 迭代.

I don't necessarily need someone to write the whole thing. I'm trying to better understand the most efficient way to know when the previous step is done, but also handle the nextPageToken iteration.

推荐答案

我不熟悉 youtube api.但是您基本上需要的是每个端点的 get 函数.这个函数也应该关心nextPageToken".

i'm not familiar with the youtube api. But what you basically need is a get function for each endpoint. This function should also care about the "nextPageToken".

类似的东西:(未测试)

Something like that: (not tested)

'use strict';

const Promise = require('bluebird');
const request = Promise.promisifyAll(require('request'));

const playlistEndpoint = '/youtube/v3/playlists';
const baseUrl = 'https://www.googleapis.com'

const channelIds = ['xy', 'ab', 'cd'];

const getPlaylist = async (channelId, pageToken, playlists) => {

  const url = `${baseUrl}${playlistEndpoint}`;
  const qs = { 
    channelId,
    maxResults: 25,
    pageToken
  };

  try {
    const playlistRequest = await request.getAsync({ url, qs });
    const nextPageToken = playlistRequest.body.nextPageToken;
    // if we already had items, combine with the new ones
    const items = playlists ? playlists.concat(playlistRequest.body.items) : playlistRequest.body.items;
    if (nextPageToken) {
      // if token, do the same again and pass results to function
      return getPlaylist(channelId, nextPageToken, items);
    }
    // if no token we are finished 
    return items;
  }
  catch (e) {
    console.log(e.message);
  }
};

const getVideos = async (playlistId, pageToken, videos) => {

  // pretty much the same as above
}

function awesome(channelIds) {

  const fancyArray = [];

  await Promise.map(channelIds, async (channelId) => {

    const playlists = await getPlaylist(channelId);

    const videos = await Promise.map(playlists, async (playlistId) => {

      const videos = await getVideos(playlistId);

      videos.forEach(videoId => {
        fancyArray.push({ channelId, playlistId, videoId })
      })
    });
  });

  return fancyArray;
}

awesome(channelIds)

//更新

这可能是很多并发请求，你可以使用

This may be a lot concurrent requests, you can limit them by using

Promise.map(items, item => { somefunction() }, { concurrency: 5 });

这篇关于node.js Get.Request &分页&异步的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！