nodejs-解析分块的twitter json [英] nodejs - parsing chunked twitter json

问题描述

nodejs服务器从Twitter获取"此JSON流并将其发送到客户端:

The nodejs server 'gets' this JSON stream from Twitter and sends it to the client:

stream.twitter.com/1/statuses/filter.json?track=gadget

返回给客户端的数据是块状" JSON，并且客户端上的JSON.parse(chunk)和eval('('+'chunk +')')都会导致解析错误. 连接被卡住的片段并等待结束"事件也不是解决方案

The data returned to the client is 'chunked' JSON and both JSON.parse(chunk) and eval('(' + chunk + ')') on the client side result in parsing errors. Concatenating the chucked pieces and waiting for the 'end' event isn't a solution either

我注意到以前的示例在客户端上使用了类似的东西，显然以前是这样的:

I noticed previous samples used something like this on the client side that apparently worked before:

  socket.onmessage = function(chunk) { 
  data = eval("(" + chunk.data + ")");
  alert(data.user.screen_name);

我在客户端使用它，它导致解析错误:

I'm using this on the client side and it results in a parsing error:

var socket = new io.Socket();
    socket.on('message', function(chunk) { 
    var data = eval('(' + chunk + ')'); // parsing error
    alert(data.screen_name): 

我知道它成功返回了带有以下内容的JSON块:

I know that its successfully returning a JSON chunk with:

  var socket = new io.Socket();
        socket.on('message', function(chunk) {  
        alert(chunk): // shows a JSON chunk

服务器:

  response.on('data', function (chunk) {
    client.each(function(e) {
      e.send(chunk);  
  });  

有什么改变还是我做错了什么?

Did something change or what else em I doing wrong?

更新:结束"事件由于其流而不会触发?

UPDATE: The 'end' event does not fire because its streaming?

http.get({
  headers: { 'content-type': 'application/json' },
  host: 'stream.twitter.com',
  path: '/1/statuses/filter.json?track...
}, function(res) {

  res.setEncoding('utf8');
  res.on('data', function (chunk) {
    client.each(function(e) {
      e.send(chunk);  
  });  
 });

  // does not fire
  res.on('end', function () {

  });  

...

在发送分块数据方面，我正在研究与http 1.0和http 1.1的区别.

I'm looking into the difference with http 1.0 and http 1.1 as far as sending chunked data.

推荐答案

查看标题为在Twitter文档中解析响应.

从Streaming API解析JSON响应很简单，每个对象都在其自己的行上返回，并以回车结尾.换行符(\ n)可能出现在对象元素(例如状态对象的文本元素)中，但回车符(\ r)不应出现.

Parsing JSON responses from the Streaming API is simple every object is returned on its own line, and ends with a carriage return. Newline characters (\n) may occur in object elements (the text element of a status object, for example), but carriage returns (\r) should not.

在服务器端，不断堆积数据块，直到看到回车符"\r".找到回车符后，提取字符串直至回车符，这将给我们一条鸣叫.

On the server side, keep accumulating chunks until you see the carriage return "\r". Once the carriage return is found, extract the string up to the carriage return, and that gives us one tweet.

var message = ""; // variable that collects chunks
var tweetSeparator = "\r";

res.on('data', function(chunk) {
    message += chunk;

    var tweetSeparatorIndex = message.indexOf(tweetSeparator);
    var didFindTweet = tweetSeparatorIndex != -1;

    if (didFindTweet) {
        var tweet = message.slice(0, tweetSeparatorIndex);
        clients.forEach(function(client) {
            client.send(tweet);
        });
        message = message.slice(tweetSeparatorIndex + 1);
    }
});

客户端变得简单.只需将套接字消息整体解析为JSON.

The client becomes simple. Simply parse the socket message as JSON in its entirety.

socket.on('message', function(data) {
    var tweet = JSON.parse(data);
});

这篇关于nodejs-解析分块的twitter json的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！