nodejs - 解析分块的 twitter json [英] nodejs - parsing chunked twitter json

问题描述

The nodejs server 'gets' this JSON stream from Twitter and sends it to the client:

stream.twitter.com/1/statuses/filter.json?track=gadget

The data returned to the client is 'chunked' JSON and both JSON.parse(chunk) and eval('(' + chunk + ')') on the client side result in parsing errors. Concatenating the chucked pieces and waiting for the 'end' event isn't a solution either

I noticed previous samples used something like this on the client side that apparently worked before:

  socket.onmessage = function(chunk) { 
  data = eval("(" + chunk.data + ")");
  alert(data.user.screen_name);

I'm using this on the client side and it results in a parsing error:

var socket = new io.Socket();
    socket.on('message', function(chunk) { 
    var data = eval('(' + chunk + ')'); // parsing error
    alert(data.screen_name): 

I know that its successfully returning a JSON chunk with:

  var socket = new io.Socket();
        socket.on('message', function(chunk) {  
        alert(chunk): // shows a JSON chunk

Server:

  response.on('data', function (chunk) {
    client.each(function(e) {
      e.send(chunk);  
  });  

Did something change or what else em I doing wrong?

UPDATE: The 'end' event does not fire because its streaming?

http.get({
  headers: { 'content-type': 'application/json' },
  host: 'stream.twitter.com',
  path: '/1/statuses/filter.json?track...
}, function(res) {

  res.setEncoding('utf8');
  res.on('data', function (chunk) {
    client.each(function(e) {
      e.send(chunk);  
  });  
 });

  // does not fire
  res.on('end', function () {

  });  

...

I'm looking into the difference with http 1.0 and http 1.1 as far as sending chunked data.

解决方案

Look at the section titled Parsing Responses in Twitter's documentation.

Parsing JSON responses from the Streaming API is simple every object is returned on its own line, and ends with a carriage return. Newline characters ( ) may occur in object elements (the text element of a status object, for example), but carriage returns ( ) should not.

On the server side, keep accumulating chunks until you see the carriage return " ". Once the carriage return is found, extract the string up to the carriage return, and that gives us one tweet.

var message = ""; // variable that collects chunks
var tweetSeparator = "
";

res.on('data', function(chunk) {
    message += chunk;

    var tweetSeparatorIndex = message.indexOf(tweetSeparator);
    var didFindTweet = tweetSeparatorIndex != -1;

    if (didFindTweet) {
        var tweet = message.slice(0, tweetSeparatorIndex);
        clients.forEach(function(client) {
            client.send(tweet);
        });
        message = message.slice(tweetSeparatorIndex + 1);
    }
});

The client becomes simple. Simply parse the socket message as JSON in its entirety.

socket.on('message', function(data) {
    var tweet = JSON.parse(data);
});

这篇关于nodejs - 解析分块的 twitter json的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！