处理连续的 JSON 流 [英] Process a continuous stream of JSON

问题描述

(现已失效)页面 http://stream.twitter.com/1/statuses/sample.json 用于返回连续不断的 JSON 数据流.

我想在我自己的网页中使用 jQuery(或 JavaScript，但最好是 jQuery)来处理它，以便能够根据推文的实时提要显示视觉效果.

因为据我所知，jQuery parseJSON 函数只会在服务器发送完所有数据后才执行回调函数，但这实际上是一个连续的数据流.我怎样才能在发生时"处理数据但仍然保持连接运行?

解决方案

这类事情最好使用 WebSockets 现在，根据 CanIUse.Com 的说法，它适用于所有主要浏览器a> Opera Mini 除外(有关旧版浏览器或所有浏览器的更多详细信息，请参阅该链接，并单击资源"选项卡以查看更多链接).作为概述，IE 10+、Firefox 11+(如果在 WebWorker 上下文中为 38+)、Chrome 16+、Opera 12.1+、Safari 7+、Android 4.4+、Opera Mobile 12.1+ 支持 websocket.

注意:您可能想了解 Service Workers 和 Web Workers 也是如此，尽管它们有不同的用例和不同的能力.

看起来像这样:

var connection = new WebSocket('ws://html5rocks.websocket.org/echo',['肥皂'，'xmpp']);

<块引用>

立即将一些事件处理程序附加到连接中，可以让您知道连接何时打开、何时收到传入消息或是否发生错误.

发送消息变得如此简单:

connection.send('你的消息');connection.send(binaryData);

请参阅介绍 WebSockets:将 Sockets 引入网络以获得完整的解释关于如何做到这一点.

ASP.Net 开发人员:如果出于某种原因您需要支持旧浏览器并且不想自己弄清楚如何处理那些不支持 WebSockets 的浏览器，请考虑使用诸如 SignalR.

旧版浏览器的旧 EventSource API 答案

大多数浏览器现在都实现了 EventSource API，这使得轮询真的很容易，只要可以使用内容类型text/event-stream 传送流.较旧的浏览器或那些出于任何原因无法设计流以具有该内容类型的开发人员可以使用一些帮助脚本 做同样的事情.

这是一个例子:

var jsonStream = new EventSource('https://example.com/yourstreamingservice')jsonStream.onmessage = 函数 (e) {var message = JSON.parse(e.data);//处理消息};

这基本上是我在下面概述的内容的完整版本.

针对非常老的浏览器的更旧的服务流媒体答案

你想要的是长轮询.您需要一个自定义的 AJAX onreadystatechange 处理函数.您无需等待整个流完成(因为它永远不会完成)，而是需要定期检查内容.请注意，要在 IE 9 及更低版本中使用 iframe，您需要做一些繁重的工作.

大致:

• 响应每个 onreadystatechange 事件并检查您已放弃给当前角色的流，以查看是否有足够的数据来消耗一个或多个离散事件.您需要使用 javascript 字符串处理函数自己解析流.可以使用 split、indexOf、正则表达式、循环等组合来完成此任务.
• 如果还没有足够的内容，则退出并等待下一个事件.
• 我很确定每次 onreadystatechange 处理程序触发时，responseText 将是迄今为止收到的所有数据.定义一个持久变量，用于保存尚未正确处理的第一个字符的位置.
• 一旦有足够的内容让一个或多个离散事件出现在流中，一次将它们取出一个并将它们传递给您的 JSON 解析器，以实际将文本呈现为对象.正常使用它们.

查看此HTTP Streaming gist以获取一项资源，或在 SoftwareAs 上将流式传输作为轮询服务器的替代方法.如果您必须支持 IE 9 或更早版本，则需要为此使用 iframe 方法.

这里是 引用Ajax 设计模式:使用编程和可用性模式创建 Web 2.0 站点:

<块引用>

总而言之，Service Streaming 使 HTTP Streaming 方法更加灵活，因为您可以流式传输任意内容而不是 Javascript 命令，并且因为您可以控制连接的生命周期.但是，它结合了两种跨浏览器不一致的技术，具有可预测的可移植性问题.实验表明，Page Streaming 技术在 IE [9 及更早版本] 和 Firefox 上都有效，但 Service Streaming 仅适用于 Firefox，无论是使用 XMLHTTPRequest 还是 IFrame.在第一种情况下，IE [9 及更早版本] 会抑制响应直到其完成，如果使用变通方法，则 IFrame 可以工作:IE [9 及更早版本] 在前 256 个字节后接受来自服务器的消息，因此唯一的事情要做的是在发送消息之前发送 256 个虚拟字节.在此之后，所有消息都将按预期到达.因此，在 IE [9 及更早版本] 中也可以实现完整的 Service Streaming！

请注意，它是 2006 年的，所以肯定已经过时了，但如果您必须支持旧浏览器，它仍然适用.

安全问题

普通的 AJAX 不能跨域，这意味着(现在我注意到你想从 Twitter 流式传输的事实)你将无法做你所要求的.这可以通过 JSONP 解决，但 JSONP 本质上不能进行服务流式传输，而且无论如何 twitter 也不提供.还有 Cross-Origin Resource Sharing (CORS) 但 twitter 不会设置这取决于您——这是他们只对附属于他们的域做的事情.而且 CORS 需要现代浏览器.

因此，您唯一的选择是在您的网络服务器上创建一个代理服务，为您执行对 twitter 的请求，然后分发数据.这只能从与提供主页的同一域中完成.这样做还允许您使用 iframe 技术创建一个适用于 IE 的版本.如果您不关心旧的 IE 版本，如果您知道将发出请求的域，您可以自己实现 CORS 来克服域限制.

如果您拥有对客户端软件的完全控制权(例如这是用于公司内部网)，则还有另一种选择:将 Web 浏览器托管在已编译的本地执行应用程序的用户表单中.我只使用 C# 做到了这一点，但我想其他语言也可以做到这一点.当您使用正确的浏览器对象时，因为它托管在 C# 应用程序中，C# 应用程序可以克服跨域安全限制，读取和写入所有页面内容，无论它来自哪个域.我怀疑你的情况是这样的，但我想把这个选项放在这里，供其他可能欣赏它的人使用.

The (now defunct) page http://stream.twitter.com/1/statuses/sample.json used to return a continuous and endless stream of JSON data.

I'd like to process it using jQuery (or JavaScript, but preferably jQuery) inside my own web page to be able to display visual effects based on the live feed of tweets.

Since as far as I know the jQuery parseJSON function will only execute the callback function after all the data has been sent by the server, but this is actually a continuous stream of data. How can I process the data "as it happens" but still keep the connection running?

解决方案

This sort of thing is best done using WebSockets now, which according to CanIUse.Com is available in all major browsers except Opera Mini (see that link for more details about older or all browsers, and click the Resources tab to see even more links). As an overview, websockets are supported in IE 10+, Firefox 11+ (38+ if within a WebWorker context), Chrome 16+, Opera 12.1+, Safari 7+, Android 4.4+, Opera Mobile 12.1+.

Note: you will likely want to learn about Service Workers and Web Workers as well, though those have different use cases and different abilities.

It looks like this:

var connection = new WebSocket(
   'ws://html5rocks.websocket.org/echo',
   ['soap', 'xmpp']
);

Attaching some event handlers immediately to the connection allows you to know when the connection is opened, when you've received incoming messages, or if an error has occurred.

Sending messages becomes as easy as this:

connection.send('your message');
connection.send(binaryData);

See Introducing WebSockets: Bringing Sockets to the Web for a full explanation on how to do this.

ASP.Net developers: if for some reason you need to support older browsers and don't want to figure out for yourself how to deal with those that don't support WebSockets, consider using a library such as SignalR.

The Old EventSource API Answer For Older Browsers

Most browsers now implement the EventSource API, which makes long polling really easy, as long as the stream can be delivered with content-type text/event-stream. Older browsers or those developers who for any reason can't engineer the stream to have that content-type can use some helper script to do the same thing.

Here's an example:

var jsonStream = new EventSource('https://example.com/yourstreamingservice')
jsonStream.onmessage = function (e) {
   var message = JSON.parse(e.data);
   // handle message
};

This is basically a full-fledged version of the exact thing that I outline below.

The Even Older Service Streaming Answer For REALLY OLD Browsers

What you want is called long polling. You'll need a custom AJAX onreadystatechange handling function. Instead of waiting until the entire stream has completed (since it never will), you'll need to examine the contents periodically. Note that you'll need to do some heavy lifting for this to work in IE 9 and lower, using an iframe.

Roughly:

• Respond to each onreadystatechange event and examine the stream you've been given up to the current character to see if there is enough data to consume one or more discrete events. You'll need to parse the stream yourself with javascript string-handling functions. A combination of split, indexOf, regular expressions, looping, and so on can be used to accomplish this task.
• If there's not enough content yet, then exit and wait for the next event.
• I am pretty sure that each time the onreadystatechange handler fires, the responseText will be all the data that has been received so far. Define a persistent variable that will hold the position of the first character that hasn't been properly processed yet.
• Once there is enough content for one or more discrete events to appear in the stream, take them out one at a time and pass them to your JSON parser to actually render the text as objects. Use them normally.

Check out this HTTP Streaming gist for one resource, or Streaming as an alternative to polling the server at SoftwareAs. If you must support IE 9 or older, then you'll need to use the iframe method for that.

Here is a quote from the book Ajax Design Patterns: Creating Web 2.0 Sites with Programming and Usability Patterns:

In summary, Service Streaming makes the HTTP Streaming approach more flexible, because you can stream arbitrary content rather than Javascript commands, and because you can control the connection's lifecycle. However, it combines two technologies that aren't consistent across browsers, with predictable portability issues. Experiments suggest that the Page Streaming technique does work on both IE [9 and older] and Firefox, but Service Streaming only works on Firefox, whether XMLHTTPRequest or IFrame is used. In the first case IE [9 and older] suppresses the response until its complete, with the IFrame it works if a workaround is used: The IE [9 and older] accepts a message from the server after the first 256 bytes so the only thing to do is to send 256 dummy Bytes before sending the messages. After this all messages will arrive as expected. So a full Service Streaming is possible in IE [9 and older], too!

Mind you that it is from 2006, so it is definitely out of date, but if you have to support older browsers, it's still relevant.

Security Issues

Normal AJAX cannot go cross-domain, meaning (now that I pay attention to the fact that you want to stream from twitter) that you won't be able to do what you're asking. This can be worked around with JSONP, but JSONP by nature can't be service streamed and moreover isn't offered by twitter anyway. There is also Cross-Origin Resource Sharing (CORS) but twitter's not going to set that up for you--that's the kind of thing they'd only do for domains affiliated with them. And CORS requires a modern browser.

Your only option is thus to create a proxy service on your web server that performs the requests to twitter for you and then hands out the data. This can only be done from the same domain as the main page was served from. Doing this would also allow you to create a version that will work for IE using the iframe technique. If you don't care about old IE versions, you can implement CORS yourself to defeat the domain restriction, if you know the domain that will be making the requests.

If you have full control of the client software (like if this is for a corporate intranet) there is another option: hosting the web browser inside of a compiled locally-executed application's user form. I have only done this using C# but I imagine it's possible from other languages. When you use the right browser object, because it's hosted inside a C# application, the C# application can defeat the cross-domain security restrictions, reading and writing all page content no matter what domain it comes from. I doubt your situation is this one but I wanted to put the option here for others who might appreciate it.

这篇关于处理连续的 JSON 流的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！