处理连续的JSON流 [英] Process a continuous stream of JSON

问题描述

（现已解散）页面 http://stream.twitter.com/1/ statuses / sample.json 用于返回连续且无穷无尽的JSON数据流。

The (now defunct) page http://stream.twitter.com/1/statuses/sample.json used to return a continuous and endless stream of JSON data.

我想使用jQuery（或JavaScript）来处理它最好是jQuery）在我自己的网页中，能够根据推文的实时信息显示视觉效果。

I'd like to process it using jQuery (or JavaScript, but preferably jQuery) inside my own web page to be able to display visual effects based on the live feed of tweets.

因为据我所知jQuery parseJSON 函数只会在服务器发送完所有数据后执行回调函数，但这实际上是一个连续的数据流。如何处理数据当它发生但仍保持连接运行？

Since as far as I know the jQuery parseJSON function will only execute the callback function after all the data has been sent by the server, but this is actually a continuous stream of data. How can I process the data "as it happens" but still keep the connection running?

推荐答案

更新2

这类事情最好用 WebSockets 现在，根据CanIUse.Com，可在所有主流浏览器中使用除了Opera Mini之外（有关旧版或所有浏览器的更多详细信息，请参阅该链接，然后单击资源选项卡以查看更多链接）。概述，IE 10 +，Firefox 11+（如果在WebWorker上下文中为38+），Chrome 16 +，Opera 12.1 +，Safari 7 +，Android 4.4 +，Opera Mobile 12.1 +支持websockets。

This sort of thing is best done using WebSockets now, which according to CanIUse.Com is available in all major browsers except Opera Mini (see that link for more details about older or all browsers, and click the Resources tab to see even more links). As an overview, websockets are supported in IE 10+, Firefox 11+ (38+ if within a WebWorker context), Chrome 16+, Opera 12.1+, Safari 7+, Android 4.4+, Opera Mobile 12.1+.

看起来像这样：

var connection = new WebSocket(
   'ws://html5rocks.websocket.org/echo',
   ['soap', 'xmpp']
);

立即将一些事件处理程序附加到连接允许您知道何时当您收到传入消息或发生错误时，将打开连接。

Attaching some event handlers immediately to the connection allows you to know when the connection is opened, when you've received incoming messages, or if an error has occurred.

发送消息变得如此简单：

Sending messages becomes as easy as this:

connection.send('your message');
connection.send(binaryData);

参见介绍WebSockets：将套接字引入Web 以获得有关如何执行此操作的完整说明。

See Introducing WebSockets: Bringing Sockets to the Web for a full explanation on how to do this.

ASP。网络开发人员：如果由于某种原因你需要支持旧浏览器并且不想自己弄清楚如何处理那些不支持WebSockets的浏览器，请考虑使用诸如 SignalR 。

ASP.Net developers: if for some reason you need to support older browsers and don't want to figure out for yourself how to deal with those that don't support WebSockets, consider using a library such as SignalR.

老事件浏览器的旧EventSource API答案

大多数浏览器现在实现 EventSource API ，只要流可以与内容类型 text /一起提供，就可以很长时间地进行长轮询事件流。较旧的浏览器或那些因任何原因无法将流设计为具有该内容类型的开发人员可以使用某些帮助脚本做同样的事情。

Most browsers now implement the EventSource API, which makes long polling really easy, as long as the stream can be delivered with content-type text/event-stream. Older browsers or those developers who for any reason can't engineer the stream to have that content-type can use some helper script to do the same thing.

以下是一个例子：

var jsonStream = new EventSource('https://example.com/yourstreamingservice')
jsonStream.onmessage = function (e) {
   var message = JSON.parse(e.data);
   // handle message
};

这基本上是我在下面概述的完整版本的完整版本。

This is basically a full-fledged version of the exact thing that I outline below.

甚至更旧的服务流回答真正的老浏览器

你想要什么称为长轮询。你需要一个自定义的AJAX onreadystatechange 处理函数。而不是等到整个流完成（因为它永远不会），你需要定期检查内容。请注意，你需要做一些繁重的工作才能在IE 9及更低版本中工作，使用 iframe 。

What you want is called long polling. You'll need a custom AJAX onreadystatechange handling function. Instead of waiting until the entire stream has completed (since it never will), you'll need to examine the contents periodically. Note that you'll need to do some heavy lifting for this to work in IE 9 and lower, using an iframe.

粗略地说：

• 回应每个 onreadystatechange 事件并检查你的流已经放弃当前字符以查看是否有足够的数据来消耗一个或多个离散事件。您需要使用javascript字符串处理函数自行解析流。 split，indexOf，正则表达式，循环等的组合可用于完成此任务。


• 如果还没有足够的内容，则退出并等待下一个事件。


• 我很确定每次 onreadystatechange 处理程序触发时， responseText 将是迄今为止收到的所有数据。定义一个持久变量，它将保留尚未正确处理的第一个字符的位置。


• 一旦有足够的内容让一个或多个离散事件出现在流中，一次取出一个并将它们传递给您的JSON解析器，以实际将文本呈现为对象。正常使用它们。

• Respond to each onreadystatechange event and examine the stream you've been given up to the current character to see if there is enough data to consume one or more discrete events. You'll need to parse the stream yourself with javascript string-handling functions. A combination of split, indexOf, regular expressions, looping, and so on can be used to accomplish this task.
• If there's not enough content yet, then exit and wait for the next event.
• I am pretty sure that each time the onreadystatechange handler fires, the responseText will be all the data that has been received so far. Define a persistent variable that will hold the position of the first character that hasn't been properly processed yet.
• Once there is enough content for one or more discrete events to appear in the stream, take them out one at a time and pass them to your JSON parser to actually render the text as objects. Use them normally.

看看这个一个资源的HTTP Streaming gist ，或在SoftwareAs上流式传输作为轮询服务器的替代方法。如果您必须支持IE 9或更早版本，那么您需要使用 iframe 方法。

Check out this HTTP Streaming gist for one resource, or Streaming as an alternative to polling the server at SoftwareAs. If you must support IE 9 or older, then you'll need to use the iframe method for that.

这是引自的书 Ajax设计模式：使用编程和可用模式创建Web 2.0站点：

总之，Service Streaming使HTTP Streaming方法更多灵活，因为您可以流式传输任意内容而不是Javascript命令，因为您可以控制连接的生命周期。但是，它结合了两种跨浏览器不一致的技术，具有可预测的可移植性问题。实验表明，Page Streaming技术适用于IE [9及更早]和Firefox，但Service Streaming仅适用于Firefox，无论是使用XMLHTTPRequest还是IFrame。在第一种情况下，IE [9及更早版本]会在响应完成之前抑制响应，如果使用了变通方法，则IFrame可以工作：IE [9及更早版本]在前256字节之后接受来自服务器的消息所以唯一的事情要做的是在发送消息之前发送256个虚拟字节。在此之后，所有消息将按预期到达。因此，在IE [9及更早版本]中也可以使用完整的服务流！

In summary, Service Streaming makes the HTTP Streaming approach more flexible, because you can stream arbitrary content rather than Javascript commands, and because you can control the connection's lifecycle. However, it combines two technologies that aren't consistent across browsers, with predictable portability issues. Experiments suggest that the Page Streaming technique does work on both IE [9 and older] and Firefox, but Service Streaming only works on Firefox, whether XMLHTTPRequest or IFrame is used. In the first case IE [9 and older] suppresses the response until its complete, with the IFrame it works if a workaround is used: The IE [9 and older] accepts a message from the server after the first 256 bytes so the only thing to do is to send 256 dummy Bytes before sending the messages. After this all messages will arrive as expected. So a full Service Streaming is possible in IE [9 and older], too!

请注意，它是从2006年开始的，所以它肯定是过时了，但是如果你必须支持旧浏览器，它仍然是相关的。

Mind you that it is from 2006, so it is definitely out of date, but if you have to support older browsers, it's still relevant.

安全问题

普通的AJAX不能跨域，这意味着（现在我注意到你想从twitter流式传输的事实），你将无法做你想要的事情。这可以与JSONP一起解决，但JSONP本质上不能进行服务流传输，而且无论如何也不是由Twitter提供的。还有跨源资源共享（CORS），但Twitter不会为你设定 - 这是他们只为附属于他们的域名所做的事情。 CORS需要一个现代化的浏览器。

Normal AJAX cannot go cross-domain, meaning (now that I pay attention to the fact that you want to stream from twitter) that you won't be able to do what you're asking. This can be worked around with JSONP, but JSONP by nature can't be service streamed and moreover isn't offered by twitter anyway. There is also Cross-Origin Resource Sharing (CORS) but twitter's not going to set that up for you--that's the kind of thing they'd only do for domains affiliated with them. And CORS requires a modern browser.

因此，您唯一的选择就是在您的网络服务器上创建代理服务，为您执行Twitter请求，然后分发数据。这只能通过与提供主页面相同的域来完成。这样做还允许您使用iframe技术创建适用于IE的版本。如果你不关心旧的IE版本，你可以自己实现CORS来破坏域名限制，如果你知道将要发出请求的域名。

Your only option is thus to create a proxy service on your web server that performs the requests to twitter for you and then hands out the data. This can only be done from the same domain as the main page was served from. Doing this would also allow you to create a version that will work for IE using the iframe technique. If you don't care about old IE versions, you can implement CORS yourself to defeat the domain restriction, if you know the domain that will be making the requests.

如果您可以完全控制客户端软件（如果这是用于企业内部网），还有另一种选择：在已编译的本地执行的应用程序的用户表单中托管Web浏览器。我只使用C＃完成了这项工作，但我认为可以使用其他语言。当您使用正确的浏览器对象时，因为它托管在C＃应用程序中，C＃应用程序可以破坏跨域安全限制，读取和写入所有页面内容，无论它来自哪个域。我怀疑你的情况是这个，但我想把这个选项放在其他可能会欣赏它的人身上。

If you have full control of the client software (like if this is for a corporate intranet) there is another option: hosting the web browser inside of a compiled locally-executed application's user form. I have only done this using C# but I imagine it's possible from other languages. When you use the right browser object, because it's hosted inside a C# application, the C# application can defeat the cross-domain security restrictions, reading and writing all page content no matter what domain it comes from. I doubt your situation is this one but I wanted to put the option here for others who might appreciate it.

这篇关于处理连续的JSON流的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！