D3:如何显示大型数据集 [英] D3: How to show large dataset

问题描述

我有一个包含10 ^ 5个数据点的大型数据集.现在，我正在考虑与大型数据集有关的以下问题:

I've a large dataset comprises 10^5 data points. And now I'm considering the following question related to large dataset:

有没有一种有效的方法可以可视化非常大的数据集?在我的情况下，我有一个用户集，每个用户有10 ^ 3个项目.共有10 ^ 5项.我想一次显示每个用户的所有项目，以实现用户之间的快速比较.有人建议使用列表，但是在处理这个大数据集时，我认为列表不是唯一的选择.

Is there any efficient way to visualize very large dataset? In my case I have a user set and each user has 10^3 items. There are 10^5 items in total. I want to show all the items for each user at a time to enable quick comparison between users. Some body suggests using a list, but I don't think a list is the only choice when dealing with this big dataset.

注意

我想一次显示每个用户的所有项目.

I want to show all the items for each user at a time.

这意味着我想在单击用户时显示所有数据点，并且当我单击两种用途时，我可以比较那里的数据点之间的差异.

This means I want to show all the datapoints when click on a user, and when I click on two uses, I can compare the difference between there datapoints.

推荐答案

问题是不渲染它们.您可以为渲染部分切换到canvas或webgl.您可以找到一些使用画布和平行坐标示例.此示例还具有渐进式渲染功能，可以加载和渲染所有数据元素.

The problem is not to render them. You could switch to canvas or webgl for the rendering part. You can find some examples of using canvas and X3DOM with D3 data-binding. But it will be slow because of the number of DOM objects, so it's better to keep them separated, as in this parallel coordinates example. This example also features progressive rendering to load and render all the data elements.

将它们保留在内存中并在客户端进行操作也不是问题. D3通常与 Crossfilter 一起使用，以快速处理百万条或更多记录"的数据.

Keeping them in memory and manipulating them client-side is not a problem neither. D3 is often used with Crossfilter for quick data manipulation of "million or more records".

10 ^ 5个数据点对于SVG交互式渲染来说只是太多点.但是，可视化中的数据点太多通常暗示您使用了错误的抽象级别或错误的绘图策略.许多点可能会重叠或在视觉上融合.那么为什么不汇总这些形状，例如使用热图(用于重叠点数量的色标)，合并(六边形，直方图)或汇总数据集呢?

10^5 data points are just slightly too many points for SVG interactive rendering. But too many data points in a visualization is often a hint that you have the wrong level of abstraction or the wrong plotting strategy. A lot of points will probably overlap or visually fuse. So why not aggregate these shapes, for example using heatmap (color scale for number of overlapping points), binning (hexbin, histogram), or summarizing the dataset?

如果您想要的是概览并比较数据集，则可能需要抽象化，例如一些汇总数据集的统计信息，然后按需查看详细信息(语义缩放，焦点+上下文，下钻).

If what you want is an overview, and comparing datasets, you probably need an abstraction, like some statistics summarizing your dataset, then see a detail on-demand (semantic zoom, focus+context, drill-down).

这篇关于D3:如何显示大型数据集的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！