如何加载大的CSV文件在ILArray的3d绘图？ [英] How to load big CSV file in ILArray for 3d plotting?
问题描述
我有几个文件在csv格式与科学数据，以检查他们在3D中绘制。
文件内容组织为datetime，频率为Hz，强度为dB，如下例所示：
03-15 18:00:00，19700000.0，-30.19
2014-03-15 18:00:00，19700781.25，-31.19
2014-03-15 18:00:00，19701562.5， - 30.43
2014-03-15 18:00:00，19702343.75，-30.37
2014-03-15 18:00:00，19703125.0，-27.06
对于绘图上下文，这三个值分别代表x，y，z。
I想要创建一个名为 datasample 的自定义类型，其中包含 datetime，float，float 是否有特定的函数或建议从文件加载ILArray绘图的方法？ ILNumerics是否可以正确地处理第一列的datetime格式，或者让我预处理文件以将其转换为其他格式？ 感谢您的支持 在回答以下源代码后UPDATED 这是一个屏幕截图： code> ILMath.csvread< T>（）。该函数允许通过通用参数指定整个元素类型： 目前没有单独的列格式规范。因此，为了包含日期（DateTime），您必须创建自己的数据类型，如您所指出的。 但是，这会引入其他问题： / p> 1）csvread（）期望csv文件中的同构元素。 你的目标是绘制数据，你将不得不将datetime转换为一些实数。 3）此外，大多数ILMath函数专用于某些真实（System.Value）元素格式的ILArray。有一个ILArray将通过这些函数带来非常有限的支持。 编辑：这怎么样？ ：） I have several files in csv format with scientific data to examine plotting them in 3D.
The file content is organized as datetime, frequency in Hz and intensity in dB as in this example: For the plotting context, the 3 values represent respectively x,y,z. I would like to create a custom type named datasample consisting of datetime,float,float types and then declare the array as Is there a specific function or suggested way of loading from file the ILArray for plotting ? Can ILNumerics correctly handle the datetime format of the first column or have I to preprocess the file to convert it to something else ? Thank you for your kind support UPDATED after the answer with the following source code
I've loaded 3.6 Million points with this and the 3d plot is rendered smoothly.
I would need some instructions on how to color the individual points based on their Z value.
Seems that using a surface with millions of point in input is a bit...heavy :-) Any performance hints I could use ? This is a screenshot:
CSV data are read using There is currently no individual column format specification. So, in order to include the dates as such (DateTime), you must create your own datatype – as you pointed out. This, however, introduces other problems: 1) csvread() expects homogeneous elements from the csv file. While 2) If your goal is to plot the data, you will have to convert the datetime to some real number anyway. The Drawing namespace expects ILArray only. 3) Furthermore, most ILMath functions are specialized for ILArray of some real (System.Value) element format. Having an ILArray would bring very limited support by means of those functions. EDIT: How about this? :) 这篇关于如何加载大的CSV文件在ILArray的3d绘图？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
ILArray< datasample>
我已加载 3.6百万分，并且3D图表平滑呈现。
我需要一些关于如何根据它们的Z值对各个点进行着色的说明。
似乎使用一个带有数百万点输入的表面是有点...重要:-)我可以使用任何性能提示？
使用ILNumerics;
使用ILNumerics.Drawing;
使用ILNumerics.Drawing.Plotting;
使用System.IO;
using System.Globalization;
private void ilPanel1_Load（object sender，EventArgs e）
{
fsin = new System.IO.StreamReader（testlong.iln）;
while（fsin.EndOfStream == false）
{
datarow = fsin.ReadLine（）;
// Console.WriteLine（datarow）;
rowvalues = datarow.Split（'，'）;
inrows = inrows + 1;
tempX.Add（float.Parse（rowvalues [0]，CultureInfo.InvariantCulture））;
tempY.Add（float.Parse（rowvalues [1]，CultureInfo.InvariantCulture））;
tempZ.Add（float.Parse（rowvalues [2]，CultureInfo.InvariantCulture））;
}
fsin.Close（）;
//现在我知道输入大小（x，y，z元组的数量），我可以
//创建和初始化ILArray：
ILArray< float> datasamples = ILMath.zeros< float>（3，（int）inrows）;
label1.Text = String.Format（Data points read：{0：N0}，inrows）;
// ...现在我可以从输入数组复制数据：
datasamples [0 ;:] = tempX.ToArray（）;
datasamples [1;：] = tempY.ToArray（）;
datasamples [2 ;:] = tempZ.ToArray（）;
//这些不再使用所以...
tempX.Clear（）;
tempY.Clear（）;
tempZ.Clear（）;
var scene = new ILScene {
new ILPlotCube（twoDMode：false）{
新ILPoints {
Positions = datasamples，
Color = Color.Blue ，
// Colors =
Size = 0.5F
}
}
};
//在所有修改结束时，调用Configure（）
ilPanel1.Scene = scene;
ilPanel1.Scene.Configure（）;
ilPanel1.Refresh（）;
}
string s =
@2014- 03-15 18:00:00，19700000.0，-30.19
2014-03-15 18:00:00，19700781.25，-31.19
2014-03-15 18:00:00，19701562.5， - 30.43
2014-03-15 18:00:00，19702343.75，-30.37
2014-03-15 18:00:00，19703125.0，-27.06;
ILArray< double> A = ILMath.csvread< double>（s）;
> A
< Double> [5,3]
（：，:) 1e + 007 *
0,00000 1,97000 0,00000
0,00000 1,97008 0,00000
0， 00000 1,97016 0,00000
0,00000 1,97023 0,00000
0,00000 1,97031 0,00000
ILArray< datasample> 很好（有用），csvread不直接支持。
private void ilPanel1_Load（object sender，EventArgs e）{
ILArray< float& A = ILMath.tosingle（ILMath.rand（4，3000））;
ilPanel1.Scene.Add（
new ILPlotCube（twoDMode：false）{
new ILPoints（）{
Positions = A [0,1,2 ;:]，
Colors = new ILColormap（Colormaps.Jet）.Map（A [2;：]）。T，
Color = null
}
}
}
> 2014-03-15 18:00:00, 19700000.0, -30.19
2014-03-15 18:00:00, 19700781.25, -31.19
2014-03-15 18:00:00, 19701562.5, -30.43
2014-03-15 18:00:00, 19702343.75, -30.37
2014-03-15 18:00:00, 19703125.0, -27.06
ILArray<datasample>
using ILNumerics;
using ILNumerics.Drawing;
using ILNumerics.Drawing.Plotting;
using System.IO;
using System.Globalization;
private void ilPanel1_Load(object sender, EventArgs e)
{
fsin = new System.IO.StreamReader("testlong.iln");
while(fsin.EndOfStream == false)
{
datarow = fsin.ReadLine();
// Console.WriteLine(datarow);
rowvalues = datarow.Split(',');
inrows = inrows + 1;
tempX.Add(float.Parse(rowvalues[0], CultureInfo.InvariantCulture));
tempY.Add(float.Parse(rowvalues[1], CultureInfo.InvariantCulture));
tempZ.Add(float.Parse(rowvalues[2], CultureInfo.InvariantCulture));
}
fsin.Close();
// now that I know the input size (number of x,y,z tuples), I can
// create and initialize the ILArray :
ILArray<float> datasamples = ILMath.zeros<float>(3, (int)inrows );
label1.Text = String.Format("Data points read: {0:N0}", inrows);
// ... and now I can copy data from the input arrays:
datasamples["0;:"] = tempX.ToArray();
datasamples["1;:"] = tempY.ToArray();
datasamples["2;:"] = tempZ.ToArray();
// these are no longer used so...
tempX.Clear();
tempY.Clear();
tempZ.Clear();
var scene = new ILScene {
new ILPlotCube(twoDMode: false) {
new ILPoints {
Positions = datasamples,
Color = Color.Blue,
// Colors =
Size = 0.5F
}
}
};
// at the end of all modifications, call Configure()
ilPanel1.Scene = scene;
ilPanel1.Scene.Configure();
ilPanel1.Refresh();
}
ILMath.csvread<T>(). The function allows the specification of the overall element type via the generic parameter:string s =
@"2014-03-15 18:00:00, 19700000.0, -30.19
2014-03-15 18:00:00, 19700781.25, -31.19
2014-03-15 18:00:00, 19701562.5, -30.43
2014-03-15 18:00:00, 19702343.75, -30.37
2014-03-15 18:00:00, 19703125.0, -27.06";
ILArray<double> A = ILMath.csvread<double>(s);
>A
<Double> [5,3]
(:,:) 1e+007 *
0,00000 1,97000 0,00000
0,00000 1,97008 0,00000
0,00000 1,97016 0,00000
0,00000 1,97023 0,00000
0,00000 1,97031 0,00000
ILArray<datasample> is fine (and useful), csvread does not support that directly.private void ilPanel1_Load(object sender, EventArgs e) {
ILArray<float> A = ILMath.tosingle(ILMath.rand(4, 3000));
ilPanel1.Scene.Add(
new ILPlotCube(twoDMode:false) {
new ILPoints() {
Positions = A["0,1,2;:"],
Colors = new ILColormap(Colormaps.Jet).Map(A["2;:"]).T,
Color = null
}
});
}
