在Java中读取IDX文件类型 [英] Reading a IDX file type in Java

问题描述

我已经用Java构建了一个图像分类器，我想根据这里提供的图像进行测试： http ：//yann.lecun.com/exdb/mnist/

I have built a image classifier in Java that I would like to test against the images provided here: http://yann.lecun.com/exdb/mnist/

不幸的是，如果您下载train-images-idx3-ubyte.gz或任何其他3个文件，它们都是文件类型：.idx1-ubyte

Unfortunately, if you download the train-images-idx3-ubyte.gz or any of the other 3 files, they are all of file type: .idx1-ubyte

第一个问题：
我想知道是否有人可以给我指示如何将.idx1-ubyte变成bitmaps（.bmp）文件？

First Question: I was wondering if anyone can give me instructions on how to make the .idx1-ubyte into bitmaps (.bmp) files?

第二个问题：
或者我一般如何阅读这些文件？

Second Question: Or just how I can read these files in general?

有关IDX文件格式的信息：
IDX文件格式是各种数字类型的向量和多维矩阵的简单格式。
基本格式为：

Information about the IDX file format: the IDX file format is a simple format for vectors and multidimensional matrices of various numerical types. The basic format is:

magic number 
size in dimension 0 
size in dimension 1 
size in dimension 2 
..... 
size in dimension N 
data

幻数是一个整数（MSB优先）。前2个字节始终为0.

The magic number is an integer (MSB first). The first 2 bytes are always 0.

第三个字节代码数据类型：

The third byte codes the type of the data:

0x08: unsigned byte 
0x09: signed byte 
0x0B: short (2 bytes) 
0x0C: int (4 bytes) 
0x0D: float (4 bytes) 
0x0E: double (8 bytes)

第4个字节对数字进行编码向量/矩阵的维数：1表示向量，2表示矩阵....

The 4-th byte codes the number of dimensions of the vector/matrix: 1 for vectors, 2 for matrices....

每个维度的大小为4字节整数（MSB优先，高端与大多数非英特尔处理器一样。）

The sizes in each dimension are 4-byte integers (MSB first, high endian, like in most non-Intel processors).

数据存储方式与C数组相同，即最后一维中的索引变化最快。

The data is stored like in a C array, i.e. the index in the last dimension changes the fastest.

推荐答案

相当直截了当，正如WPrecht所说：URL描述了你必须解码的格式。这是我的idx文件的ImageSet导出器，不是很干净，但它做了它必须做的事情。

Pretty Straightforward, as WPrecht said: "The URL describes the format you have to decode". This is my ImageSet exporter for the idx file, not very clean, but does what it has to do.

public class IdxReader {

    public static void main(String[] args) {
        // TODO Auto-generated method stub
        FileInputStream inImage = null;
        FileInputStream inLabel = null;

        String inputImagePath = "CBIR_Project/imagesRaw/MNIST/train-images-idx3-ubyte";
        String inputLabelPath = "CBIR_Project/imagesRaw/MNIST/train-labels-idx1-ubyte";

        String outputPath = "CBIR_Project/images/MNIST_Database_ARGB/";

        int[] hashMap = new int[10]; 

        try {
            inImage = new FileInputStream(inputImagePath);
            inLabel = new FileInputStream(inputLabelPath);

            int magicNumberImages = (inImage.read() << 24) | (inImage.read() << 16) | (inImage.read() << 8) | (inImage.read());
            int numberOfImages = (inImage.read() << 24) | (inImage.read() << 16) | (inImage.read() << 8) | (inImage.read());
            int numberOfRows  = (inImage.read() << 24) | (inImage.read() << 16) | (inImage.read() << 8) | (inImage.read());
            int numberOfColumns = (inImage.read() << 24) | (inImage.read() << 16) | (inImage.read() << 8) | (inImage.read());

            int magicNumberLabels = (inLabel.read() << 24) | (inLabel.read() << 16) | (inLabel.read() << 8) | (inLabel.read());
            int numberOfLabels = (inLabel.read() << 24) | (inLabel.read() << 16) | (inLabel.read() << 8) | (inLabel.read());

            BufferedImage image = new BufferedImage(numberOfColumns, numberOfRows, BufferedImage.TYPE_INT_ARGB);
            int numberOfPixels = numberOfRows * numberOfColumns;
            int[] imgPixels = new int[numberOfPixels];

            for(int i = 0; i < numberOfImages; i++) {

                if(i % 100 == 0) {System.out.println("Number of images extracted: " + i);}

                for(int p = 0; p < numberOfPixels; p++) {
                    int gray = 255 - inImage.read();
                    imgPixels[p] = 0xFF000000 | (gray<<16) | (gray<<8) | gray;
                }

                image.setRGB(0, 0, numberOfColumns, numberOfRows, imgPixels, 0, numberOfColumns);

                int label = inLabel.read();

                hashMap[label]++;
                File outputfile = new File(outputPath + label + "_0" + hashMap[label] + ".png");

                ImageIO.write(image, "png", outputfile);
            }

        } catch (FileNotFoundException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } finally {
            if (inImage != null) {
                try {
                    inImage.close();
                } catch (IOException e) {
                    // TODO Auto-generated catch block
                    e.printStackTrace();
                }
            }
            if (inLabel != null) {
                try {
                    inLabel.close();
                } catch (IOException e) {
                    // TODO Auto-generated catch block
                    e.printStackTrace();
                }
            }
        }
    }

}

这篇关于在Java中读取IDX文件类型的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！