python pickle UnicodeDecodeError [英] python pickle UnicodeDecodeError

问题描述

我正在尝试加载mnist字符数据集(按照此处概述的教程进行操作: http://neuralnetworksanddeeplearning.com /chap1.html )

I'm trying to load the mnist character dataset (following the tutorial outlined here: http://neuralnetworksanddeeplearning.com/chap1.html )

当我运行load_data_wrapper函数时，我得到了错误.

when I run the load_data_wrapper function I get the error.

UnicodeDecodeError: 'ascii' codec can't decode byte 0x90 in position 614: ordinal not in range(128)

运行的代码是:

import numpy as np
import gzip


def load_data():
    f = gzip.open('../data/mnist.pkl.gz', 'rb')
    training_data, validation_data, test_data = pickle.load(f)
    f.close()
    return (training_data, validation_data, test_data)

def load_data_wrapper():
    tr_d, va_d, te_d = load_data()
    training_inputs = [np.reshape(x, (784,1)) for x in tr_d[0]]
    training_results = [vectorized_result(y) for y in tr_d[1]]
    training_data = zip(training_inputs, training_results)
    validation_inputs = [np.reshape(x,(784, 1))for x in va_d[0]]
    validation_data = zip(validation_inputs, va_d[1])
    test_inputs = [np.reshape(x, (784, 1)) for x in te_d[0]]
    test_data = zip(test_inputs, te_d[1])
    return(training_data, validation_data, test_data)

def vectorized_result(j):
    e = np.zeros((10,1))
    e[j] = 1.0
    return e

更新:问题似乎是我正在尝试使用python 2.x腌制的python 3.6进行修复.

UPDATE: The problem seems to be that I am trying to unpickle with python 3.6 which was pickled with python 2.x.

推荐答案

如上所述，主要问题原来是python 2.x cPickle和python 3.x pickle之间的不兼容.

As stated the main problem turned out to be incompatibility between python 2.x cPickle and python 3.x pickle.

将编码设置为"latin-1"似乎有效.

setting the encoding to 'latin-1' seems to work.

training_data, validation_data, test_data = pickle.load(f, encoding='latin1')

这里的答案很有帮助: Pickle不兼容Python 2和3之间的numpy数组

Answer here helped a lot: Pickle incompatability of numpy arrays between Python 2 and 3

这篇关于python pickle UnicodeDecodeError的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！