Google Colab TPU中未实现文件系统方案"[本地]" [英] File system scheme '[local]' not implemented in Google Colab TPU
问题描述
我在Google Colab中使用TPU运行时，但是在读取文件时遇到问题(不确定).我使用以下方法初始化了TPU:
I am using TPU runtime in Google Colab, but having problems in reading files (not sure). I initialized TPU using:
import tensorflow as tf
import os
import tensorflow_datasets as tfds
resolver = tf.distribute.cluster_resolver.TPUClusterResolver(tpu='grpc://' + os.environ['COLAB_TPU_ADDR'])
tf.config.experimental_connect_to_cluster(resolver)
# This is the TPU initialization code that has to be at the beginning.
tf.tpu.experimental.initialize_tpu_system(resolver)
print("All devices: ", tf.config.list_logical_devices('TPU'))
我在Google Colab存储的一个文件夹中有很多图像(例如'/content/train2017/000000000009.jpg').我运行以下代码:
I have many images in a folder in Google Colab storage ( e.g. '/content/train2017/000000000009.jpg'). I run the following code:
import tensorflow as tf
def load_image(image_path):
img = tf.io.read_file(image_path)
img = tf.image.decode_jpeg(img, channels=3)
img = tf.image.resize(img, (299, 299))
img = tf.keras.applications.inception_v3.preprocess_input(img)
return img, image_path
load_image('/content/train2017/000000000009.jpg')
但是，我收到以下错误消息:
But, I am getting the following error:
---------------------------------------------------------------------------
UnimplementedError Traceback (most recent call last)
<ipython-input-33-a7fbb45f3b76> in <module>()
----> 1 load_image('/content/train2017/000000000009.jpg')
5 frames
<ipython-input-7-862c73d29b96> in load_image(image_path)
2 img = tf.io.read_file(image_path)
3 img = tf.image.decode_jpeg(img, channels=3)
----> 4 img = tf.image.resize(img, (299, 299))
5 img = tf.keras.applications.inception_v3.preprocess_input(img)
6 return img, image_path
/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/image_ops_impl.py in resize_images_v2(images, size, method, preserve_aspect_ratio, antialias, name)
1515 preserve_aspect_ratio=preserve_aspect_ratio,
1516 name=name,
-> 1517 skip_resize_if_same=False)
1518
1519
/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/image_ops_impl.py in _resize_images_common(images, resizer_fn, size, preserve_aspect_ratio, name, skip_resize_if_same)
1183 with ops.name_scope(name, 'resize', [images, size]):
1184 images = ops.convert_to_tensor(images, name='images')
-> 1185 if images.get_shape().ndims is None:
1186 raise ValueError('\'images\' contains no shape.')
1187 # TODO(shlens): Migrate this functionality to the underlying Op's.
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py in get_shape(self)
1071 def get_shape(self):
1072 """Alias of Tensor.shape."""
-> 1073 return self.shape
1074
1075 def _shape_as_list(self):
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py in shape(self)
1065 self._tensor_shape = tensor_shape.TensorShape(self._shape_tuple())
1066 except core._NotOkStatusException as e:
-> 1067 six.raise_from(core._status_to_exception(e.code, e.message), None)
1068
1069 return self._tensor_shape
/usr/local/lib/python3.6/dist-packages/six.py in raise_from(value, from_value)
UnimplementedError: File system scheme '[local]' not implemented (file: '/content/train2017/000000000009.jpg')
我应该如何解决?我发现了一个类似gs桶的东西，但是它是有偿的.还有其他解决方法吗?
How should I solve it? I found something like a gs bucket, but it is paid. Is there any other way to solve this?
推荐答案
在使用TPU时从本地文件加载文件-将其作为普通python file.read()读取(而不是tf.io).就您而言:
For loading file from local file when using TPU - read them as normal python file.read() (not tf.io). In your case:
def load_image(image_path):
with open(image_path, "rb") as local_file: # <= change here
img = local_file.read()
img = tf.image.decode_jpeg(img, channels=3)
img = tf.image.resize(img, (299, 299))
img = tf.keras.applications.inception_v3.preprocess_input(img)
return img, image_path
load_image('/content/train2017/000000000009.jpg')
这篇关于Google Colab TPU中未实现文件系统方案"[本地]"的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
