如何使用python csv模块分割双管分隔数据 [英] How to use python csv module for splitting double pipe delimited data

问题描述

我得到的数据类似：

"1234"||"abcd"||"a1s1"

我试图使用Python的csv读写器来读写。
因为csv模块的分隔符限制为单个字符，有没有办法检索数据干净？我不能删除空列，因为它是一个庞大的数据集，以时间有限的方式处理。任何想法都会有所帮助。

I am trying to read and write using Python's csv reader and writer. As the csv module's delimiter is limited to single char, is there any way to retrieve data cleanly? I cannot afford to remove the empty columns as it is a massively huge data set to be processed in time bound manner. Any thoughts will be helpful.

推荐答案

文档和实验证明只允许使用单字符分隔符。

The docs and experimentation prove that only single-character delimiters are allowed.

由于 cvs.reader 接受任何支持迭代器协议的对象，可以使用生成器语法将 || -s替换为 | -s，然后将此生成器馈送给读者：

Since cvs.reader accepts any object that supports iterator protocol, you can use generator syntax to replace ||-s with |-s, and then feed this generator to the reader:

def read_this_funky_csv(source):
  # be sure to pass a source object that supports
  # iteration (e.g. a file object, or a list of csv text lines)
  return csv.reader((line.replace('||', '|') for line in source), delimiter='|')

此代码非常有效，因为它一次只在一个CSV行上运行，只要您的CSV源产生的行不超过可用的RAM：）

This code is pretty effective since it operates on one CSV line at a time, provided your CSV source yields lines that do not exceed your available RAM :)

这篇关于如何使用python csv模块分割双管分隔数据的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！