awk:超出程序限制:最大字段数= 32767 [英] awk: program limit exceeded: maximum number of fields size=32767
本文介绍了awk:超出程序限制:最大字段数= 32767的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
当我在ubuntu 14.04中运行我的shell脚本时,出现如下错误
when i am running my shell script in ubuntu 14.04 i am getting an error like below
awk: program limit exceeded: maximum number of fields size=32767
FILENAME="ensemble_features/Training_BOOKS_POS_Bigram_with_stemming_BOOLEAN_FVT.csv" FNR=1 NR=1
cut: invalid byte, character or field list
Try 'cut --help' for more information.
-1
cut: invalid byte, character or field list
Try 'cut --help' for more information.
6656
user@user-Lenovo-IdeaPad-Z410:~/Thesis/BOOKS$ bash Training_POS_Uni_Bi.sh
awk: program limit exceeded: maximum number of fields size=32767
FILENAME="ensemble_features/Training_BOOKS_POS_Bigram_with_stemming_BOOLEAN_FVT.csv" FNR=1 NR=1
cut: invalid byte, character or field list
Try 'cut --help' for more information.
-1
cut: invalid byte, character or field list
Try 'cut --help' for more information.
6656
我正在下面添加我的脚本
i am adding my script below
cd /home/user/Thesis/BOOKS/Features/Training/POSITIVE/
fname="ensemble_features"
mkdir $fname
cp /home/user/Thesis/BOOKS/Features/Training/POSITIVE/Training_BOOKS_POS_unigram_FVT_with_stemming_BOOLEAN.csv ensemble_features/
cp /home/user/Thesis/BOOKS/Features/Training/POSITIVE/Training_BOOKS_POS_Bigram_with_stemming_BOOLEAN_FVT.csv ensemble_features/
mkdir "proces"
cnt=0
for file in $fname/*
do
#Number of columns
num=`awk 'BEGIN {FS=",";c=0};{if (c==0 ){print NF; c=1}}END{}' $file`
if [[ cnt -eq 0 ]];then
cut -d, -f $num $file >class.csv
cnt=1;
fi
num=$((num-1))
echo $num
nfname=`basename $file`
#Cut the columns
cut -d',' -f1-$num $file > proces/cutlast$nfname
done
#Paste multiple csv
paste -d',' proces/* > comb.csv
paste -d, comb.csv class.csv > Training_BOOKS_Unigram_Bigram_POS_Ensemble_Features_BOOLEAN.csv
rm comb.csv
rm class.csv
rm -r proces
rm -r ensemble_features
我的输入文件分别包含38453列和6656列.有人可以帮助我更正此错误吗?
my input files contain 38453 columns and 6656 columns respectively.Anybody can help me to correct this error?
推荐答案
在ubuntu上,awk是awk某些变体的软链接,如今默认为mawk.尝试安装gawk. gawk对记录中的字段数没有限制.
On ubuntu awk is a soft link to some variant of awk, nowadays by default it is mawk. Try to install gawk. gawk does not have a limitation on the number of fields in a record.
顺便说一句,如果您有时间学习python,python可能是一个更好的长期解决方案.
BTW, python may be a better long term solution, if you got the time to learn it.
这篇关于awk:超出程序限制:最大字段数= 32767的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
查看全文