在 shell 脚本中迭代地从列中过滤掉值 [英] filtering out values from a column iteratively in shell script

问题描述

这里是shell scipt 新手.我在一个文件夹中有一组 csv 文件，我想要的是以迭代方式从每个文件中选择 1000 个不同的用户 ID，以便从下一个文件中选择的下一组用户 ID 不包含从以前的文件.我从第一个文件中选择了不同的 1000 个用户 ID，并将其存储在一个临时文件中.下面是命令:

shell scipt newbie here. I have a set of csv files in a folder, What I want is to select 1000 distinct user Ids from each file in an iterative way so that the next set of user ids picked from the next file does not contain the user ids filtered from the previous files. I have selected distinct 1000 user id from the first file and stored it in a temp file. Below is the command:

sort -u -t, -k1,8 file1.csv|head -1000 > temp.txt

这里的 8 是用户 ID 列.现在我想要来自 file2 的下 1000 个用户 ID，以便从 file2 中排除来自 file1(存储在 temp.txt 中)的用户 ID.有没有一种优雅的方法来实现这一目标?

Here 8 is the user id column. Now I want next 1000 user ids from file2 such that the user ids from file1( stored in temp.txt) are excluded from file2. Is there an elegant way to achieve this?

推荐答案

-k1,8 使用第 1 列和第 8 列.你不想只使用 -k8 吗?根据您的问题，尝试:

-k1,8 use 1st and then 8th column. Don't you want to use just -k8? According to your question, try:

cut -d"," -f 8 file2.csv | grep -v -f temp.txt | sort -u | head -1000 > temp2.txt 

顺便说一句，您可以在排序中使用通配符:sort -u -t, -k8 file*.csv |头...

BTW you can use wildcard in sort: sort -u -t, -k8 file*.csv | head ...

这篇关于在 shell 脚本中迭代地从列中过滤掉值的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！