通过bash脚本获取域头 [英] get header of domains by bash script

问题描述

我在test.csv中编写了10个域，并试图获取这10个域的标头.但不会一一读取内容.

I write 10 domains in test.csv and trying to get the header of these 10 domains. but it won't read contents 1 by 1.

并运行该脚本

for j in test.csv
do

awk -F',' '{ print "$1" }' $j | curl -Is | cat >> b.txt

done

我大约有1000万个域，并尝试通过脚本获取标头.有可能吗?

I have about 10 million domains, and trying to get header by script. Any way possible ?

推荐答案

curl不会告诉您来自stdin的URL，除非您告知这样做.而且您不需要为此进行循环(假设您有多个文件).您正在寻找的是:

curl won't read URLs from stdin unless you tell it to do so. And you don't need a loop for this (assuming you have multiple files). What you're looking for is:

awk -F',' '{ print "url=" $1 }' file1 file2 file3 ... | curl -s -I -K- > out

注意-K-，它告诉curl从stdin读取命令行参数.

Notice the -K-, it tells curl to read command line arguments from stdin.

这篇关于通过bash脚本获取域头的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！