在Bash中并行迭代IP地址 [英] Parallel Iterating IP Addresses in Bash
问题描述
我处理的是一个大型的私有/ 8网络，需要枚举所有正在侦听端口443并具有在其HTTP HEADER响应中指定的特定版本的网络服务器。
首先我想通过输出文件连接扫描和grep自己运行 nmap ，但结果是抛出许多假阳性，其中 nmap 表示端口被过滤，而实际上是打开（使用连接扫描： nmap -sT -sV -Pn -n -oA foo 10.0.0.0 / 8 -p 443 ）。
现在我想用bash和 curl - 伪代码将是：
对于10.0.0.0/8中的每个IP
do：
curl --head https：// {IP}：443 | grep -iE（Server \：\ Target）> {IP} _info.txt;
done
由于我不熟悉bash我不知道如何请妥善处理这个问题 - 我必须：
- 循环浏览所有IP
- X威胁并行运行
- 最好将输出剪切到一个文件中只记下匹配主机的IP
- 匹配的服务器版本记录在下面
任何建议或指向的方向都非常感激。
建议按如下方式进行迭代：
for ip in 192.168.1。 do ...
如类似问题。
大规模 - 并行！
您的问题涉及巨大 IP地址范围，您应该考虑采用不同的方法。
此乞求使用 gnu并行。
使用 gnu parallel 在bash中并行迭代大量IP地址需要拆分逻辑到几个文件（用于并行命令使用）。
ip2int
＃！/ bin / bash
设置-e
函数ip_to_int（）
{
本地IP =$ 1
本地A = $（echo $ IP | cut -d。-f1）
local B = $（echo $ IP | cut -d。-f2）
local C = $（echo $ IP | cut -d 。-f3）
local D = $（echo $ IP | cut -d。-f4）
local INT
INT = $（expr 256*256* 256*$ A）
INT = $（expr 256*256*$ B + $ INT）
INT = $（expr 256*$ C + $ INT）
INT = $（expr $ D + $ INT）
echo $ INT
}
函数int_to_ip（）
{
local INT =$ 1
local D = $（expr $ INT％256）
local C = $（expr'（'$ INT - $ D'）'/ 256 ％256）
local B = $（expr'（'$ INT - $ C - $ D'）'/ 65536％256）
local A = $（expr'（'$ INT - $ B - $ C - $ D'）'/ 16777216％256）
echo$ A. $ B. $ C. $ D
}
scan_ip
＃！/ bin / bash
设置-e
源ip2int
$ b b if [[$＃-ne 1]];那么
echo用法：$（basename$ 0）ip_address_number
exit 1
fi
CONNECT_TIMEOUT = 2＃以秒为单位
IP_ADDRESS = $（int_to_ip $ {1}）
set + e
data = $（curl --head -vs -m $ {CONNECT_TIMEOUT} https：// $ {IP_ADDRESS}：443 2& 1）
exit_code =$？
data = $（echo -e$ {data}| grepServer：）
＃不确定你在你的服务器中找什么
set -e
if [[$ {exit_code} -eq 0]];那么
if [[-n$ {data}]];那么
echo$ {IP_ADDRESS} - $ {data}
else
echo$ {IP_ADDRESS} - 获取服务器的空数据！
fi
else
echo$ {IP_ADDRESS} - 无服务器。
fi
scan_range
＃！/ bin / bash
set -e
source ip2int
START_ADDRESS =10.0.0.0
NUM_OF_ADDRESSES =16777216＃256 * 256 * 256
start_address_num = $（ip_to_int $ {START_ADDRESS}）
end_address_num = $（（start_address_num + NUM_OF_ADDRESSES））
seq $ {start_address_num} $ {end_address_num} |并行-P0 ./scan_ip
＃这个并行调用与下面的一样：
＃
＃用于$（seq $ {start_address_num} $ {end_address_num}）中的ip_num。 do
＃./scan_ip $ {ip_num}
＃done
＃
＃只有一个更快！
迭代方法：
原生for循环（估计需要200天的256 * 256 * 256个地址）的运行时间是根据@skrskrskr改进为一天。
I'm dealing with a large private /8 network and need to enumerate all webservers which are listening on port 443 and have a specific version stated in their HTTP HEADER response.
First I was thinking to run nmap with connect scans and grep myself through the output files, but this turned out to throw many false-positives where nmap stated a port to be "filtered" while it actually was "open" (used connect scans: nmap -sT -sV -Pn -n -oA foo 10.0.0.0/8 -p 443).
So now I was thinking to script something with bash and curl - pseudo code would be like:
for each IP in 10.0.0.0/8
do:
curl --head https://{IP}:443 | grep -iE "(Server\:\ Target)" > {IP}_info.txt;
done
As I'm not that familiar with bash I'm not sure how to script this properly - I would have to:
- loop through all IPs
- make sure that only X threats run in parallel
- ideally cut the output to only note down the IP of the matching host in one single file
- ideally make sure that only matching server versions are noted down
Any suggestion or pointing into a direction is highly appreciated.
Small Scale - iterate
for a smaller IP address span it would probably be recommended to iterate like this:
for ip in 192.168.1.{1..10}; do ...
As stated in this similar question.
Big Scale - parallel !
Given that your problem deals with a huge IP address span you should probably consider a different approach.
This begs for the use of gnu parallel.
Parallel iterating a big span of IP addresses in bash using gnu parallel requires splitting the logic to several files (for the parallel command to use).
ip2int
#!/bin/bash
set -e
function ip_to_int()
{
local IP="$1"
local A=$(echo $IP | cut -d. -f1)
local B=$(echo $IP | cut -d. -f2)
local C=$(echo $IP | cut -d. -f3)
local D=$(echo $IP | cut -d. -f4)
local INT
INT=$(expr 256 "*" 256 "*" 256 "*" $A)
INT=$(expr 256 "*" 256 "*" $B + $INT)
INT=$(expr 256 "*" $C + $INT)
INT=$(expr $D + $INT)
echo $INT
}
function int_to_ip()
{
local INT="$1"
local D=$(expr $INT % 256)
local C=$(expr '(' $INT - $D ')' / 256 % 256)
local B=$(expr '(' $INT - $C - $D ')' / 65536 % 256)
local A=$(expr '(' $INT - $B - $C - $D ')' / 16777216 % 256)
echo "$A.$B.$C.$D"
}
scan_ip
#!/bin/bash
set -e
source ip2int
if [[ $# -ne 1 ]]; then
echo "Usage: $(basename "$0") ip_address_number"
exit 1
fi
CONNECT_TIMEOUT=2 # in seconds
IP_ADDRESS="$(int_to_ip ${1})"
set +e
data=$(curl --head -vs -m ${CONNECT_TIMEOUT} https://${IP_ADDRESS}:443 2>&1)
exit_code="$?"
data=$(echo -e "${data}" | grep "Server: ")
# wasn't sure what are you looking for in your servers
set -e
if [[ ${exit_code} -eq 0 ]]; then
if [[ -n "${data}" ]]; then
echo "${IP_ADDRESS} - ${data}"
else
echo "${IP_ADDRESS} - Got empty data for server!"
fi
else
echo "${IP_ADDRESS} - no server."
fi
scan_range
#!/bin/bash
set -e
source ip2int
START_ADDRESS="10.0.0.0"
NUM_OF_ADDRESSES="16777216" # 256 * 256 * 256
start_address_num=$(ip_to_int ${START_ADDRESS})
end_address_num=$(( start_address_num + NUM_OF_ADDRESSES ))
seq ${start_address_num} ${end_address_num} | parallel -P0 ./scan_ip
# This parallel call does the same as this:
#
# for ip_num in $(seq ${start_address_num} ${end_address_num}); do
# ./scan_ip ${ip_num}
# done
#
# only a LOT faster!
Improvement from the iterative approach:
The run time of the naive for loop (which is estimated to take 200 days for 256*256*256 addresses) was improved to under a day according to @skrskrskr.
这篇关于在Bash中并行迭代IP地址的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
