BOT/蜘蛛陷阱的想法 [英] BOT/Spider Trap Ideas

问题描述

我有一个客户端，该客户端的域似乎受到DDoS的严重冲击.在日志中，它是具有随机IP的正常用户代理，但它们浏览页面的速度太快了，无法让人理解.他们似乎也没有要求任何图像.我似乎找不到任何模式，我怀疑这是Windows僵尸舰队.

I have a client whose domain seems to be getting hit pretty hard by what appears to be a DDoS. In the logs it's normal looking user agents with random IPs but they're flipping through pages too fast to be human. They also don't appear to be requesting any images. I can't seem to find any pattern and my suspicion is it's a fleet of Windows Zombies.

过去，客户遇到过SPAM攻击的问题-甚至不得不将MX指向Postini，以获得每天6.7 GB的垃圾来停止服务器端.

The clients had issues in the past with SPAM attacks--even had to point MX at Postini to get the 6.7 GB/day of junk to stop server-side.

我想在robots.txt不允许的目录中设置BOT陷阱...以前从未尝试过类似的尝试，希望有人在那里有捕获BOT的创意！

I want to setup a BOT trap in a directory disallowed by robots.txt... just never attempted anything like this before, hoping someone out there has a creative ideas for trapping BOTs!

编辑:我已经有了很多捕捉一个主意的方法.当落入陷阱时，应该采取的措施.

I already have plenty of ideas for catching one.. it's what to do to it when lands in the trap.

推荐答案

您可以设置一个robots.txt明确禁止其URL的PHP​​脚本.在该脚本中，您可以(通过$ _SERVER ['REMOTE_ADDR'])拉出可疑的bot击中您的源IP，然后将该IP添加到数据库黑名单表中.

You can set up a PHP script whose URL is explicitly forbidden by robots.txt. In that script, you can pull the source IP of the suspected bot hitting you (via $_SERVER['REMOTE_ADDR']), and then add that IP to a database blacklist table.

然后，在主应用程序中，您可以检查源IP，在黑名单表中查找该IP，如果找到它，则抛出403页. (也许会显示我们已检测到您IP的滥用，如果您认为这是错误的，请通过...与我们联系".)

Then, in your main app, you can check the source IP, do a lookup for that IP in your blacklist table, and if you find it, throw a 403 page instead. (Perhaps with a message like, "We've detected abuse coming from your IP, if you feel this is in error, contact us at ...")

从好的方面来说，您会自动将恶意机器人列入黑名单.不利的一面是，它效率不高，而且很危险. (一个人出于好奇而无辜地检查该页面可能会导致大量用户被禁止.)

On the upside, you get automatic blacklisting of bad bots. On the downside, it's not terribly efficient, and it can be dangerous. (One person innocently checking that page out of curiosity can result in the ban of a large swath of users.)

另外(或者，我想)，您可以相当简单地添加 GeoIP 检查到您的应用，然后根据来源国家/地区拒绝匹配.

Alternatively (or additionally, I suppose) you can fairly simply add a GeoIP check to your app, and reject hits based on country of origin.

这篇关于BOT/蜘蛛陷阱的想法的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！