I have a large Mediawiki site (the open source software behind Wikipedia) that's getting killed by data mining bots. The bots are eating up all my server resources. Mediawiki has lots of extensions that kill spam bots, but none that I can find that kill data mining bots. I need the following:
1. I need a script that blocks data mining bots by creating a bot trap. All bots that go to the trap need to be blacklisted from the site (unless their on the whitelist)
2. The script to have a whitelist that allows the major search engine bots (google, yahoo, bing,etc).
3. The script to be written as an extension to the Mediawiki software so I can still update the software Mediawiki (in other words not a hack that changes the fundamental code and locks up the ability for easy updates).
Basically I'm looking for something similar to:[url removed, login to view] But once a bot goes to a trap it's blocked from the entire website forever.