• 0

[IPB] Biggest Bot List Ever v5.0.1 Released!


Question

After neglecting the bot lists for over a year, I have taken it upon myself to rebuild the lists from scratch. This has led to the creation of 5.0.0. Many new bots have been added, with some removed as well. Also, a new technique for IDing bots has been implemented allowing one ID on the list to catch dozens or more bots IDs from the same site.

I've created a new site dedicated for all this bot-goodness which can be found at IPBots.info!

On the site you will find current download links, changelogs, and everything you need!

I've only had time to finish the Biggest Bot List, but others will be coming soon!

If you really do love my work, please consider making a small (or large!) donation with the link on the homepage.

>> Get what you need now at IPBots.info <<

Edited by Fred Derf

Recommended Posts

  • 0

Bot/Spider List 4.0.0 Released! Whats new?

  • 5 New bot lists.
  • 150 Dead bots removed.
  • 18 Bot entries beginning with "Mozilla" have been removed due to issues with the list logging members as bots.
  • 27 Updated bots (mostly campitilaztion errors.)
  • Changed list name.

The removal of (exactly) 150 dead bots should speed up the board - although you probably won't notice it.

All lists are now avaliable in .docx format.

It's quite an amazing format! Reduced the size of the .doc Bot List from 92KB to 25KB. It's even smaller than the .rtf format sitting at 46KB.

Enjoy! :D

  • 0
if i add the Spider List is this will archive my site in the search engines ?

No.

Any updates coming?

Yes. (see below)

Hey cooldude.

I was wondering if I could post a link to this guide in my Complete IPB Guide topic?

Thanks,

SadJoker

Sure, but please give credit.

Is this still alive?

Yes!

I've just started work on version 5.0.0. Going to add a BUNCH of new bots, remove dead ones, make a "Most Common" list for those who don't want all the extra tracking, and make the document prettier!

  • 0

Thanks Fred!

This is my first time posting this here at IPSBeyond, but many of you probably know about this from other sites like Invisionize or the official IPS forums.

After neglecting the bot lists for over a year, I have taken it upon myself to rebuild the lists from scratch. This has led to the creation of 5.0.0. Many new bots have been added, with some removed as well. Also, a new technique for IDing bots has been implemented allowing one ID on the list to catch dozens or more bots IDs from the same site.

I've created a new site dedicated for all this bot-goodness which can be found at IPBots.info!

On the site you will find current download links, changelogs, and everything you need!

I've only had time to finish the Biggest Bot List, but others will be coming soon!

If you really do love my work, please consider making a small (or large!) donation with the link on the homepage.

>> Get what you need now at IPBots.info << :thumbs:

  • 0

Before you use this list, I strongly recommend you run the following command on your IPB install:

TRUNCATE TABLE `ibf_spider_logs`

This will empty your bot logs and will remove old logs and remove some empty tables you will likely have as a result of many bot entries being removed.

Within 10 seconds of me running this command, my board was already logging 25+ new spider hits.

Again - this is highly recommended!

  • 0

I've drafted up a list of popular bots that seem to visit often and search engines that people use often.

Get it at http://ipbots.info/download.htm - only .doc and .docx are available atm.

Please let me know here if you feel anything should be added or taken off. The list will be final later this week if no one has anything to say.

  • 0

Biggest Bot List 5.0.3 has been released!

  • Changes:
  • 9 New bots have been added

Simply download the list from ipbots.info and grab the changes off the first page if you are already using 5.0.2!

Core Bot List 1.0.1 has been released!

  • Changes:
  • Fixed typo in Gsa-Crawler entry. (Thanks NetPanther!)

Simply download the list from ipbots.info and grab the changes off the first page if you are already using 1.0.0!

(Can a mod please change the 5.0.1 in the title to 5.0.3?)

  • 0

5.0.4 has been released. The only difference is the Ack entry has been removed due to it causing legitimate guests to be treated as bots if they had the ImageShack toolbar installed.

Get it at ipbots.info

  • 0

Biggest Bot List 5.0.5 has been released!

  • Changes:
  • Fixed Accoona typo (thanks Axel Wers!)
  • Removed duplicate entry of eBot (thanks Axel Wers!)
  • 4 New bots (thanks to GreasyTech for 2 contributions!)

Simply download the list from ipbots.info and grab the new list!

(You are invited to say "thanks!" if you like this!)

This topic is now closed to further replies.
  • Recently Browsing   0 members

    • No registered users viewing this page.
  • Posts

    • But it opens the floodgates to anyone who gets a refund instead of a replacement (since refund will buy you a 1/3rd of the capacity it did years ago)
    • He has planned to file a lawsuit in small claims court so it'll only be a $1000 lesson assuming he wins. That's likely a fraction of what Samsung spends on toilet paper on a daily basis.
    • Windows Server gets DNS over HTTPS (DoH) support by Usama Jawad For the past few months, Microsoft has been previewing DNS over HTTPS (DoH) for Windows DNS Server, touting it as a foundational upgrade for zero-trust enterprise networks. It essentially introduces encrypted, authenticated DNS for the networks rather than transmitting DNS traffic in clear. Now, the company has introduced the general availability (GA) of this feature. The GA of DoH encourages organizations to deploy the solution in production environments without implementing a new client-to-resolver architecture. DoH helps improve the overall security of the network and reduces the risk of spoofing due to its zero-trust design. This is a significant change because pretty much every interaction with the network requires interfacing with DNS. DoH offers several advantages over standard DNS traffic, such as encryption using HTTPS, preventing unauthorized inspection, man-in-the-middle attacks, and traffic analysis. Since it leverages TLS certificates so that clients can verify the identity of the DNS server, it prevents spoofing through this authentication mechanism. Additionally, it's built on the DoH standard defined by the Internet Engineering Task Force (IETF), which means that it should work with modern RFC 8484-compliant clients. Finally, it integrates into the existing network architecture seamlessly and can even run in parallel with standard DNS, so that customers can migrate to the new technology at their own pace. Microsoft says that in the past few months of preview, DoH has become more stable, and customers can confidently deploy it in production environments with proper guidance. Microsoft has emphasized that migrating to DoH is necessary for organizations that are moving toward zero-trust DNS solutions. Windows clients already support DoH, but the latest availability on Windows Server provides encrypted DNS to all endpoints. The company has also mentioned that "while this release focuses on encrypting client-to-resolver communication, support for encrypted communication between Windows DNS Server and upstream DNS resolvers is planned for a future update." You can follow Microsoft's guidance to deploy DoH here, but keep in mind that you need a Windows Server 2025 installation with the latest Patch Tuesday updates installed.
    • Lol I had one of these turn faulty in Jan, guess it wasn't just bad luck lol
    • I'm team Rossmann all the way. I have the exact same NVME, altough not in an array like him.
  • Recent Achievements

    • Week One Done
      davidbazooked earned a badge
      Week One Done
    • One Month Later
      Jamswaz earned a badge
      One Month Later
    • Week One Done
      Jamswaz earned a badge
      Week One Done
    • Rookie
      Marzoid went up a rank
      Rookie
    • Community Regular
      coch went up a rank
      Community Regular
  • Popular Contributors

    1. 1
      +primortal
      511
    2. 2
      PsYcHoKiLLa
      184
    3. 3
      +Edouard
      159
    4. 4
      Steven P.
      83
    5. 5
      ATLien_0
      75
  • Tell a friend

    Love Neowin? Tell a friend!