| 217 different robots | Hits | Bandwidth | Last visit |
|---|
| Yahoo Slurp | 5472201+36443 | 449.87 GB | 31 Dec 2008 - 20:58 |
| MSNBot | 3256761+53509 | 175.07 GB | 31 Dec 2008 - 20:58 |
| Googlebot | 2690239+3014 | 187.68 GB | 31 Dec 2008 - 20:57 |
| Unknown robot (identified by 'crawl') | 1737368+13444 | 87.65 GB | 31 Dec 2008 - 20:56 |
| Voila | 662024+256087 | 51.18 GB | 31 Dec 2008 - 19:27 |
| Unknown robot (identified by 'robot') | 667739+11832 | 37.02 GB | 31 Dec 2008 - 20:53 |
| HTTrack off-line browser | 578593+528 | 36.42 GB | 31 Dec 2008 - 20:56 |
| Yandex bot | 549467+953 | 39.35 GB | 31 Dec 2008 - 19:33 |
| MSNBot-media | 470107+24000 | 43.44 GB | 31 Dec 2008 - 20:57 |
| WebVac | 342392+45 | 48.74 GB | 20 Dec 2008 - 00:17 |
| Ask | 249192+5203 | 21.23 GB | 31 Dec 2008 - 20:45 |
| Ocelli | 232841+98 | 14.40 GB | 31 Dec 2008 - 04:32 |
| GigaBot | 226199+5706 | 54.40 GB | 31 Dec 2008 - 20:52 |
| vse | 215589+31 | 1.01 GB | 10 Dec 2008 - 03:57 |
| Ultraseek | 187263+3509 | 17.55 GB | 01 Dec 2008 - 13:05 |
| java | 187191+504 | 13.44 GB | 31 Dec 2008 - 20:47 |
| Heritrix | 164843+771 | 17.32 GB | 30 Dec 2008 - 05:16 |
| Unknown robot (identified by hit on 'robots.txt') | 0+148991 | 48.41 MB | 31 Dec 2008 - 20:53 |
| \wbot[\/\-] | 139969+7421 | 95.49 GB | 31 Dec 2008 - 20:38 |
| Nutch | 132605+4746 | 22.58 GB | 31 Dec 2008 - 20:51 |
| DoCoMo | 109123+213 | 18.30 GB | 31 Dec 2008 - 19:34 |
| Unknown robot (identified by 'spider') | 95562+6059 | 3.43 GB | 31 Dec 2008 - 20:54 |
| Seekbot | 66338+289 | 513.81 MB | 31 Dec 2008 - 14:30 |
| Exabot | 64438+1324 | 659.98 MB | 31 Dec 2008 - 20:44 |
| psbot | 53383+3744 | 1.08 GB | 17 Dec 2008 - 21:39 |
| MJ12bot | 55278+1016 | 6.23 GB | 31 Dec 2008 - 09:48 |
| yahoo!\sslurp\schina | 47826+2761 | 2.80 GB | 31 Dec 2008 - 20:52 |
| Yahoo-MMCrawler | 46211+2756 | 52.67 MB | 23 Dec 2008 - 22:29 |
| Turn It In | 43586+225 | 4.10 GB | 05 Dec 2008 - 06:26 |
| noxtrumbot | 43110+62 | 7.81 GB | 15 May 2008 - 16:59 |
| W3C Link Checker | 20385+16479 | 902.16 MB | 31 Dec 2008 - 20:43 |
| BaiDuSpider | 32311+1780 | 2.64 GB | 31 Dec 2008 - 20:49 |
| BecomeBot | 30625+1220 | 591.66 MB | 31 Dec 2008 - 02:30 |
| fast\senterprise\scrawler | 30889+245 | 2.36 GB | 31 Dec 2008 - 20:58 |
| UbiCrawler | 29967+115 | 3.96 GB | 24 May 2008 - 21:48 |
| Vagabondo | 27197+699 | 3.09 GB | 31 Dec 2008 - 20:27 |
| ichiro | 25743+86 | 2.13 GB | 16 Dec 2008 - 12:13 |
| Speedy Spider | 20692+3120 | 6.34 GB | 31 Dec 2008 - 16:44 |
| larbin | 21722+1815 | 553.39 MB | 31 Dec 2008 - 15:43 |
| Alexa (IA Archiver) | 19038+862 | 9.00 GB | 31 Dec 2008 - 19:03 |
| Gaisbot | 19285+337 | 935.79 MB | 27 Dec 2008 - 16:50 |
| MSIECrawler | 14311+1622 | 1.08 GB | 31 Dec 2008 - 17:28 |
| web\sdownloader | 15616 | 763.65 MB | 31 Dec 2008 - 02:55 |
| MOMspider | 15585+16 | 345.05 MB | 21 Dec 2008 - 22:57 |
| WebCopy | 15298 | 7.92 MB | 31 Dec 2008 - 10:38 |
| yacy | 14276+522 | 2.76 GB | 31 Dec 2008 - 18:38 |
| boitho.com-dc | 13169+562 | 103.04 MB | 26 Dec 2008 - 06:35 |
| Walhello appie | 12847+456 | 948.84 MB | 01 Nov 2008 - 21:56 |
| tencenttraveler | 10814 | 2.50 GB | 31 Dec 2008 - 08:53 |
| StackRambler | 8334+145 | 100.10 MB | 29 Dec 2008 - 18:12 |
| Python-urllib | 5502+2738 | 6.44 GB | 31 Dec 2008 - 20:58 |
| Webdup | 7308 | 785.48 MB | 22 Oct 2008 - 12:55 |
| Grub.org | 5789+587 | 170.36 MB | 31 Dec 2008 - 20:54 |
| Geniebot | 6280+59 | 157.73 MB | 01 Aug 2008 - 13:36 |
| NG 1.x (Exalead) | 5277+51 | 84.81 MB | 16 Dec 2008 - 09:12 |
| Jakarta commons-httpclient | 4618+504 | 568.39 MB | 31 Dec 2008 - 18:38 |
| ConveraCrawler | 4618+15 | 1.04 GB | 29 Nov 2008 - 07:50 |
| Sunrise | 4556 | 74.02 MB | 23 Nov 2008 - 01:43 |
| aipbot | 4541+11 | 558.12 MB | 18 Nov 2008 - 02:33 |
| WebReaper | 4336+8 | 775.24 MB | 28 Dec 2008 - 20:09 |
| The web archive (IA Archiver) | 0+3843 | 1.32 MB | 31 Dec 2008 - 20:01 |
| Biglotron | 3698+37 | 195.57 MB | 29 Sep 2008 - 10:30 |
| YahooSeeker Yahoo! Blog crawler | 3285+248 | 9.55 MB | 27 Dec 2008 - 22:09 |
| xenu\slink\ssleuth | 2855 | 101.32 MB | 31 Dec 2008 - 14:02 |
| CFNetwork | 2628+1 | 2.99 GB | 31 Dec 2008 - 09:00 |
| Powermarks | 2555 | 53.32 MB | 31 Dec 2008 - 17:06 |
| Voyager | 2302+90 | 148.59 MB | 23 Dec 2008 - 17:51 |
| microsoft\surl\scontrol | 2120 | 121.11 MB | 31 Dec 2008 - 18:32 |
| LinkChecker | 1695+395 | 1.63 MB | 29 Dec 2008 - 12:04 |
| Checkbot | 2089 | 9.93 MB | 31 Dec 2008 - 05:50 |
| SuperBot | 1959 | 144.41 MB | 31 Dec 2008 - 05:24 |
| BSpider | 1564+256 | 118.20 MB | 31 Dec 2008 - 19:23 |
| up\.browser | 1788+9 | 26.65 MB | 31 Dec 2008 - 10:24 |
| KnowItAll | 1721+31 | 155.15 MB | 26 Sep 2008 - 02:55 |
| NG 2.x (Exalead) | 1370+278 | 424.62 MB | 28 Dec 2008 - 23:01 |
| Netluchs | 1520+111 | 19.63 MB | 20 Dec 2008 - 06:33 |
| LinkScan | 1604 | 2.36 GB | 31 Dec 2008 - 13:48 |
| NutchCVS | 1535+58 | 90.06 MB | 09 Dec 2008 - 16:50 |
| Internet Shinchakubin | 1383+12 | 265.48 MB | 30 Nov 2008 - 10:41 |
| Holmes | 779+318 | 43.28 MB | 31 Dec 2008 - 09:26 |
| EasyDL | 968 | 58.46 MB | 27 Dec 2008 - 02:26 |
| MSRBOT | 816+131 | 414.42 MB | 15 Nov 2008 - 11:30 |
| SBIder | 509+433 | 7.62 MB | 23 Mar 2008 - 05:16 |
| SeznamBot | 699+237 | 37.09 MB | 31 Dec 2008 - 13:06 |
| WordPress | 883 | 150.70 MB | 29 Dec 2008 - 07:08 |
| wells\ssearch | 881 | 18.03 MB | 30 Dec 2008 - 04:42 |
| WWWC Ver 0.2.5 | 864 | 5.74 MB | 26 Dec 2008 - 00:57 |
| InfoBot | 847+13 | 22.76 MB | 30 Dec 2008 - 14:15 |
| Ingrid | 329+312 | 2.46 MB | 29 Dec 2008 - 05:36 |
| ASPseek | 625 | 5.08 MB | 31 Dec 2008 - 18:09 |
| IBM Almaden Research Center WebFountain™ | 352+266 | 1.01 GB | 09 Dec 2008 - 18:11 |
| Netcraft | 589+2 | 168.71 MB | 31 Dec 2008 - 15:44 |
| Custo | 512+18 | 166.75 MB | 30 Dec 2008 - 20:03 |
| ms\ssearch\s4\.0\srobot | 337+188 | 13.01 MB | 31 Dec 2008 - 07:29 |
| HTMLParser | 511 | 25.50 MB | 24 Nov 2008 - 03:47 |
| MojeekBot | 358+120 | 7.39 MB | 16 Dec 2008 - 08:22 |
| T-H-U-N-D-E-R-S-T-O-N-E | 427+22 | 7.35 MB | 10 Dec 2008 - 05:01 |
| oracle\sultra\ssearch | 435 | 54.54 MB | 18 Dec 2008 - 02:35 |
| Fish search | 430 | 33.41 MB | 30 Dec 2008 - 07:42 |
| BBot | 424+3 | 200.09 MB | 30 Dec 2008 - 20:06 |
| lwp-request | 359 | 864.93 KB | 31 Dec 2008 - 12:13 |
| Spiderman | 302+54 | 4.17 MB | 25 Nov 2008 - 19:46 |
| lwp-trivial | 328+23 | 13.66 MB | 23 Dec 2008 - 06:41 |
| WebCollage | 351 | 11.18 MB | 31 Dec 2008 - 05:55 |
| lycos_ | 223+118 | 1.05 MB | 13 Mar 2008 - 14:19 |
| archive.org bot | 305+31 | 182.66 MB | 15 Dec 2008 - 17:40 |
| Dumbot | 318+4 | 45.50 MB | 25 Aug 2008 - 18:31 |
| IRLbot | 278+15 | 7.31 MB | 20 Apr 2008 - 21:57 |
| Emacs-w3 Search Engine | 261 | 4.38 MB | 27 Dec 2008 - 19:18 |
| JoBo Java Web Robot | 229+1 | 816.22 KB | 23 Nov 2008 - 01:44 |
| The World Wide Web Worm | 194 | 681.93 KB | 25 Dec 2008 - 16:26 |
| Robozilla | 190 | 4.08 MB | 01 Dec 2008 - 02:45 |
| Web Core / Roots | 176+7 | 2.65 MB | 21 Dec 2008 - 04:35 |
| UCSD Crawl | 155+10 | 3.66 MB | 29 Oct 2008 - 11:29 |
| Perman surfer | 157+3 | 4.38 MB | 23 Oct 2008 - 03:01 |
| omniexplorer\_bot | 96+56 | 240.82 MB | 19 Nov 2008 - 23:12 |
| arks | 117+8 | 271.19 KB | 29 Dec 2008 - 15:41 |
| Google AdSense | 119 | 5.08 MB | 26 Dec 2008 - 11:51 |
| Combine System | 55+36 | 80.01 MB | 27 Nov 2008 - 03:33 |
| LinkWalker | 45+45 | 17.54 KB | 26 Dec 2008 - 07:41 |
| Girafabot | 0+86 | 30.15 KB | 22 Dec 2008 - 16:49 |
| php\sversion\stracker | 83 | 0 | 23 Nov 2008 - 05:45 |
| Metaspinner | 76+7 | 1.62 MB | 21 Jul 2008 - 16:14 |
| FavOrg | 75 | 6.39 MB | 23 Dec 2008 - 09:36 |
| test\scrawler | 43+21 | 1.19 MB | 19 Apr 2008 - 00:52 |
| Motor | 61 | 766.73 KB | 26 Nov 2008 - 06:50 |
| TestBot | 54+6 | 23.40 MB | 20 Nov 2008 - 03:16 |
| GoForIt.com | 51+5 | 1.03 MB | 11 Feb 2008 - 07:02 |
| W3C Validator | 55 | 571.74 KB | 29 Dec 2008 - 13:33 |
| Y!J Yahoo Japan | 44+10 | 2.25 MB | 19 Dec 2008 - 11:12 |
| Cfetch | 52 | 0 | 20 Feb 2008 - 07:53 |
| SurveyBot | 0+51 | 17.88 KB | 31 Dec 2008 - 02:25 |
| nicebot | 32+17 | 524.53 KB | 03 Oct 2008 - 22:09 |
| missigua\slocator | 48 | 1.03 MB | 19 Dec 2008 - 23:45 |
| ASpider (Associative Spider) | 39 | 87.13 KB | 20 Oct 2008 - 20:21 |
| LinkBot | 33 | 0 | 16 Nov 2008 - 20:42 |
| ZyBorg | 30 | 220.52 KB | 08 Sep 2008 - 03:23 |
| Muncher | 25+1 | 234.23 KB | 15 Jul 2008 - 05:02 |
| tcl\shttp\sclient\spackage | 25 | 131.09 KB | 15 Jan 2008 - 15:31 |
| Steeler | 21+2 | 190.76 KB | 16 Dec 2008 - 09:55 |
| Scooter | 7+16 | 170.40 KB | 03 Dec 2008 - 09:33 |
| CyberSpyder Link Test | 22 | 0 | 11 Mar 2008 - 20:23 |
| ht://Dig | 0+20 | 5.26 KB | 21 Nov 2008 - 18:16 |
| Susie | 20 | 259.37 KB | 25 Dec 2008 - 02:21 |
| Feedburner | 19 | 0 | 27 Nov 2008 - 15:53 |
| Digger | 17+2 | 4.21 MB | 23 Sep 2008 - 21:57 |
| HKU WWW Octopus | 18 | 30.75 KB | 28 Feb 2008 - 18:26 |
| Fluid Dynamics Search Engine robot | 10+7 | 163.06 KB | 14 Sep 2008 - 11:37 |
| Sleek | 17 | 0 | 31 Dec 2008 - 20:28 |
| lmspider | 11+6 | 625.72 KB | 14 Nov 2008 - 20:10 |
| Alkaline | 16 | 0 | 04 Aug 2008 - 05:17 |
| xenu's\slink\ssleuth | 16 | 0 | 18 Dec 2008 - 23:03 |
| Zeus Webster Pro | 16 | 103.11 KB | 09 Dec 2008 - 06:28 |
| mnoGoSearch search engine software | 12+4 | 83.76 KB | 10 Dec 2008 - 09:07 |
| z\-add\slink\schecker | 15 | 0 | 23 Nov 2008 - 05:34 |
| Miva | 15 | 16.98 KB | 04 Jul 2008 - 05:05 |
| link\svalet\sonline | 14 | 0 | 20 Oct 2008 - 13:55 |
| NimbleCrawler | 10+4 | 58.76 KB | 10 May 2008 - 00:10 |
| yahoo!\smindset | 14 | 22.76 MB | 17 Mar 2008 - 12:14 |
| Cursor | 13 | 1.03 MB | 29 Dec 2008 - 10:32 |
| Kevin | 12 | 20.63 KB | 10 Jul 2008 - 15:16 |
| Feedfetcher-Google | 10 | 27.04 KB | 15 Nov 2008 - 11:07 |
| GetURL | 9 | 79.07 KB | 22 Sep 2008 - 15:02 |
| Boris | 9 | 1.96 MB | 15 Dec 2008 - 13:37 |
| Snappy | 8+1 | 245.41 KB | 22 Nov 2008 - 23:09 |
| TITAN | 8+1 | 324.73 KB | 30 Apr 2008 - 18:36 |
| nasa\ssearch | 7 | 27.73 MB | 10 Aug 2008 - 17:50 |
| w3c\_css\_validator\_jfouffa | 7 | 74.09 KB | 29 Dec 2008 - 13:33 |
| SpiderBot | 4+3 | 206.28 KB | 08 Jan 2008 - 09:56 |
| sogou\sspider | 6 | 195.04 KB | 10 May 2008 - 19:18 |
| Fast-Webcrawler | 6 | 99.14 KB | 14 Jul 2008 - 05:19 |
| SpiderView(tm) | 5 | 16.82 KB | 31 Dec 2008 - 00:36 |
| Openfind data gatherer | 5 | 231.27 KB | 04 May 2008 - 05:43 |
| Sven | 3+2 | 97.39 KB | 30 Oct 2008 - 06:46 |
| ActiveBookmark | 4 | 6.52 KB | 23 Nov 2008 - 01:44 |
| HPPrint | 4 | 24.58 KB | 08 Sep 2008 - 23:31 |
| Wild Ferret Web Hopper #1, #2, #3 | 4 | 519.10 KB | 02 Dec 2008 - 11:34 |
| The Web Wombat | 2+2 | 61.60 KB | 09 Oct 2008 - 07:33 |
| Orbiter | 4 | 48.68 KB | 28 Nov 2008 - 14:10 |
| ComputingSite Robi/1.0 | 4 | 44.69 KB | 22 Feb 2008 - 10:59 |
| Pioneer | 4 | 1.44 MB | 21 Nov 2008 - 11:45 |
| gpu\sp2p\scrawler | 2+2 | 2.55 KB | 13 Jun 2008 - 23:01 |
| RoboCrawl Spider | 0+3 | 1.05 KB | 20 Jun 2008 - 07:17 |
| INFOMINE VLCrawler | 3 | 9.41 KB | 30 Dec 2008 - 17:44 |
| Harvest | 2+1 | 15.81 KB | 15 Dec 2008 - 21:54 |
| Felix IDE | 3 | 61.20 KB | 08 Mar 2008 - 10:50 |
| MagpieRSS | 3 | 23.04 KB | 30 Dec 2008 - 10:12 |
| PhpDig | 1+2 | 359 Bytes | 21 Jan 2008 - 14:16 |
| VORTEX | 3 | 10.09 KB | 06 Oct 2008 - 15:11 |
| Findlinks | 1+1 | 3.95 KB | 28 Oct 2008 - 13:54 |
| arianna.libero.it | 0+2 | 718 Bytes | 12 Nov 2008 - 18:09 |
| NetMechanic | 1+1 | 359 Bytes | 06 Oct 2008 - 06:18 |
| Bookmark-Manager | 2 | 0 | 27 Dec 2008 - 07:16 |
| google-sitemaps | 2 | 30 Bytes | 20 Mar 2008 - 22:15 |
| Suke | 2 | 79.31 KB | 14 Nov 2008 - 18:54 |
| UdmSearch | 2 | 2.32 KB | 28 Dec 2008 - 17:29 |
| Bloglines | 2 | 5.27 KB | 11 Jul 2008 - 09:29 |
| RoadHouse Crawling System | 1 | 24.84 KB | 21 Sep 2008 - 18:49 |
| JBot Java Web Robot | 1 | 43.80 KB | 02 Oct 2008 - 04:40 |
| gammaSpider, FocusedCrawler | 1 | 38.97 KB | 14 Nov 2008 - 15:20 |
| xirq | 1 | 20.37 KB | 08 Aug 2008 - 09:03 |
| sohu agent | 1 | 30.42 KB | 26 Aug 2008 - 22:45 |
| ets\sv | 1 | 55.66 KB | 24 May 2008 - 04:41 |
| EARTHCOM.info | 1 | 173.71 KB | 14 Jul 2008 - 07:49 |
| XGET | 1 | 79.51 KB | 14 Jul 2008 - 22:47 |
| Accoona-AI-Agent | 1 | 1.04 KB | 10 Jul 2008 - 13:19 |
| Bumblebee (relevare.com) | 1 | 2.35 KB | 14 Jul 2008 - 15:34 |
| ExactSeek Crawler | 1 | 1.04 KB | 11 Jul 2008 - 17:15 |
| BlogSearch | 1 | 2.25 KB | 15 Jul 2008 - 13:28 |
| Argus | 1 | 1.04 KB | 11 Jul 2008 - 12:38 |
| RoJo aggregator | 1 | 34.02 KB | 23 Nov 2008 - 09:41 |
| legs | 1 | 47.99 KB | 05 Jun 2008 - 20:23 |
| Fooky.com/ScorpionBot/ScoutOut | 1 | 6.64 KB | 10 Jul 2008 - 10:22 |
| XYLEME Robot | 1 | 1.47 KB | 13 Jul 2008 - 11:30 |
| Sift | 1 | 43.12 KB | 16 Jul 2008 - 05:49 |
| fido | 1 | 30.64 KB | 17 Nov 2008 - 04:02 |
| Spry Wizard Robot | 1 | 47.49 KB | 22 Mar 2008 - 03:11 |