Projects / grub-client


Grub-client is a distributed crawling client, used to create an infrastructure that provides URL update status information for Web pages on the Internet. Grub's distributed crawler network will enable Web sites, content providers, and individuals to notify others that changes have occurred in their content, all in real time. Clients are ranked by the numbers of URLs that are crawled, both on their own machines and other servers.

Operating Systems

Recent releases

  •  21 Nov 2002 01:11

    Release Notes: The speed of the crawling process has been increased. Proxy functionality, bandwidth limiting, and a man page have been added. The GUI look has been improved. The entire project has been converted to use pthreads. Several bugs related to memory leaks, handling of threads, the GUI, crawler, and GUI shutdown have been fixed.

    •  01 Nov 2002 13:31

      Release Notes: This is the initial native Windows GUI/Screensaver client.

      •  13 Apr 2002 03:00

        Release Notes: Use of cURL with threads, rewriting the archiver to use MetaKit, tons of bugfixes, a new UIFace Class for a Windows GUI, fixes to the status information class, new per run features and nice kill handling, and bad or down host detection.

        •  21 Nov 2001 21:41

          Release Notes: Lots of new features, including a text GUI with stats, better bandwidth limiting, host protection, a completely rewritten crawler class, and an order of magnitude increase in crawling capabilities. The new crawler displays your daily ranking, so you can see where stand in comparison to the other Grubsters.


          Project Spotlight


          A Fluent OpenStack client API for Java.


          Project Spotlight

          TurnKey TWiki Appliance

          A TWiki appliance that is easy to use and lightweight.