All Versions
13
Latest Version
Avg Release Cycle
79 days
Latest Release
3477 days ago

Changelog History
Page 1

  • v0.7.2 Changes

    May 30, 2012
    • ๐Ÿ› Bug fixes

      • Fix bug causing anchor links to have '#' converted to '%23'
  • v0.7.1 Changes

    January 20, 2012
    • Minor enhancements

      • Switch from robots gem (which people reported problems with) to new robotex gem
    • ๐Ÿ› Bug fixes

      • Fix incorrect default file extension for KyotoCabinet
  • v0.7.0 Changes

    January 19, 2012
    • Major enhancements

      • Added support for SQLite3 and Kyoto Cabinet storage
    • Minor enhancements

      • Added Page#base to use base HTML element
      • Use bundler for development dependencies
    • ๐Ÿ› Bug fixes

      • Encode characters in URLs
      • Fix specs to run under rake
      • Fix handling of redirect_to in storage adapters
  • v0.6.1 Changes

    February 24, 2011
    • ๐Ÿ› Bug fixes

      • Fix a bug preventing SSL connections from working
  • v0.6.0 Changes

    February 17, 2011
    • Major enhancements

      • Added support for HTTP Basic Auth with URLs containing a username and password
      • Added support for anonymous HTTP proxies
    • Minor enhancements

      • Added read_timeout option to set the HTTP request timeout in seconds
    • ๐Ÿ› Bug fixes

      • Don't fatal error if a page request times out
      • Fix double encoding of links containing %20
  • v0.5.0 Changes

    September 01, 2010
    • Major enhancements

      • Added page storage engines for MongoDB and Redis
    • Minor enhancements

      • Use xpath for link parsing instead of CSS (faster) (Marc Seeger)
      • Added skip_query_strings option to skip links with query strings (Joost Baaij)
    • ๐Ÿ› Bug fixes

      • Only consider status code 300..307 a redirect (Marc Seeger)
      • Canonicalize redirect links (Marc Seeger)
  • v0.4.0 Changes

    April 08, 2010
    • Major enchancements

      • Cookies can be accepted and sent with each HTTP request.
  • v0.3.2 Changes

    February 04, 2010
    • ๐Ÿ› Bug fixes

      • Fixed issue that allowed following redirects off the original domain
  • v0.3.1 Changes

    January 22, 2010
    • Minor enhancements

      • Added an attr_accessor to Page for the HTTP response body
    • ๐Ÿ› Bug fixes

      • Fixed incorrect method calls in CLI scripts
  • v0.3.0 Changes

    December 15, 2009
    • Major enchancements

      • Option for persistent storage of pages during crawl with TokyoCabinet or PStore
    • Minor enhancements

      • Options can be set via methods on the Core object in the crawl block