All Versions
13
Latest Version
Avg Release Cycle
79 days
Latest Release
4346 days ago

Changelog History
Page 1

  • v0.7.2 Changes

    May 30, 2012

    • 🐛 Bug fixes

      • Fix bug causing anchor links to have '#' converted to '%23'

  • v0.7.1 Changes

    January 20, 2012

    • Minor enhancements

      • Switch from robots gem (which people reported problems with) to new robotex gem

    • 🐛 Bug fixes

      • Fix incorrect default file extension for KyotoCabinet

  • v0.7.0 Changes

    January 19, 2012

    • Major enhancements

      • Added support for SQLite3 and Kyoto Cabinet storage

    • Minor enhancements

      • Added Page#base to use base HTML element
      • Use bundler for development dependencies

    • 🐛 Bug fixes

      • Encode characters in URLs
      • Fix specs to run under rake
      • Fix handling of redirect_to in storage adapters

  • v0.6.1 Changes

    February 24, 2011

    • 🐛 Bug fixes

      • Fix a bug preventing SSL connections from working

  • v0.6.0 Changes

    February 17, 2011

    • Major enhancements

      • Added support for HTTP Basic Auth with URLs containing a username and password
      • Added support for anonymous HTTP proxies

    • Minor enhancements

      • Added read_timeout option to set the HTTP request timeout in seconds

    • 🐛 Bug fixes

      • Don't fatal error if a page request times out
      • Fix double encoding of links containing %20

  • v0.5.0 Changes

    September 01, 2010

    • Major enhancements

      • Added page storage engines for MongoDB and Redis

    • Minor enhancements

      • Use xpath for link parsing instead of CSS (faster) (Marc Seeger)
      • Added skip_query_strings option to skip links with query strings (Joost Baaij)

    • 🐛 Bug fixes

      • Only consider status code 300..307 a redirect (Marc Seeger)
      • Canonicalize redirect links (Marc Seeger)

  • v0.4.0 Changes

    April 08, 2010

    • Major enchancements

      • Cookies can be accepted and sent with each HTTP request.

  • v0.3.2 Changes

    February 04, 2010

    • 🐛 Bug fixes

      • Fixed issue that allowed following redirects off the original domain

  • v0.3.1 Changes

    January 22, 2010

    • Minor enhancements

      • Added an attr_accessor to Page for the HTTP response body

    • 🐛 Bug fixes

      • Fixed incorrect method calls in CLI scripts

  • v0.3.0 Changes

    December 15, 2009

    • Major enchancements

      • Option for persistent storage of pages during crawl with TokyoCabinet or PStore

    • Minor enhancements

      • Options can be set via methods on the Core object in the crawl block