10

8

6

4

2


8.8

4.7

7.5
0.0

6.9
0.0

5.7
1.8

5.5

4.9

5.1
0.0

9 Web Crawling gems and projects

  • Mechanize

    8.8 4.7 C Ruby
    Mechanize is a ruby library that makes automated web interaction easy.
  • anemone

    7.5 0.0 A Ruby
    Ruby library and CLI for crawling websites.
  • Upton

    6.9 0.0 A Ruby
    A batteries-included framework for easy web-scraping.
  • Wombat

    5.7 1.8 A Ruby
    Web scraper with an elegant DSL that parses structured data from web pages.
  • FastImage

    5.5 4.9 A Ruby
    FastImage finds the size or type of an image given its uri by fetching as little as needed
  • pismo

    5.1 0.0 A Ruby
    Extracts machine-readable metadata and content from Web pages
  • MetaInspector

    5.0 5.1 A Ruby
    Ruby gem for web scraping purposes.
  • LinkThumbnailer

    3.6 4.3 A Ruby
    Ruby gem that generates thumbnail images and videos from a given URL. Much like popular social website with link preview.
  • Screencap

    2.3 0.0 JavaScript
    A gem to screencap webpages in ruby. Uses Phantom.js under the hood.

Add another 'Web Crawling' Gem