10

8

6

4

2


9.0

6.5
Mechanize

7.5
0.0
anemone

7.2
0.0
Upton

6.7

4.3
FastImage

6.7

5.0
Wombat

6.3

8.3
MetaInspector

17 Web Crawling gems and projects

  • Mechanize

    9.0 6.5 L4 Ruby
    Mechanize is a ruby library that makes automated web interaction easy.

  • anemone

    7.5 0.0 L5 Ruby
    Anemone web-spider framework

    • Kargo automates promotion across dev, staging, and prod with approval gates and verification. Open source, built by the team behind Argo CD. Download now.
    Promo akuity.io
    Kargo Logo

  • Upton

    7.2 0.0 L5 HTML
    A batteries-included framework for easy web-scraping. Just add CSS! (Or do more.)

  • FastImage

    6.7 4.3 L5 Ruby
    FastImage finds the size or type of an image given its uri by fetching as little as needed

  • Wombat

    6.7 5.0 L5 Ruby
    Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.

  • MetaInspector

    6.3 8.3 L5 Ruby
    Ruby gem for web scraping purposes. It scrapes a given URL, and returns you its title, meta description, meta keywords, links, images...

  • Spidr

    5.8 5.5 Ruby
    A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

  • pismo

    5.6 0.0 L5 Ruby
    Extracts machine-readable metadata and content from Web pages

  • LinkThumbnailer

    5.1 5.5 L5 Ruby
    Ruby gem that fetches images and metadata from a given URL. Much like popular social website with link preview.

  • Vessel

    5.0 4.2 Ruby
    Fast high-level web crawling Ruby framework

  • Screencap

    3.5 0.0 JavaScript
    A gem to screencap webpages in ruby. Uses Phantom.js under the hood.

  • 🕷 Tanakai

    3.4 5.0 Ruby
    Tanakai is a modern web scraping framework written in Ruby. A fork of Kimurai.

  • instabot.rb

    3.0 0.0 Ruby
    An instagram bot works without instagram api, only needs your username and password. written in ruby

  • Google Search Results in Ruby

    2.3 2.0 Ruby
    Google Search Results via SERP API Ruby Gem

  • The Hawker Ruby gem

    2.1 0.0 Ruby
    The Hawker gem is a web scraper which allows you to pull the basic information for given social media profile URL

  • Supplejack API

    1.5 2.5 Ruby
    Supplejack API Mountable Engine

  • Kimurai

    0.6 2.1 Ruby
    DISCONTINUED. Kimurai is a modern web scraping framework written in Ruby which works out of box with Headless Chromium/Firefox, PhantomJS, or simple HTTP requests and allows to scrape and interact with Javascript rendered websites
AppSignal knows why the f*#k it crashed.
sponsored www.appsignal.com

Popular Comparisons

SaaSHub - Software Alternatives and Reviews
sponsored www.saashub.com

Add another 'Web Crawling' Gem