Ruby Web Crawling

14 Web Crawling gems and projects

• Mechanize

  8.9 0.0 L4 Ruby

  Mechanize is a ruby library that makes automated web interaction easy.

• anemone

  7.5 0.0 L5 Ruby

  Ruby library and CLI for crawling websites.

Get performance insights in less than 4 minutes

Scout APM uses tracing logic that ties bottlenecks to source code so you know the exact line of code causing performance issues and can get back to building a great product faster.

Sponsored scoutapm.com



• Upton

  7.2 0.0 L5 HTML

  A batteries-included framework for easy web-scraping.

• Wombat

  6.5 2.0 L5 Ruby

  Web scraper with an elegant DSL that parses structured data from web pages.

• FastImage

  6.3 5.0 L5 Ruby

  FastImage finds the size or type of an image given its uri by fetching as little as needed

• MetaInspector

  6.0 4.3 L5 Ruby

  Ruby gem for web scraping purposes.

• pismo

  5.6 0.0 L5 Ruby

  Extracts machine-readable metadata and content from Web pages

• LinkThumbnailer

  4.8 1.9 L5 Ruby

  Ruby gem that generates thumbnail images and videos from a given URL. Much like popular social website with link preview.

• Screencap

  3.2 0.0 JavaScript

  A gem to screencap webpages in ruby. Uses Phantom.js under the hood.

• instabot.rb

  2.6 0.0 Ruby

  An instagram bot works without instagram api, only needs your username and password. written in ruby

• Vessel

  2.6 4.0 Ruby

  Fast high-level web crawling Ruby framework

• The Hawker Ruby gem

  1.6 0.0 Ruby

  Get the profile information from Github, Twitter or Instagram without the API

• Supplejack API

  1.3 6.8 Ruby

  Supplejack API Mountable Engine

• Kimurai

  0.6 2.1 Ruby

  Kimurai is a modern web scraping framework written in Ruby which works out of box with Headless Chromium/Firefox, PhantomJS, or simple HTTP requests and allows to scrape and interact with Javascript rendered websites

* Code Quality Rankings and insights are calculated and provided by Lumnify.
They vary from L1 to L5 with "L5" being the highest. Visit our partner's website for more details.

Popular Comparisons

• Wombat
  vs
  Mechanize
• anemone
  vs
  Mechanize
• Mechanize
  vs
  MetaInspector
• Mechanize
  vs
  Upton
• Wombat
  vs
  anemone

SaaSHub - Software Alternatives and Reviews Sponsored www.saashub.com