Ruby Web Crawling

8.9

3.4

Mechanize

7.6

0.0

anemone

7.2

0.0

Upton

6.6

6.1

Wombat

6.5

0.0

FastImage

6.2

5.6

MetaInspector

16 Web Crawling gems and projects

Mechanize

8.9 3.4 L4 Ruby

Mechanize is a ruby library that makes automated web interaction easy.
anemone

7.6 0.0 L5 Ruby

Anemone web-spider framework

Build time-series-based applications quickly and at scale.

InfluxDB is the Time Series Platform where developers build real-time applications for analytics, IoT and cloud-native services. Easy to start, it is available in the cloud or on-premises.

Promo www.influxdata.com

Upton

7.2 0.0 L5 HTML

A batteries-included framework for easy web-scraping. Just add CSS! (Or do more.)
Wombat

6.6 6.1 L5 Ruby

Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.
FastImage

6.5 0.0 L5 Ruby

FastImage finds the size or type of an image given its uri by fetching as little as needed
MetaInspector

6.2 5.6 L5 Ruby

Ruby gem for web scraping purposes. It scrapes a given URL, and returns you its title, meta description, meta keywords, links, images...
pismo

5.6 0.0 L5 Ruby

Extracts machine-readable metadata and content from Web pages
Spidr

5.6 10.0 Ruby

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
LinkThumbnailer

4.9 0.0 L5 Ruby

Ruby gem that fetches images and metadata from a given URL. Much like popular social website with link preview.
Vessel

3.9 2.1 Ruby

Fast high-level web crawling Ruby framework
Screencap

3.2 0.0 JavaScript

A gem to screencap webpages in ruby. Uses Phantom.js under the hood.
instabot.rb

2.8 0.0 Ruby

An instagram bot works without instagram api, only needs your username and password. written in ruby
The Hawker Ruby gem

1.8 0.0 Ruby

The Hawker gem is a web scraper which allows you to pull the basic information for given social media profile URL
Google Search Results in Ruby

1.3 0.0 Ruby

Google Search Results via SERP API Ruby Gem
Supplejack API

1.3 7.4 Ruby

Supplejack API Mountable Engine
Kimurai

0.6 2.1 Ruby

Kimurai is a modern web scraping framework written in Ruby which works out of box with Headless Chromium/Firefox, PhantomJS, or simple HTTP requests and allows to scrape and interact with Javascript rendered websites

* Code Quality Rankings and insights are calculated and provided by Lumnify.
They vary from L1 to L5 with "L5" being the highest.

Popular Comparisons

Add another 'Web Crawling' Gem